JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 28.09.2023, 12:38
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default Exhentai/E-hentai plugin doesn't always know how to handle mirror links

That issue actually been around for a long time but I never thought of reporting about it,

In some situations where one copy a Exhentai's/E-hentai's gallery link to Jdownloader, if some files got the same names few more times, it will auto exclude them as mirrored links and won't download them,

It for some reason also show them in the previews as if they got the same file size when they don't, and as result think they're a mirrors of the same file and just skip them rather than ask me if to download and rename them


EDIT:
Umm seems like changing the settings to always auto rename files solve it (???)
But it still doesn't explained why it originally detected the files in the wrong size as well as why it didn't asked the user what to do despite being set to ask by default ...

Last edited by Doom; 28.09.2023 at 12:44.
Reply With Quote
  #2  
Old 05.10.2023, 14:25
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,260
Default

Hi,
keep in mind that "File with same name exists" and "JD treats a file as mirror of another file" are two separate conditions.

In this case you either need to provide some example links for me so I can make sure that this won't happen anymore for this specific plugin and/or you need to change the JDownloader settings so that such files will not be treated as mirrors anymore:
Settings -> Advanced Settings -> GeneralSettings.mirrordetectiondecision -> FILENAME_FILESIZE
-> That might help you but we do not recommend changing this setting!

About the same/wrong filesize: This is just the result of the wrongly detected mirror so I do not consider this as a bug.
Detecting a mirror as mirror if it isn't a mirror, that however should not happen.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

-----------------------------------

On Vacation / Im Urlaub
Start: 2024-02-08
End: TBA


Last edited by pspzockerscene; 05.10.2023 at 14:25. Reason: Fixed typo
Reply With Quote
  #3  
Old 04.04.2024, 06:35
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Sorry for the huge delay and about promting this post again (a lot happened in my life in recent time),

I will give a try to these settings

Quote:
-> That might help you but we do not recommend changing this setting!
Since it happens a lot with with exhentai, even if the sizes doesn't match,

Why do you not recommand to change it though?
I rather Jdownloader always download the same file twice (with a name change like adding (1)) than skip file due to it being stupid not being able to tell when a file is a mirrored or not.
Reply With Quote
  #4  
Old 04.04.2024, 11:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,260
Default

Quote:
Originally Posted by Doom View Post
Since it happens a lot with with exhentai, even if the sizes doesn't match,
You misunderstood what it will do, in fact it looks like you haven't even tried it. Explanation:
When enabled, files need to have a matching filesize and name to be detected as a mirror.
In the end, your files with the same filenames which are not mirrors will be detected as "files with the same name but not mirror" and thus the auto rename (if configured) will do its job.

Quote:
Originally Posted by Doom View Post
[...]than skip file due to it being stupid not being able to tell when a file is a mirrored or not.
There are reasons for why JDownloaders internals work in the way they do.
I'am open for discussions and improvement suggestions but simply calling the existing behavior stupid won't help anyone to improve it.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

-----------------------------------

On Vacation / Im Urlaub
Start: 2024-02-08
End: TBA

Reply With Quote
  #5  
Old 05.04.2024, 03:31
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Quote:
You misunderstood what it will do, in fact it looks like you haven't even tried it. Explanation:
When enabled, files need to have a matching filesize and name to be detected as a mirror.
In the end, your files with the same filenames which are not mirrors will be detected as "files with the same name but not mirror" and thus the auto rename (if configured) will do its job.
I've actually tried it few secs after posting the old comment and it solved my issue (at least with the link I've tested it on), the auto rename failed me so many times even if I've set it right so I rather try this option,
When I tell JD to download something like an entire gallery, I want it to download everything, and I rather let it make duplications of the same files which I can later remove manually, than see JD just mistake it for a mirror and auto skip it,


Quote:
There are reasons for why JDownloaders internals work in the way they do.
I'am open for discussions and improvement suggestions but simply calling the existing behavior stupid won't help anyone to improve it.
Surely even if I don't understand or appriciate the technical depth behind the settings as the creator does, it still not means I shouldn't use it if at the end of the day it (may) solve my issue,

I'm not experinced enough to know how it work to give suggestions about it, but as I've said above I rather let JD to download the same file twice if I can, over letting it mistake the file as mirror and skip it,

If what you showed here

Quote:
Settings -> Advanced Settings -> GeneralSettings.mirrordetectiondecision -> FILENAME_FILESIZE
Will do that then I'm more than fine with it, as long as there aren't any other possible issues I should be aware of,

Also don't take the "stupid" part too serious, it was mostly meant to be a sarcastic joke in a manner of relief rather than being part of a sentence some mad kiddo who doesn't appriciate the app or the support they got for it will say ahah xD,

So you can ignore it.

Last edited by Doom; 05.04.2024 at 03:42.
Reply With Quote
  #6  
Old 05.04.2024, 11:53
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,260
Default

Quote:
Originally Posted by Doom View Post
the auto rename failed me so many times even if I've set it right so I rather try this option
The auto rename didn't fail, it wasn't even attempted.
I'll try one last time but you do not need to understand my answer; it's more important that it will work in the end which it seems like it does now:
In your case, those files were detected as a mirror but it was not(!) a mirror.
If JDownloader detects e.g. 4 files as a mirror and one gets downloaded, the other ones will be skipped.
TRhe auto rename will only happen if JD thinks sees "ohh two different files with the same filename".
The "fix" in this case is to make JD not detect those files as mirror.

Quote:
Originally Posted by Doom View Post
When I tell JD to download something like an entire gallery, I want it to download everything, and I rather let it make duplications of the same files which I can later remove manually, than see JD just mistake it for a mirror and auto skip it,
I understand your use case.

Quote:
Originally Posted by Doom View Post
Surely even if I don't understand or appriciate the technical depth behind the settings as the creator does, it still not means I shouldn't use it if at the end of the day it (may) solve my issue,
Sure and that is not what I wanted to say - in fact you should use the setting I told you about but you shouldn't need to use it.
Please post some links to ehentai galleries which contain multiple items with the same default filenames so I can look into this to make it work fine with default settings.

Quote:
Originally Posted by Doom View Post
Also don't take the "stupid" part too serious, it was mostly meant to be a sarcastic joke in a manner of relief rather than being part of a sentence some mad kiddo who doesn't appriciate the app or the support they got for it will say ahah xD,
No worries
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

-----------------------------------

On Vacation / Im Urlaub
Start: 2024-02-08
End: TBA

Reply With Quote
  #7  
Old 09.04.2024, 06:06
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Alright and thx,
I will try look for one,

But it's usually happens most of the times if you're trying to download galleries that got many files in them in general.
Reply With Quote
  #8  
Old 09.04.2024, 10:59
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,918
Default

Quote:
Originally Posted by Doom View Post
I will try look for one,
But it's usually happens most of the times if you're trying to download galleries that got many files in them in general.
We need to be able to reproduce the issue, so best is to provide example links where this issue happens for you, then we can look into it
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 03.05.2024, 06:05
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Sorry for the long delay, had to finish something with Jdownloader before change the settings back and test things,

So far after changing the settings to FILENAME_FILESIZE for a month now I didn't got a single issue,


But it seems like you updated the exhentai plugin recently?
Cause today I didn't managed to find a single mirrored link after changing the settings back, and there's also a new issue that appeared now:
Jdownloader no longer separate the archive rar files (Which cost site points) and the images (which don't cost points),

So now you can download by mistake something that can potentially ban your account unless you will separate it again.


As for the mirror issue, I need to keep testing till I've bump into it cause while it was easy to do before, I think some recent change might've fixed it (?)


One more additonal issue I actually want to bring up that been there for years is a situation where Jdownloader can't find an image in exhentai and will enter a loop where it's saying that the file is temporarily unavaliable,
And if you look on said links that it claim to be unavaliable, you sometimes will find out the image there is indeed dead or still working,
However exhentai got a "Reload broken image" in cases like this to reload the image, so will it be possible to make it so if Jdownloader can't find the pic from the link, it will look for the pic that is in the Reload broken image url instead?


Reply With Quote
  #10  
Old 03.05.2024, 12:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,260
Default

Quote:
Originally Posted by Doom View Post
Cause today I didn't managed to find a single mirrored link after changing the settings back
I haven't really changed anything regarding this on purpose but I did change the way, the imagepositions are generated so maybe that somehow avoided the problem.
Before:
Code:
[FANBOX] Zerodo⁄Degree123 (2023.3.7~2024.4.26)_2906046-0001.jpg
(Fixed 4-digit number padding)
Now:
Code:
[FANBOX] Zerodo⁄Degree123 (2023.3.7~2024.4.26)_2906046-001.jpg
(Padding now depends on max number of images in gallery [as long as that can be determined] e.g. up to 999 images, padding is 3 digits.)
URL used for testing:
Code:
e-hentai.org/g/2906046/5de8ba6bef/
Quote:
Originally Posted by Doom View Post
Jdownloader no longer separate the archive rar files (Which cost site points) and the images (which don't cost points),
You mean that archive- and loose images now go intop one package?
Indeed I did change this since the fact that .zip archives went into a separate package was just a mistake by me.
As before, I'am open for suggestions but before doing further changes to the plugin you may want to tell me why you are adding the .zip archives if you do not plan on downloading them?
You can just turn that off under Settings -> e-hentai.org -> Gallery crawl mode -> Do not crawl zip archive

Quote:
Originally Posted by Doom View Post
One more additonal issue I actually want to bring up that been there for years is a situation where Jdownloader can't find an image in exhentai and will enter a loop where it's saying that the file is temporarily unavaliable,
And if you look on said links that it claim to be unavaliable, you sometimes will find out the image there is indeed dead or still working,
However exhentai got a "Reload broken image" in cases like this to reload the image, so will it be possible to make it so if Jdownloader can't find the pic from the link, it will look for the pic that is in the Reload broken image url instead?
Please provide a log and example URLs:

Please post your log-ID here
If your report is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.
If your report is related to a login specific problem with a plugin supported website, enable debug mode before creating logs, see previously linked instructions.

Bitte poste deine Log-ID hier.
Falls dein Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.
Geht es um einen nicht funktionierenden Account-Login, aktiviere vor dem Erstellen deines Logs den debug Modus (siehe zuvor verlinkte Anleitung).

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

-----------------------------------

On Vacation / Im Urlaub
Start: 2024-02-08
End: TBA

Reply With Quote
  #11  
Old 05.05.2024, 09:25
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Quote:
I haven't really changed anything regarding this on purpose but I did change the way, the imagepositions are generated so maybe that somehow avoided the problem.
Umm it may did the trick, though I will need to test it more to tell.


Quote:
You mean that archive- and loose images now go intop one package?
Indeed I did change this since the fact that .zip archives went into a separate package was just a mistake by me.
As before, I'am open for suggestions but before doing further changes to the plugin you may want to tell me why you are adding the .zip archives if you do not plan on downloading them?
You can just turn that off under Settings -> e-hentai.org -> Gallery crawl mode -> Do not crawl zip archive

I think it should be separate by default even if it was a mistake by you,
The rars from the archives don't contain the same files, they contains the original full size ver of the images, which cost site points,
I downloaded a rar by mistake after the last changes you did and stopped it midway cause I knew it could get me into troubles,
Usually download the archive cost you points so I was surprised it download for me at all, but stopped it just in case cause I was afarid I may get banned if the site will detect I somehow bypass their point system,

I recommand separate it like before so that people could easily decide which one (full size or compressed ver) they want, and even if they want to download both, if they're already separated things will be better orginized between folders from the git-go.


Quote:
Please provide a log and example URLs:

Please post your log-ID here
If your report is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.
If your report is related to a login specific problem with a plugin supported website, enable debug mode before creating logs, see previously linked instructions.

That issue tend to happend to me often when I'm download from a gallery that got many GCs, but sometimes can happend on random galleries too,
The issue seem to rise from a load issue image on the site server end I think, but the site got a button to reload the image from the backup server if the user need to,

So rather than send a log, I was wondering if you can try to download these two links (which I recall to trigger the issue for me) first to see if it's triggerred on your end,

If you won't be able to trigger it yourself I will collect the logs,
The main reason I'm asking if you can spare me from the logs is that it's take time to download stuff from this gallery and the issue been triggerred for me many times on at least each one of the [Escu:de] Re;Lord galleries when I was testing things, so I think you should be able to trigger it fast enough by try to download from one or two of them and could see things firsthand.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Doom; 05.05.2024 at 09:27.
Reply With Quote
  #12  
Old 06.05.2024, 14:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,260
Default

Quote:
Originally Posted by Doom View Post
I think it should be separate by default even if it was a mistake by you,
Okay I've reverted that change so the .zip and loose images will go into separate packages again.
Said change will go live with the release of our next set of plugin updates.

Quote:
Originally Posted by Doom View Post
The rars from the archives don't contain the same files, they contains the original full size ver of the images, which cost site points,
That depends on the settings you are using.
If you configure JD to download the original images, the downloaded single images will be the same (original) files as inside the .zip file.

Quote:
Originally Posted by Doom View Post
So rather than send a log, I was wondering if you can try to download these two links (which I recall to trigger the issue for me) first to see if it's triggerred on your end,
Checking a log is much faster and easier for me.
Your links lead to galleries with over 2000 images in total.
I've test-downloaded 500 of them and I wasn't able to reproduce the problem.
Please provide a log.

I'll be on vacation and tomorrow is the last day I'm online before said vacation so sorry if my next answer will be late.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

-----------------------------------

On Vacation / Im Urlaub
Start: 2024-02-08
End: TBA

Reply With Quote
  #13  
Old 08.05.2024, 04:01
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Quote:
Okay I've reverted that change so the .zip and loose images will go into separate packages again.
Said change will go live with the release of our next set of plugin updates.
Thx.


Quote:
That depends on the settings you are using.
If you configure JD to download the original images, the downloaded single images will be the same (original) files as inside the .zip file.
I was referring to how the rars that JD will catch always be the one that got the original size no matter the settings (there are also rars for low size images too), and either way all rars cost points so it's better to download low size images instead.


Quote:
Checking a log is much faster and easier for me.
Your links lead to galleries with over 2000 images in total.
I've test-downloaded 500 of them and I wasn't able to reproduce the problem.
Please provide a log.
Will post soon, but I know this URL give me the issue currently so you can also try download form it while I wait for some of my stuff to finish download and then posting the log
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Also if I'm already on that, Jdownloader got the option to show download url when you right click on the file,
In exhentai even if you copy the file url from there it will always load the gallery page instead of the image page, that could've spare me a lot of time if it could really open the exact image url and lead me straight to the image that don't want to load when I've got the issue I've mentioed above.

Quote:
I'll be on vacation and tomorrow is the last day I'm online before said vacation so sorry if my next answer will be late.
It's alright, have fun o/
Reply With Quote
  #14  
Old 08.05.2024, 05:27
Doom Doom is offline
Ultra Loader
 
Join Date: Apr 2019
Posts: 46
Default

Alright here the log

08.05.24 06.14.40 <--> 08.05.24 06.27.49 jdlog://1372411370661/
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 23:49.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.