JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 07.12.2022, 04:46
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default Exhentai/E-hentai 5000 images ghost limit

So I've looked a bit into this issue in the forums and it seems to be somewhat persistent and has already been reported multiple times. However I decided to make a new thread to avoid potentially sending notifications to other users who might be still be subscribed to that thread just to be good mannered and not necro the thread again.

To keep things short, it seems that the plugin still works fine with e-hentai links, however it runs into the ghost image limit issue whenever an exhentai link is started after it reaches the default 5000 limit (i have a 25000 limit on my account and JD recognizes that just fine). I also ran a couple of tests by stopping the downloads and switching the package order around (exhentai pack first followed by e-hentai package and viceversa) to confirm that it was just the exhentai package that didn't want to cooperate.

here's the log 06.12.22 22.11.36 <--> 06.12.22 22.25.13 jdlog://8361311370661/

I'm willing to share my account details if necessary again (think I've done so in the past)

also not related to the limit but still relevant to the plugin, it seems like the "image size too small" error pops up randomly on some downloads for some reason even though the error doesn't really apply to the image that it reports as such. currently i've only managed to reproduce that behavior while downloading and connected through a vpn (proton), but the images seem to download just fine when not connected via vpn. the behavior should also be reflected in that log also
Reply With Quote
  #2  
Old 07.12.2022, 19:55
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

Quote:
Originally Posted by serveral View Post
however it runs into the ghost image limit issue whenever an exhentai link is started after it reaches the default 5000 limit (i have a 25000 limit on my account and JD recognizes that just fine). I also ran a couple of tests by stopping the downloads and switching the package order around (exhentai pack first followed by e-hentai package and viceversa) to confirm that it was just the exhentai package that didn't want to cooperate.
In case JDownloader did download that ghost image, can you send it to us to support@jdownloader.org ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 07.12.2022, 19:56
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

Quote:
Originally Posted by serveral View Post
also not related to the limit but still relevant to the plugin, it seems like the "image size too small" error pops up randomly on some downloads for some reason even
The server doesn't report content lenght and thus the *not a picture* detection fails to handle it. We have to check if this is new sort of *ghost image* or just missing header and workaround required.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 07.12.2022, 19:58
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

Quote:
Originally Posted by serveral View Post
currently i've only managed to reproduce that behavior while downloading and connected through a vpn (proton), but the images seem to download just fine when not connected via vpn. the behavior should also be reflected in that log also
Maybe the limit also somehow connected to IP or it's explicit VPN blocking/rate limit try? Maybe with VPN connection, your account won't be used /server side and it's acting like a normal(no/limited account) user? What happens when it starts to happen in JDownloader when you test in browser with same IP?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 07.12.2022, 20:22
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
In case JDownloader did download that ghost image, can you send it to us to support@jdownloader.org ?
sorry, should've phrased it better. it's not a "ghost image" issue per se, it's the same issue that has been reported previously and marked as fixed before: the 5000 image/token/item limit that e-hentai/exhentai has for default accounts. mine has a 25000 limit and it works fine whenever i'm downloading images linked from e-hentai, however the exhentai linked ones get stuck at the 5000 limit with the "limit reached/time out" errors.

Quote:
Originally Posted by Jiaz View Post
The server doesn't report content lenght and thus the *not a picture* detection fails to handle it. We have to check if this is new sort of *ghost image* or just missing header and workaround required.
Quote:
Originally Posted by Jiaz View Post
Maybe the limit also somehow connected to IP or it's explicit VPN blocking/rate limit try? Maybe with VPN connection, your account won't be used /server side and it's acting like a normal(no/limited account) user? What happens when it starts to happen in JDownloader when you test in browser with same IP?
maybe it's some sort of anti-botting measure from their servers since they don't seem to be too VPN friendly anyways. it just seemed weird that it JD returned a "image size too small" error from random links whenever it tried to download them, everything seemed just fine on the browser however.

either way i'd like to clarify that the VPN/"image too small" is a separate/minor issue that i just felt like pointing out as well. the main issue at hand is the 5000 limit bug that doesn't seem to be completely ironed out yet, and sorry if made it sound too complicated by mentioning that minor one as well, since they're both unrelated anyways.
Reply With Quote
  #6  
Old 08.12.2022, 01:26
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,045
Default

Quote:
Originally Posted by serveral View Post
it's the same issue that has been reported previously and marked as fixed before: the 5000 image/token/item limit that e-hentai/exhentai has for default accounts. mine has a 25000 limit and it works fine whenever i'm downloading images linked from e-hentai, however the exhentai linked ones get stuck at the 5000 limit with the "limit reached/time out" errors.
@Jiaz
Context:
https://board.jdownloader.org/showth...456#post477456
I've never found a solution for this issue although I always said "it can't be that hard".
It is still to this day unclear, whether the problem happened due to serverside detection/block/limit of e(x)hentai or is a "bug/small oversight in our plugin" though I'm voting for the latter.

Quote:
Originally Posted by serveral View Post
it just seemed weird that it JD returned a "image size too small" error from random links whenever it tried to download them, everything seemed just fine on the browser however.
We'd need a log of that so we could try to fix this/improve that check.

Quote:
Originally Posted by serveral View Post
the main issue at hand is the 5000 limit bug that doesn't seem to be completely ironed out yet
I'm definitely out of this I've wasted too much time looking into this back then.
My guess is that it fails due to a small oversight of us like one missing request or a request which is currently done differently in our plugin than in browser resulting in that problem.
To work on this, we will definitely need your ehentai login credentials.
Please send them to: support@jdownloader.org and please don't have high hopes for this.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 08.12.2022 at 01:27. Reason: Fixed spelling mistakes
Reply With Quote
  #7  
Old 08.12.2022, 02:04
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by pspzockerscene View Post
@Jiaz
Context:
**External links are only visible to Support Staff**...
I've never found a solution for this issue although I always said "it can't be that hard".
It is still to this day unclear, whether the problem happened due to serverside detection/block/limit of e(x)hentai or is a "bug/small oversight in our plugin" though I'm voting for the latter.

I'm definitely out of this I've wasted too much time looking into this back then.
My guess is that it fails due to a small oversight of us like one missing request or a request which is currently done differently in our plugin than in browser resulting in that problem.
To work on this, we will definitely need your ehentai login credentials.
Please send them to: support@jdownloader.org and please don't have high hopes for this.
Yeah, besides not wanting to necrobump the thread I saw that in, I abstained from asking if the issue was fixed since the other users also stopped communicating, so I just started thinking this was something on my end or something... those news are a little disheartening though.

Still, appreciate the support given either way, at the least it has cleared a bunch of uncertainties I had.

I've also just sent my account details.
Reply With Quote
  #8  
Old 08.12.2022, 14:55
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

@serveral: I can try to dig deeper and maybe be able to find the cause. Am I allowed to use account to use up 5k and more so I try to reach the limit in JDownloader?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 08.12.2022, 17:17
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
@serveral: I can try to dig deeper and maybe be able to find the cause. Am I allowed to use account to use up 5k and more so I try to reach the limit in JDownloader?
Sure, no problem. Just make sure to not manually reset the limit through **External links are only visible to Support Staff****External links are only visible to Support Staff** if possible, since that uses up the site's internal currency to reset it.

Edit: I just checked my mail. The easiest way to reach the limit would be by checking the "prefer original quality" option in the e-hentai plugin's settings, and then just grab a random gallery with a large amount of pictures (1k plus pictures should reach the 5k limit quite easily)

Last edited by serveral; 08.12.2022 at 17:22. Reason: Additional info
Reply With Quote
  #10  
Old 08.12.2022, 17:22
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

@serveral: just to make sure, does visiting the /home.php do any reset or is there some sort of button to click? I'm not familiar with that site
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 08.12.2022, 17:26
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
@serveral: just to make sure, does visiting the /home.php do any reset or is there some sort of button to click? I'm not familiar with that site
There's a "Reset Limit" button at the top of that page under a box that says "Image Limits", just make sure not to press that unless it's truly necessary.

The image limits are IP based either way, so a simple IP change should reset the limit too.
Reply With Quote
  #12  
Old 08.12.2022, 17:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

Quote:
Originally Posted by serveral View Post
The image limits are IP based either way, so a simple IP change should reset the limit too.
Maybe that's the reason as you can only reproduce the issue with being connected to VPN? https://board.jdownloader.org/showpo...06&postcount=4
__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 08.12.2022, 17:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,236
Default

@serveral: just to make sure to look for the right issue
JDownloader fails after about 5000 images? whereas the browser works fine?
also check the "image too small" issue, right?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 08.12.2022, 17:39
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
@serveral: just to make sure to look for the right issue
JDownloader fails after about 5000 images? whereas the browser works fine?
also check the "image too small" issue, right?
Yes, links grabbed from exhentai.org fail around the 5k mark, whereas the e-hentai.org grabbed ones don't. Checking pictures on exhentai.org in the browser after the 5k limit is reached works fine.

The "image too small" issue might be a little harder to reproduce since I only got it to happen when downloading via VPN (proton i was using then) so I can't really provide anymore info beyond that. Could just be a simple connection issue on my end either way.
Reply With Quote
  #15  
Old 08.12.2022, 17:54
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,045
Default

Quote:
Originally Posted by serveral View Post
The "image too small" issue might be a little harder to reproduce since I only got it to happen when downloading via VPN (proton i was using then) so I can't really provide anymore info beyond that. Could just be a simple connection issue on my end either way.
Sounds easy.
Just reactivate your VPN, reproduce the error and upload a fresh log for us so in the best case we can fix it without needing to be able to reproduce it locally.

Quote:
Originally Posted by serveral View Post
Could just be a simple connection issue on my end either way.
No.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #16  
Old 08.12.2022, 18:31
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

08.12.22 12.22.42 <--> 08.12.22 12.25.15 jdlog://1761311370661/ here, I managed to reproduce the small file error in this log. The file that JD flagged is shown to be "316.20 KiB" of size, which shouldn't trigger that error from my experience.
Reply With Quote
  #17  
Old 09.12.2022, 13:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,045
Default

This is the http response of the image download attempt:
Code:
----------------Response------------------------
HTTP/1.1 200 OK
Server: Genetic Lifeform and Distributed Open Server 1.6.1
Date: Thu, 08 Dec 2022 16:24:33 GMT
Content-Type: image/jpeg
Cache-Control: public, max-age=31536000
Transfer-Encoding: chunked
Connection: close
------------------------------------------------
While I got no idea which file that would have lead to and I can't check as the direct-URL from your log is not working anymore, we could update the check to only trigger if "Content-Length" is given && value < X while X could be a hardcoded minimum size or, if parsed correctly, the expected filesize parsed from html code before.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #18  
Old 09.12.2022, 13:47
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

If I recall correctly (I've deleted the package already) it should be this picture **External links are only visible to Support Staff****External links are only visible to Support Staff** (or it's e-hentai mirror here **External links are only visible to Support Staff****External links are only visible to Support Staff**).
Reply With Quote
  #19  
Old 09.12.2022, 14:16
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,045
Default

I know. Finding that in your log was not a problem.
We'll look into it. Thanks for the log.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #20  
Old 11.12.2022, 07:35
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Just wanted to mention that the "image too small error" doesn't seem to be VPN dependent as I had originally thought since I just replicated it when I had it turned off. I did manage to download the image in the end by changing my IP through with the VPN, and then, just as a test, I deleted the image with the reset option in JD, disconnected from the VPN and then tried downloading it again and it went just fine...

10.12.22 22.36.39 <--> 11.12.22 01.01.11 jdlog://9271311370661/ here's the log for that session.

I do want to add something else to this issue though, which is that I've had something somewhat similar happen in the past, where a few images at random intervals just failed to download and just gave a "host offline" error and remained like that for some good 15 to 30 minutes unless I changed my IP with a VPN or went into my router's settings to get a new IP (dynamic). I always blamed it on my ISP, as I had packet loss issues on my ADSL line for a good while then (due to poor maintenance and crappy infrastructure where I live), and I sort of guessed that it should've been that, since I also had issues while browsing galleries through the browser where pictures just wouldn't load sometimes (unless I clicked on the "click here if image fails loading" button they have there), which mirrored the issues on JD with those particular pictures that wouldn't download.

However I've started thinking it might actually just be an issue related to their hentai@home service failing to provide the requested images due to it's reliance on third party users to help out with their bandwidth issues. That's just a theory though, as the issue is now somewhat different and the behavior isn't really mirrored anymore between the browser and JD, though I guess that could be just a user agent/cookie thing too.

Last edited by serveral; 11.12.2022 at 07:37. Reason: syntax
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 12:50.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.