JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 07.12.2022, 04:46
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default Exhentai/E-hentai 5000 images ghost limit

So I've looked a bit into this issue in the forums and it seems to be somewhat persistent and has already been reported multiple times. However I decided to make a new thread to avoid potentially sending notifications to other users who might be still be subscribed to that thread just to be good mannered and not necro the thread again.

To keep things short, it seems that the plugin still works fine with e-hentai links, however it runs into the ghost image limit issue whenever an exhentai link is started after it reaches the default 5000 limit (i have a 25000 limit on my account and JD recognizes that just fine). I also ran a couple of tests by stopping the downloads and switching the package order around (exhentai pack first followed by e-hentai package and viceversa) to confirm that it was just the exhentai package that didn't want to cooperate.

here's the log 06.12.22 22.11.36 <--> 06.12.22 22.25.13 jdlog://8361311370661/

I'm willing to share my account details if necessary again (think I've done so in the past)

also not related to the limit but still relevant to the plugin, it seems like the "image size too small" error pops up randomly on some downloads for some reason even though the error doesn't really apply to the image that it reports as such. currently i've only managed to reproduce that behavior while downloading and connected through a vpn (proton), but the images seem to download just fine when not connected via vpn. the behavior should also be reflected in that log also
Reply With Quote
  #2  
Old 07.12.2022, 19:55
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by serveral View Post
however it runs into the ghost image limit issue whenever an exhentai link is started after it reaches the default 5000 limit (i have a 25000 limit on my account and JD recognizes that just fine). I also ran a couple of tests by stopping the downloads and switching the package order around (exhentai pack first followed by e-hentai package and viceversa) to confirm that it was just the exhentai package that didn't want to cooperate.
In case JDownloader did download that ghost image, can you send it to us to support@jdownloader.org ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 07.12.2022, 19:56
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by serveral View Post
also not related to the limit but still relevant to the plugin, it seems like the "image size too small" error pops up randomly on some downloads for some reason even
The server doesn't report content lenght and thus the *not a picture* detection fails to handle it. We have to check if this is new sort of *ghost image* or just missing header and workaround required.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 07.12.2022, 19:58
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by serveral View Post
currently i've only managed to reproduce that behavior while downloading and connected through a vpn (proton), but the images seem to download just fine when not connected via vpn. the behavior should also be reflected in that log also
Maybe the limit also somehow connected to IP or it's explicit VPN blocking/rate limit try? Maybe with VPN connection, your account won't be used /server side and it's acting like a normal(no/limited account) user? What happens when it starts to happen in JDownloader when you test in browser with same IP?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 07.12.2022, 20:22
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
In case JDownloader did download that ghost image, can you send it to us to support@jdownloader.org ?
sorry, should've phrased it better. it's not a "ghost image" issue per se, it's the same issue that has been reported previously and marked as fixed before: the 5000 image/token/item limit that e-hentai/exhentai has for default accounts. mine has a 25000 limit and it works fine whenever i'm downloading images linked from e-hentai, however the exhentai linked ones get stuck at the 5000 limit with the "limit reached/time out" errors.

Quote:
Originally Posted by Jiaz View Post
The server doesn't report content lenght and thus the *not a picture* detection fails to handle it. We have to check if this is new sort of *ghost image* or just missing header and workaround required.
Quote:
Originally Posted by Jiaz View Post
Maybe the limit also somehow connected to IP or it's explicit VPN blocking/rate limit try? Maybe with VPN connection, your account won't be used /server side and it's acting like a normal(no/limited account) user? What happens when it starts to happen in JDownloader when you test in browser with same IP?
maybe it's some sort of anti-botting measure from their servers since they don't seem to be too VPN friendly anyways. it just seemed weird that it JD returned a "image size too small" error from random links whenever it tried to download them, everything seemed just fine on the browser however.

either way i'd like to clarify that the VPN/"image too small" is a separate/minor issue that i just felt like pointing out as well. the main issue at hand is the 5000 limit bug that doesn't seem to be completely ironed out yet, and sorry if made it sound too complicated by mentioning that minor one as well, since they're both unrelated anyways.
Reply With Quote
  #6  
Old 08.12.2022, 01:26
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by serveral View Post
it's the same issue that has been reported previously and marked as fixed before: the 5000 image/token/item limit that e-hentai/exhentai has for default accounts. mine has a 25000 limit and it works fine whenever i'm downloading images linked from e-hentai, however the exhentai linked ones get stuck at the 5000 limit with the "limit reached/time out" errors.
@Jiaz
Context:
https://board.jdownloader.org/showth...456#post477456
I've never found a solution for this issue although I always said "it can't be that hard".
It is still to this day unclear, whether the problem happened due to serverside detection/block/limit of e(x)hentai or is a "bug/small oversight in our plugin" though I'm voting for the latter.

Quote:
Originally Posted by serveral View Post
it just seemed weird that it JD returned a "image size too small" error from random links whenever it tried to download them, everything seemed just fine on the browser however.
We'd need a log of that so we could try to fix this/improve that check.

Quote:
Originally Posted by serveral View Post
the main issue at hand is the 5000 limit bug that doesn't seem to be completely ironed out yet
I'm definitely out of this I've wasted too much time looking into this back then.
My guess is that it fails due to a small oversight of us like one missing request or a request which is currently done differently in our plugin than in browser resulting in that problem.
To work on this, we will definitely need your ehentai login credentials.
Please send them to: support@jdownloader.org and please don't have high hopes for this.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 08.12.2022 at 01:27. Reason: Fixed spelling mistakes
Reply With Quote
  #7  
Old 08.12.2022, 02:04
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by pspzockerscene View Post
@Jiaz
Context:
**External links are only visible to Support Staff**...
I've never found a solution for this issue although I always said "it can't be that hard".
It is still to this day unclear, whether the problem happened due to serverside detection/block/limit of e(x)hentai or is a "bug/small oversight in our plugin" though I'm voting for the latter.

I'm definitely out of this I've wasted too much time looking into this back then.
My guess is that it fails due to a small oversight of us like one missing request or a request which is currently done differently in our plugin than in browser resulting in that problem.
To work on this, we will definitely need your ehentai login credentials.
Please send them to: support@jdownloader.org and please don't have high hopes for this.
Yeah, besides not wanting to necrobump the thread I saw that in, I abstained from asking if the issue was fixed since the other users also stopped communicating, so I just started thinking this was something on my end or something... those news are a little disheartening though.

Still, appreciate the support given either way, at the least it has cleared a bunch of uncertainties I had.

I've also just sent my account details.
Reply With Quote
  #8  
Old 08.12.2022, 14:55
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@serveral: I can try to dig deeper and maybe be able to find the cause. Am I allowed to use account to use up 5k and more so I try to reach the limit in JDownloader?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 08.12.2022, 17:17
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
@serveral: I can try to dig deeper and maybe be able to find the cause. Am I allowed to use account to use up 5k and more so I try to reach the limit in JDownloader?
Sure, no problem. Just make sure to not manually reset the limit through **External links are only visible to Support Staff****External links are only visible to Support Staff** if possible, since that uses up the site's internal currency to reset it.

Edit: I just checked my mail. The easiest way to reach the limit would be by checking the "prefer original quality" option in the e-hentai plugin's settings, and then just grab a random gallery with a large amount of pictures (1k plus pictures should reach the 5k limit quite easily)

Last edited by serveral; 08.12.2022 at 17:22. Reason: Additional info
Reply With Quote
  #10  
Old 08.12.2022, 17:22
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@serveral: just to make sure, does visiting the /home.php do any reset or is there some sort of button to click? I'm not familiar with that site
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 08.12.2022, 17:26
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
@serveral: just to make sure, does visiting the /home.php do any reset or is there some sort of button to click? I'm not familiar with that site
There's a "Reset Limit" button at the top of that page under a box that says "Image Limits", just make sure not to press that unless it's truly necessary.

The image limits are IP based either way, so a simple IP change should reset the limit too.
Reply With Quote
  #12  
Old 08.12.2022, 17:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by serveral View Post
The image limits are IP based either way, so a simple IP change should reset the limit too.
Maybe that's the reason as you can only reproduce the issue with being connected to VPN? https://board.jdownloader.org/showpo...06&postcount=4
__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 08.12.2022, 17:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@serveral: just to make sure to look for the right issue
JDownloader fails after about 5000 images? whereas the browser works fine?
also check the "image too small" issue, right?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 08.12.2022, 17:39
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by Jiaz View Post
@serveral: just to make sure to look for the right issue
JDownloader fails after about 5000 images? whereas the browser works fine?
also check the "image too small" issue, right?
Yes, links grabbed from exhentai.org fail around the 5k mark, whereas the e-hentai.org grabbed ones don't. Checking pictures on exhentai.org in the browser after the 5k limit is reached works fine.

The "image too small" issue might be a little harder to reproduce since I only got it to happen when downloading via VPN (proton i was using then) so I can't really provide anymore info beyond that. Could just be a simple connection issue on my end either way.
Reply With Quote
  #15  
Old 08.12.2022, 17:54
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by serveral View Post
The "image too small" issue might be a little harder to reproduce since I only got it to happen when downloading via VPN (proton i was using then) so I can't really provide anymore info beyond that. Could just be a simple connection issue on my end either way.
Sounds easy.
Just reactivate your VPN, reproduce the error and upload a fresh log for us so in the best case we can fix it without needing to be able to reproduce it locally.

Quote:
Originally Posted by serveral View Post
Could just be a simple connection issue on my end either way.
No.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #16  
Old 08.12.2022, 18:31
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

08.12.22 12.22.42 <--> 08.12.22 12.25.15 jdlog://1761311370661/ here, I managed to reproduce the small file error in this log. The file that JD flagged is shown to be "316.20 KiB" of size, which shouldn't trigger that error from my experience.
Reply With Quote
  #17  
Old 09.12.2022, 13:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

This is the http response of the image download attempt:
Code:
----------------Response------------------------
HTTP/1.1 200 OK
Server: Genetic Lifeform and Distributed Open Server 1.6.1
Date: Thu, 08 Dec 2022 16:24:33 GMT
Content-Type: image/jpeg
Cache-Control: public, max-age=31536000
Transfer-Encoding: chunked
Connection: close
------------------------------------------------
While I got no idea which file that would have lead to and I can't check as the direct-URL from your log is not working anymore, we could update the check to only trigger if "Content-Length" is given && value < X while X could be a hardcoded minimum size or, if parsed correctly, the expected filesize parsed from html code before.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #18  
Old 09.12.2022, 13:47
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

If I recall correctly (I've deleted the package already) it should be this picture **External links are only visible to Support Staff****External links are only visible to Support Staff** (or it's e-hentai mirror here **External links are only visible to Support Staff****External links are only visible to Support Staff**).
Reply With Quote
  #19  
Old 09.12.2022, 14:16
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I know. Finding that in your log was not a problem.
We'll look into it. Thanks for the log.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #20  
Old 11.12.2022, 07:35
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Just wanted to mention that the "image too small error" doesn't seem to be VPN dependent as I had originally thought since I just replicated it when I had it turned off. I did manage to download the image in the end by changing my IP through with the VPN, and then, just as a test, I deleted the image with the reset option in JD, disconnected from the VPN and then tried downloading it again and it went just fine...

10.12.22 22.36.39 <--> 11.12.22 01.01.11 jdlog://9271311370661/ here's the log for that session.

I do want to add something else to this issue though, which is that I've had something somewhat similar happen in the past, where a few images at random intervals just failed to download and just gave a "host offline" error and remained like that for some good 15 to 30 minutes unless I changed my IP with a VPN or went into my router's settings to get a new IP (dynamic). I always blamed it on my ISP, as I had packet loss issues on my ADSL line for a good while then (due to poor maintenance and crappy infrastructure where I live), and I sort of guessed that it should've been that, since I also had issues while browsing galleries through the browser where pictures just wouldn't load sometimes (unless I clicked on the "click here if image fails loading" button they have there), which mirrored the issues on JD with those particular pictures that wouldn't download.

However I've started thinking it might actually just be an issue related to their hentai@home service failing to provide the requested images due to it's reliance on third party users to help out with their bandwidth issues. That's just a theory though, as the issue is now somewhat different and the behavior isn't really mirrored anymore between the browser and JD, though I guess that could be just a user agent/cookie thing too.

Last edited by serveral; 11.12.2022 at 07:37. Reason: syntax
Reply With Quote
  #21  
Old 19.12.2022, 14:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Our ehentai can't really automatically use that alternative/fallback image at this moment though that shouldn't matter as it at least shouldn't ever download corrupt/dummy images.

Regarding the "image too small" problem:
The changes we did haven't been released yet.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #22  
Old 21.05.2023, 04:57
HydroLogic HydroLogic is offline
Linkgrabbing Monster
 
Join Date: Sep 2019
Posts: 84
Default

Has there been any progress on this ghost limit issue?
Reply With Quote
  #23  
Old 22.05.2023, 14:54
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

No.
EDIT

I've changed the thread title back to "See Bugtracker" as no one is actively working on this.
We're open source so anyone is free to investigate this further, see:
https://support.jdownloader.org/Know...up-ide-eclipse
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #24  
Old 09.08.2023, 13:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

If someone is willing to provide an account for testing, I'm willing to give this one more shot.
Send logins to: psp@jdownloader.org
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #25  
Old 12.08.2023, 09:07
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

I've just sent the account details, however, there have been some changes to their site since this issue was originally posted that may impact some of the tests. I'm going to quote some of relevant details from **External links are only visible to Support Staff**this post in the site's forum.

First off, they introduced a "peak hours" system that completely ignores the "image limits" system during those specified hours, instead using one of their in-site currencies called "GP" during those hours to download the original images from a gallery (which, in my experience, is the fastest way to reach the image limit quota).
Quote:
2022-08-07

- Adjusted the "peak hours" used by original image downloading to better match the actual measured peak periods. It is now between 14:00 and 20:00 UTC Monday-Saturday, and between 05:00 and 20:00 UTC on Sundays.
Now this wouldn't because the downloader would just stop working during those hours, however due to this following change...

Quote:
2023-07-11
- The archiver and full image downloads now both auto-convert Credits to GP using the "High" rate from the last 24 hours in the exchange if you have a Credit balance but insufficient GP. (In other words, you can still usually eke out a 1-2% better rate by converting manually, but doing so is no longer essential.)
To avoid extending myself explaining the website's weird economy: credits are sort of time consuming to obtain so most people would rather not spend them.

To summarize: whoever is going to do the testing, just keep in mind not do it during the peak hours periods. And to also only use newer galleries for the tests, since galleries that are more than 1 year old only use that GP download system as well.
Reply With Quote
  #26  
Old 14.08.2023, 16:16
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Not sure how I could test it now as your account only has the default/5000 limit now.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #27  
Old 14.08.2023, 18:08
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

That's odd. My account settings show that I have a 50k limit
Reply With Quote
  #28  
Old 14.08.2023, 18:32
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I see.
I got that old "bug" in browser where it is 5000 until you visit "/hathperks.php" once.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #29  
Old 14.08.2023, 20:04
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

I don't know how helpful this might be but this **External links are only visible to Support Staff**script and gallery-dl's exhentai **External links are only visible to Support Staff**extractor have workarounds to guess an user's account actual image limits.

I'm aware they're both written in different languages to JDownloader's, and that they may also include libraries or dependencies that aren't available in JD's code base, but still thought they may provide some insight in solving this issue.
Reply With Quote
  #30  
Old 14.08.2023, 22:49
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I've now used more than 10 000 GP and I can still download via JD.
I'm stopping now so I'm not wasting all of your credits.
Could it be that this issue fixed itself?

Also I've implemented some measurements to try to avoid wasting GP when downloading original images though it's probably something which only people with a bad internet connection will notice (re-use directurls on resume).
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #31  
Old 14.08.2023, 23:11
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

I'll try downloading some galleries and get back to you in a bit.
Reply With Quote
  #32  
Old 15.08.2023, 00:14
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Can confirm that the issue is no longer present. Gone past the 5k image limit mark and downloads are not being stopped.

Edit: Nevermind, I seem to have hit the limit at 10k now, here's the log 14.08.23 15.56.26 <--> 14.08.23 17.40.31 jdlog://7907311370661/

Last edited by serveral; 15.08.2023 at 00:51.
Reply With Quote
  #33  
Old 15.08.2023, 13:18
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Your log looks good:
Code:
[jd.plugins.hoster.EHentaiOrg(getNormalmageDownloadurl)] -> Failed to get around limit - limit is definitely reached!
--ID:2750TS:1692048559736-8/14/23 5:29:19 PM -  [] -> Exception thrown at jd.controlling.downloadcontroller.SingleDownloadController.download(SingleDownloadController.java:441):
jd.plugins.AccountUnavailableException: Downloadlimit reached->ERROR_PREMIUM|Value:0
After that, your e-hentai account will get a yellow status in JD for 5 minutes.
Once that time is over, JD will check it again.
If 0 GP is available, the account will stay active but with 0 bytes of traffic-left.
--> Accounts with zero traffic won't be used for downloading.
If more than 0 GP is available again, the account will be used again until one download is running into that limit-message again.

While the account can't be used, JD will continue in free mode which will allow the download of non-original images.
If you wish to avoid this completely, you might want to create an account usage rule for e-hentai.org and disable free mode there so whenever account downloads for original images are not possible, all items will get a wait status until the account can be used again.
EDIT

I've updated the plugin so all GP related limit error-messages will temporary deactivate your account (yellow status). I think this is the way to go.

EDIT2
If you can't reproduce the "Ghost limit bug", I will close all related tickets.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 15.08.2023 at 13:24.
Reply With Quote
  #34  
Old 15.08.2023, 20:03
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

The peak hours period is in place as of right now so I'll get back to you in some hours after it's lifted. I'll update this post to inform you whether I can still replicate the ghost limit or not.

Edit: Seems like an IP ban (white page) was triggered a couple of minutes after I began my test. Log: 15.08.23 16.22.52 <--> 15.08.23 16.44.40 jdlog://2117311370661/ my guess is that their system noticed some sort of bot-like behavior coming from newly implemented GP/credit checks.

Last edited by serveral; 15.08.2023 at 23:50.
Reply With Quote
  #35  
Old 16.08.2023, 12:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

White page where?
Single image pages, account overview or even main page of e-hentai?
Also, what am I supposed to do then?

Quote:
Originally Posted by serveral View Post
my guess is that their system noticed some sort of bot-like behavior coming from newly implemented GP/credit checks.
If you mean the JD plugin:
No:
1. No additional checks have been implemented.
2. Upon refactoring the plugin, I even removed some unnecessary checks so the plugin is doing less http requests.
EDIT

So if you want me to somehow detect this block - yes I could try to do this.
Your log contains a lot of download activity, also of other filehosts.
Please provide a fresh log - in the ideal case it is only from one session and contains only e-hentai activity.
Keep in mind that if a website really wants to block public tools like JDownloader, they can do this at any point of time.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 16.08.2023 at 12:27.
Reply With Quote
  #36  
Old 16.08.2023, 16:47
simon_7617118 simon_7617118 is offline
JD Alpha
 
Join Date: Oct 2015
Posts: 24
Default

Quote:
Originally Posted by pspzockerscene View Post
After that, your e-hentai account will get a yellow status in JD for 5 minutes.
Once that time is over, JD will check it again.
If 0 GP is available, the account will stay active but with 0 bytes of traffic-left.
I don't quite understand what's going on here, I'm sorry if I missed some changes on EH's end. But I'd much prefer the old plugin where downloading original images counts towards the 5000 daily image points, rather than GP. This is forcing people to actually play their game to earn GP. This is far more tedious compared to just using the daily points system which, if one manages it correctly, ends up being about 500mb of downloads per day.
This attempt to circumvent their bandwidth usage limit is making things way worse. Please don't mess with GP conversions.

Last edited by simon_7617118; 16.08.2023 at 16:53.
Reply With Quote
  #37  
Old 16.08.2023, 17:15
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

@simon_7617118
To make this absolutely clear:
  • The way how JD downloads original images (if preferred by the user) was never changed
  • JD does not convert / auto "buy" GP at any time
  • The account handling I described has been this way before - I didn't change any behavior here either. Wherever I wrote "GP" I ment the points that are displayed on their website as "You are currently at 0 towards a limit of 50000."
Maybe I've unknowingly mixed up the wording for "GP" and "image points" here - I can only repeat: I'm not using e-hentai myself.

Quote:
Originally Posted by simon_7617118 View Post
This attempt to circumvent their bandwidth usage limit is making things way worse. Please don't mess with GP conversions.
There is not any attempt to circumvent their bandwidth usage implemented into our plugin.
If you mean the log-quote I posted in post #33:
That was a badly worded log output in a function which simply checked if the limit was really reached after the first time it looked like itwas reached.
You can verify that information by looking into our source code:
Code reference:
github.com/mirror/jdownloader/blob/5f50ef81a535c64d31713ed043587eac7a3da057/src/jd/plugins/hoster/EHentaiOrg.java#L493C14-L493C14

In the meanwhile the wording of that log-output was changed as well.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 16.08.2023 at 17:18. Reason: Fixed typo and updated wording
Reply With Quote
  #38  
Old 16.08.2023, 18:26
simon_7617118 simon_7617118 is offline
JD Alpha
 
Join Date: Oct 2015
Posts: 24
Default

So is this change coming from EH itself? Because right now I can't download any image at original resolution, even though my image points usage is at 0. It's counting towards GP, and I've run out quickly because I don't play that game.
Sorry if that's the case. I'm not a coder, I can't review anything. Is there any chance that a recent change in the plugin may have triggered this behavior? It has never used GP before, and I can't imagine how EH is able to discern traffic from JD. Previously, the only way they could detect it was if the simultaneous download was set too high.
Reply With Quote
  #39  
Old 16.08.2023, 19:47
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by pspzockerscene View Post
White page where?
Single image pages, account overview or even main page of e-hentai?
Also, what am I supposed to do then?
Anywhere. Got it after downloading for some minutes with JD (with the original quality option checked, only one picture at a time, and after the peak hours period had passed) when I noticed downloads had stopped with the "account missing" error. When I went to check on the browser, exhentai just gave me a blank/white page (IP ban behavior which I could confirm by using a VPN), didn't try accessing e-hentai though I'm guessing it would've also shown me a white page. It could've been a one off thing since I didn't try to replicate the behavior again See Edits

Quote:
Originally Posted by pspzockerscene View Post
So if you want me to somehow detect this block - yes I could try to do this.
Your log contains a lot of download activity, also of other filehosts.
Please provide a fresh log - in the ideal case it is only from one session and contains only e-hentai activity.
Keep in mind that if a website really wants to block public tools like JDownloader, they can do this at any point of time.
Will do. I'm going to try downloading a bunch of galleries without the "original quality" setting on to see if I can trigger then ban as well.

Quote:
Originally Posted by simon_7617118 View Post
So is this change coming from EH itself? Because right now I can't download any image at original resolution, even though my image points usage is at 0. It's counting towards GP, and I've run out quickly because I don't play that game.
Yes, they introduced a "peak hours" system about a year ago where from Monday to Saturday around 14:00 to 20:00 UTC (during Sundays, it's almost the whole day), GP would be used instead of image limits if you're downloading original images, and galleries older than a year will also only use GP as of july if you try downloading their original images too regardless of whether it's peak hours or not. Their "News & Updates" section of their forum has more info.

Edit
No bans triggered so far when downloading resized pictures (original quality setting unchecked) but I seem to have hit the image limit near 5k. Kinda odd given the last time I ran into the limit was at 10k instead (the one I reported in one of the posts above), but that was with the original quality settings ticked. Here's the log 16.08.23 12.24.44 <--> 16.08.23 14.40.02 jdlog://9217311370661/
There are probably some errors/download failures scattered across the whole log but those are most likely local network issues, the important stuff should be near the end of the log session.

Edit 2
Seems like the white page/ban thing was a one time only thing, maybe something I did on my end but so far I haven't managed to replicate it. However I did try once more to download some more galleries, this time with the original image quality setting ticked on to see if the image limit bug was still there, and this time I got to around 8k before JD started saying that I had reached the limit. Log of this session: 16.08.23 20.31.43 <--> 16.08.23 20.42.52 jdlog://4317311370661/ . This last log is not as clean as the previous one because my system crashed, (unrelated to JD) but prior to the crash I had already passed the 5k image limit mark and downloads kept going (until the crash of course). After the crash, I was able to download about a dozen more pictures before it finally stopped at about a little over 8k.

Last edited by serveral; 17.08.2023 at 04:21. Reason: Update
Reply With Quote
  #40  
Old 16.08.2023, 20:32
simon_7617118 simon_7617118 is offline
JD Alpha
 
Join Date: Oct 2015
Posts: 24
Default

Quote:
Originally Posted by serveral View Post
galleries older than a year will also only use GP as of july if you try downloading their original images too regardless of whether it's peak hours or not.
Okay, that must be it, although I've only started running into this problem this week. Thanks for the explanation.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 01:06.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.