[pspzockerscene]

@serveral
I really don't need new logs unless you're sure you found some kind of bug.
Also the limit is really easy to reach (for testing) so there is no reason to test it while you're not on your computer and let me check the logs for possible bugs later.
I did check them anyways and all I can say is that "exhentai.org/img/509.gif" happened, the limit handling was correctly triggered.
This is btw. the same detection also used by one of the Pythob projects you linked.

Best is to simply be there once you expect the limit to be reached and then check if your points are also empty on the website.

Changing default setting of "Prefer original quality"
The post of user "simon_7617118" makes me agree to your idea of changing the default setting for "original download" though again:
Depending on how many users are using this plugin, some backlash may happen if I'm going to do that.

[serveral]

Sorry if seem a little pedantic at this point, but the issue is that while there has been some progress and JD clearly (only seldomly though) goes past the default image limit of 5k, the issue is that I have an account with 50k meaning only 10% of my account's limit is only being realistically used most of the times. I can also check that the limit hasn't been reached because the 509.gif doesn't show up whenever I open an image on the browser (so long as I'm logged in of course), with the 509/limit reached only being reflected on JD according to the logs.

I did notice something odd this morning though, after leaving JD downloading a gallery for the whole night, when I came to check it after waking up, the 50k limit almost had been reached (it was around 48k), however, the total progress made during that time was around 700MB worth of pictures (I had original quality checked), which in my experience isn't nearly enough to make up for the 40k or so of image limits consumed.

At this point my only guess is that JD is sending image limit requests without the account details attached or something, i.e just sending a "default" no-account request which would obviously return with a 509.gif/limit reached response at 5k (at least in my case apparently), hence why it would wrongly assume I had already reached the limit when I shouldn't have. Mind you, I'm only arriving at this conclusion due to the new behavior I saw after I woke up.

Originally Posted by pspzockerscene

@serveral
Changing default setting of "Prefer original quality"
The post of user "simon_7617118" makes me agree to your idea of changing the default setting for "original download" though again:
Depending on how many users are using this plugin, some backlash may happen if I'm going to do that.

Also forgive me if I'm misunderstanding this, but this doesn't mean you are removing the original quality option and is only in regards to the recent GP workaround, right? Because if it's the later then I do agree that JD is better off defaulting to image limit reached instead of consuming GP when the original quality option is checked, since dealing with GP consumption is more trouble than it's worth for most e-hentai users.

[pspzockerscene]

Originally Posted by serveral

At this point my only guess is that JD is sending image limit requests without the account details attached or something, i.e just sending a "default" no-account request which would obviously return with a 509.gif/limit reached response at 5k (at least in my case apparently), hence why it would wrongly assume I had already reached the limit when I shouldn't have. Mind you, I'm only arriving at this conclusion due to the new behavior I saw after I woke up.

Okay so there is still an issue present?
But I mean before it clearly stopped at 5k but now it goes past it and as long as you are not there to watch JD you can't see what it'sdoing: Maybe the e-hentai website went offline for some hours in the night or something else happened which prevented JD from downloading more. It doesn't need to be related to that "phantom limit bug".

To improve your tests:
Create an account usage rule for e-hentai.com and disable free mode. Instructions:
https://support.jdownloader.org/Know...-free-download
-> Then try again.

Also / again for clarification:
Behavior of JD whenever "509.gif" (limit reached) happens:
If if happens in account mode: Temporarily disable whole e-hentai account in JD (yellow status) for 5 minutes with statustext "Downloadlimit reached".

If it happens in non-account mode: Set a wait status on all e-hentai links with text "Downloadlimit reached" and wait of 5 minutes.

Originally Posted by serveral

Also forgive me if I'm misunderstanding this, but this doesn't mean you are removing the original quality option and is only in regards to the recent GP workaround, right?

No - we're talking past each other again. I was only talking about deactivating the "Prefer original quality" setting by default and maybe forcing a reset of it once for all users.

Which "GP workaround" are you talking about?

Originally Posted by serveral

Because if it's the later then I do agree that JD is better off defaulting to image limit reached instead of consuming GP when the original quality option is checked, since dealing with GP consumption is more trouble than it's worth for most e-hentai users.

Again for dummies please:
There is these "image points" which your account has 50 000 of.
Then there is those "GP" which is a virtual currency which can be used to buy stuff there. It can also be used to buy those "image points", yes?
Just to be absolutely clear: The current plugin does not auto-exchange/use any of those "GP" - it doesn't even have any functions which can do this!

If my post sounds rude: Sorry - wasn't my intention!

[serveral]

Originally Posted by pspzockerscene

But I mean before it clearly stopped at 5k but now it goes past it and as long as you are not there to watch JD you can't see what it'sdoing: Maybe the e-hentai website went offline for some hours in the night or something else happened which prevented JD from downloading more. It doesn't need to be related to that "phantom limit bug".

I did notice something new yesterday during another test while downloading with the original quality settings ticked, everything seemed fine, downloads were happening while I was doing other stuff on my PC with JD on the background, then when I went to check JD's settings tab (the plugin settings more specifically to check on the image limits) everything stopped with the limit reached yellow warning. Enabling and disabling the account did nothing to change the behavior, plugin reported image limits at 10343/50000 but whenever I tried to resume the downloads it just kept saying "limit reached" immediately. Just found it kinda odd that everything was fine until that point.

I'll try doing what you suggested and disabling free mode in the account usage settings to see if there's any difference and report in a few hours.

Originally Posted by pspzockerscene

No - we're talking past each other again. I was only talking about deactivating the "Prefer original quality" setting by default and maybe forcing a reset of it once for all users.

Oh, no worries then, was just a minor misunderstanding on my part.

Edit
Did as suggested, disabled free mode, reached the image limit at 5k, disabled the account in JD and waited 5 mins, re-enabled it and tried to resume downloads but it kept saying limit reached.

Now at this point I decided to try something else out of curiosity, because for the past few days I've been doing these tests, I've always been downloading galleries grabbed from exhentai.org, instead of e-hentai.org, so I went a grabbed a random e-hentai gallery and disabled the exhentai one I was testing on and.... Voila, the e-hentai gallery was doing fine past the 5k mark. I then stopped it for a second and re-enabled the exhentai gallery and, for some reason, this time it didn't give me the limit reached error and instead downloads resumed like normal on the exhentai gallery. I'd continue the tests to see if the e-hentai gallery remained unaffected but it's getting a little late, however I think I found the actual reason behind the random limit reached behavior I been experiencing as of late.

Edit 2
Ok so after nearly 20k image limits consumed (and still going with no issues so far) when only downloading a e-hentai.org grabbed gallery I can now confirm that the limit reached issue only affects galleries grabbed from exhentai.org exclusively, and even then the issue isn't 100% consistent since sometimes the limit is reached at 5k, 10k or something in between, and then by starting an e-hentai gallery right after an exhentai one "reaches" the limit exhentai galleries are able to be resumed again for some reason... (though I don't know if they can wrongly "reach" the limit again since I haven't tested that).

[pspzockerscene]

Originally Posted by serveral

Enabling and disabling the account did nothing to change the behavior, plugin reported image limits at 10343/50000 but whenever I tried to resume the downloads it just kept saying "limit reached" immediately. Just found it kinda odd that everything was fine until that point.

So in browser you were still able to continue downloading?
I mean I don't want to point out possible mistakes of others but it's impossible for me to verify your information without checking via browser at the time the problem is happening because again: In the logs I will see that the limit is reached according to the e-hentai website but they could also be a JD-specific block which they introduced on purpose -> I can't verify it without checking via website at the time the problem has happened.

Originally Posted by serveral

I'll try doing what you suggested and disabling free mode in the account usage settings to see if there's any difference and report in a few hours.

This will not fix any issue - it will only prevent JD from downloading without account if a failure in account mode happens.

Originally Posted by serveral

I've always been downloading galleries grabbed from exhentai.org, instead of e-hentai.org

ood to know.
I did all my tests only with images from e-hentai.org.

[HydroLogic]

I can't even download a single gallery even though JDownloader *and* E-hentai are telling me I have 0/5000.

I try to start the gallery and it instantly goes into the 5 minute time out and in account manager it says "Downloadlimit reached." But if I refresh it, it goes back to 0/5000. Then the process repeats.

[pspzockerscene]

Please post your log-ID here
If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.

Bitte poste deine Log-ID hier.
Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.

-psp-

[serveral]

Originally Posted by pspzockerscene

So in browser you were still able to continue downloading?

Yes, I was still able to continue downloading/viewing pictures in the browser at the moment JD reported it had reached the limit, and as I pointed out in the edits of my post, I can confidently say that the issue only occurs whenever the gallery being downloaded is grabbed from exhentai.org, with the only consistent behavior being that the "limit reached" error pops up at some point after reaching 5k image limits. E-hentai galleries don't seem to run into this issue as far I've been able to notice in my tests, meaning it's not a JD block they implemented or anything of that sort.

If you wish to do some tests yourself then just make sure to be logged in before trying to access exhentai.org, otherwise the site won't let you in unless you clear the site's cookies (only exhentai's is necessary) and reload it.

Also to clarify some stuff regarding both sites in case it's needed: exhentai is sort of a mirror of e-hentai, with the main difference being that exhentai has content that is hidden on e-hentai, and exhentai can only be accessed from accounts that are 2 years old or older.

Other differences are the fact that some pages cannot be accessed from exhentai since they're not mirrored, like e-hentai.org/home.php has no equivalent in exhentai (exhentai.org/home.php just gives out a 404), which could maybe be one of the issues behind the ghost limit only occurring on exhentai galleries? Though I'm not really sure of that and would have to fall back on you guys.

[pspzockerscene]

Originally Posted by serveral

Yes, I was still able to continue downloading/viewing pictures in the browser at the moment JD reported it had reached the limit, and as I pointed out in the edits of my post, I can confidently say that the issue only occurs whenever the gallery being downloaded is grabbed from exhentai.org, with the only consistent behavior being that the "limit reached" error pops up at some point after reaching 5k image limits. E-hentai galleries don't seem to run into this issue as far I've been able to notice in my tests

Okay so all I need to do is to download from exhentai until JD claims that the limit is reached.
Anything I need to take into account such as those peak hours or can I just investigate this right now?

Originally Posted by serveral

Also to clarify some stuff regarding both sites in case it's needed: exhentai is sort of a mirror of e-hentai, with the main difference being that exhentai has content that is hidden on e-hentai, and exhentai can only be accessed from accounts that are 2 years old or older.

Thanks for the explanation.

Originally Posted by serveral

(exhentai.org/home.php just gives out a 404), which could maybe be one of the issues behind the ghost limit only occurring on exhentai galleries?

No. The plugin explicitely calls that page with the e-hentai domain.

[serveral]

Originally Posted by pspzockerscene

Okay so all I need to do is to download from exhentai until JD claims that the limit is reached.
Anything I need to take into account such as those peak hours or can I just investigate this right now.

Nothing else that I'm aware off, feel free to start whenever.

[pspzockerscene]

Hm okay so here is my results:
I did run into the "509.gif" Limit at some point of time though then I also had it in browser.
I also noticed that my "original download must always be available" assumption was wrong:
Some items do not have an extra "original" download for example:

exhentai.org/s/f448e4f296/2652144-204

My guess is that we're still kinda talking past each other but I've identifier two main issues which could be issues, you also dealt with:

• E-hentai and exhentai do have different (account-)limits but once either of both gets reached, the account in JD will be temporarily deactivated. If for example your image points of e-hentai.org are empty, JD also wouldn't let you continue to download from exhentai.org.
• -> If you're running into the "509.gif" limit fo exhentai.org, you also won't be able to continue downloading from ehentai.org

If that is [also] what you wanted to tell me:
The only way for me to solve/fix this in a good way is to split ehentai into two plugins: One for ehentai and one for exhentai but both sharing the same codebase.
I'm still looking into other solutions because if all content is available on both platforms, this fact could be used to auto circumvent limits.

Right now I've purposely reached the exhentai limit again.
The funny thing is if I wanted to, I could just continue downloading the same galleries via e-hentai as I haven't reached any limit there yet.
Also why does e-hentai use up your "image points" when downloading original images while you can download the same(?) originals via exhentai.org "without" limits?
EDIT

So are all exhentai.org items also available on e-hentai.org?
I think you've already answered this question before but I'm unable to find that post now.

[pspzockerscene]

Further small changes:

• Fixed wrong file-extension when original file has different file-extension than the one in the filename displayed on website
• Added different errormessage for exhentai items which can't be downloaded due to the 509 limit and also can't be downloaded via e-hentai

I think that to further optimize this and avoid wasting tries, I should implement a probe into the gallery crawler which will check if a gallery is also available on e-hentai (or vise-versa) and bsed on that, allow different fallbacks later.

[Zaagan]

Hi
downloading samplesize doesnt work on ex-hentai anymore, did work Yesterday.
Today with and or without Account only Prefer original quality checked it alwasy uses original filesize pls fix.

[pspzockerscene]

Please post your log-ID here
If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.

Bitte poste deine Log-ID hier.
Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.

-psp-

[Zaagan]

24.08.23 16.18.01 <--> 24.08.23 16.20.16 jdlog://2827311370661/

[pspzockerscene]

Fixed.
Changes:
- updated handling of those image points: If none are available, account can still be used [This change hasn't been finished 100% yet!]
- fixed regression: filesize of original was always set regardless of user preference

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[Zaagan]

works now thank you for fast fix

[pspzockerscene]

Thanks for your feedback and sorry for that small regression.
It was only the displayed filesize. The downloaded file was still the one according to the settings.

[serveral]

Originally Posted by pspzockerscene

E-hentai and exhentai do have different (account-)limits but once either of both gets reached, the account in JD will be temporarily deactivated.

I can confidently say that this wouldn't be the case as the limit is account wide and shared across both sites. A way to prove this is via other tools that I've used for downloading off of both e-hentai and exhentai that show limit consumption is the same across both sites and only stop when the limit reflected at home.php page is maxed. Keep in mind that the limit is also IP based, so if you change your IP via a VPN the limit would be reset (or be set to the limit that other IP had, kind of like MEGA's bandwidth cap system)



Lighter background is a screenshot from e-hentai and the darker one is from exhentai. (script source **External links are only visible to Support Staff****External links are only visible to Support Staff**)

I'm not saying that you should download the script to check it out for yourself, instead I'm doing to this in order to try and better explain how their system works.

Originally Posted by pspzockerscene

Also why does e-hentai use up your "image points" when downloading original images while you can download the same(?) originals via exhentai.org "without" limits?

I'm not sure I understand this question, but limits are consumed equally regardless of which site they're being downloaded from.

Originally Posted by pspzockerscene

So are all exhentai.org items also available on e-hentai.org?
I think you've already answered this question before but I'm unable to find that post now.

No, all e-hentai content is available on exhentai, however there is some content that can only be accessed through exhentai and is not visible on e-hentai.

[pspzockerscene]

Originally Posted by serveral

I can confidently say that this wouldn't be the case as the limit is account wide and shared across both sites.

I can't confirm that or again we're talking past each other.
You can easily test this:

1. Download a lot of items from exhentai.org with JD (original image download enabled!).
2. At some point, the exhentai.org website will display the "509.gif" limit so you can't view images on exhentai.org anymore but you can still download originals if that image is available as original.
3. Also at the same time: If an image is not available as original, you can access it via e-hentai website (where the 509 limit hasn't kicked in yet) and download it from there.

Originally Posted by serveral

I'm not sure I understand this question, but limits are consumed equally regardless of which site they're being downloaded from.

Downloading originals from exhentai.org that does not cost any e-hentai image points.
Example:

exhentai.org/s/2227165461/2653717-2

-> Download original from exhentai and check your remaining image points -> No change

e-hentai.org/s/2227165461/2653717-2

-> Download original from e-hentai and check your remaining image points -> Some image points were needed

Originally Posted by serveral

I'm not saying that you should download the script to check it out for yourself, instead I'm doing to this in order to try and better explain how their system works.

No further explanation needed.
I think I understood most of it.
You can just continue to test the JDownloader plugin.

I think this thread is a prime example of what happens if users and programmers are talking past each other.
Now that I've overcome this barricate, the e-hentai plugin can only get better.

[serveral]

Originally Posted by pspzockerscene

exhentai.org/s/2227165461/2653717-2

-> Download original from exhentai and check your remaining image points -> No change

e-hentai.org/s/2227165461/2653717-2

-> Download original from e-hentai and check your remaining image points -> Some image points were needed

I tried downloading that same picture from the "download original" link at the bottom of the page, and in both cases limits were consumed on my end when I checked on the home.php page.

I also tried downloading them via JD and results were the same, limits were consumed when the picture was downloaded from e-hentai and it's exhentai mirror.

Originally Posted by pspzockerscene

1. Download a lot of items from exhentai.org with JD (original image download enabled!).
2. At some point, the exhentai.org website will display the "509.gif" limit so you can't view images on exhentai.org anymore but you can still download originals if that image is available as original.
3. Also at the same time: If an image is not available as original, you can access it via e-hentai website (where the 509 limit hasn't kicked in yet) and download it from there.

I've also tried to test this but I did a little cheating by downloading a bunch of very heavy images with the maximum amount of simultaneous downloads that JD allows, and ended up reaching a 22k image limit without the "limit reached" warning popping up, and all those pictures were from exhentai too, so... I guess the issue is sort of fixed?

Now I'll be honest, their whole image limit system thing that they have for anti-scrapping purposes is VERY convoluted and it's probably one of the reasons as to why, as you've previously said, we end up talking past each other. However there's definitely been a lot of progress done in order the mitigate the original issue at hand and, even though it's not been completely ironed out, I'd have no objections to marking the issue as solved at this point, so long as no other users have anything else to report in regards to this issue of course.

[pspzockerscene]

Originally Posted by serveral

I also tried downloading them via JD and results were the same, limits were consumed when the picture was downloaded from e-hentai and it's exhentai mirror.

Hm okay I'm still not able to confirm this but it's fine as it's nothing I can influence either way.

Originally Posted by serveral

ended up reaching a 22k image limit without the "limit reached" warning popping up, and all those pictures were from exhentai too, so... I guess the issue is sort of fixed?

Yes that's how it should be.
I'm still going to add some changes but as of now and for exhentai.org content (not e-hentai.org yet), limits will not temporary-disable the whole e-hentai account anymore which was the core issue of the problems you've reported.

Originally Posted by serveral

However there's definitely been a lot of progress done in order the mitigate the original issue at hand and, even though it's not been completely ironed out, I'd have no objections to marking the issue as solved at this point, so long as no other users have anything else to report in regards to this issue of course.

Thanks for your feedback.
I'll mark this thread as "Solved" once I'm done with my final changes.

[pspzockerscene]

Next CORE-upödate will contain updated e-hentai/exhentai plugin settings.
More information:
https://board.jdownloader.org/showthread.php?t=94170

Important:
This will reset the "add zip file of gallery" settings one time for all users!

[pspzockerscene]

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-

[pspzockerscene]

I've just pushed another small change/bugfix:
Even without account, the e-hentai plugin was trying to download original images if wanted by the user.
This would often leave such users with error message "Skipped - account missing".
While some images can be downloaded as original without account, for most of all, an account is required.

I've updated the plugin so that original images will never attempted to be downloaded when no e-hentai account is present.
Not only is this better for all users with e-hentai default settings and no e-hentai account: It will also reduce the number of unsuccessful download attempts greatly and thus will reduce the chances of getting IP-blocked by e-hentai.
Additionally, this change also goes in line with the wording in the e-hentai "Original image" setting:

Account downloads: Prefer original quality (better quality, counts towards image points limit)?

Other than that we didn't get more feedback since the last bigger changes of the e-hentai plugin.

[HydroLogic]

Ever since the new changes, nothing downloads.

I get this error: Downloading original files of this gallery (during peak hours) requires GP, and you do not have enough. (e-hentai.org)

Even if I let my account reset back to 0/5000, I still get this error message.

[pspzockerscene]

Please post your log-ID here
If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.

Bitte poste deine Log-ID hier.
Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.

-psp-

[pspzockerscene]

Minor change:
e-hentai now displays an "adult content warning" for some galleries.
I've updated the crawler to be able to handle that.

[Mely]

(Translate by ChatGPT)
I'm not sure if I should reply to this question here, but I'm currently facing a situation. I am a free member of ehentai, and when I download original images, it exceeds the 5000 image download points. JDownloader continues to attempt to download images, leading to an infinite deduction of points because the download of original images is calculated based on the file size, with 1MB costing 10 points. One can only earn 3 points per minute, but JDownloader retries downloading after a failure in 5 minutes. This results in an unlimited deduction of points whenever I download images exceeding 1.5MB when reaching the points limit. I hope one of the following three solutions can be implemented:
1.Check if there are enough points before attempting to download.
2.Do not retry downloading after a failure.
3.Allow the user to set a specific time for retrying downloads.

Thank you.

[pspzockerscene]

This should not happen.

Also, as long as no downloads are happening no points should be used at all so I'm very confused on why JD is using your points while the downloads keep failing.

Please post your log-ID here
If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.

Bitte poste deine Log-ID hier.
Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.

-psp-