JDownloader Community - Appwork GmbH
 

Closed Thread
 
Thread Tools Display Modes
  #1  
Old 06.11.2020, 20:28
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default Logging in to E-hentai through account manager triggers bans from site.

As the title says. For some reason whenever I activate my E-hentai account through the account manager feature it results in their system handing out an IP ban and disallowing me from being able to grab/download anything. This behavior disappears as soon as I disable that account on the account manager (and after I get a fresh IP to circumvent the ban of course) and I'm back to being able to grab and download galleries.

At the moment, there's two ways I know of that are able to reproduce this behavior:
1. Log onto your account via the account manager and try to fetch a link with linkgrabber (I.e **External links are only visible to Support Staff****External links are only visible to Support Staff**) . The action will result in an instant IP ban (Link will have the plugin defect tag on JDownloader and if you try to open the link on any browser, only a blank page will be shown)
2. Log onto your account via the account manager, wait some 2-5 minutes, IP will eventually be banned without even taking any actions.

My guess is that they changed something on their back end and the method that the account manager uses to check the account's status is somehow triggering something on their system which in turn triggers an IP ban.

Also, just to clarify, the IP bans are temporary (though they're still incremental, ie 1st ban 1 hour, 2nd 24h, 3rd 1 week, etc) and it's only the IPs that are banned, not the accounts.
  #2  
Old 06.11.2020, 20:37
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

We need username/password to check this. Most likely something has changed on their login and triggering this protection. Send details to support@jdownloader.org and we will check by next week
__________________
JD-Dev & Server-Admin
  #3  
Old 09.11.2020, 18:55
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Ticket:


This is really strange!

I was not able to completely reproduce it but indeed the crawler started failing here once I was logged in into your account.

... but still, by disabling your account, then adding that gallery and enabling it again, I was able to download the "original quality" which is only available for logged-in users.

It may not be a JD related ban but something else hmmm

-psp-
EDIT

Are any other users able to reproduce this issue?
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 09.11.2020 at 18:59.
  #4  
Old 10.11.2020, 00:44
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Yes, the behavior is indeed odd as it only happens when I'm logged in through JD, otherwise the crawler functions just as intended. This wouldn't be so much of an issue if it wasn't for the fact that you need an account to crawl their other sister/mirror site exhentai.org.

Also I had contacted their staff earlier when I was still figuring out what was triggering the bans, and they told me their system should return a "you've been banned" message of sorts (which should precede the site returning blank pages) explaining the reason of the ban, duration, etc. However since the ban is triggered by some sort of interaction between the crawler and the JD log-in system, (as opposed to it being triggered manually through user activity) that message gets kinda lost since it only shows up once, and I lack the technical knowledge as to how one would go around trying to catch that initial message (does the crawler log these sort of messages/interactions with the servers?). Just thought of adding this as well since the message might contain some extra technical data, I don't know...
  #5  
Old 10.11.2020, 13:53
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Fixed!

Explanation if you're interested in it:
Not all e-hentai account are allowed to use exhentai.org.
I had built our crawler to modify all added gallery URLs internally to the exhentai domain as I assumed this was the way to go.
Unfurtunately, I was wrong.
Seems like some accounts either don't have access to exhentai at all or only after some time has passed after initial account registration.
Source:
smdc-translations.com/exhentai-without-plugins-bypass-the-sad-panda/
When I looked through the logs, I've overseen that it said "exhentai" although I've added your "e-hentai" URL.
I've changed it to always use the added URL asnd never modify it --> Hope that won't cause any other bad side-effects.

That also explains why there was only one report about this issue

You can change the password of your e-hentai account now!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #6  
Old 12.11.2020, 18:37
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Quote:
Originally Posted by pspzockerscene View Post
Fixed!

Explanation if you're interested in it:
Not all e-hentai account are allowed to use exhentai.org.
I had built our crawler to modify all added gallery URLs internally to the exhentai domain as I assumed this was the way to go.
Unfurtunately, I was wrong.
Seems like some accounts either don't have access to exhentai at all or only after some time has passed after initial account registration.
Source:
smdc-translations.com/exhentai-without-plugins-bypass-the-sad-panda/
When I looked through the logs, I've overseen that it said "exhentai" although I've added your "e-hentai" URL.
I've changed it to always use the added URL asnd never modify it --> Hope that won't cause any other bad side-effects.

That also explains why there was only one report about this issue

You can change the password of your e-hentai account now!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser **External links are only visible to Support Staff**...! | Please read our **External links are only visible to Support Staff**...!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser **External links are only visible to Support Staff**...! | Please read our **External links are only visible to Support Staff**...!


-psp-
Hey, sorry for the lateness of this answer but the issue seems to still be present, worst of all is that now exhentai links wont work at all because of the new "no valid account present" implemented (back then most galleries from exhentai were still "crawlable" it's just that some very specific ones were locked behind that "special access" thing they got going), which although is a step in the right direction, their system keeps handing out IP bans whenever you're logged in through JD regardless.

Also it seems that the "no valid account present" crawl error is still given even when you're logged in through JD. I can send you guys another account with full exhentai access if you need so, just tell me and I'll be happy to help.
  #7  
Old 12.11.2020, 18:49
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

So all remaining issues you are having are exhentai related?
If so, I will definitely need an account with exhentai access to check this.

If you are still having issues with e-hentai URLs, you may first post these URLs so I can check with my own test-account.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #8  
Old 13.11.2020, 21:26
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Just sent the a mail with the login details.

As for the issues, yes now they're mostly exhentai related. However attempting to crawl both exhentai and e-hentai links while logged in through JD still results in IP bans (blank pages).

Here's a random gallery link from exhentai **External links are only visible to Support Staff****External links are only visible to Support Staff** and the same gallery on e-hentai **External links are only visible to Support Staff****External links are only visible to Support Staff** (links are interchangeable for most of the part so long as the galleries themselves don't have some particular tags)

Here's another gallery that requires exhentai access to be viewed.
Exhentai: **External links are only visible to Support Staff****External links are only visible to Support Staff**
E-hentai: **External links are only visible to Support Staff****External links are only visible to Support Staff** (this one will just show you a "gallery not available message" for a bit then redirect you back to the main page when opened through an internet browser)

Also, this is just my guess but I guess it's still worth nothing: Could the bans be related to the speed or amount of requests sent at one time by JD to e-hentai's login servers? Because I knew they had some measures in place to try and combat botting/scripting to some extent (a staff member later confirmed this). Obviously enough the requests from the crawler seem to be doing just fine, it's just the requests from JD's account management system that seem to trigger the bans. Again this is just my guess since I don't really have a way to prove this.
  #9  
Old 13.11.2020, 21:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

I've forwared ticket/mail to pspzocker, he will check as soon as he finds time
__________________
JD-Dev & Server-Admin
  #10  
Old 16.11.2020, 18:47
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Hmm I still don't think this is any kind of JD ban.
I've just logged in into your account via browser and straight away I was not able to access that exhentai URL.
I haven't even added your account to JD.
Even using a private browser window, I'am experiencing the same issue.

So my guesses are:
- Your account has been banned (e.g. because of JD) BEFORE and now doesn't work anymore at all fo exhentai
- There is a bug on their side --> I think this is more likely the issue

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #11  
Old 16.11.2020, 19:12
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Well I just opened up exhentai on my end and I was able to see the page, so it's definitely not the account. Mind you, they have this weird little thing going on with their cookies where you have to clean them before being able to access the site (just the exhentai cookies though) and you can test if that's the case by opening any e-hentai URL, and if it loads, then it's the cookies for exhentai need to be deleted.

If that's isn't the case however and you're still getting blank pages then it might just be that the IP you're trying to log in through is banned, which would be the most likely case if you guys are working on static IPs. I've more or less ended up tracing back the issue to JD's account manager by using a VPN and multiple proxies.

Again, this issue probably has something to do with a back-end change they've done a couple of weeks ago most likely, which ends up associating the JD account manager's queries as a botting script or something. Just my thoughts.
  #12  
Old 16.11.2020, 19:21
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Indeed it was old exhentai cookies ... strange!

It's workinf fine here now via browser and JD.
I was unable to find any issues.
One of the cookie differences was that, before clearing the older ones there was a cookie with the name "ingneous" and the value "mystery" which now has a hash as value.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #13  
Old 16.11.2020, 19:56
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Odd, I just tried to crawl an exhentai gallery while logged in through JD and it was met with the same behavior (blank page on browser, no valid account on JD)

Here's the logs just in case they could provide some insight 16.11.20 13.12.13 <--> 16.11.20 13.24.41 jdlog://8174425302851/
  #14  
Old 16.11.2020, 20:00
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

I need a debug log in this case.

... so you still think this is caused by JD?
It could also be a really simple "multi session ban" - maybe they try to prohibit users from opening multiple sessions at the same time/accountsharing.

Really no idea.
Jiaz can look at this again if he wants but:
- I'm unable to reproduce it
- So far you're the only user with this issue and afaik this plugin is quite popular so I expect more reports soon in case more users are having the same issue ...

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #15  
Old 16.11.2020, 20:41
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Here's the new logs 16.11.20 14.01.27 <--> 16.11.20 14.05.10 jdlog://0474425302851/

I've seen the blank page behavior reported by other users in other places these past couple of weeks (couldn't confirm if they were using JD though), so it's a given they (exhentai/e-hentai admins) did change something.

It's just that I find it weird that the IP ban behavior is somehow consistent on my end whenever I use JD's account manager, as I have not managed to reproduce the blank page/IP ban behavior in any other way.
  #16  
Old 17.11.2020, 00:36
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

I'm pretty confident that I got it fixed!
When logging in for the first time, JD never accessed exhentai.org right away.
I'm guessing that when not doing this in time, you have to re-login to e-hentai or the "blank page bug" happens.

I was still unable to see any side-effects in a real browser after logging in via JD and I still don't think it is/was a ban!

I guess it somehow worked without issues until now - they may have changed some things.
Anyways - after the next CORE-update please remove- and re-add your e-hentai account and test it again.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #17  
Old 17.11.2020, 06:32
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Will do and shall report back if it's all good on my end. Cheers
  #18  
Old 17.11.2020, 18:19
serveral serveral is offline
DSL Light User
 
Join Date: Aug 2020
Posts: 32
Default

Yep, the issue is solved now. No errors to report on my end anymore.

One thing of note however is that back then (when I was still having the issue) the image limit number wasn't being displayed like in the image below, instead, the account status just ended at "Free Account".
2020-11-17_12-14-17.png

Also thank you very much for your kind support.
  #19  
Old 17.11.2020, 18:31
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

Thanks for the feedback.
I will let pspzocker know to check about this information

@serveral:Maybe provide a new debug log from account check and post logID, might be language/geo stuff that causes the credit parser to fail
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 17.11.2020 at 19:15.
  #20  
Old 17.11.2020, 20:58
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

@serveral
Thanks for your feedback I'm glad I finally found the issue!
I'm not able to reproduce your other issue though neither with my nor with your account.
Please post a DEBUG-log that shows the login process.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 07:32.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.