JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #21  
Old 11.10.2020, 22:06
daffdaemon daffdaemon is offline
Mega Loader
 
Join Date: Jan 2011
Posts: 61
Default Imagefap rate limits status?

I just tried some downloads today after updating. Here is what I noticed, hope it is helpful.

Previously loaded & failed links still failed with message:
"Try restarting this link (plug-in outdated)"

Re-grabbed links successfully downloaded 50 files then failed with same message. JD did not pause at 100 either: was at 108 50 good, 58 failed when I disabled the links.

Linkgrabber was attempting to get some links and stopped working at about same time file downloads stopped working with message:
"Plugin Defect!7045892"

Log ID
11.10.20 14.15.21 <--> 11.10.20 14.03.52 jdlog://9404325302851/
Reply With Quote
  #22  
Old 12.10.2020, 16:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 55,297
Default

Hi,

please post a lot of example URLs and another log.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #23  
Old 13.10.2020, 20:41
daffdaemon daffdaemon is offline
Mega Loader
 
Join Date: Jan 2011
Posts: 61
Default Imagefap rate limits status?

Here are a bunch of links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Here is new log file:
13.10.20 12.36.55 <--> 13.10.20 13.22.16 jdlog://4454325302851/

8018292 29 files successfully downloaded
7045892 Previously failed links failed
7319035 46 files successfully downloaded
7045892 58 files regrabbed & successfully downloaded
6890859 54 files successfully downloaded then continuous fail

Total files downloaded 187
Total MB downloaded 171.93
JD kept trying to download without pausing
I kept it running till it had tried over 100 files from 890859 put it just kept trying to download so I shut it down.
Reply With Quote
  #24  
Old 14.10.2020, 15:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 55,297
Default

@daffdaemon
Thanks for your report!

I've explored- and partly implemented new rate limit handling along with that captcha.
For users who do not want to solve captchas at all, I've also implemented a plugin setting which will disable the captchas and wait/reconnect instead.
See Settings -> Plugins -> imagefap.com

Screenshot:


-psp-

EDIT

Current imagefap ticket & status:



Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 14.10.2020 at 16:47.
Reply With Quote
  #25  
Old 14.10.2020, 17:32
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 55,297
Default

Updates have been released - we're waiting for feedback ...

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #26  
Old 15.10.2020, 01:39
Trenton Trenton is offline
Baby Loader
 
Join Date: Nov 2019
Posts: 8
Default

Quote:
Originally Posted by pspzockerscene View Post
Updates have been released - we're waiting for feedback ...

-psp-
you're awesome!

I tried and I notice the first time I try to donwload a gallery I get 5% of the links "file not found" and if I do a "check online status" on the whole folder it returns the same results, if I do it on the single links it finds them and then force-download them all with no issues.

I am not able to replicate the issue because the latest 2 galleries I've downloaded have been removed from direct link but the original images still stands. also it seems they are doing stuff on the beta site and galleries overall so it might be that.

I'm not sure if I will be able to generate a debug before this weekend.

I do not mind leaving my pc online at night to slowly download like 1 image every 10 seconds to reduce the load on their side (I see their point) so an option to schedule and space out single links in the plugin would be nice IMO
Reply With Quote
  #27  
Old 15.10.2020, 13:50
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 55,297
Default

Please post a log if you're able to reproduce the issue.

Also keep in mind that our crawler (= the module that grabs complete galleries first) does not yet have proper "rate limit" handling implemented - we're still working on it ...

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #28  
Old 15.10.2020, 17:29
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

Wait for next plugin update. I did some more changes and even further reduced number of requests
__________________
JD-Dev & Server-Admin
Reply With Quote
  #29  
Old 15.10.2020, 17:35
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

In case you encounter issues, please provide more logs
__________________
JD-Dev & Server-Admin
Reply With Quote
  #30  
Old 15.10.2020, 19:07
denylx denylx is offline
Baby Loader
 
Join Date: Aug 2019
Posts: 8
Default

Today I tried downloading a gallery with 210 images and while most of the images downloaded just fine, 51 images were downloaded as just "file" with no extension.
I have attached the log below.

15.10.20 21.13.46 <--> 15.10.20 21.31.39 jdlog://2205325302851/
Reply With Quote
  #31  
Old 15.10.2020, 19:38
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

When you rightclick reset those and try again, still downloaded as file?
Filename changed to file on download or when you were crawling/adding the galleries?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #32  
Old 15.10.2020, 20:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

New update with changes, please check. I was not able to reproduce the file issue
__________________
JD-Dev & Server-Admin
Reply With Quote
  #33  
Old 16.10.2020, 15:35
denylx denylx is offline
Baby Loader
 
Join Date: Aug 2019
Posts: 8
Default

Quote:
Originally Posted by Jiaz View Post
When you rightclick reset those and try again, still downloaded as file?
Filename changed to file on download or when you were crawling/adding the galleries?
Yes, even after resetting they still got downloaded as file. I later tried a gallery with a 73 images and it got downloaded correctly. And yes the filename changed to file on crawling/adding the galleries.

I think this problem arises when a gallery contains 100+ images but I'm not sure.

Last edited by denylx; 16.10.2020 at 15:38.
Reply With Quote
  #34  
Old 16.10.2020, 16:03
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

I was not able to reproduce the issue with your provided example links. Please let debug mode enabled in JDownloader and once you encounter/can reproduce the "file" issue again, create log and post here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #35  
Old 16.10.2020, 20:56
daffdaemon daffdaemon is offline
Mega Loader
 
Join Date: Jan 2011
Posts: 61
Default Imagefap rate limits status?

I updated this AM and downloaded both the two largest galleries from the bunch of links previously submitted. No real problems noticed. Nice job!

I noticed it moves through the file list a little slower than before.
I got a handful of "Problems with server" which resolved on reset.
At one point I started getting captchas... 4 of them in succession then JD proceeded with no more interruptions.

Total files downloaded: about 1,100.
Here is log of session:
16.10.20 13.10.22 <--> 16.10.20 13.10.23 jdlog://4145325302851/

Thank you.

PS I am curious...how did you get it to download without pausing.
Reply With Quote
  #36  
Old 17.10.2020, 12:21
denylx denylx is offline
Baby Loader
 
Join Date: Aug 2019
Posts: 8
Default

Quote:
Originally Posted by Jiaz View Post
I was not able to reproduce the issue with your provided example links. Please let debug mode enabled in JDownloader and once you encounter/can reproduce the "file" issue again, create log and post here
So today I tried with bunch of galleries with images in them ranging from 50 to 100 plus. This time while certain images were crawled as "file", they changed to appropriated jpg/png format while downloading which was not the case last time (I don't know what has changed in between). A few files started to show host problems but resetting them fixed it.

During the download a particular gallery started asking for captcha for every other file but an IP change along with file reset fixed this.

Overall I set 627 images for download and everything was downloaded correctly although speed was a bit on the slower side.

I have enabled debug mode and have attached the log file from the session:

17.10.20 14.15.32 <--> 17.10.20 14.46.53 jdlog://8955325302851/
Reply With Quote
  #37  
Old 18.10.2020, 10:49
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

@denylx:
So the name "file" happened while crawling the file, right? Not on actual download? It was already "file" in linkgrabber, right?
File extension was fixed with last update, some were missing correct one.
The host issues were server problems, 500 Internal Server Error
__________________
JD-Dev & Server-Admin
Reply With Quote
  #38  
Old 18.10.2020, 10:51
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

We've set request limit to 1 request every 750ms, that's why the whole process is slower but should less often hit the limits
__________________
JD-Dev & Server-Admin
Reply With Quote
  #39  
Old 19.10.2020, 14:43
denylx denylx is offline
Baby Loader
 
Join Date: Aug 2019
Posts: 8
Default

Quote:
Originally Posted by Jiaz View Post
We've set request limit to 1 request every 750ms, that's why the whole process is slower but should less often hit the limits
Everything is working okay now
Reply With Quote
  #40  
Old 19.10.2020, 15:19
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 68,457
Default

Thanks for the feedback! Still need to find/reproduce/fix the "file" naming issue.
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 04:23.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.