JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 14.09.2019, 19:00
contadorpagador contadorpagador is offline
Junior Loader
 
Join Date: Dec 2017
Posts: 10
Post Spankbang plugin improve and suggestion for linkgrabber

Hi, I've noticed recently that if I copy more than 10 up to 15 links at same time(when selecting all the videos on 1 profile page by example, which have a maximum of 40 videos), JDownloader starts to check all the links, I don't know if it tries to check their status at same time or one by one, but when are more than 10-15 links, JD starts to say "PluginDefectSpankbang.com".

So, I changed from selectin 40 videos at same time, to give right click, copy link to clipboard with 1-2 seconds difference, and all the links were able to be confirmed online.

In fact, I think this is caused by a config on the site to prevent work overloads, if I try to open 15 videos en Chrome, Edge, Opera or any other browser in shorts periods of time, the site becomes to show me a "503 Service Temporarily Unavailable" if I wait 1 minute or less, eventually will let me access any link or video again.

So, the suggestion is, to put like a timer before checks on files, like 1 or 2 seconds between links until they are confirmed, that way the site won't block the filegrabber tasks, and so on, wont mark as "Plugin defect"

I think, could be good to have the timer setting modifiable by user on plugin setup, to select which amount of seconds to use, or disable it too.

If there is needed any information, I will be glad to share it and help to improve that amazing plugin.
Reply With Quote
  #2  
Old 16.09.2019, 19:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,340
Default

The decrypter plugin already only processes the links 1 by 1 but
then the hoster plugin also processes the resulting links and does linkcheck for each one and then multiple requests happening and can easily result in request limit reached which then will cause plugin error for next links.
Go to Settings-Plugins-search for plugin name and disable/enable fast linkcheck and check again
or wait for next plugin update because in settings it's shown as enabled but it isn't.
I've also added request limit to avoid high number of requests -> this might reduce grabbing speed!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 17.09.2019, 07:29
contadorpagador contadorpagador is offline
Junior Loader
 
Join Date: Dec 2017
Posts: 10
Default

Thanks Jiaz for the quick answer, in fact I had tried unchecking the box "fast linkcheck" before post this suggestion here, I forgot to mention it. In fact after uncheck that box, its when I can grab links using the 1 by 1 method with manual/irregular timing of 1 up to 5 seconds between links.

I have also setup a linkgrabber filter for any unnecesary file like JPG, JS, JPEG, PNG and a specific **External links are only visible to Support Staff****External links are only visible to Support Staff** domain items like the previews (which most of them has a weight no more than 1024Kb).

But, I will be waiting paciently if you decide to add the suggested function.
For now, I will try a bandwidth limiter to specifcly limit JDownloader when grabbing any link from spankbang, to see if it produces a higher artificial latency, and of corse, a slower link checking tasks.

Thanks again for the response!
Reply With Quote
  #4  
Old 17.09.2019, 09:16
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,340
Default

The update is already live and adds global rate limit to this site
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 10:33.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.