#1
|
|||
|
|||
[Cloudflare] Spankbang.com some videos unable to download
t
Last edited by ES9038PRO; 10.06.2022 at 01:02. |
#2
|
||||
|
||||
Could not reproduce any issue, worked fine here
Please create a log, see https://support.jdownloader.org/Know...d-session-logs and post shown logID here
__________________
JD-Dev & Server-Admin |
#3
|
|||
|
|||
t
Last edited by ES9038PRO; 10.06.2022 at 01:02. |
#4
|
|||
|
|||
Spankbang works absolutely fine for me.
While link-grabbing JD doesn't know the filesizes yet but this is no problem at all. |
#5
|
||||
|
||||
@ES9038PRO: log doesn't contain any download activity, nor any error.
Please create a debug log. Enable Settings->Advanced Settings->Log.debugmodeenabled and restart JDownloader. Now try to add the link again and create new log
__________________
JD-Dev & Server-Admin |
#6
|
|||
|
|||
I can confirm, there is a problem with spankbang.com. When I copy a lot of links (100+) only some of them are found or some other weird stuff happens.
I just copied 180 links and 92 JD classified as offline and created one package with them. The other time only 100 were found and also weird stuff with offline packages happened. You can try yourself with these:
Spoiler:
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#7
|
|||
|
|||
It probably is a new hcaptcha issue, because when I try open these links in a browser it gets me an hcaptcha page.
|
#8
|
||||
|
||||
Can you please provide a debug log. Enable Settings->Advanced Settings->Log.debugmodeenabled
restart JDownloader and wait for issue to happen again and create a log, see https://support.jdownloader.org/Know...d-session-logs and post shown logID here
__________________
JD-Dev & Server-Admin |
#9
|
|||
|
|||
g
Last edited by ES9038PRO; 09.06.2022 at 20:44. |
#10
|
||||
|
||||
Hi,
English / Englisch the issue you've reported has been caused by our current Cloudflare issues. Please read the first post of the linked thread and post in this thread if you have further questions! In case you've posted a new thread and it will either get merged with the linked Cloudflare thread or we will lock it. Please post Cloudflare related questions in the above linked thread only! Attention: The first post of that linked thread contains useful information for you AND website-owners and also the hint that website-owners can solve this issue on their side which would also make our plugin work again! There is no ETA for a Cloudflare fix from our side so it might be faster to ask website admins! Please read the first post of the linked thread completely and consider sending it to the admins of the website you are having issues with! Deutsch / German Das von dir gemeldete Problem wurde durch die aktuellen Cloudflare Probleme verursacht. Bitte lies den ersten Post des Cloudflare Threads. Falls du einen neuen Thread dazu geöffnet hast wurde dieser entweder in den bestehenden Cloudflare Thread verschoben oder mit diesem Post von mir beantwortet und geschlossen. Weitere Fragen bitte im oben verlinkten Cloudflare Thread posten! Achtung: Im ersten Post ist u.a. beschrieben, dass auch Webseiten-Admins das Problem auf ihrer Seite lösen können --> Unser Plugin würde dann sofort wieder funktionieren - das bitte beachten, den ersten Post vollständig lesen und ggf. an den Support der betroffenen Webseite weiterleiten! Wie können derzeit nicht sagen, wann es von unserer Seite aus eine Lösung für das Cloudflare Problem geben wird! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
|||
|
|||
6
Last edited by ES9038PRO; 10.06.2022 at 01:02. |
#12
|
||||
|
||||
@ES9038PRO: It depends on many different factors if/when Cloudflare will kick in. It can be caused by your IP or simply by the number of links you are trying to add/download
__________________
JD-Dev & Server-Admin |
#13
|
||||
|
||||
All of these questions are answered in the linked Cloudflare thread/FAQ...
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#14
|
|||
|
|||
Duplicate post
Last edited by ES9038PRO; 26.07.2021 at 17:00. Reason: Duplicate post |
#15
|
|||
|
|||
spankbang.com linkgrabber fails on large number of links
f
Last edited by ES9038PRO; 09.06.2022 at 20:44. |
#16
|
||||
|
||||
Merged duplicated posts.
No reason to open up a separate thread and sending a duplicated post. We also watch threads marked as "Declined" or "Solved"! Quote:
For most of all other plugins however, "do link check" will make a difference. Quote:
Please provide example URLs so we can check this. Quote:
Please provide a log. Quote:
Cloudflare also got "completely invisible" Java Script challenges which can also break our plugins. EDIT With your huge list of links I was indeed able to reproduce the issue with URLs appearing as "offline". Again the issue is Cloudflare which seems to kick in after X requests. I wasn't able to make it work without getting a new IP...
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by Jiaz; 26.07.2021 at 14:45. |
#17
|
||||
|
||||
Quote:
Spankbang crawler plugin handles the quality selection and can return multiple qualities whereas other plugins, eg xhamster, does not provide this feature because no crawler plugin is being involved in process. The hoster plugin can only download in certain quality but cannot provide/list all qualities
__________________
JD-Dev & Server-Admin |
#18
|
|||
|
|||
y
Last edited by ES9038PRO; 10.06.2022 at 01:01. |
#19
|
|||||||||
|
|||||||||
No.
We have different plugins handling this differently. A lot of websites will block powerusers - this case is no different. Quote:
Quote:
A lot of crawlers are designed to find multiple items e.g. photo albums. There is no way to process those without accessing the webpage first Quote:
Quote:
Quote:
Quote:
Quote:
It is not planned to add extra request-limit plugin settings for such/all of our supported websites... Most of all should work just fine out of the box. Quote:
Only for a hand full of websites. For all others, the "real" Cloudflare protection kicks in and JD will not be able to handle this at all at this moment! See our Cloudflare thread... Quote:
See our Cloudflare thread. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#20
|
||||
|
||||
I don't have unlimited time available to test this but here are my results so far:
Test with 1000ms request limit: Block after ~260 URLs This tells me that there is either no way around the block or it will maybe work with 500ms although this would slow down the crawl process significantly ... also for non-powerusers. I'll run some more tests once I find the time but I can't promise anything. I'm sorry to say this but as a poweruser it might sometimes be better to use tools specifically designed to download from specific websites. I'm unsure if there are any available for spankbang. I also want to mention that we're open source so you're free to contribute code and improve what you think can be improved. Please don't take this as an insult but we just don't have the time to optimize every single crawler for every single website. As long as it works for the majority of our users we're usually also happy. There are more examples for such rate-limiting websites where we did not optimize our plugins e.g. twitter. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#21
|
||||
|
||||
2nd Test:
Request limit: 3000ms I was able to successfully add all 594 of your 752 URLs. 125 of them are offline. I've set that request-limit for the next update. The sweet-spot might be somewhere in between but it's better to wait longer than not enough. Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#22
|
|||
|
|||
a
Last edited by ES9038PRO; 10.06.2022 at 01:00. |
#23
|
|||||
|
|||||
Quote:
Quote:
If you find a better/smaller value e.g. 2500 I can use that instead. Quote:
https://jdownloader.org/knowledge/wiki/glossary/cnl2 ... or via myjd API: https://my.jdownloader.org/developers/ Quote:
They're only on that list to prevent more tickets from ppl asking again and again. Sometimes Cloudflare is only used to e.g. block VPN users or all users of a specified country. Quote:
Again: You can use external tools to set individual request-limits for specified hosts which will work just fine. Also again: The plugin update I've mentioned in my last post will be released with our next CORE-update: Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|