JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 05.07.2019, 20:21
Takhen Takhen is offline
Super Loader
 
Join Date: Mar 2018
Posts: 28
Default Pornhub and capchas

Hi, I hope I'm in the right section. I'm using a script that each 10 seconds scans the page of a channel on Pornhub searching for videos. It's a test, and the time I choose is a test too, but the issue I'm testing about is that after some minutes, JD ask for a capcha for each found video, each time it scans the link. The issue with this is that it mean opening each x seconds x capchas. The following is the specific request I receive, it also seems I need the chrome extension when I open in the browser:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
I need to remove this problem, but I don't know the possible ways, niether the exact conditions that starts the capcha's requirements. Someone is able to help?
Reply With Quote
  #2  
Old 06.07.2019, 07:05
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,143
Default

reduce the request rate? every 10 seconds probably the cause for the captchas?
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 06.07.2019, 11:29
Takhen Takhen is offline
Super Loader
 
Join Date: Mar 2018
Posts: 28
Default

It's possible, but there is to know the rapport requests/time, because also an higher interval would be too much if the site resets its "without capcha availability" after a lot of hours. My interval today have not a maximum value, but after the test it will have, I don't know which yet
Reply With Quote
  #4  
Old 06.07.2019, 14:40
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 63,945
Default

I would say pornhub detects your crawling/scanning and then the bot protection kicks in and that's why you see the captcha. I would also reduce the request rate.
every channel scan results in many many requests for videos on that site -> so 10 secs is not enough wait time
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 06.07.2019, 18:12
Takhen Takhen is offline
Super Loader
 
Join Date: Mar 2018
Posts: 28
Default

Someone knows the maximum videos/time to avoid capchas? So I would see if it's not too much for the purpose
Reply With Quote
  #6  
Old 07.07.2019, 00:55
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,143
Default

@Takhen
we don't know either, otherwise we would have it already within the plugins. you will need to experiment and try and find the threshold. When you figure it out, let us know =]
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #7  
Old 08.07.2019, 10:07
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 63,945
Default

I would start with high timeout and then slowely reduce it and that way you should *fast* find a good threshold
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 08.07.2019, 14:45
Takhen Takhen is offline
Super Loader
 
Join Date: Mar 2018
Posts: 28
Default

I'm trying. With 60.000ms for one video, so 1 video/minute after some time the capcha arrives. With 120.000 it seems doesn't. Now I'm trying 90.000, and if it does I will retry 120.000 for more time to be sure.

Jiaz, could you help with flexget in the dedicated thread? It's for the same purpose of this one
Reply With Quote
  #9  
Old 08.07.2019, 17:21
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 63,945
Default

what thread? I can try but can't promise because I'm not familiar with flexget
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 19:04.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.