#1
|
|||
|
|||
Pornhub and capchas
Hi, I hope I'm in the right section. I'm using a script that each 10 seconds scans the page of a channel on Pornhub searching for videos. It's a test, and the time I choose is a test too, but the issue I'm testing about is that after some minutes, JD ask for a capcha for each found video, each time it scans the link. The issue with this is that it mean opening each x seconds x capchas. The following is the specific request I receive, it also seems I need the chrome extension when I open in the browser:
**External links are only visible to Support Staff****External links are only visible to Support Staff** I need to remove this problem, but I don't know the possible ways, niether the exact conditions that starts the capcha's requirements. Someone is able to help? |
#2
|
||||
|
||||
reduce the request rate? every 10 seconds probably the cause for the captchas?
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#3
|
|||
|
|||
It's possible, but there is to know the rapport requests/time, because also an higher interval would be too much if the site resets its "without capcha availability" after a lot of hours. My interval today have not a maximum value, but after the test it will have, I don't know which yet
|
#4
|
||||
|
||||
I would say pornhub detects your crawling/scanning and then the bot protection kicks in and that's why you see the captcha. I would also reduce the request rate.
every channel scan results in many many requests for videos on that site -> so 10 secs is not enough wait time
__________________
JD-Dev & Server-Admin |
#5
|
|||
|
|||
Someone knows the maximum videos/time to avoid capchas? So I would see if it's not too much for the purpose
|
#6
|
||||
|
||||
@Takhen
we don't know either, otherwise we would have it already within the plugins. you will need to experiment and try and find the threshold. When you figure it out, let us know =]
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#7
|
||||
|
||||
I would start with high timeout and then slowely reduce it and that way you should *fast* find a good threshold
__________________
JD-Dev & Server-Admin |
#8
|
|||
|
|||
I'm trying. With 60.000ms for one video, so 1 video/minute after some time the capcha arrives. With 120.000 it seems doesn't. Now I'm trying 90.000, and if it does I will retry 120.000 for more time to be sure.
Jiaz, could you help with flexget in the dedicated thread? It's for the same purpose of this one |
#9
|
||||
|
||||
what thread? I can try but can't promise because I'm not familiar with flexget
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|