#1
|
|||
|
|||
WatchFolder crawljob skipping some download links. As individual jobs they all work.
Hello,
I have i have a script on another computer that is generating crawljob files and placing them in the JDownerloader2's watch folder. Depending on other unrelated factors in the script, it may put up to 12 different download jobs in a single crawljob file. It seemed like everything was working but now under closer inspection i am noticing that there are some links that will not get parse if included in a multi-download crawjob file. However if i put each download into individual crawljob files then all the links get parsed. In this crawljob file JDownloader2 will not download the url ending in "vAw". The only way i can download this is to put it in its own crawljob file. I do not understand what is special about that download job and why it doesn't work yet other similar crawljobs with 12 unique links in the same format and options work fine. Sorry for the weird links. This is just what was at the top of my error list that demonstrates the intermittent problem i am having. Last edited by Jiaz; 05.06.2019 at 19:34. |
#2
|
||||
|
||||
will be fixed with next update. rate limiting by mega and too *fast* crawling caused issue in plugin
__________________
JD-Dev & Server-Admin |
#3
|
|||
|
|||
Sorry I cannot verify if the problem is fixed since the original links are offline now and i only happened to noticed them out of chance. I will take it as a good sign that i have not seen the issue again and leave this as solved unless i can find the problem again.
Thanks for taking a look! |
Thread Tools | |
Display Modes | |
|
|