JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 31.12.2015, 08:26
computermonster
Guest
 
Posts: n/a
Default Entire Youtube Channel Not Grabbed by Linkgrabber

**External links are only visible to Support Staff**www.youtube.com/user/RussiaToday channel has over 27000 videos. But, linkgrabber is grabbing only around 6500.
Reply With Quote
  #2  
Old 31.12.2015, 08:34
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@computermonster
please post your logs from that session so we can see why, I suspect timeout or too many requests in short succession

https://support.jdownloader.org/Know...d-session-logs

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 31.12.2015, 09:21
computermonster
Guest
 
Posts: n/a
Default

31.12.15 06.17.31to31.12.15 06.05.29 jdlog://9421154739341/
Reply With Quote
  #4  
Old 01.01.2016, 07:32
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

ok sorry about the delay, large log took some time to download and then I forgot to come back to it, thankfully I left your tab open in opera. From what I could determine is, decryption list, no more continuation links where provided by youtube and the decryption of the list stopped and videos started processing.

Not fault of JDownloader as I see it.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #5  
Old 04.01.2016, 09:58
computermonster
Guest
 
Posts: n/a
Default

I want to download all videos of Putin from RussiaToday channel.

I searched Putin on RussiaToday channel to obtain following link:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

1. Will it grab links of entire RussiaToday
2. Will it grab links present till "Load More" tab at the bottom of the page
3. Will it grab all links that would be present if I click "Load More" tab infinite times
Reply With Quote
  #6  
Old 04.01.2016, 10:04
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@computermonster
its not infinite times, button only shows or auto loads more videos based on the previous AJAX request. In your 200+meg log that you uploaded, it no longer gave a continue link.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #7  
Old 04.01.2016, 17:25
computermonster
Guest
 
Posts: n/a
Default

What will happen under following circumstance:
1. I search Putin on RussiaToday channel to obtain following link:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

2. I press "Load More" button as long as it shows up when I scroll down the window

3. I let JD2 linkgrabber check this link

Will JD2 check all links on RussiaToday channel or just those present on that page?
Reply With Quote
  #8  
Old 05.01.2016, 01:33
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

we support /user/username but not parameters query=value
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 05.01.2016, 12:50
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

I guess that either that channel no longer has so many videos or youtube prevents mass crawling of such many videos. Even showing that many videos in browser won't work
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 21:07.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.