JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 24.06.2022, 03:04
ralph88 ralph88 is offline
I will play nice!
 
Join Date: Mar 2022
Posts: 10
Default Pornhub.com video crawling issue

Searched the forum, but could not find anything on this. I am running the latest version.

When trying to download videos from a models site, the crawler only seems to be going thru the first page, even when I manually copy in the URL to each page. Just the other day, could put in the first page URL and it would crawl thru all the pages.

Here's an example:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Used to be it would crawl all the videos (in this case ~188) with just the first URL, but even if I manually copy in all the URLs, it just comes back with the 40 videos on the first page.

If it matters, I am a premium member and my log-in is saved in Jdownloader

Thanks in advance for any help!
Reply With Quote
  #2  
Old 24.06.2022, 11:24
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

@ralph88: cannot reproduce the issue. Yes, it might be caused by the premium account/logged in state.
Either send logins to support@jdownloader.org so we can try to reproduce it
or provide a debug log.
Enable Settings->Advanced Settings->Log.debugmodeenabled and restart JDownloader, then reproduce the issue and then create a log, see https://support.jdownloader.org/Know...d-session-logs and post shown logID here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 24.06.2022, 16:41
ralph88 ralph88 is offline
I will play nice!
 
Join Date: Mar 2022
Posts: 10
Default

Went to send logs, but I am getting an exception error when trying to send. See below.

I will go ahead and just send the log folder file to the support email with a link to this post unless there is some trouble shooting you want me to do on my side for the log sending error?

For the log - I recreated the issue from the single video page, as well as another example for each of the "video page=x". Hope this helps. Thanks!

Last edited by Jiaz; 24.06.2022 at 17:36.
Reply With Quote
  #4  
Old 24.06.2022, 17:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

@ralph88: restart JDownloader. do not download anything but just add the link and wait for error, then try again to upload the log. by restarting JDownloader you keep the log fresh/small.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 24.06.2022, 18:33
ralph88 ralph88 is offline
I will play nice!
 
Join Date: Mar 2022
Posts: 10
Default

Ok, i restarted and tried submitting.

First is just from the restart, but no link crawled. Probably not what you need.

24.06.22 10.03.37 <--> 24.06.22 10.05.21 jdlog://4211035302851/


Second one: restarted again, added link, tried to send but still error-ed out.

Here's the log after the error, with a restart, that contains the link:

24.06.22 10.32.12 <--> 24.06.22 10.32.43 jdlog://5211035302851/

Hope that helps. Sorry if i am doing something wrong.
Reply With Quote
  #6  
Old 24.06.2022, 21:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

@ralph88: I've replied by mail
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 25.06.2022, 12:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

check again with next plugin update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 26.06.2022, 04:57
ralph88 ralph88 is offline
I will play nice!
 
Join Date: Mar 2022
Posts: 10
Default

Fixed, thank you!
Reply With Quote
  #9  
Old 26.06.2022, 09:17
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

Quote:
Originally Posted by ralph88 View Post
Fixed, thank you!
Thanks for the feedback and you're welcome!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #10  
Old 31.08.2022, 09:04
Thorman Thorman is offline
Modem User
 
Join Date: Nov 2019
Posts: 2
Default

getting the exact same issue as above. do you need logs or can you reproduce?
Reply With Quote
  #11  
Old 31.08.2022, 12:42
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

@Thorman: thanks for the feedback. the fix is not yet working as intended. working on it.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 31.08.2022, 13:46
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,343
Default

@Thorman: should be working with next plugin update
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 13:11.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.