JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 18.09.2022, 23:35
heinrichb heinrichb is offline
I will play nice!
 
Join Date: Sep 2022
Posts: 2
Default Reddit user profile crawl sometimes not working

Hey JD team,

I'm experiencing a weird issue, where certain Reddit.com User profiles are not getting crawl. I had a quick look at the logs and it looks like the crawler does find some URLs, but somehow they do not get added.

An example URL: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2  
Old 19.09.2022, 12:47
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 77,828
Default

@heinrichb: working fine here/can't reproduce any issue
have you setup a quick filter in linkgrabber right side? eg hide hide imgur?
have you setup any linkgrabber filter? eg blocking imgur or anything else that might cause this issue

Else please provide a debug log, see
https://support.jdownloader.org/Know...d-session-logs
Enable debug mode, restart JDownloader and try to add this link again and then create log and post logID here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 20.09.2022, 00:31
heinrichb heinrichb is offline
I will play nice!
 
Join Date: Sep 2022
Posts: 2
Default

Hey,

thanks for having a look.

No, I don't have much configured, it's basically a vanilla JD2 installation. With few Reddit plugins setting + packagizer rules.

Just testing a bit around here before sending logs. I noticed 2 things.

- I do have have set "crawl complete user profiles" set. Could you confirm that it's working for you with this setting as well?
- In the logs I noticed java.lang.NullPointerException at a very specific time in the crawl. This seems to be the post that generates the Exception **External links are only visible to Support Staff****External links are only visible to Support Staff** In fact just trying to add this link alone to JD doesn't work in throws the same Exception. Could you try if this link is working for you?
Reply With Quote
  #4  
Old 20.09.2022, 10:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 77,828
Default

Quote:
Originally Posted by heinrichb View Post
- I do have have set "crawl complete user profiles" set. Could you confirm that it's working for you with this setting as well?
I have same default setting, yes
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 20.09.2022, 10:28
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 77,828
Default

Quote:
Originally Posted by heinrichb View Post
- In the logs I noticed java.lang.NullPointerException at a very specific time in the crawl. This seems to be the post that generates the Exception
I can reproduce this one
Update: I can't stress enough that I hate unicode within a URL Will be fixed with next update
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 20.09.2022 at 10:39.
Reply With Quote
  #6  
Old 20.09.2022, 10:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 77,828
Default

@heinrichb; most likely this causes the crawling to fail when other links have unicode in url as well. Please wait for plugin update
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 23:32.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.