JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 05.07.2024, 16:29
senormal senormal is offline
Baby Loader
 
Join Date: Mar 2024
Posts: 9
Default Deviantart description scraping

Hello! one more we really appreciate the work the devs are putting into JD.

I am not able to get the deviantart plugin to scrape the links contained within the description of each post. Would it be possible to help me figure this out? The images are being correctly scraped and I am indeed logged in.

Thank you.
Reply With Quote
  #2  
Old 06.07.2024, 19:39
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 80,976
Default

@senormal: please provide example links. Best would be links you can check without being logged in. Else please send links and logins to support@jdownloader.org

Just to be sure, you want JDownloader to analyze description for links and process/add those as well?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 09.07.2024, 18:01
senormal senormal is offline
Baby Loader
 
Join Date: Mar 2024
Posts: 9
Default

Thank you for looking into this.

For example:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

does not scrape the the individual pages contained within it like

**External links are only visible to Support Staff****External links are only visible to Support Staff**

and in this link above, it would scrape the image file of the post (the one that has typeof="ImageObject" parameter in class="_1OOyM"). It seems the plugin is currently broken.

However, the content of the post underneath the image (class="_1CFPz") is not parsed, including the gdrive link (class="_3zM-5")

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Thank you very much once again.
Reply With Quote
  #4  
Old 09.07.2024, 22:59
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 80,976
Default

Quote:
Originally Posted by senormal View Post
does not scrape the the individual pages contained within it like

It seems the plugin is currently broken.
can't reproduce any issue, working perfectly fine here.
Please provide a debug log, see https://support.jdownloader.org/de/k...d-session-logs
Enable Debug mode, restart JDownloader, reproduce the issue (adding the /all link and try to download) and then create log and post shown logID here.



Quote:
Originally Posted by senormal View Post
However, the content of the post underneath the image (class="_1CFPz") is not parsed, including the gdrive link (class="_3zM-5")
Not yet support, but thanks for example links
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 10.07.2024, 11:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,894
Default

I do not see us implementing this any time soon since this would require extensive changes to the existing plugin.
However, I've just updated the plugin so that the description will be set as comment (if no comment was set before by the user) so you will have the following options to make use of this, following the next update:
  • Manually look into the comment and add such links to JD
  • Create an EventScripter script which is automatically doing this for you

EventScripter forum thread:
https://board.jdownloader.org/showthread.php?t=70525
EventScripter help article:
https://support.jdownloader.org/Know...event-scripter
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #6  
Old 15.07.2024, 18:35
senormal senormal is offline
Baby Loader
 
Join Date: Mar 2024
Posts: 9
Default

Thank you very much for providing a solution to this issue!

Much appreciated
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 17:25.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.