JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 18.05.2019, 09:32
mjolnir870 mjolnir870 is offline
Wind Gust
 
Join Date: Oct 2012
Posts: 42
Default [LinkCrawler Rule] Redtube Photo Gallery Support

Example link here: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Pasting this link into JD right now downloads dozens of unrelated videos.
Reply With Quote
  #2  
Old 18.05.2019, 18:59
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 67,320
Default

When you copy link into clipboard and JDownloader doesn't react, then it's sign that it's not supported. Pasting a link makes JDownloader crawl the whole site and add whatever it finds Not yet supportedv
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 28.05.2019, 21:10
mjolnir870 mjolnir870 is offline
Wind Gust
 
Join Date: Oct 2012
Posts: 42
Default

Thanks I figured it wasn't supported, that's why I requested it.
Reply With Quote
  #4  
Old 31.07.2020, 16:19
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 54,410
Default

Hi,

this is easily possible via LinkCrawler Rules.

A dedicated plugin is not required.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #5  
Old 01.08.2020, 00:47
mjolnir870 mjolnir870 is offline
Wind Gust
 
Join Date: Oct 2012
Posts: 42
Default

Quote:
Originally Posted by pspzockerscene View Post
Hi,

this is easily possible via LinkCrawler Rules.

A dedicated plugin is not required.

-psp-
When I paste the above link in JDownloader, the linkgrabber parsed 2050 links. It also crawled a twitter link it found.

If there is a way to avoid this using the default JDownloader abilities can you please point me to where? I don't see anything called LinkCrawler. Do you mean Linkgrabber filters? I use those, but it seems like they aren't really meant for this. A dedicated plugin would avoid scraping the entire page instead of just grabbing the specific links needed.
Reply With Quote
  #6  
Old 01.08.2020, 06:32
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,196
Default

its located in settings > advanced settings >

so to answer your query, yes you can avoid content by providing location to the content that you want to scan the html via regular expression pattern.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #7  
Old 01.08.2020, 09:02
tony2long's Avatar
tony2long tony2long is offline
English Supporter
 
Join Date: Jun 2009
Posts: 6,381
Default

@mjolnir870
Code:
[ {
  "logging" : true,
  "rule" : "DEEPDECRYPT",
  "maxDecryptDepth" : 1,
  "pattern" : "https?://www\\.redtube\\.com/gallery/\\d+",
  "packageNamePattern" : "<title>(.+?)</title>",
  "deepPattern" : "data-fullsize=\"(https?://[^<>\"]*?\\.jpg)\""
} ]
__________________
FAQ: How to upload a Log
Reply With Quote
  #8  
Old 04.08.2020, 15:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 54,410
Default

@tony2long
I recommend you to always add a pastebin URL to link crawler rules to get round our forums' URL censoring so that the user can simply copy & paste the full rule e.g.:
pastebin.com/E8Y9tj68

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #9  
Old 04.08.2020, 21:51
mjolnir870 mjolnir870 is offline
Wind Gust
 
Join Date: Oct 2012
Posts: 42
Default

Quote:
Originally Posted by tony2long View Post
@mjolnir870
Code:
[ {
  "logging" : true,
  "rule" : "DEEPDECRYPT",
  "maxDecryptDepth" : 1,
  "pattern" : "https?://www\\.redtube\\.com/gallery/\\d+",
  "packageNamePattern" : "<title>(.+?)</title>",
  "deepPattern" : "data-fullsize=\"(https?://[^<>\"]*?\\.jpg)\""
} ]
Thanks, this worked, and now that I've been introduced to the concept I already wrote another one for another site. Is there a wiki or guide for the different options? I've tried searching the forum but it seems like it's not officially documented?
Reply With Quote
  #10  
Old 05.08.2020, 12:48
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 54,410
Default

No it really isn't well documented.
However it is on my TODO to at least add an overview and some simple examples to our help database.

What exactly are you looking for at this moment?

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 17:05.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.