JDownloader Community - Appwork GmbH
 

Notices

Closed Thread
 
Thread Tools Display Modes
  #1  
Old 08.04.2018, 21:16
jdmani jdmani is offline
JD Fan
 
Join Date: Oct 2010
Posts: 73
Question LinkCrawler Rules help

Hi, is there any tutorial or faq showing how to create rules?
I search the forum and i'm not finding it. For example the different values that "rule" parameter (example "DIRECTHTTP") could have and the meaning of them.
And specially how to assign cookies.

Thanks in advance
  #2  
Old 10.04.2018, 14:07
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

LinkCrawler Rules support different modes

REWRITE -> rewrite/change URL
SUBMITFORM -> submit a FORM
DIRECTHTTP -> treat URL as direct downloadable and download the content/response
DEEPDECRYPT -> load the URL and parse the html source (no js execution) to find inner links
FOLLOWREDIRECT -> follow URL redirect
for example: "rule" : "DEEPDECRYPT"

with
"pattern" : "pattern"
you specify the pattern that matches the rule

you can specify cookies by
"cookies: [ ["key1","value1"], ["key2","value2"],["key2","value2"]..]
pay attention for correct json escaping!

you can enable/disable rule by
"enabled" : true or false

you can specify max depth (how deep the same rule can go)
"maxDecryptDepth" : 0

you can specify name (for yourself or when you share rules)
"name" : "Test"

you can specify pattern (DEEPDECRYPT) that returns the name of the package for all found links
"packageNamePattern" : "pattern"

you can specify pattern (SUBMITFORM) that selects the form to submit
"formPattern" : "pattern"

you can specify pattern (DEEPDECRYPT) that returns inner links/html to parse
"deepPattern" : "pattern"

you can specify pattern (REWRITE) to rewrite URL ($1 is first match, $2 is second match...)
"rewriteReplaceWith" : "$1"


In case you need further help of have additional questions, please don't hesitate and ask
__________________
JD-Dev & Server-Admin
  #3  
Old 22.05.2018, 04:30
jdmani jdmani is offline
JD Fan
 
Join Date: Oct 2010
Posts: 73
Default

You're the best Jiaz! I'll play with these and tell you if there is any doubt
  #4  
Old 22.05.2018, 12:42
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

You're welcome! Just ask if you have further questions about this or need help
__________________
JD-Dev & Server-Admin
  #5  
Old 17.08.2020, 17:37
pystein pystein is offline
Fibre Channel User
 
Join Date: Sep 2010
Posts: 129
Default

How exactly do I enter a new rule?

I miss an menu "New rule" or similar.

And there is no context menu in "LinkGrabber" tab as well.

Where do I have to enter it?
  #6  
Old 17.08.2020, 17:46
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hallo pystein,

LinkCrawler Regeln habern kein eigenes Menü - diese findest du in den Profieinstellungen --> Suche nach "link crawler rules" --> Zweites Ergebnis

Siehe auch Forensuche zu Beispielen von LinkCrawler Regeln.
Wir können nur konkret mit Regeln helfen, wenn wir richtige Testlinks haben und eine genaue Beschreibung, was die Regel tun soll.

Grüße, psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #7  
Old 07.01.2021, 20:03
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Wir haben nun einen detailierten Support Artikel zu LinkCrawler Regeln.

As of now we have a more detailed help article regarding LinkCrawler rules.


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #8  
Old 08.01.2021, 22:33
Student im ersten Jahr Student im ersten Jahr is offline
Black Hole
 
Join Date: Nov 2020
Posts: 281
Default

@Jiaz This is a very good extended tutorial!

Last edited by Student im ersten Jahr; 09.01.2021 at 00:04.
  #9  
Old 09.01.2021, 10:08
Student im ersten Jahr Student im ersten Jahr is offline
Black Hole
 
Join Date: Nov 2020
Posts: 281
Default

EDIT 1:
pspzockerscene - Thank you for writing the rule!!! pastebin.com/9XMKenSi, but still get error (decrypt and download issue):
I found the cause of the problem "Error 502" or "Server Error Time Out"
"Checksum" should be automatically deleted in the generated link,
to make the link work.
&md5=...................
JD2 should try to download such a link (without end md5=......) to avoid error 502

Code:
static.flibusta.is:443/converter/get/convert?url=http%3A%2F%2Fflibusta.is%2Fb%2Fb031825c64d401e1e68e022dee7517f0%2Fdownload&out=epub
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Example Screenshot:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

File:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff**
08.01.21 20.49.43 <--> 08.01.21 21.49.43 jdlog://0691625302851/


EDIT 2:
I found another error (wrong File Extension):
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Should be EPUB extension (not ZIP)

Example Screenshot 2:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Student im ersten Jahr; 09.01.2021 at 11:08.
  #10  
Old 09.01.2021, 15:03
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

@Student im ersten Jahr
This is again not a JD issue - it is related to that particular website in its limits.

Please stop wasting our time!

closed!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #11  
Old 12.01.2021, 18:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

You have create LinkCrawler rule to tell JDownloader that those are DirectHTTP urls, then file extension is correct (just tested)
__________________
JD-Dev & Server-Admin
  #12  
Old 12.01.2021, 18:40
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I've done that for him already HERE.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 20:25.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.