[academyoffetishes]

I'm trying to create linkcrawler rules without asking staff for help, but there are some things that confuse me in the documentation.

1. If you scroll down to "Basic example of the structure of a LinkCrawler Rule:", you'll see that example looks like this:
   
   

   [
   {
   "enabled": true,
   "cookies": [
   [
   "key",
   "value"
   ]
   ],
   "updateCookies": true,
   "logging": false,
   "maxDecryptDepth": 1,
   "id": 1000001540111,
   "name": "example rule",
   "pattern": "**External links are only visible to Support Staff**,
   "rule": "DEEPDECRYPT",
   "packageNamePattern": "<title>(.*?)</title>",
   "passwordPattern": null,
   "formPattern": null,
   "deepPattern": null,
   "rewriteReplaceWith": "**External links are only visible to Support Staff**
   }
   ]

   So the rule is "DEEPDECRYPT", but it has a "rewriteReplaceWith". According to the table below it, this is only allowed in "rule": "REWRITE". So what does this do? Where does the field get its $1 from? I know what $1 means, but I don't know which field it's coming from.
2. Using the same example, "packageNamePattern": "<title>(.*?)</title>" But in the table below, that field uses this example: **External links are only visible to Support Staff** So does this regex look at the URL or the html, or both? I have the same question about all the other "pattern" fields.
   
3. How do you set the name of the file in a rule? e.g., the URL //example.com/video/-210372192_456239067 should download as the page's <h1> value on the page it came from.
   
4. What kind of rule would I use for this example? **External links are only visible to Support Staff****External links are only visible to Support Staff** I want the name of the file to be "Smokie - Needles and Pins 1977.<whatever>". If you look at the <source src= on that page, you will see the url of the file to download (it has no extension).

[Jiaz]

@academyoffetishes:

Originally Posted by academyoffetishes

So the rule is "DEEPDECRYPT", but it has a "rewriteReplaceWith". According to the table below it, this is only allowed in "rule": "REWRITE". So what does this do? Where does the field get its $1 from? I know what $1 means, but I don't know which field it's coming from.

it depends on the rule type (eg DEEPDECRYPT) on which fields are used for this rule. There are not different type of objects/jsons for different types of rules and thus you can set any field you want but only those used by that specific type of rule will actually get used.

Originally Posted by academyoffetishes

Using the same example, "packageNamePattern": "<title>(.*?)</title>" But in the table below, that field uses this example: **External links are only visible to Support Staff** So does this regex look at the URL or the html, or both? I have the same question about all the other "pattern" fields.

pattern(URL) specifies the pattern on which the rule matches on, rewriteReplaceWith references to pattern(URL), all other pattern match on HTML, see https://support.jdownloader.org/Know...kcrawler-rules

Originally Posted by academyoffetishes

How do you set the name of the file in a rule? e.g., the URL //example.com/video/-210372192_456239067 should download as the page's <h1> value on the page it came from.

That's not possible from within a linkcrawler rule, please use packagizer for this, see https://support.jdownloader.org/Know...paketverwalter

Originally Posted by academyoffetishes

What kind of rule would I use for this example?

wait for next plugin update, this site matches existing plugin template and videos are hosted on vk.com

[academyoffetishes]

Thank you for the help!

Originally Posted by Jiaz

@academyoffetishes:

it depends on the rule type (eg DEEPDECRYPT) on which fields are used for this rule. There are not different type of objects/jsons for different types of rules and thus you can set any field you want but only those used by that specific type of rule will actually get used.

Okay, so what I think that example is, is just showing off the structure, but the example wouldn't really work.

Originally Posted by Jiaz

pattern(URL) specifies the pattern on which the rule matches on, rewriteReplaceWith references to pattern(URL), all other pattern match on HTML, see **External links are only visible to Support Staff**...

Okay that's really useful to know. Is that said anywhere in the documentation, because it's news to me. I don't know if you have any interest in updating the documentation, but I do notice a lot of people asking questions about link crawler rules. If you would have any interest, I could give some suggestions on some edits that I think would reduce forum questions. For example, I think multiple examples for each rule would help me a lot.

Regardless, thank you!

[Jiaz]

Originally Posted by academyoffetishes

Okay, so what I think that example is, is just showing off the structure, but the example wouldn't really work.

I'm sorry but I don't understand?! The examples are all good, see https://support.jdownloader.org/Know...kcrawler-rules
What example do you mean?

Originally Posted by academyoffetishes

Okay that's really useful to know. Is that said anywhere in the documentation

Yes, see here, https://support.jdownloader.org/Know...kcrawler-rules

[academyoffetishes]

Originally Posted by Jiaz

The examples are all good, see **External links are only visible to Support Staff**...
What example do you mean?

The examples are good, but more examples would be more helpful. I guess I am requesting more examples of different scenarios for each rule.

[pspzockerscene]

It depends.
In a lot of cases you probably want more examples of regular expressions -> You should learn that separately (use e.g. regex101.com).
Also, you will find a lot more examples in our forum, just search for "link crawler rule".

If you do not find answers for your questions, you can still ask here.
At this moment I don't really see the need for more examples as we got one example for each rule-type and also a general explanation of all types and fields.
However, feedback is always welcome!