[kingxerjsaeg]

I've made two json schemas for validating JD2's crawler rule format.
They're available at github.com/sergxerj/jdownloader2-crawler-rule-json-schema including a config folder for working locally with them in visual studio code. Further instructions and a few tips are in the repo's README.md.

One scheme is for validating single rules, the other is for validating an array of rules. The latter is what you can "Ctrl+A, Ctrl+C; Ctrl+A, Ctrl+V" from the editor into JD2's "Link Crawler: Link Crawler Rules".

I'll make a PR to add it to schemastore.org.

[Jiaz]

@kingxerjsaeg: VERY NICE Thumbs up! Also like the good explanations!
Maybe add info to where copy/paste (Settings->Advanced Settings->LinkCrawler.linkcrawlerrules) those rules here

This is the format that you can "Ctrl+A, Ctrl+V" in the current JD2 version as of latest commit.

maybe change

with the Java 8 engine

to

with the Java 8 FLAVOR

as it matches the wording on the website

[kingxerjsaeg]

Thanks! Suggestions applied. Also fixed an extension misplacement in the Usage section.

[Jiaz]

@kingxerjsaeg: would be great if you chould mix schemes, so eg also define value xy must be valid regex pattern

[kingxerjsaeg]

@Jiaz

would be great if you chould mix schemes

You mean for instance "Have different types of rules (DEEPCRYPT, DIRECTHTTP, etc.) in the same multi-rule array"? Because if it's that, then it's already accounted for.

The single-rule scheme validates a rule object only to internal consistency. The multi rule-scheme just reuses this as the array-item definition and validates the listed rule objects to their individual internal consistency.
Unless you mean some other scenario(?)

define value xy must be valid regex pattern

Reading this I remembered that I tried to do it in the definition I was working on, but "format" doesn't seem to be configured as an assertion expression for all possible values stated to be 'builtin' according to spec for the case of VScode's implementation of draft07-onwards. "format":"uri" or "format":"email" work as assertion, "format":"regex" appears to be among the annotation ones. Which is was a strong motivator for me including the "- Type: HTML RegEx" bit in the "description" properties of each rule property. On the "it's not that relevant at this point of user workflow" side, the likely fact that if there was incorrect behaviour from the rule, users would end up putting the regex against the url in the online regex validator itself, which would tell them if the expression was off anyway. So I just left it out.
A small aside is that "format":"regex" validates explicitly against the ES262 regex dialect, I'm not too knowledgeable of the Java flavor but there's probably some construct out there that doesn't translate to the point of breaking the expresion from ES262's viewpoint, hopefully nothing but edge cases that will be caught by regex101.

However, right now, mid-explanation, it prompted my curiosity to see if the online validators had it in their implementation... And lo! They have it. So I'm gonna add a bunch of "format":"regex" to the definition right away.

[pspzockerscene]

Nice one!

[Jiaz]

@kingxerjsaeg
VERY nice to see it being available on jsonschemavalidator.net
Very cool to see the validator in action!

you could optimize to avoid having to set enabled, because enable=true is default value. so you can set it but also leave away for shorter rules

[pspzockerscene]

I've also noticed some possible mistakes in the scheme checking:

Rule:

[
  {
    "deepPattern": "(https?://mega\\.nz/[^\"<>]+)",
    "formPattern": null,
    "maxDecryptDepth": 1,
    "name": "test",
    "packageNamePattern": "<title>(.*?)</title>",
    "passwordPattern": null,
    "pattern": "https?://(www\\.)?test\\.org/topic/[0-9]+-[a-z0-9\\-]+.*",
    "rewriteReplaceWith": null,
    "rule": "DEEPDECRYPT",
    "enabled": true,
    "logging": true,
    "updateCookies": true
  }
]

Errors:

Found 3 error(s)
Message:
Invalid type. Expected String but got Null.
Schema path:
**External links are only visible to Support Staff**
Message:
Invalid type. Expected String but got Null.
Schema path:
**External links are only visible to Support Staff**
Message:
Invalid type. Expected String but got Null.
Schema path:
**External links are only visible to Support Staff**

I'd say:
- formPattern can be null
- passwordPattern can be null
- rewriteReplaceWith can be null / is only needed for rule type "REWRITE"

Please let me know if I made a mistake.

[pspzockerscene]

I've linked jsonschemavalidator.net in our LinkCrawler rules article.
I've included credits to you @kingxerjsaeg.