#1
|
|||
|
|||
Link Crawler cookies gets ignored
I have a basic Link Crawler rule and I am not sure why it is not working properly
I have the following rule: { "cookies" : [ ["name1", "very_long_value_208_characters"], ["name2", "value2"], ..., ["name17", "value17"]], "deepPattern" : null, "formPattern" : null, "id" : [somenumber], "name" : "fancy_rule_name", "packageNamePattern" : null, "passwordPattern" : null, "pattern" : "**External links are only visible to Support Staff**, "rewriteReplaceWith" : null, "rule" : "DIRECTHTTP", "enabled" : true, "logging" : true, "updateCookies" : true } I know the cookies are correct because in python I can successfully download a file using a requests.session after setting the session's cookies. Based on the logs the Link Crawler Rule gets activated but not all the cookies set in rule gets propagated,
|
#2
|
|||
|
|||
Please ignore my question, I found an old rule with the same pattern that was triggered before my new rule was considered. All good folks!
|
#3
|
|||
|
|||
Another slightly related question. Is there a way to update the rules, e.g. override a file or something?
It seems that I can override `jd.controlling.linkcrawler.LinkCrawlerConfig.linkcrawlerrules.json` is there a way to avoid restarting JDownloader for the changes to take effect? Last edited by egamal; 07.09.2024 at 13:35. |
#4
|
||||
|
||||
Quote:
You want to update rules from outside JDownloader?
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#5
|
|||
|
|||
> You want to update rules from outside JDownloader?
Correct. After some searching, I found a way via MyJDownloader API to set the 'LinkCrawlerRules' key of the 'jd.controlling.linkcrawler.LinkCrawlerConfig' interface. I am still facing an issue with the `LinkCrawlerRules` and I am not sure how to debug it. The rule gets activated (the request includes the rule cookies) for a link like this one [prefix_that_matches_the_pattern]/filename.zip but it does not get activated (the rule cookies are not part of the request) for a link like this one [prefix_that_matches_the_pattern]/1080p/filename.mp4 |
#6
|
||||
|
||||
Quote:
- Wrong pattern in rule - or: the 2nd link is handled by a plugin and thus cannot be handled by the rule Without real life example URLs and the real/full rule I will not be able to provide further help.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#7
|
|||
|
|||
> Without real life example URLs and the real/full rule I will not be able to provide further help.
Happy to share the rule, links and the full log. How can I securely share that? |
#8
|
||||
|
||||
You can send that to me via PN or to psp@jdownloader.org.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#9
|
|||
|
|||
Quote:
Everything works as expected if I placed my rule before the generic broad one. This is the generic broad rule I am referring to { "cookies" : null, "deepPattern" : null, "formPattern" : null, "id" : 1628106034293, "maxDecryptDepth" : 0, "name" : "Learned file extension: mp4", "packageNamePattern" : null, "passwordPattern" : null, "pattern" : "(?i)https?://.*\\.mp4($|\\?.*$)", "rewriteReplaceWith" : null, "rule" : "DIRECTHTTP", "enabled" : true, "logging" : false, "updateCookies" : true } I don't recall creating that rule but based on the name "Learned file extension" it is probably auto-generated. If this is the case, where can I find more info on automatically learnt rules (I have a quite a few of them)? |
#10
|
||||
|
||||
Quote:
Settings -> Advanced Settings: Code:
LinkCrawler.autolearnextensionorigins Code:
[]
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
||||
|
||||
Far as I understand the mp4 is already listened to by the DIRECTHTTP plugin, there is no need for such learned rule, unless I'm missing something here.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#12
|
||||
|
||||
This might have been a super old entry from when the "auto learn file extensions" feature has been added for the first time.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|