JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 08.10.2019, 18:17
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default I need help with linkcrawler rule for newgrounds

Since Jdownloader doesn't seem to recognize links like **External links are only visible to Support Staff****External links are only visible to Support Staff** for some reason, I tried to make Jdownloader recognize it and grab the download links e.g.:**External links are only visible to Support Staff****External links are only visible to Support Staff** (which by themselves get recognized by jdownloader) from these links **External links are only visible to Support Staff****External links are only visible to Support Staff** via linkcrawler rule.
I searched pretty much every thread on linkcrawler rules in here and ended up with this code, which I put into the link crawler rule in the advanced settings (I also made sure to tick that link crawler rules checkbox):
Code:
[ {
  "enabled" : true,
  "cookies" : null,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 10,
  "id" : 1570539992812,
  "name" : "newgrounds.com",
  "pattern" : "**External links are only visible to Support Staff**,
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "**External links are only visible to Support Staff****External links are only visible to Support Staff**[A-Za-z0-9-]+/[A-Za-z0-9-]+/[A-Za-z0-9-_.?]+",
  "rewriteReplaceWith" : null
} ]
I did this according to regex rules, but had to throw out the escaping of some characters, since that gives me an error message from jdownloader. (despite regex101 telling me that my original regex, that had the escaping of characters via \ for these links are correct and that they match with all the desired links)

Well my current code for this does nothing. Just a looking for downloads popping up for half a second and disappearing without grabbing any links once I add the links manually.

Are there any jdownloader specific regex variations I need to consider? Since regular escaping of characters with a \ doesn't work here, I'd like to know how to do it right then please. Or do I have to do something else for this to work as intended? (I couldn't find any jdownloader specific tutorial for this either and since links are not visible to me I can't really look it up or learn that from any examples in other threads...)
Reply With Quote
  #2  
Old 08.10.2019, 19:16
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

was caused by plugin error, will be fixed with next update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 08.10.2019, 19:17
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by Loberon View Post
S
I did this according to regex rules, but had to throw out the escaping of some characters, since that gives me an error message from jdownloader. (despite regex101 telling me that my original regex, that had the escaping of characters via \ for these links are correct and that they match with all the desired links)
Are there any jdownloader specific regex variations I need to consider? Since regular escaping of characters with a \ doesn't work here, I'd like to know how to do it right then please. Or do I have to do something else for this to work as intended? (I couldn't find any jdownloader specific tutorial for this either and since links are not visible to me I can't really look it up or learn that from any examples in other threads...)
The syntax is json and that causes additional escaping
You can use json validator (searchengine of your choice) and it will tell you where escaping is wrong
most important " -> " and \ -> \\
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 08.10.2019, 19:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

The reason why your custom rule doesn't work, because rules are processed AFTER all plugins had a chance and the plugin had a bug and that's why it didn't work
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 08.10.2019, 19:37
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default

Oh lol I didn't expect it to be a bug, me being dumb was more likely to me tbh. xD I'll look into that json validator thing. Thank you very much!
Reply With Quote
  #6  
Old 08.10.2019, 20:10
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Update is already live and your link should work fine again
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 08.10.2019, 20:55
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default

I tested it and I noticed that the grabbing of multi-image posts e.g. the third link from my original post: **External links are only visible to Support Staff****External links are only visible to Support Staff** doesn't work correctly. Only the first image-link gets grabbed and the others don't, which as I found out have e.g. **External links are only visible to Support Staff****External links are only visible to Support Staff** a different link pattern compared to the top picture. (my bad for not noticing that beforehand) My custom rule, which, now that I fixed it with that json validator, should be working covers that different pattern. But it doesn't seem to come into play again, could that be another issue with the plugin? Is there any way for me to turn off the plugin or change priorization, so the custom rule is prioritized so I can test, if it's a plugin issue or an issue with the custom rule?

Last edited by Loberon; 08.10.2019 at 20:58.
Reply With Quote
  #8  
Old 10.10.2019, 19:06
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Yes,thanks for the follow up. The current plugin doesn't support multiple images for single art/view link

can you provide more example links?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 10.10.2019, 19:09
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by Loberon View Post
Is there any way for me to turn off the plugin or change priorization, so the custom rule is prioritized so I can test, if it's a plugin issue or an issue with the custom rule?
I'm sorry, not at the moment, but I can add support for it
__________________
JD-Dev & Server-Admin
Reply With Quote
  #10  
Old 10.10.2019, 19:54
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default

2 pics: **External links are only visible to Support Staff****External links are only visible to Support Staff**


1 pic 1 gif: **External links are only visible to Support Staff****External links are only visible to Support Staff**


8 pics: **External links are only visible to Support Staff****External links are only visible to Support Staff**


2 pics: **External links are only visible to Support Staff****External links are only visible to Support Staff**


2 pics: **External links are only visible to Support Staff****External links are only visible to Support Staff**


4 pics: **External links are only visible to Support Staff****External links are only visible to Support Staff**


I hope these are sufficient. If not, let me know. Thanks for the help!

Last edited by Jiaz; 10.10.2019 at 19:59.
Reply With Quote
  #11  
Old 10.10.2019, 20:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Thanks
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 03.02.2020, 20:44
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Fixed in existing newgrounds crawler.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #13  
Old 18.02.2020, 15:19
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default

Thanks, the pattern of secondary pics from multi picture posts is now getting picked up. Unfortunately it seems the single pic pattern recognition was broken by this or some other recent change. (at least for .png .jpg .zip and .gif files other file types seem to work) It just displays "Plugin error" at the linkcrawler now when trying to retrieve those files from /art/view/ links. (this also includes the first file of multi pic posts not being picked up, since the first file has a different link pattern from the others)

Here some links: gif **External links are only visible to Support Staff****External links are only visible to Support Staff**
png **External links are only visible to Support Staff****External links are only visible to Support Staff**
jpg **External links are only visible to Support Staff****External links are only visible to Support Staff**
multi pic post where it now skips picking up the upper file **External links are only visible to Support Staff****External links are only visible to Support Staff**

For the following file type I get a "Plugin outdated" error when attempting to download them, so they get picked up the linkcrawler and the files are displayed as .zip files in the linkcrawler, but as soon at I try to download them the error pops up.
game **External links are only visible to Support Staff****External links are only visible to Support Staff**
game **External links are only visible to Support Staff****External links are only visible to Support Staff**
game **External links are only visible to Support Staff****External links are only visible to Support Staff**

Let me know if you need more links or something. Thanks for the help!

Last edited by Loberon; 18.02.2020 at 15:26.
Reply With Quote
  #14  
Old 18.02.2020, 16:19
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Fixed issues with single pictures and "missing first item".

Regarding .zip downloads:
From the webpage it seems like the content you liked is either not downloadable at all anymore or only for registered users(?)

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #15  
Old 18.02.2020, 17:32
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default

Thank you, I'll try it as soon as the update is available.

From what I see I don't have access to any download button/option on the webpage of the games that get grabbed as .zip downloads, even while being logged in there. The same goes for any other games there as well, but for example .swf files (flash games) there get grabbed and downloaded without problems, despite having no visible download option. Due to this I assumed there's some issue with the .zip format.
Reply With Quote
  #16  
Old 18.02.2020, 17:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I will update our plugin ti display another errormessage in this case.

If they ever fix their website in the future, let me know and I'll check it again.

-psp-
EDIT

Done.


Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 18.02.2020 at 18:13.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 09:22.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.