JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 22.03.2021, 17:55
verheiratet1952 verheiratet1952 is offline
Peta Loader
 
Join Date: Jan 2016
Posts: 180
Default wikifeet.com how to DEEPCRAWL for profile links?

how to DEEPCRAWL for profile links?

example URL
**External links are only visible to Support Staff****External links are only visible to Support Staff**


profile links is all links at **External links are only visible to Support Staff****External links are only visible to Support Staff**
10 on each page


(by the way:
profile links like **External links are only visible to Support Staff****External links are only visible to Support Staff** do work perfect...)
Reply With Quote
  #2  
Old 22.03.2021, 18:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,357
Default

Fixed existing crawler picking up "/celebs" URLs as profileURLs and thus failing.
--> Deepdecrypt of e.g. "/celebs?page=22" will work fine after the next update.

If you want JD to auto DEEPDECRYPT such URLs, you can make a simple LinkCrawler Rule.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 22.03.2021, 18:23
verheiratet1952 verheiratet1952 is offline
Peta Loader
 
Join Date: Jan 2016
Posts: 180
Default

Quote:
Originally Posted by pspzockerscene View Post
Fixed existing crawler picking up "/celebs" URLs as profileURLs and thus failing.
--> Deepdecrypt of e.g. "/celebs?page=22" will work fine after the next update.

If you want JD to auto DEEPDECRYPT such URLs, you can make a simple **External links are only visible to Support Staff**....
thank you

creating a linkcrawler rule does ONLY work if there is NO existing plugin for that website?
Reply With Quote
  #4  
Old 22.03.2021, 18:31
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,357
Default

Quote:
Originally Posted by verheiratet1952 View Post
creating a linkcrawler rule does ONLY work if there is NO existing plugin for that website?
Kind of.
To be more precise:
It works as long as there is no plugin that handles the URLs you want to crawl.
In this case, our plugin accepts URLs of single profiles but ignores "/celebs..." URLs -> Yes you could create a LinkCrawler rule for those.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #5  
Old 22.03.2021, 18:49
verheiratet1952 verheiratet1952 is offline
Peta Loader
 
Join Date: Jan 2016
Posts: 180
Default

Quote:
Originally Posted by pspzockerscene View Post
Kind of.
To be more precise:
It works as long as there is no plugin that handles the URLs you want to crawl.
In this case, our plugin accepts URLs of single profiles but ignores "/celebs..." URLs -> Yes you could create a LinkCrawler rule for those.

-psp-
can you help with linkcrawler rule for those links?



you say "...will work fine after the next update" ?
Reply With Quote
  #6  
Old 23.03.2021, 12:57
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,357
Default

Quote:
Originally Posted by verheiratet1952 View Post
you say "...will work fine after the next update" ?
The mentioned update has already been released yesterday.

Quote:
Originally Posted by verheiratet1952 View Post
can you help with linkcrawler rule for those links?
Here is an example LinkCrawler Rule for "wikifeet.com/celebs?page=XXX":
Code:
[ {
  "enabled" : true,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "wikifeet.com/celebs example rule",
  "pattern" : "**External links are only visible to Support Staff**,
  "rule" : "DEEPDECRYPT",
  "deepPattern" : "class=boxcont><a href=\"(/[^\"]+)\""
} ]
Rule as plaintext for easier copy/paste:
pastebin.com/tZ2BgVaD
See this support-article on how to add LinkCrawler Rules to JD.

Please keep in mind that LinkCrawler Rules are designed to be made by users.
We'll give you some examples and help out but if you need more you'll have to learn how to create them yourself.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 15:58.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.