JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 02.03.2022, 15:10
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 325
Default [LinkCrawler Rule] tits-guru.com website

plugin request for **External links are only visible to Support Staff****External links are only visible to Support Staff**


I was trying with linkcrawler rule


{
"packageNamePattern" : "<title>(.+?)</title>",
"maxDecryptDepth" : 0,
"rewriteReplaceWith" : null,
"passwordPattern" : null,
"updateCookies" : true,
"formPattern" : null,
"deepPattern" : null,
"cookies" : [],
"pattern" : "**External links are only visible to Support Staff**,
"name" : null,
"enabled" : true,
"logging" : false,
"rule" : "DEEPDECRYPT"
}

but it does not crawl all images...
small thumbnails can be removed using linkfilter, but it does not get all images, most images are hidden because of infinite scroll and pagination...

?page=2
?page=3
?page=4
...
...
...

example links
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2  
Old 02.03.2022, 15:18
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,920
Default

I'm sorry but websites with dynamically reloading elements/"infinite scroll" are not easily possible via LinkCrawler Rules.
You can manually collect/create the URLs containing "?page=..." and then add them to JD.
You can also text editors like "Notepad++" and/or browser addons such as "Link Gopher" to help with that.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 02.03.2022, 17:35
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 325
Default

Quote:
Originally Posted by pspzockerscene View Post
I'm sorry but websites with dynamically reloading elements/"infinite scroll" are not easily possible via LinkCrawler Rules.
You can manually collect/create the URLs containing "?page=..." and then add them to JD.
You can also text editors like "Notepad++" and/or browser addons such as "Link Gopher" to help with that.

-psp-

you have any idea how to create full link list for every link with

?page=2
?page=3
?page=4
...
?page=200


it more than 30'000 links with **External links are only visible to Support Staff****External links are only visible to Support Staff**


bulk mode
Reply With Quote
  #4  
Old 02.03.2022, 18:12
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,920
Default

Quote:
Originally Posted by verheiratet1952 View Post
you have any idea how to create full link list for every link with

?page=2
?page=3
?page=4
...
Sure.
Use "Notepad++" or similar powerful text editors, work yourself into it and then you should be able to do generate those 2XX URLs easily.

Quote:
Originally Posted by verheiratet1952 View Post
bulk mode
Such a thing does not exist for our LinkCrawler rules.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 18:27.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.