JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 27.06.2020, 13:52
SourceStryker SourceStryker is offline
Baby Loader
 
Join Date: Jun 2020
Posts: 5
Default nhentai.net digit increase linkcrawler rule

So I wanted to create a linkcrawler rule that allowed me to supply the first page and to search through only a specified number of pages. I've been looking around and found this piece of code here, but since all the external links are censored I'm not really sure how the regex works. I've also searched around but not really found any tutorials on it, although I am very willing to say I've missed them.

Code:
[{
"enabled": true,
"maxDecryptDepth": 10,
"name": "nhentai.net",
"pattern": https://nhentai.net/search/?q=femdom&sort=popular&page=,
"rule": "DEEPDECRYPT",
"deepPattern": "(class=\"mp4\"><a href=\"([^\"]+)\"|<a href=\"(/modelSearch/.*?/page/\\d+)\")"
}]
For example if I supplied this link: **External links are only visible to Support Staff****External links are only visible to Support Staff** I would like to be able to specify crawling through the pages 25-100 (hopefully I can change those numbers myself when I need to) without typing each of the links individually.

Further example links look like
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

So essentially they're just page=1, page=2, page=3 at the end and all I want to do is know how to increment the ending for a link crawl up to a certain amount.

I did write some c++ code that generates the links myself, but was hoping there was a more official way to do it through JDownloader 2 through the deepPattern.

Thank you.

Last edited by SourceStryker; 27.06.2020 at 13:59.
Reply With Quote
  #2  
Old 27.06.2020, 19:47
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 55,219
Default

Hi,

what you want is not possible using linkcrawler rules.
Here are your possibilities:
- EventScripter script
- External script and und LinkCrawler rule to auto-parse all galleries of a single search page
- Modify our official nhentai.net plugin and add support to crawl search pages

We do officially not add support for search pages as this simply creates too many requests and we don't want people to abuse our plugins to create so called "site rips".

We are open source - you can find our source code here:
http://jdownloader.org/knowledge/wik...nt/get-started

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 18:01.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.