JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 18.01.2021, 00:19
georgegalily georgegalily is offline
Junior Loader
 
Join Date: Oct 2018
Posts: 12
Smile Related to pics.vc LinkCrawler Rule

Hi everyone,
I want to download the following gallery in the righ order according to the page"
Code:
**External links are only visible to Support Staff**
I looked at a previous topic and used this auto-crawl Rule:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Code:
[ {
  "enabled" : true,
  "cookies" : null,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "pics.vc example rule 1",
  "pattern" : "https?://(?:www\\.)?pics\\.vc/watch\\?g=[a-f0-9]{32}",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "(https?://s\\d+\\.pics\\.vc/pics/s/[^\"]+\\.jpg)",
  "rewriteReplaceWith" : null
}, {
  "enabled" : true,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "pics.vc example rule 2: Convert thumbnail URLs to fullsize URLs",
  "pattern" : "(https?://s\\d+\\.pics\\.vc)/pics/s/([^\"]+\\.jpg)",
  "rule" : "REWRITE",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "rewriteReplaceWith" : "$1/pics/o/$2"
} ]
First problem is that the crawler does not go throughout the pages which are:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
h**External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**


Second one, the link crawler crawl the links in a not?? specific order, can I force ordering according to image order in the page
and renaming and renumber to: image 001, image 002, image 003
Reply With Quote
  #2  
Old 18.01.2021, 00:36
georgegalily georgegalily is offline
Junior Loader
 
Join Date: Oct 2018
Posts: 12
Default Renumbering links

Hi ALL

I wand to crawl this links to one package say "AAA" while renamin the files to image 001, image 002, image 004...,

Code:
http://s5.pics.vc/pics/n/9e6/903/a5c/029/9e6903a5c029a00ec4663755f5cdf17e.jpg
http://s5.pics.vc/pics/n/572/17c/a1e/7af/57217ca1e7af932bea8caa2506b0b46a.jpg
http://s5.pics.vc/pics/n/531/b89/c0c/bcb/531b89c0cbcb1af164b9a47de900c248.jpg
**External links are only visible to Support Staff**
I used this packagizer rule:


Code:
[ {
  "filetypeFilter" : {
    "enabled" : true,
    "audioFilesEnabled" : false,
    "matchType" : "IS",
    "useRegex" : true,
    "hashEnabled" : false,
    "videoFilesEnabled" : false,
    "docFilesEnabled" : false,
    "subFilesEnabled" : false,
    "exeFilesEnabled" : false,
    "archivesEnabled" : false,
    "imagesEnabled" : false,
    "customs" : "rev"
  },
  "staticRule" : true,
  "originFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "origins" : null
  },
  "hosterURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "onlineStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "onlineStatus" : "OFFLINE"
  },
  "created" : 1610919658756,
  "sourceURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "conditionFilter" : {
    "enabled" : false,
    "matchType" : "IS_TRUE",
    "conditions" : null
  },
  "iconKey" : "extract",
  "enabled" : true,
  "linkEnabled" : false,
  "valid" : true,
  "packagenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "filenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "name" : "Disable Archive *.rev Files",
  "matchAlwaysFilter" : {
    "enabled" : false
  },
  "id" : "DisableRevFilesPackageRule",
  "pluginStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "pluginStatus" : "PREMIUM"
  },
  "filesizeFilter" : {
    "enabled" : false,
    "from" : 0,
    "matchType" : "BETWEEN",
    "to" : 0
  }
}, {
  "filetypeFilter" : {
    "enabled" : true,
    "audioFilesEnabled" : false,
    "matchType" : "IS",
    "useRegex" : true,
    "hashEnabled" : false,
    "videoFilesEnabled" : false,
    "docFilesEnabled" : false,
    "subFilesEnabled" : false,
    "exeFilesEnabled" : false,
    "archivesEnabled" : false,
    "imagesEnabled" : false,
    "customs" : "par,par2"
  },
  "staticRule" : true,
  "originFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "origins" : null
  },
  "hosterURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "onlineStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "onlineStatus" : "OFFLINE"
  },
  "created" : 1610919658762,
  "sourceURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "conditionFilter" : {
    "enabled" : false,
    "matchType" : "IS_TRUE",
    "conditions" : null
  },
  "iconKey" : "extract",
  "enabled" : true,
  "linkEnabled" : false,
  "valid" : true,
  "packagenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "filenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "name" : "Disable Parity Archive *.par(2) Files",
  "matchAlwaysFilter" : {
    "enabled" : false
  },
  "id" : "DisableParFilesPackageRule",
  "pluginStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "pluginStatus" : "PREMIUM"
  },
  "filesizeFilter" : {
    "enabled" : false,
    "from" : 0,
    "matchType" : "BETWEEN",
    "to" : 0
  }
}, {
  "filetypeFilter" : {
    "enabled" : false,
    "audioFilesEnabled" : false,
    "matchType" : "IS",
    "useRegex" : true,
    "hashEnabled" : false,
    "videoFilesEnabled" : false,
    "docFilesEnabled" : false,
    "subFilesEnabled" : false,
    "exeFilesEnabled" : false,
    "archivesEnabled" : false,
    "imagesEnabled" : false,
    "customs" : null
  },
  "downloadDestination" : "<jd:packagename>",
  "staticRule" : true,
  "originFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "origins" : null
  },
  "hosterURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "onlineStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "onlineStatus" : "OFFLINE"
  },
  "created" : 1610919658769,
  "sourceURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "conditionFilter" : {
    "enabled" : false,
    "matchType" : "IS_TRUE",
    "conditions" : null
  },
  "iconKey" : "folder",
  "enabled" : true,
  "valid" : true,
  "packagenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "filenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "name" : "Create Subfolder by Packagename",
  "matchAlwaysFilter" : {
    "enabled" : true
  },
  "id" : "SubFolderByPackageRule",
  "pluginStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "pluginStatus" : "PREMIUM"
  },
  "filesizeFilter" : {
    "enabled" : false,
    "from" : 0,
    "matchType" : "BETWEEN",
    "to" : 0
  }
}, {
  "filetypeFilter" : {
    "enabled" : false,
    "audioFilesEnabled" : false,
    "matchType" : "IS",
    "useRegex" : true,
    "hashEnabled" : false,
    "videoFilesEnabled" : false,
    "docFilesEnabled" : false,
    "subFilesEnabled" : false,
    "exeFilesEnabled" : false,
    "archivesEnabled" : false,
    "imagesEnabled" : false,
    "customs" : null
  },
  "downloadDestination" : "<jd:subfolderbyplugin>",
  "staticRule" : true,
  "originFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "origins" : null
  },
  "hosterURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "onlineStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "onlineStatus" : "OFFLINE"
  },
  "created" : 1610919658786,
  "sourceURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "conditionFilter" : {
    "enabled" : false,
    "matchType" : "IS_TRUE",
    "conditions" : null
  },
  "iconKey" : "folder",
  "enabled" : true,
  "valid" : true,
  "packagenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "filenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : null,
    "useRegex" : true
  },
  "name" : "Adopt folder structure",
  "matchAlwaysFilter" : {
    "enabled" : true
  },
  "id" : "SubFolderByPluginRule",
  "pluginStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "pluginStatus" : "PREMIUM"
  },
  "filesizeFilter" : {
    "enabled" : false,
    "from" : 0,
    "matchType" : "BETWEEN",
    "to" : 0
  }
}, {
  "filetypeFilter" : {
    "enabled" : true,
    "audioFilesEnabled" : false,
    "matchType" : "IS",
    "useRegex" : true,
    "hashEnabled" : false,
    "videoFilesEnabled" : false,
    "docFilesEnabled" : false,
    "subFilesEnabled" : false,
    "exeFilesEnabled" : false,
    "archivesEnabled" : false,
    "imagesEnabled" : true,
    "customs" : null
  },
  "originFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "origins" : [ ]
  },
  "hosterURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : "",
    "useRegex" : true
  },
  "onlineStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "onlineStatus" : "OFFLINE"
  },
  "chunks" : -1,
  "created" : 1610922545180,
  "sourceURLFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : "",
    "useRegex" : true
  },
  "conditionFilter" : {
    "enabled" : false,
    "matchType" : "IS_TRUE",
    "conditions" : [ ]
  },
  "iconKey" : "addContainer",
  "testUrl" : "**External links are only visible to Support Staff**,
  "enabled" : true,
  "valid" : true,
  "filename" : "Image_",
  "packagenameFilter" : {
    "enabled" : false,
    "matchType" : "CONTAINS",
    "regex" : "*<.#=(.+)><jd:orgfiletype>",
    "useRegex" : true
  },
  "filenameFilter" : {
    "enabled" : true,
    "matchType" : "CONTAINS",
    "regex" : ".jpg",
    "useRegex" : true
  },
  "name" : "imageAutoNum",
  "matchAlwaysFilter" : {
    "enabled" : false
  },
  "pluginStatusFilter" : {
    "enabled" : false,
    "matchType" : "IS",
    "pluginStatus" : "PREMIUM"
  },
  "filesizeFilter" : {
    "enabled" : false,
    "from" : 0,
    "matchType" : "BETWEEN",
    "to" : 0
  }
} ]
bu it does not work


2nd using folder watcher can renumbering be done:

->NEW ENTRY<-
chunks=8
enabled=true
packageName=AAA
autoStart=true
priority=highest
addOfflineLink=true
autoConfirm=false
forcedStart=false
downloadFolder = V:\dsd\<jd:packagename>
text=
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by georgegalily; 18.01.2021 at 00:43. Reason: image not showing
Reply With Quote
  #3  
Old 18.01.2021, 16:03
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,190
Default

Hi,

Quote:
Originally Posted by georgegalily View Post
First problem is that the crawler does not go throughout the pages which are:
You'd have to modify the first rule to grab- and accept such "pagination" URLs too.
Quote:
Originally Posted by georgegalily View Post
Second one, the link crawler crawl the links in a not?? specific order, can I force ordering according to image order in the page
and renaming and renumber to: image 001, image 002, image 003
No.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #4  
Old 18.01.2021, 18:07
georgegalily georgegalily is offline
Junior Loader
 
Join Date: Oct 2018
Posts: 12
Default

Thanks U very much, Modified

Last edited by georgegalily; 18.01.2021 at 18:10.
Reply With Quote
  #5  
Old 18.01.2021, 18:19
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,190
Default

Merged pics.vc threads.

I will write a crawler plugin for this website as it has been requested multiple times.
You won't need any LinkCrawler rules anymore then.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #6  
Old 18.01.2021, 19:08
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,190
Default

A crawler plugin for this website will be available after the next update.
You do not need pics.vc related LinkCrawler rules anymore then.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #7  
Old 19.01.2021, 22:06
georgegalily georgegalily is offline
Junior Loader
 
Join Date: Oct 2018
Posts: 12
Default

Thanks a million
Worked well.
Reply With Quote
  #8  
Old 20.01.2021, 00:23
georgegalily georgegalily is offline
Junior Loader
 
Join Date: Oct 2018
Posts: 12
Default

Surprisingly, the plugin sort the result according to their occurrence on the page, but with original names that are long and not ordered by any mean.
Can you update it to rename images to image 001, image 002, image 003
Or Best, please, can You provide me a rule to rename jpg images to image 001, image 002, image 003 in the crawler.
Reply With Quote
  #9  
Old 20.01.2021, 15:00
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,190
Default

Hi again,
Here is a Packagizer rule that changes the filenames according to your needs.
Add it to Settings -> Packagizer
Screenshot:


Please keep in mind that these rules are only applied to all items added AFTER adding the rule!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #10  
Old 20.01.2021, 18:01
georgegalily georgegalily is offline
Junior Loader
 
Join Date: Oct 2018
Posts: 12
Default

Thanks A lot, that is magnificent work,
These are the codes used above, Hehehe I forget to put a ] and did not notice forgetting it, of course this was rejected by regx conditions:
(.*)_[a-f0-9]{32}.jpg$

<jd:orgfilename:1>.<jd:orgfiletype>
this will rename files to numbers 001, 002, 003

or You can use this:
Image <jd:orgfilename:1>.<jd:orgfiletype>
this will rename files to Image 001, Image 002, Image 003

Last edited by georgegalily; 20.01.2021 at 18:04.
Reply With Quote
  #11  
Old 20.01.2021, 18:22
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,190
Default

Quote:
Originally Posted by georgegalily View Post
or You can use this:
Image <jd:orgfilename:1>.<jd:orgfiletype>
this will rename files to Image 001, Image 002, Image 003
Sure - I've just overread that you wanted to have it named "Image XXX.jpg".

Thanks for your feedback.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 04:21.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.