Quote:
Originally Posted by annodominus
As far as I understand how this site works, it basically acts like an aggregator and archiver of nsfw content on reddit. Other than its image-focused UI, its main features that make it work better than normal reddit browsing (and likewise JD's crawler for reddit):
|
Ahh okay so I guess it doesn't make any sense to crawl the original reddit links since:
- That website is sorted better
- Some of it may already be down on reddit
Quote:
Originally Posted by annodominus
I'd appreciate any pointers you can give to better automate such a process?
|
As explained in my first reply, you could use LinkCrawler Rules:
https://support.jdownloader.org/Know...kcrawler-rules
I've created and very basic example for you which will process those "/post/" URLs and grab all images.
It will still find unwanted stuff such as thumbnails but you can tweak it further in order to avoid this.
Please keep in mind that I didn't test it with a lot of content so it might not find all expected results for all URLs.
Rule:
Code:
[
{
"enabled": true,
"updateCookies": true,
"logging": false,
"maxDecryptDepth": 1,
"name": "example rule for nsfw.xxx pictures in single posts",
"pattern": "**External links are only visible to Support Staff**,
"rule": "DEEPDECRYPT",
"packageNamePattern": "<title>(.*?)</title>",
"deepPattern": "<img src=\"(https?://[^\"]+)"
}
]
Rule as plaintext for easier copy & paste:
pastebin.com/raw/TZPnEjSF
-psp-