JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 01.08.2022, 00:56
400 400 is offline
Modem User
 
Join Date: Jul 2022
Posts: 4
Lightbulb [LinkCrawler Rules] Request to add fapello

Hi guys
The site is not supported See screenshot
I want to add fapello
To download photos and videos
and everything
examples
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
The site is full of everything
Attached Images
File Type: png 43643.PNG (107.0 KB, 1 views)
Reply With Quote
  #2  
Old 01.08.2022, 14:06
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

Hi,
a plugin for this simple website is not needed.
You can easily make it work using 1-2 LinkCrawler rules.

Please read the linked docs and if you're having trouble understanding it, let us know If you need help with that.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 02.08.2022, 06:56
400 400 is offline
Modem User
 
Join Date: Jul 2022
Posts: 4
Default

My friend I don't know I tried I didn't succeed
Can you help me This is my first time using LinkCrawler rules
If you want to download this full profile how?
**External links are only visible to Support Staff****External links are only visible to Support Staff**

If you want to download one image or one video, how?

**External links are only visible to Support Staff****External links are only visible to Support Staff** or **External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #4  
Old 02.08.2022, 06:57
400 400 is offline
Modem User
 
Join Date: Jul 2022
Posts: 4
Default

I have another question
Do I have to make a rule for each profile?
Reply With Quote
  #5  
Old 02.08.2022, 11:57
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

Quote:
Originally Posted by 400 View Post
Can you help me This is my first time using LinkCrawler rules
Yes.

Quote:
Originally Posted by 400 View Post
Do I have to make a rule for each profile?
No.

Here is a simple rule which can crawl images/videos from links to single posts:
Code:
[
  {
    "enabled": true,
    "updateCookies": true,
    "logging": false,
    "maxDecryptDepth": 1,
    "name": "example rule for fapello.com single posts",
    "pattern": "**External links are only visible to Support Staff**,
    "rule": "DEEPDECRYPT",
    "packageNamePattern": "<title>(.*?)</title>",
    "deepPattern": "\"(**External links are only visible to Support Staff**]+)"
  }
]
Rule as plaintext for easier copy & paste:
pastebin.com/raw/Hx2jm3Wr

Add this rule to JD and test it.
If your JD auto-recognizes images/videos whenever you add single post links, it's working as it should.

You need to do the following steps to crawl complete profiles:
1. Open the profile you want to crawl in your browser.
2. Scroll all the way down so you can see all posts.
3. Press CTRL + A followed by CTRL + C to copy all links to all posts and add them to JD.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #6  
Old 03.08.2022, 05:04
400 400 is offline
Modem User
 
Join Date: Jul 2022
Posts: 4
Default

Thanks for your help

I put
The code is as it is in this place Is it correct as in the picture (1)

After placing the code, I tried to experiment

When crawling a photo or video
He doesn't crawl until after two or three tries until he crawls to something else As in picture (2)

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**


And when I scroll down
and copy all links
Crawls into her low-resolution photos
Example
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Not high quality pictures
+ other photos
As in picture (3)
This is the code for the page
Code:
**External links are only visible to Support Staff**www.w3.org%2F2000%2Fsvg%22%3E%0A%20%20%20%20%3Ccircle%20fill%3D%22none%22%20stroke%3D%22%23e5e5e5%22%20stroke-width%3D%222%22%20cx%3D%2210%22%20cy%3D%2210%22%20r%3D%227%22%20%2F%3E%0A%3C%2Fsvg%3E%0A
**External links are only visible to Support Staff**
Attached Images
File Type: png 1.PNG (40.0 KB, 1 views)
File Type: png 2.PNG (20.9 KB, 1 views)
File Type: png 3.PNG (114.4 KB, 1 views)
Reply With Quote
  #7  
Old 03.08.2022, 15:07
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

You didn't make any mistakes!

Indeed that rule missed some .mp4 files and didn't grab all available image qualities.
I've updated the rule accordingly:
Code:
[
 {
  "cookies"            : [],
  "deepPattern"        : "\"(**External links are only visible to Support Staff**]+)|[^\"]+\\.mp4)",
  "formPattern"        : null,
  "id"                 : 1659528266827,
  "maxDecryptDepth"    : 1,
  "name"               : "example rule for fapello.com single posts",
  "packageNamePattern" : "<title>(.*?)</title>",
  "passwordPattern"    : null,
  "pattern"            : "**External links are only visible to Support Staff**,
  "rewriteReplaceWith" : null,
  "rule"               : "DEEPDECRYPT",
  "enabled"            : true,
  "logging"            : false,
  "updateCookies"      : true
 }
]
Plaintext for easier copy & paste:
pastebin.com/raw/CHRurTN8
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 25.10.2022 at 17:42.
Reply With Quote
  #8  
Old 24.10.2022, 00:43
CatHypno CatHypno is offline
Modem User
 
Join Date: Oct 2022
Posts: 3
Default

Hi, I just tried this script but it's not working. Not for me at least. All that it does is to grab some (not all) mp4 vids. The rest of the vids and all the pics are offline (but they aren't, of course). I hope you can help me. Thank you!
Reply With Quote
  #9  
Old 24.10.2022, 18:06
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

Please provide example URLs which you've used to test that rule and describe exactly, which items were grabbed and which ones were missing.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #10  
Old 25.10.2022, 16:47
CatHypno CatHypno is offline
Modem User
 
Join Date: Oct 2022
Posts: 3
Default

I used the script as you posted it and I tried with several links but it is the same for all.
As example **External links are only visible to Support Staff****External links are only visible to Support Staff**
or **External links are only visible to Support Staff****External links are only visible to Support Staff**

but all i get is:


A bit better with some account like
**External links are only visible to Support Staff****External links are only visible to Support Staff**



I tried some random with vids and, like I said, only a few vids were downloaded
Reply With Quote
  #11  
Old 25.10.2022, 17:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

@CatHypno
Maybe we should start from the beginning and go through this one by one:
The rule I've posted works on URLs that go to single posts and it will extract the individual images/videos.
Before doing anything we should make sure that the rule is still working. Websites can change at any point of time, rendering such rules/scripts/plugins useless
Example for such a single post:
fapello.com/jennifer-lopez/507/

As we can see it looks like the rule is returning only broken links while that image is clearly available and should be downloadable.
The direct URL to that picture is:
fapello.com/content/j/e/jennifer-lopez/1000/jennifer-lopez_0507.jpg
-> Immediately I see that the "deepPattern" of that rule does not allow links with "j" but only with "i" so we'll update the pattern in that rule accordingly:
Code:
 "deepPattern": "\"(**External links are only visible to Support Staff**j|l|k)/[^\"]+)|[^\"]+\\.mp4)",
Now that rule should work and you can continue testing and update it until it's working for all single post URLs.


Next step:
You seem to be wanting to crawl complete profiles.
While you can add a profile URL and the JDownloader deep-parser will find some single post URLs which will then be handled by said rule, JDownloader will only find all posts of the first page since it does not have a crawler for this website.
In this case I'd recommend the following method(s) to collect all single post links and let the LinkCrawler rule do the rest then:
https://support.jdownloader.org/Know...orted-websites

I'll soon add a followup post to this one in which I will try to improve said LinkCrawler rule even further.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 25.10.2022 at 17:44.
Reply With Quote
  #12  
Old 25.10.2022, 17:52
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

Now additionally to what I wrote in my last post I've noticed two things with that rule that could be improved:
  1. Package name always ends with " - Fapello"
  2. The deepPattern is quite complicated and prone to failure. Also it would return broken image URLs resulting in more results than you'd want to have.
Both possible issues have been addressed in the rule below:
Code:
[
  {
    "deepPattern": "=\"(https?://[^\"]+)\"( target=\"_blank\" class=\"uk-align-center\">| /></video>)",
    "maxDecryptDepth": 1,
    "name": "example rule for fapello.com single posts",
    "packageNamePattern": "<title>(.*?)(  - Fapello)?</title>",
    "pattern": "**External links are only visible to Support Staff**,
    "rule": "DEEPDECRYPT",
    "enabled": true,
    "logging": false,
    "updateCookies": true
  }
]
Rule as plaintext for easier copy & paste:
pastebin.com/raw/DZsZ16qB

Edit by Jiaz: small updated rule, see pastebin.com/f0ncSPcX
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by Jiaz; 26.10.2022 at 12:41.
Reply With Quote
  #13  
Old 01.11.2022, 01:11
CatHypno CatHypno is offline
Modem User
 
Join Date: Oct 2022
Posts: 3
Default

Did some tests and everything seems to be working very well. Thank you!
Reply With Quote
  #14  
Old 02.11.2022, 12:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

Thanks for your feedback.
Just keep in mind that that website can change at any time, rendering that rule useless.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #15  
Old 05.11.2022, 03:50
the first the first is offline
I will play nice!
 
Join Date: Nov 2022
Posts: 2
Default

Welcome pspzockerscene
Why don't you add it to Plugin
Greetings pspzockerscene and Jiaz you are kings
Reply With Quote
  #16  
Old 07.11.2022, 17:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 64,016
Default

Quote:
Originally Posted by the first View Post
Why don't you add it to Plugin
Because:
- Its a simple website so a pugin is not needed
- The demand for a plugin for this website did not seem to be very high
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 10:44.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.