JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 04.09.2024, 22:12
fileawaytwitter fileawaytwitter is offline
Vacuum Cleaner
 
Join Date: May 2024
Posts: 19
Default [LinkCrawler rules] ashemaletube.com profile crawling

The plugin for the site only grabs the first page of videos in a given profile. Any videos on pages past that seem to be ignored. However, a large number of unrelated videos are downloaded in addition to the target videos.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2  
Old 05.09.2024, 12:51
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,132
Default

Hi,
we do not have a profile crawler for ashemaletube.com and it isn't planned to add one.

Before you ask:
Why does JDownloader find the videos of the first profile page if there is no plugin?
Because they're visible in plain html so the generic deep-parser finds them.
Before you ask: Why doesn't this work here?
Because the Archivebate.com website is doing asynchronous requests so the items/links you are looking for are not in the html which gets loaded when accessing that page (as also described here, search for word "dynamically"). This means that they are invisible to our deepCrawler

How to find all links in this case?

Option 1
Automatically: Create a LinkCrawler Rule of type DEEPDECRYPT for ashemaletube.com.

Option 2
Manually: Append "videos/" to your links so that it ends with "/videos/".
That looks to possible tell the website to load all elemenrts as one html page.
The deep-parser might then just find all items when you add that link.

Option 3
Use these instructions.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 05.09.2024 at 13:01. Reason: Added Option 3
Reply With Quote
  #3  
Old 05.09.2024, 15:21
fileawaytwitter fileawaytwitter is offline
Vacuum Cleaner
 
Join Date: May 2024
Posts: 19
Default

I see, when I try & grab all of the links on a profiles page via LinkClump jDownloader only detects 40 out of the 56 videos that are selected on the page. Do you know why that is? The videos that are being ignored all have a similar starting name convention (3/16 examples linked below), could that be the issue?

**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #4  
Old 05.09.2024, 15:46
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,132
Default

Quote:
Originally Posted by fileawaytwitter View Post
The videos that are being ignored all have a similar starting name convention
The plugins' pattern didn't allow for underscores.
Current/old regular expression:
Code:
ashemaletube\\.com/videos/\\d+/[a-z0-9\\-]+/
New:
Code:
ashemaletube\\.com/videos/\\d+/[a-z0-9\\-_]+/
Fixed.

If you need this to work right now (without the need for updates), here are your options:
  • Remove all underscores from those links before adding them - the links are still valid and the plugin will eat them. You can use a text editor to do a search & replace or just an external script which automatically does a search & replace on stuff copied into your clipboard.
  • Use a LinkCrawler Rule of type REWRITE that automatically cuts of the links before that underscore, effectively removing the underscore and allowing the current plugins' pattern to match to accept them

For all code changes mentioned in this reply, the following information applies:

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 05.09.2024 at 15:50. Reason: Added clearer instructions
Reply With Quote
  #5  
Old 06.09.2024, 15:34
fileawaytwitter fileawaytwitter is offline
Vacuum Cleaner
 
Join Date: May 2024
Posts: 19
Default

The plugin is now grabbing all the links correctly via LinkClump. However, the videos are downloading at 480p instead of 720p (the highest resolution available for them). Do you why that is? (Reference below)

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #6  
Old 06.09.2024, 16:48
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,132
Default

Quote:
Originally Posted by fileawaytwitter View Post
Do you why that is? (Reference below)
The website has changed since the plugin has been created.

I've adopted it to said changes to find the best quality again.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 07.09.2024, 15:35
fileawaytwitter fileawaytwitter is offline
Vacuum Cleaner
 
Join Date: May 2024
Posts: 19
Default

Everything is working perfectly now, thank you.
Reply With Quote
  #8  
Old 09.09.2024, 12:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,132
Default

Thanks for your feedback
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 02:15.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.