JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 29.01.2022, 11:41
leroyeur leroyeur is offline
JD Beta
 
Join Date: Apr 2021
Posts: 58
Default Xhamster.com Channel Scraping

Copying **External links are only visible to Support Staff**this channel's URL for example, does not feed the LinkGrabber all of the channel's videos. The channel has 900 videos, yet I am left with only 285 links in the linkgrabber, most of them are not videos.
Spoiler:


I do however get a message box telling me that "this is a playlist/channel, do you want to process it?"
Spoiler:


But regardless of whether I choose "Do nothing" or "Process playlist", the result is the same

In contrast, the Xvideos plugin works tremendously in this regard. I pasted a channel's URL that contains over 20,000 videos and it has scraped them all flawlessly.

Here are more example links of channels. All of them have the same problem > when the channel URL is added to JD, it does not fetch only the videos, and it certainly does not fetch all of the videos

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2  
Old 31.01.2022, 16:58
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hi,

Quote:
Originally Posted by leroyeur View Post
The channel has 900 videos, yet I am left with only 285 links in the linkgrabber, most of them are not videos.
Those "/channels/<channelname>" URLs were not yet supported.
I've added support for them for our next set of CORE-updates.

Quote:
Originally Posted by leroyeur View Post
I do however get a message box telling me that "this is a playlist/channel, do you want to process it?"
As you can clearly see in your screenshot, that dialog is from our youtube plugin.
As you've added unsupported URLs, JD scraped the html code behind that "/channels/" URLs to find anything downloadable.
That page must have contained a link to a youtube channel

Quote:
Originally Posted by leroyeur View Post
Here are more example links of channels. All of them have the same problem > when the channel URL is added to JD, it does not fetch only the videos, and it certainly does not fetch all of the videos
All "/users" URLs are working fine here.
If they still fail for you, please provide a log.
As mentioned at the beginning of this post, I've added support for "/channels/" URLs for our next set of CORE-updates.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 07:07.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.