JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 07.09.2022, 12:32
Beer Beer is offline
Giga Loader
 
Join Date: Oct 2019
Posts: 99
Default teenfilipina.com

Hello,

I am new to siterip's. Was looking for ways i could do this. One the options, which is familiar, however rudimentar, is to use JD2 + HTML extractor, such as the LinkGrabber extension.

The site was added to basic authentication, and i even tried using the Linkgrabber Filter to see if it would save me some time during the deep scan craws.

I think i might need some tips on this. I wanted to do both the videos and photosets (not archived). As a bonus for expertise, i would also like to have the filenames properly renamed. All while using automation.

This is more of a personal side project for me, because i don't have any real experience on the field, other than the analog method already mentioned.

1. Do you have any tips on the subject of adult siterips? (don't really need to answer this)
2. Would it be possible to create a plugin on the aforementioned site? Seems simple enough for you.

The renaming of files is not mandatory. Only if you wanted to. Don't want you spending time going out of your way for something that is already borderline inappropriate of me to ask.

Some links:

Videos (alphabethical order) - Over 500

Spoiler:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

>

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**


Photosets (alphabethical order) - Around 150

Spoiler:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

>

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**


To whom shall i send the credentials if you decide to looking into it?
Reply With Quote
  #2  
Old 07.09.2022, 12:38
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 77,893
Default

@Beer: send logins to support@jdownloader.org
then we can check and provide further help/hints on this
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 07.09.2022, 12:45
Beer Beer is offline
Giga Loader
 
Join Date: Oct 2019
Posts: 99
Default

Quote:
Originally Posted by Jiaz View Post
@Beer: send logins to support@jdownloader.org
then we can check and provide further help/hints on this
Done. Will await for feedback. Thank you.
Reply With Quote
  #4  
Old 07.09.2022, 16:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 63,127
Default

General information: We're not adding crawlers that auto perform "site rips".
Also:
I'm not planning to add a plugin for this website but I can give you some hints.

Image galleries.
Example:
Code:
teenfilipina.com/members/index.php?n=869&page=1
Afaik it is possible to change their thumbnail URLs to fullsize URLs.
This is usually the fastest way to crawl galleries and it is possible in this case.
Example:
Thumbnail:
Code:
teenfilipina.com/members/content/pictures/tf705/tn_DSC_0002.jpg
Real URL:
Code:
teenfilipina.com/members/content/pictures/tf705/DSC_0002.jpg
I guess you know what I mean.
Then those galleries also got pagination/multiple pages.

I'm not 100% sure but at least poarts of this should be possible using 1-2 LinkCrawler Rules depending on what you want and what is actually possible:
- one rule that crawls all thumbnails and maybe is also able to find/walk through all pages
- one REWRITE rule that changes those thumbnail URLs to fullsize URLs
Example 2nd page:
Code:
teenfilipina.com/members/index.php?page=2&view=20&start=20&n=869
Alternatively you could of course also prepare a text document that contains all links to all pages.
That is of course only possible if you know the max number of pages per gallery before.

Videos:
Looks like those are behind the same type of URLs like image galleries e.g.:
teenfilipina.com/members/index.php?page=1&n=1427
Looks like the website is simple and the direct-URL to those streams is contained in the html code e.g.:
Code:
teenfilipina.com/members/content/movies/tfm262/tfamatuer.mp4
This means that one DEEPDECRYPT LinkCrawler Rule would be enough to find single videos.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #5  
Old 07.09.2022, 19:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 77,893
Default

@Beer: it seems there is an issue making JDownloader fail to download files from that server. I'm working on it.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 08.09.2022, 14:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 63,127
Default

@Beer
He is talking about the direct-URLs of this- and other websites using Basic Authentication.
You can already start making LinkCrawler Rules but the files won't be downloadable in the current JD build.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #7  
Old 10.09.2022, 06:35
Beer Beer is offline
Giga Loader
 
Join Date: Oct 2019
Posts: 99
Default

@pspzockerscene

Changing thumbnail URL's to fullsize URL's is good tip.

@Jiaz

Still not downloadable from what i can see.

Well, thank you both so much for trying. This issue with direct-URLs and other websites using Basic Authentication needs to be addressed, but its probably best if you close this request ticket.

If i do happen to be needing any specific tips in regard to the linkcrawler rules once the downloading issue is fixed i will be sure to ask.
Reply With Quote
  #8  
Old 12.09.2022, 12:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 63,127
Default

Quote:
Originally Posted by Beer View Post
Still not downloadable from what i can see.
The fix for this issue hasn't been released yet.

Quote:
Originally Posted by Beer View Post
best if you close this request ticket.
No ticket has ever been opened.

I recommend waiting for the CORE-update.
After that (but also already now), you can start creating the mentioned rules and see where that gets you.
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 20:20.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.