JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 26.03.2020, 17:08
Chuckybuck Chuckybuck is offline
Baby Loader
 
Join Date: Mar 2020
Posts: 6
Default Downloading a whole website and maybe more

I would like to say hi ya'll. My over-priced subscription to IDM is soon coming to an end. I really want to change even though I have enjoyed it. I decided to try out JD because thumbing through the community I found that I really liked the interaction. I also like the fact that it is open source as well. Now to my question.

Looking for a function to download an entire site like with IDM. I am an information junky like no other. I have a pebibyte of storage devoted to a huge library. I am looking to download a whole site (would be really nice to also follow links to other sites also) to find where they have a certain type of information stored. Normally this is mostly what I do. I have a 25.3 Mbps connection devoted just to this effort. Will run it for a couple weeks then fish out what I want to save deleting everything else and then do it all over. So can someone explain/link me to a set of instructions where I can perform this?
Reply With Quote
  #2  
Old 26.03.2020, 17:25
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 51,093
Default

Hi,

we are not actively supporting the download of entire websites, also called "site-rips".
You are free to use our existing crawlers and also custom "link crawler rules" and EventScripter scripts to make your life easier but what we do not support is:
- Downloading entire websites
- Downloading so called "search query URLs" e.g. "google.com/jdownloader+is+thebest" --> Download all result pages --> We do not support this

We do not have detailed instructions for either of what I've linked so you'll have to stick to searching our forum and asking other users or our staff.

-psp-
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 26.03.2020, 17:36
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,684
Default

All I would add is, JDownloader is used to download specific content and not complete websites. what you're interested in is spider software or at least used to be called that 15+ years ago.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #4  
Old 26.03.2020, 17:40
Chuckybuck Chuckybuck is offline
Baby Loader
 
Join Date: Mar 2020
Posts: 6
Default

Great, gives me a good start. Thanks! Is anyone aware of someone who is working on this function. Could someone out there that maybe knows someone who is capable to code this function and either put him in touch with me or me in touch with him?
Thanks again!!
Reply With Quote
  #5  
Old 26.03.2020, 17:44
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 51,093
Default

Nope - none of us is doing any kind of things like this.

-psp-
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #6  
Old 26.03.2020, 17:45
Chuckybuck Chuckybuck is offline
Baby Loader
 
Join Date: Mar 2020
Posts: 6
Default

Comment to raztoki, yes you are correct. Crawlers, Spiders to scrapers. But was looking for something like this to play with. I also use HTTrack. It kinda locks up at max speed though, IDM cannot keep up either so I tune back.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 18:53.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.