JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 16.09.2018, 01:39
yuppicide yuppicide is offline
DSL Light User
 
Join Date: Nov 2013
Posts: 30
Default JDownloader2 always confuses me..

I scraped the web for some images to download. Not sure how many maybe 400,000+. Direct links to jpg files.

When I go into Jdownloader2 is does Link Checking. It was on like 15 hours and nowhere near done. So I disabled "LinkChecker: Do Link Check" and restarted.

I tried again, this time LinkGrabber Activity says Crawling and it's taking a long time.

I turned off Clipboard Observer (and as a result it keeps flashing, which is annoying).

How the heck do I get it to just take the links I gave it and download without doing days of checking or crawling?
Reply With Quote
  #2  
Old 17.09.2018, 12:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 58,940
Default

You can disable the flashing clipboard icon in Settings-Advanced Settings-GraphicalUserInterfaceSettings.clipboarddisabledwarningflashenabled

Adding 400k direct images will result in 400k requests and depending on the latency/host/response speed of the server, this may result between 1 and 10 images per second on average. In case the server enforces request limits, then even less than 1 image per second is possible.
Disabling the *do link check* will bypass those requests and you will be able to add them very fast

Scraping websites will make JDownloader add everything it can find on that website. That means in case the website is not supported by a plugin, adding/deep decrypting it might result in many *garbage*. You can try to use Settings-Filter and only allow wanted content/links or make use of Linkcrawler Rules and teach JDownloader how to parse the website and what links you're looking for.

You can provide example links here or to support@jdownloader.org and we can take a look at it and try to help
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 02:26.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.