View Single Post
  #46  
Old 22.03.2017, 15:32
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 325
Default

Quote:
Originally Posted by raztoki View Post
dlc is just another container like csv with more downside than upside, its crypted and would require more resources to add links.

As stated earlier I would remove old links when downloaded, I would add new batch of csv when required. This is the best and really the only solution when talking about sheer volume of links you are dealing with.

Also I would recommend that you disable linkchecking in linkgrabber, to speed up adding of links, settings > advanced > LinkCollector.dolinkcheck
This will reduce a lot of overheads with the required browser requests just to check filesize (if your links are directlink you should already know the filename) and online availability. linkchecking will then take place within download routine.

raztoki

disable linkchecking is a good idea, definitely when doing those 1 million and more links jobs...


-Xmx12g or -Xmx8g I think was best for some massive performance push...

but it's really wicked to see 7,33gb 11,86gb 11,86gb
(screenshot)

what does that mean exactly? why three different specs?

I am just trying to get the most out of it, so I can handle most issues...
Attached Thumbnails
jd-about-12gb-2nd.JPG  
Reply With Quote