Quote:
Originally Posted by raztoki
dlc is just another container like csv with more downside than upside, its crypted and would require more resources to add links.
As stated earlier I would remove old links when downloaded, I would add new batch of csv when required. This is the best and really the only solution when talking about sheer volume of links you are dealing with.
Also I would recommend that you disable linkchecking in linkgrabber, to speed up adding of links, settings > advanced > LinkCollector.dolinkcheck
This will reduce a lot of overheads with the required browser requests just to check filesize (if your links are directlink you should already know the filename) and online availability. linkchecking will then take place within download routine.
raztoki
|
disable linkchecking is a good idea, definitely when doing those 1 million and more links jobs...
-Xmx12g or -Xmx8g I think was best for some massive performance push...
but it's really wicked to see 7,33gb 11,86gb 11,86gb
(screenshot)
what does that mean exactly? why three different specs?
I am just trying to get the most out of it, so I can handle most issues...