#1
|
|||
|
|||
Extraction extremely slow with huge number of archives
Hi,
In JD2 beta I have been downloading in one haul the huge number of 5448 .zip archives which the DWD (Deutsche Wetterdienst) site (ftp protocol) has made available for free download. I had automatic extraction on. After completing the download JD2 started extracting as planned, which went well for a while I guess for about 2000 archives. But then the extraction dropped to a very slow crawl of maybe 20 archives per minute. Cpu usage was very high (about 90%) on all four cpu-cores, which is extremely rare on my computer (and I do use some very demanding software sometimes). So after having lost patience I exited JD2 and extracted the remainder of about 3000 archives by hand within a few minutes using WinRAR. These archives are all quite small, each being less than 200 KB in size. The total of the 5448 archives is about 380 MB in size. I'm on Windows Vista, with an AMD Phenom II X4 810 4-core cpu, 4 GB RAM and plenty of harddisk space. The link to the folder containing the archives is: **External links are only visible to Support Staff**ftp://ftp-cdc.dwd.de/pub/CDC/observa...ip/historical/ This link cannot be used directly in JD2, it is the link to the folder on the DWD ftp page where the links to the archives can be collected. It seems more sensible to present this link than the more than 5000 direct links, which would use far too much space in this post. I hope you can do something with this observation. Maybe it is something that should be fixed in JD2. Thanks for looking into this.;) Last edited by rockwater; 17.06.2015 at 01:46. |
Thread Tools | |
Display Modes | |
|
|