[netgearjd]

Here's the situation - there are folders on different file hosts (MediaFire, ZippyShare etc.) where files are added off and on. So every month or so I add the folder links to JD, let the LinkGrabber do the crawling and it gives me a list of all the files in those folders, ready for downloading. So far, so good.

However a problem arises because every month I obviously want to catch up with only the latest added content, i.e. I want to download only the files I haven't already downloaded earlier.

Now to do this I add all the links to the Downloads list, then set JD to skip files if they already exist. Again, this does work and the existing files are all skipped, but the major issue is that the checking and skipping is so damn slow!

As far as I can tell, JD first starts every link, connects to the server, goes through the captcha/waiting process if required, then retrieves the file details and possibly even initiates the download before it realizes that oh wait, a file with that same name already exists in the download folder, so the file must be skipped. How crazy!

Why can't JD simply check for the existence of a file with the same name as the current one in the list before it goes through the laborious and slow process of connecting to the server? That way it can simply skip an existing file in a fraction of the time it currently takes.

Moreover it's not just a simple time penalty as described above, because there's another terrible side-effect of the current process too. JD connecting to the server for every file, whether it's to be skipped or not, inevitably results in the file host erroneously imposing limits because according to it so many downloads have been initiated (irrespective of the fact that no data was actually downloaded for so many of the files). This is ridiculous but there's no way around it at present.

The end result is that even if a mere 5 files have been added since my last download session, due to the server-imposed limits, checking the rest of the files takes hours or even days right now (not kidding!), whereas with the short-circuit file check it would probably take not more than a minute at most, which would be a massive improvement.

Final note: I understand that some plugins do not retrieve the actual file name till the download actually starts. The easy solution is for JD to always perform the the short-circuit file check first, then if that fails (no match found) it can continue with the existing slow process. So best-case scenario would be a file match before the server needs to be contacted which would save a huge amount of time, while worst-case scenario would be that the server does need to be contacted for a particular file, resulting in the same time penalty as exists now. Seems like a win-win situation to me!