#1
|
|||
|
|||
Bug: Check whether the file exists BEFORE the download begins
Hello jD team!
I found a bug in latest jD 2. If you have a list of files to download, jD2 tries to download the file before it checks whether the file already exits. That means that you'll get a captcha dialog (and you will get blocked by some file hosting sites) even if the file exits and you won't download anything. jD2 should check whether the file exits before it contacts the server, presents you with a captcha dialog ( and refuses to download if the file exits and blocks you from the server for some amount of time). |
#2
|
||||
|
||||
Please post some example links, also include your log.
__________________
» Setup JD2 / Instalador de JD2 «
Spoiler:
Installer for Windows XP/Vista/Seven/Eight || JD2 x86 - x64 (Beta) || Installer for Mac || JD2 (Beta) || Installers for Linux || JD2 (Beta) x86 || <---> || JD2 (Beta) x64 || How to Create a Log -» Click Here «- ¿Cómo crear un registro? -» Click Aquí «- Support Chat / Chat de Soporte -» Click Here / Click Aquí «- |
#3
|
||||
|
||||
@Mikebell
It is "exists" not "exits" JD2 has to contact the server in order to determine if the files exists, or not. Last edited by MrVideo; 18.09.2014 at 11:33. |
#4
|
||||
|
||||
Depending on your user settings when file exists AND if the hoster doesn't provide VERIFIED FILESIZE its not trusted source. Download routine is needed in order to verify that the file you're about to download is the same/smaller/larger in size.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#5
|
|||
|
|||
Quote:
If jD sees that a file exists in the specified downloads directory, it should not connect to the file server, present you with a captcha, and then refuse to download because the file exists. jD does NOT download anything from the server right now to verify that the files are the same (i.e. it doesn't download the file and then check the hash to verify they're the same) yet jD does connect to the server, initiates the download transfer and then just refuses to download because the file already exists. If the file exists, it should let you know ahead of time so you can decide what to do. It shouldn’t connect to these sites and waste your time by "poisoning" the IP and initiating wait times on file sharing sites. As for links, it will work with ANY link. I've used NitoFlare link but it's the same behavior on all links. It's just that NitroFlare flags you as "downloaded file" even if you don't download anything and just solve captcha. |
#6
|
||||
|
||||
As said before, post the links to try to reproduce the problem.
__________________
» Setup JD2 / Instalador de JD2 «
Spoiler:
Installer for Windows XP/Vista/Seven/Eight || JD2 x86 - x64 (Beta) || Installer for Mac || JD2 (Beta) || Installers for Linux || JD2 (Beta) x86 || <---> || JD2 (Beta) x64 || How to Create a Log -» Click Here «- ¿Cómo crear un registro? -» Click Aquí «- Support Chat / Chat de Soporte -» Click Here / Click Aquí «- |
#7
|
||||
|
||||
I think I misunderstood the OP's original question. By file exists, I was thinking about the file existing on the server. After the above postings, I now believe that the OP meant the file already existing on his/her computer.
My bad. |
#8
|
|||
|
|||
Here's a link but links are completely irrelevant. It happens with every hoster.
**External links are only visible to Support Staff****External links are only visible to Support Staff** Quote:
That's right. If the file is available locally, jD2 still connects and tries to download and then it stops. "File exits" check should happen a lot sooner in the process. |
#9
|
||||
|
||||
Its complicated because of the mirror handling...
GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#10
|
||||
|
||||
Advanced settings:
GeneralSettings: Mirror Detection Decision --> FILENAME GeneralSettings: Force Mirror Detection File Size Check --> DISABLE GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
|||
|
|||
@pspzockerscene - Can you clarify what those two settings do? They only seem to effect how JD2 decides if something is a duplicate (filename and/or file size). There is no indication that it effects when it checks for duplicates.
I agree this behavior is undesirable, the default settings should always check if there is a file with the same name in the designated directory before any attempt is made to contact a file service. If there is, regardless of size, boom, done. File size and name are already checked when links are added so rechecking is extraneous either way. Finally, filesize is too fluid between different services to be a useful identifier. Besides, if I've got a file by the same name and a different size, I'm still not going to want it over-written. I can't image anyone else would either. Last edited by Lram32; 22.11.2014 at 04:18. Reason: removed Caps |
#12
|
||||
|
||||
@CuF
Those settings basically switch from verified checking to filename only. 98% hosts do not provide a verified way of linkchecking, its all estimated (they say 2.3GiB or even worse 2GiB, yet you don't know if they have rounded up or down) each mirror could be done differently. Switching it off means if you have a file that's 1MiB downloaded on your disk, will prevent another download from trying to download even though new download is larger.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#13
|
|||
|
|||
Quote:
Edit: I should point out that skipped files don't get counted in percentage completed. E.g. You have a 3 part file, 1 part you download manually, JD2 knows to skip the manually downloaded part, but percent completed won't pass 66% when you are finished. I'll try those settings out. Edit 2: Tested with settings found in post #10. It does not change this behavior. Even if a file exists of the same name, JD2 still contacts the file service before skipping. Last edited by CuF; 24.11.2014 at 18:18. |
#14
|
|||
|
|||
This is a big change from the old JD.
I have just given JD2 a list of files to download from fileboom, using premium bought through JD. I have most of the files and want to complete the set so I was relying on JD2 bypassing the files that exist on my hard disk and downloading the missing ones. It started to do this but then I ran out of download allowance. The problem is that each of the contacts made before bypassing an existing file debited my download allowance and the result is I consumed my 20 GB daily download allowance on files I did not download. Working around this problem looks like it will be a lot of work for me. Any suggestions would be welcome. |
#15
|
||||
|
||||
Ticket:
GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#16
|
|||
|
|||
JD team.
Thanks for all your efforts. I thought that the goal in resolving this problem was to bypass downloads where a file of the same name exists in the download directory without querying the hosting service. That problem is not fixed. If the file already exists then querying the hosting service in unnecessary and and that unnecessary step caused the problem i described in my earlier post. I understand the the design of JD2 has been greatly improved by having a more sophisticated ways of detecting duplicate files and mirrors. The implementation is creating problems and is still worthy of a bug report. |
#17
|
|||
|
|||
Error
or putting it less politely, marking this problem as fixed without modifying JD2 is an error, incorrect and simply wrong. I love your work but you have screwed up on this one.
|
#18
|
||||
|
||||
@1971ti
This is simply wrong! My mistake was that I did not realize that it is already possible and yours that you only read the "closed" status but did not read the response of our developer jiaz: Quote:
Select a more unsafe mirrorDecision and a less than 100% MirrorDetectionFileSizeEquality. Disable ForceMirrorDetectionFileSizeCheck (if enabled). I did not check this but this is what I assume from the developers' comment on this! GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#19
|
|||
|
|||
Bugs
I made the advanced settings changes as you interpreted them:mirror decision = filename and force mirror detection file size check = not ticked.
Even with this setting, JD2 queries the hoster before skipping the download. I have not done a controlled test but with this setting it looks like that query initiates a download and debits my fileboom daily download allowance. This is the problem I reported in an earlier post where I consumed 20GB download allowance without downloading a file. Given that the file name is discovered in link grabber and can be compared to file names in the download directory, there is no need to query the hoster (consuming my download quota) before skipping the download. The setting mostly works but the implementation is flawed. I say mostly because I grabbed a couple of mega links where the files were already in the download directory. Mega deleted the files after I grabbed the links but before JD2 got to downloading them. Instead of bypassing the files JD2 went into loop retrying the downloads at 15 minute intervals. I would say this indicates a small problem in the mega plugin as well. |
#20
|
||||
|
||||
I'll let Jiaz know, and ask for some feedback.
I would personally take up the fileboom data usage issue with fileboom, the file hoster should only debit your useage on download of data (http server logs of throughput vs generating final download link with 0 data transfer). They shouldn't take data off your account, unless you actually physically received such data.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
Thread Tools | |
Display Modes | |
|
|