JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 08.04.2018, 20:33
user748912 user748912 is offline
JD Adviser
 
Join Date: Jun 2014
Posts: 103
Default JD starts the same Download of a File twice

Why JD dont know itself that it started Downloading File1 from share-online?
It just try to Download File1 from rapidgator.

If the rapidgator captcha is requested first, i just get a requester file exists->skip,
forced to enter another annoying captcha from share-online, SO now loads File2.

If share-online captche is requested first, the problem is bigger, requester exists->skip can first start next Download after 2 hours.

This happend not always, but if it happend to a packet it does it over and over again. All files are in the same packet, File1 has the same name(47 chars) for different hosters, no files are downloaded by Hand.
It may be the Problem that MD5 for File1 is SHOWN different, but this is not true, cause its RAR and it always decrunch successful with right CRC32 doesnt care which part comes from which hoster.
Reply With Quote
  #2  
Old 09.04.2018, 16:24
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

As JDownloader has not detected both file as mirror, then either filename or filesize wasn't *safe*. You can customize mirror conditions in Settings-Advanced Settings-mirror
For example you can change GeneralSettings.mirrordetectiondecision to FILENAME only, but this will cause side effects with multiple files but same filename.
Without *safe* information, JDownloader has to start the download in order to get *safe* filename and filesize
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 22.04.2018, 19:15
user748912 user748912 is offline
JD Adviser
 
Join Date: Jun 2014
Posts: 103
Default

I catched the Files via JD Link-collector, they are in the same Package & same folder. Mirror has the same Name&Filesize(Before Download)

How can JD assume that the user want download a Package, that is in the same folder, with different Files but the same Filename? Its clear that this cause a Collision.
Reply With Quote
  #4  
Old 23.04.2018, 09:44
dabrown dabrown is offline
Black Hole
 
Join Date: Jun 2015
Location: North America
Posts: 264
Default

Mirror detection breaks when the file name "changes" between detection, download, and completion.

Rapidgator is one that changes almost every time- it detects with spaces turned into underscores but those change to spaces when the file actually downloads. Keep2share and fileboom, they detect the full filename on link add, but on link start it truncates if the file name is fairly long. If the download doesn't start, it changes the link's file name to the truncated one, breaking mirror detection, then changes it back to the full name only when the download actually starts.

I complained about this a year ago, and was told to work around it by renaming every single mirrored link manually. Which I do, because it ticks me off when it tries to dl the same file in the same package twice.

Apparently dynamic filenames is a feature. Which I can understand when the filename isn't properly detected (like in some gallery links). But it would be nice to turn that feature off for archives, since there's no way to manually force mirroring on dissimilarly named files other than to rename them.
Reply With Quote
  #5  
Old 23.04.2018, 15:55
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

@user748912 if filename/size would be identical and safe, then those files would already be treated as mirrors.

You can change detection in Settings-Advanced Settings-searc for mirror
for example set GeneralSettings.mirrordetectiondecision to FILENAME if you only want to incorporate filename into mirror decision or enable GeneralSettings.forcemirrordetectionfilesizecheck to enforce filesize validation for modes that make use of filesize as well.

Filename/sizes must be *safe* for use in mirror decision, as for example the host may truncate the filename on website and only provide full name on actual download. For example
The Example Movie...rar
The Example Movie...rar
on download it becomes
The Example Movie.part01.rar
The Example Movie.part02.rar

Also there are services that may name two different images the same
image.png and image.png and only have different filesize which also might only be available on actual download.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 23.04.2018, 15:57
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Quote:
Originally Posted by dabrown View Post
Rapidgator is one that changes almost every time- it detects with spaces turned into underscores but those change to spaces when the file actually downloads. Keep2share and fileboom, they detect the full filename on link add, but on link start it truncates if the file name is fairly long.
It highly depends if you use website/api mode on those plugins. As explained above website and api responses might be different and JDownloader trusts the one that is given on actual download the most.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 05.05.2018, 09:39
user748912 user748912 is offline
JD Adviser
 
Join Date: Jun 2014
Posts: 103
Default

I tryed Filename_Filesize and Filename only. Both dont solve the Problem.
Reply With Quote
  #8  
Old 05.05.2018, 11:18
dabrown dabrown is offline
Black Hole
 
Join Date: Jun 2015
Location: North America
Posts: 264
Default

The only way I've found to make mirror detection nearly foolproof is 1) set Mirror Detection Decision to FILENAME and 2) rename all the links in the package manually.

Otherwise, mirror detection breaks if JD changes the filename (even temporarily) due to it's lack of a default filename lock setting. When you change the link's filename manually, it always uses that. No glitchy temporary incorrect filenames that break the mirror detection.

Jiaz: I get that it's trying to find a trusted name, but when I dump a bunch of files in one package with the sole intent is that they are mirrors, I'd like them to stay mirrors. Not suddenly decide that one file isn't a mirror anymore if it got skipped or postponed due to a captcha timeout or a post-captcha wait time detection. If it was a package of mirrors when I moved the package from linkgrabber to the download list, I expect it to remain a mirror.

Here's a long rundown of what happens: Say I have 5 links. One from Rapidgator, one from Depositfiles, one from Fileboom, one from Filejoker, and one from Filefactory. They all ended up in the same package on link detect, with the name this_is_a_really_long_filename.zip. This name was what linkgrabber detected for all 5 files.

I have JD running unattended, so it skips the RG, DF, FB, and FJ ones in order due to unsolved captchas. It ends up downloading the FF one since no captcha was needed.

But instead of having a 100% complete package with one DL'd file marked "Finished" and the rest marked "Finished(Mirror)", I have one "Finished" file (FileFactory), 3 "Finished (Mirror)" links, and one that's "Skipped (Captcha Required)" (Fileboom). 4 of the 5 files still show the name to be this_is_a_really_long_filename.zip. Fileboom now is called this_is_a_really_...me.zip. So, when I hit Unskip, it tries to download that file AGAIN, and only when the download starts, it now complains that the file exists (and the name is changed BACK to this_is_a_really_long_filename.zip. Now I get a time penalty for the aborted download, since it initiated the download even though I had to tell it to skip the file due to being a duplicate. Of course, some times I catch that problem before (I see the skipped files in what should be a completed package) but with a download list of over 150,000 links and thousands of packages, sometimes I'll miss a couple.

As I said, I can avoid this by manually renaming every single "mirrored" link manually. But that's a real pain in the caboose (not to mention a waste of time) when the detected filename happens to be what I want the completed file to be called. I have to rename every link temporarily to something else, then back to the actual link, or I get the mirror mismanagement problem above.

There are literally only 2 use cases where I would ever want JD to rename the file from what was detected: when it didn't detect the filename at all (the name shows up as a partial URL of the link) or when downloading images from a gallery. Neither of which involve mirror handling. I realize that other users may prefer getting finalized archive filenames, but when I drop an archive into the download list, I expect that filename is what it is going to be on the hard drive when it's done.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 21:28.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.