[Solved] Mark files as mirrors

[plaintext]

Hey people,

Sometimes I do get the same files with different hosters and different filenames. I tried to take these files and put them into a package, but then still the same file gets downloaded several times, if it has a different filename.

So I'd like to know:
- Is there a way to mark a file as mirror of another file?
- Is there a possibility to make JD2 download only one file of a package?
- any other solution for this situation?

If not, that might be a useful feature. Sorry if that has been brought up before; I searched and didn't find anything for that issue. Thanks in advance for any insight.

[Jiaz]

You have to rename the files to have same filename. Without having the same name, they won't be recognized as mirrors at all.
Different filenames cannot work as it will fail in resume/extraction scenarios

[dabrown]

Yeah, I brought it up before and got the same answer. I can understand the problem if it's a multi-part archive, but in the case of a single file archive it's an annoyance. I'll often have files with 7 or 8 "mirrors"- they aren't real mirrors, it's actually the same content from different uploaders with different filenames and sometimes different extensions. And even if I do rename the files to an identical name with identical extensions and the size is nearly the same, sometimes it downloads the same file 2 or 3 times if I don't catch it. Worse, if the file name is the same at the time the link was added, sometimes JD renames the file 2, 3 times (one on add, once on an attempted download start, and once on file complete) meaning that even though it said all files in the package were the same (showing only the size of one file for the package size) it ends up trying to download all of them.

[Jiaz]

The Problem is..how to treat non real mirrors?
If they are no mirrors -> What is the correct filename? How to detect them as mirror when they don't have same filename/size/hash? Resume doesn't work if they don't have same name/size.

Can you please provide some example links, then I can check and provide further feedback.

[dabrown]

Here's an example:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

These 5 files are the same content, just from different uploaders (2 different ones, I believe). I MANUALLY renamed all files to a name none of them had before.

All are in the same package, and they are the ONLY links in that package.

It started the Rapidgator download (because I had no RG downloads in my queue at the time, I rarely use RG) and it marked, on startup, all but the fboom one as "Mirror Rapidgator.net is loading". So, I had to MANUALLY disable the fboom one to prevent it from trying to start- when the RG one was downloading. I have to do that all the time, when I see one file, even with the same name, for some reason doesn't mark is as mirror loading.

Normally, if I rename files before downloading it works fine, but occasionally it does the above- mirror detection simply misses a file. But it's totally random if I don't manually rename files- it might DETECT the filenames were the same when I added them via linkgrabber (say, I have 3 files, all mirrors from the same uploader, and JD shows the package filesize = the largest detected filesize. That tells me JD thinks they're a mirror on add). But then the host sends a different, usually truncated, filename when it tries to start the link and fails due to captcha timeout. JD then renames the link to the truncated link (this happens with long filenames on Keep2share and Fileboom extremely often). Then, if one DOES start because I answered the captcha, it renames it AGAIN on file download, either back to the long filename, or it strips out the underscores it detected on file add and replaces them with spaces. So now, I could have up to 3 different files, all the same, when it detected them as mirrors originally. Pain in the rear. I try to remember to manually rename them- add an extra space, or something- just to make sure JD doesn't auto rename them, but even that isn't 100% effective in getting it to always detect as a mirror.

I'm not looking for JD to detect it as a mirror automatically- I'm trying to FORCE JD into accepting that it's a mirror. I have my system set for mirror detection on filename only. I don't care about the size. Normally I download whatever is the smallest, though not always. In those cases I manually enable/disable as appropriate. Right now I always disable Rapidgator and Uploaded mirrors if they're over 200MB because those hosts are so slow. But I keep them in the list rather than delete them in case the other links die, or to be sure I don't accidentally add them again later.

I realize it's a tough logic path. My system is set up as such:

All duplicates (true mirrors or identical content) go in their own package. Nothing else. I currently manually rename the files in those packages to try to force mirror detection, either by using the most descriptive filename that was detected, OR by making my own filenames based on the uploader's description.

All multipart archives go in their own package. I never, EVER add mirror links to multipart archives unless I know they are true mirrors, because they'd be incompatible. I do, however, often rename them manually if the filename is too generic/undescriptive. That has never yet resulted in a broken archive, since I do name them the same with part1, part2, .001, .001, etc if they came that way.

All SINGLE links (links with no mirrors and no multiple parts) go into "Various Files" packages, which eventually get combined into massive ones after the packages finish to try to keep the # of packages manageable.


Because mirror detection is so flaky when the hosts don't give accurate filenames on link add, I'd like to be able to mark, on a package by package basis, packages as "mirror package" which means that it would download one, and only one, file in that package and automatically mark every other file in that package as a mirror when it starts/finishes. Regardless of size, filename, extension, etc, and it doesn't matter which file started first- whichever had a slot to go, it's the only one that downloads. It would have to be a flag at the package level and not set automatically, or it would break other things. But just for that package. Unmarked packages would operate as they do now.

As far as filename goes, what annoys me is when JD changes the filename without my consent. I realize that the host may have sent a different name on link add than what the REAL, post-downloaded name is, but I wish there was a setting that JD never change the name once it's added. An exception is if NO filename was ever detected (I've seen that happen, it just shows the past part of the URL). Folks that don't mind JD changing the filename post-download can always leave it as-is. Me, once JD has detected a filename, I don't want it to change unless I manually rename it. If it did that and the mirror rules always worked (and ignored non-multipart extensions) I wouldn't need the special mirror package idea I had above. Right now, because some uploaders make their mirrors RAR and some ZIP, I just rename all of them to .rar since winrar doesn't care if the extension is wrong. Not a great solution on my part, but it sometimes work sin forcing mirror detection.

Resume: I don't see how that would work anyway. As far as I know you can only resume from the same host. If I had a file, even a true mirror, "die" midway through a Rapidgator DL, I can't restart it using the Depositfile mirror. Unless JD has some sort of hidden torrent abilities.

Sorry for rambling, that's just my idea of how I'd like it to work. And it would have to be something you could enable/disable in advanced settings, because it's not how most folks would want it to work unless they download a lot of archives as I do. It would be awful trying to make this work when pulling down individual files, like galleries or such.

[plaintext]

I do understand the problem of marking mirrors/duplicates automatically in these situations.
But like I wrote: In these circumstances it would be a solution, if I had a feature like "download only one file of this package". I could effectively prevent multiple downloads of the same content even in the most stupid of upload circumstances. Would it be recommended to file this as a feature request then?