#1
|
|||
|
|||
False-positive mirror downloads and Url Display Order
Hello
First of all, let me say that I've been using JDownloader 2 for several years now and I can no longer imagine not having it installed at any OS I'm running. Today however, I'd like to report - I believe - a bug I stumbled upon just recently. All links below lead to explicitly mature content and are given only as example. --- Gist of the problem: 1. Downloading whole gallery off from xHamster incorrectly flags some files as mirrors due to URL display order. 2. Advanced setting "GeneralSettings: Url Order" doesn't work. --- I'm running Windows 7 Pro x64 Build date of JD2: Tue Apr 25 02:19:30 CEST 2017 Source Revisions: Core: #36667 Launcher: #3836 AppWork Utilities: #2884 Browser: #36630 Updater: #565 The matter is about downloading whole galleries of images from xhamster.com. Sometimes JDownloader 2 wrongly marks some of the images as "mirrors" of other files and refuses to download them. Such files are marked with "mirror directhttp is loading"(sic) while downloading is pending and then "Finished(Mirror)" once the download is complete. This however leads to missing some of the files after the downloading is done since the presumably "mirror" files are actually unique and different from their respective "copies" This happens because JDownloader 2 validates files by their filename and "Download from" string. In case of xHamster multiple files in single gallery can have identical filename. Now because JD2 uses URL of the whole gallery rather than specific image file for validation it leads to false-positive collisions. An example, this gallery has two pages of images: **External links are only visible to Support Staff****External links are only visible to Support Staff** On page one, you can find the following entry: **External links are only visible to Support Staff****External links are only visible to Support Staff** Image displayed under this entry is: **External links are only visible to Support Staff****External links are only visible to Support Staff** On page two, you can find the following entry **External links are only visible to Support Staff****External links are only visible to Support Staff** Image displayed under this entry is: **External links are only visible to Support Staff****External links are only visible to Support Staff** As you can see, both files have same filename, but different direct URL. One of them will fail to download because they will be recognized as both being downloaded from "**External links are only visible to Support Staff**. --- I've tried to add another custom Packagizer rule to rename files depending on their download url. This however didn't seem to work either due to my lack of skill or the fact that JD2 doesn't differentiate between those false "mirror" files. Next I attempted to see if this flagging can be overridden in some way. While messing with settings, I noticed that clicking any file in Downloader/Linkgrabber lets me select: Properties -> Show Download URLs -> Double-click on item in "Download from" column. This displayed both "Container" link (the "bad" link to whole gallery) and "Data" link (the direct link to this particular image). Opening this also displayed friendly message that the JD2 can let me control which URL is displayed and used for copying. Hoping that this will solve my problem ("Data" type link probably would let me make valid Packagizer rule) I navigated to given option. Settings -> Advanced -> GeneralSettings: Url Display Order Only to find out it doesn't exist, but there is a very similar one under name "GeneralSettings: Url Order". The field by default was set to "null". The field description informs that valid values are: CONTAINER, CONTENT, CUSTOM, ORIGIN, etc. None of those values are accepted in the given field and rolls back to null value. Error message reads (after entering "CONTENT" as value): Code:
'CONTENT' is not a valid 'UrlDisplayEntry[]' I tried putting in the very same value embraced with single and then double quotes then braces and finally formatted as json. None of this was accepted by the field. At this point I've hit a stopping point and I have no idea what else could I do to resolve this issue. Thanks in advance for any support you could provide. Please let me know if I can/should provide any additional info. |
#2
|
||||
|
||||
URL Order is only used for GUI/UserInterface. It has no influence on Mirror Detection.
Check Settings-Advanced Settings-search for mirror Either enable GeneralSettings.forcemirrordetectionfilesizecheck or set GeneralSettings.mirrordetectiondecision to SAFE
__________________
JD-Dev & Server-Admin |
#3
|
||||
|
||||
Quote:
It uses Filename/FileSize/FileHash where available. See my post above.
__________________
JD-Dev & Server-Admin |
#4
|
||||
|
||||
Quote:
Syntax is JSON and the value expected is an Array of UrlDisplayEntry Objects Code:
[ { "type" : "ORIGIN", "enabled" : true }, { "type" : "REFERRER", "enabled" : false }, { "type" : "CONTENT", "enabled" : false }, { "type" : "CONTAINER", "enabled" : false }, { "type" : "CUSTOM", "enabled" : false } ]
__________________
JD-Dev & Server-Admin |
#5
|
|||
|
|||
Quote:
The advanced settings you listed work PERFECTLY for my needs and the core problem I had was basically resolved with switching either one of them. Thanks a lot again. Quote:
Ah, the example you have given works perfectly. I admit I never tried to input whole array as an option. Description of that advanced setting lead me to believe it was "single" value only field. But with this, this advanced option works just fine as well. Thank you once more. |
#6
|
||||
|
||||
Thanks for the feedback Glad I could be of help to you
__________________
JD-Dev & Server-Admin |
#7
|
|||
|
|||
Quote:
Like this: Code:
[{"type":"CONTENT","enabled":true},{"type":"REFERRER","enabled":false},{"type":"ORIGIN","enabled":false},{"type":"CONTAINER","enabled":false},{"type":"CUSTOM","enabled":false}] |
#8
|
||||
|
||||
@botmtl: Thanks for the info. Will be fixed with next update!
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|