JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 19.03.2012, 15:30
Davidh2k
Guest
 
Posts: n/a
Default Mirror handling, duplicated downloads, duplicated files, dupes, online status

I have another question about merging mirrors in the alpha.

I merged an episode from SJ.de with one mirror from netload and one from rapidshare. I startet the download and he began to downlod all files from rapidshare, in this example all files were available on rapidshare so he asked right at the first part from netload what to do - if he should skip, or redownload the file. I choose skip and remember for the package but then he don't began to unpack the package.
I checked that auto extract package is still marked.

One other Question is, it seems that the "Remove finished downloads..." option seems to be a bit buggy, i switched back to 'immediately' after i tried 'when package is ready' but the jdownloader did not remove a package after an extraction. So, when is an package ready, and now the 'immediately' option isn't working either.
Reply With Quote
  #2  
Old 25.05.2012, 19:04
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default Mirror handling, duplicated downloads, duplicated files, dupes, online status

Spinning off of my last post in this thread:

http://board.jdownloader.org/showpos...13&postcount=7

I'm still having an issue where sometimes mirrors are starting when another mirror is already downloading.

Once a mirror starts for a part I would think the other ones should be disabled right then, but they don't always. Sometimes they later get disabled and sometimes not. Also, if the chosen mirror for a part fails, then that part should be reset with the failed mirror disabled and the other mirrors re-enabled automatically.

Today I had 3 mirrors show that they downloaded the same file 100% and unfortunately a fourth mirror started but had a plugin error and must have deleted the part because I was left with no file at all.

I don't know if the problem exists with just one package of downloads or if it requires other items to be downloaded first from the same hosters since I'm usually downloading multiple packages. Never had this issue with the release version.

Here's one package worth of parts and mirrors:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Jiaz; 04.06.2012 at 10:51.
Reply With Quote
  #3  
Old 25.05.2012, 19:24
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

I wasn't able to get the original instance's log since and auto-update restart cleared the log, but here's the same issue where two part 4 mirrors started and the other part 4 mirrors were still enabled.

http://jdownloader.org/pastebin/84913
Reply With Quote
  #4  
Old 26.05.2012, 20:59
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,103
Default

Hi
For me this works just fine.
Put all links in one package->Same downloadfolder->JD always only starts one of the mirrors and deactivates the other 2.

I know that if a part of one hoster fails JD should try the mirror (other link on other hoster) but as far as i know that logic isn't implemented yet

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #5  
Old 27.05.2012, 03:23
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

It's happening often enough for me to be a problem.

Is there any setting that might affect this. I seem to notice it when there's multiple parts with multiple mirrors.
Reply With Quote
  #6  
Old 27.05.2012, 11:47
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,103
Default

No, it should always work correct without any setting.

Greez pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 27.05.2012, 16:04
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

I just tried the links again and it appears to be working better. I'm thinking the issue requires other packages to be previously downloading from the same hosters first and then when the parts become active in the latter package is where the problem starts to occur.

Did my previous log show anything because it should show part4 being downloaded more than once.
Reply With Quote
  #8  
Old 27.05.2012, 19:13
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,103
Default

As far as i could see the log didn't show any of your problems.
If that situation happens again, please post it.
Maybe a screenshot also helps.

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #9  
Old 27.05.2012, 22:21
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

Weird that the log didn't show anything since at the time I was watching at least 2 part 4s downloading. I'll send some more info the next time i catch something.
Reply With Quote
  #10  
Old 02.06.2012, 23:56
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

One reliably broken variation is when you reset a package with a few mirrors. They all start counting down and going through their process and some will both start downloading the same files, but sometimes 1 or 2 mirrors will deactivate so there must be a stage where they're not detecting an already in progress download has begun. Maybe during the captcha phase.
Reply With Quote
  #11  
Old 04.06.2012, 10:50
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

this is a known bug

__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 04.06.2012, 10:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 28.06.2012, 16:12
invisibleman78 invisibleman78 is offline
JD VIP
 
Join Date: May 2009
Posts: 326
Question Doppelte Downloads gleicher Datei im gleichen Paket?

Wenn man in einem Paket Links zur gleichen Teildatei auf verschiedenen Hostern hat, lädt JD2 diese parallel und dadurch mehrfach runter.
Bisher merkte JD spätestens beim Start des Downloads, dass der gleiche Teil schon von irgendeinem anderen Hoster am Downloaden ist. Aktuell scheint JD2 das nicht zu merken und lädt im dümmsten Fall die gleiche Datei x mal runter.

Wie kann ich das verhindern?

Last edited by Jiaz; 29.06.2012 at 11:59.
Reply With Quote
  #14  
Old 28.06.2012, 20:11
EliteLucker
Guest
 
Posts: n/a
Default

Passiert mir auch ab und an, manchmal werden sie aber als bereits geladen markiert
Reply With Quote
  #15  
Old 29.06.2012, 11:59
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

ist bekannt und wird noch fixed
__________________
JD-Dev & Server-Admin
Reply With Quote
  #16  
Old 30.06.2012, 09:08
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default serious error

:unsure:I have set. Skip the file if it exists. Jdownloader rather than ignore it it modifies (OVERWRITE). Only now I noticed it! For me, a lot of unnecessary files have been replaced!!!








Do you fix it?



It should be a SKIP! The program treats it as REPLACE(OVERWRITE)!

Last edited by Jiaz; 02.07.2012 at 13:12.
Reply With Quote
  #17  
Old 30.06.2012, 09:39
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

This has been reported many times now, if the file starts at the same time it doesn't skip. That's my observation though the appwork guys have not been able to reproduce this so cant fix something they can't debug.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #18  
Old 30.06.2012, 09:45
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

Not at the same time! Old file was downloaded in 2011 and was replaced with the date 2012(overwrite download!)

No offense but to me it is not acceptable.

Besides. If I recall it was once (long ago), I wrote about this problem but none too much into the not took over and has so far not been in this direction nothing done. Ech...


If someone updates the files in a folder that's clear is that the program only download the missing files and old files I want to have remained intact (without modification)
Other programs are able to skip the existing file, why Jdownloader not can do this?

NIGHTLY there was not a problem!

I hope that developers will try and fix it?

Last edited by djmakinera; 30.06.2012 at 10:31.
Reply With Quote
  #19  
Old 30.06.2012, 10:48
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

stay clam, wait for jiaz to have a look at it. I dont have the time to look into core workings of JD
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #20  
Old 30.06.2012, 18:50
editestowy's Avatar
editestowy editestowy is offline
Polish Supporter
 
Join Date: May 2009
Location: PL
Posts: 3,026
Default

Never got this problem.
Do you mean normal files or files extracted from the archive?
Reply With Quote
  #21  
Old 30.06.2012, 18:55
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

normal files(JPG) not archives

Do not try to convince me that this problem is not as if there were they would not have overwrite the files.

If you think there is no problem then why replaces me the files (modify?)




Last edited by djmakinera; 30.06.2012 at 19:04.
Reply With Quote
  #22  
Old 30.06.2012, 19:31
editestowy's Avatar
editestowy editestowy is offline
Polish Supporter
 
Join Date: May 2009
Location: PL
Posts: 3,026
Default

No one is convincing you anything.
Read my post: I NEVER got this problem and I'm using the same opiton for my downloads.

Make a copy of the files, download them again and check if they were replaced (with skip option set).
If the error is repeatable then it will be possible for developers to reproduce and fix it.

BTW: date/time of the folder are changed each time Windows makes deleting, modifying or ADDING new files in this folder.
Reply With Quote
  #23  
Old 30.06.2012, 20:15
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

to 1000% are modified because I see with the naked eye in a folder that is re-saved in the same place!

Well, unless no one is able to see this error in the program and anyone that does not stop it on to completion.

Last edited by djmakinera; 01.07.2012 at 15:22.
Reply With Quote
  #24  
Old 02.07.2012, 13:12
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

we know about this, I will fix this as soon as possible
same issue as this one
__________________
JD-Dev & Server-Admin
Reply With Quote
  #25  
Old 02.07.2012, 13:43
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

Multi Link does not occur at the same time (not the same links in linkgrabber). The problem occurs when updating a folder of files at a later time.
Reply With Quote
  #26  
Old 02.07.2012, 15:06
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

same issue....it's the same code causing the problem. I'm on it
__________________
JD-Dev & Server-Admin
Reply With Quote
  #27  
Old 19.07.2012, 10:39
antihate
Guest
 
Posts: n/a
Default Does't skips already downloaded files

Hey,
Jdownloader2 does't skips files which are already downloaded. it start downloading them again.

I imported link list of jdownloader into jdownloader 2. Its skips some files and for mostly files it starts to download.

I tried to change option in general settings. "if the file already exists" to skip link and ask for each file. but it does't work.

btw jdownloader 2 is really coool

Last edited by Jiaz; 19.07.2012 at 11:55.
Reply With Quote
  #28  
Old 19.07.2012, 11:55
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

its a known issue when linkstatus is not known (eg after jd restart). you have to rightclick on package, check online status as temp. workaround, then *link exists*check works again.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #29  
Old 19.07.2012, 12:18
antihate
Guest
 
Posts: n/a
Default

I have hundreds of links and I've to restart jd2 often (for updates).
selecting all packages (ctrl+A) and check online status for all packages, will it work? I guess it will take hours to check status again for all links? right?

I just did that for one package and it start skipping files in it then I've enabled all pakages, selected all, check online status and start the download and this workaround does't worked.
Reply With Quote
  #30  
Old 21.07.2012, 17:53
Aquifel
Guest
 
Posts: n/a
Default

I have the same issue, and the problem persists even on freshly added downloads. The workaround doesn't work for me after restarts either :/. I've uninstalled and reinstalled a few times, it works exactly one time right after jdown has been installed and updated then never again.
Reply With Quote
  #31  
Old 21.07.2012, 19:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

after jd restart you have to rightclick on the package and online check again
this bug will be fixed soon!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #32  
Old 22.07.2012, 23:57
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

editestowy - Jdownloader download again the same files that exist in the folder and it is precisely this problem that the existing files in the folder does not skip.
Reply With Quote
  #33  
Old 23.07.2012, 15:35
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

you have to linkcheck links first before *safely* load them with *existing* check enabled.
after a restart of jd you have to rightclick onlinecheck the package again!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #34  
Old 23.07.2012, 16:11
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

editestowy - Please translate it better.
To nie ma nic wspólnego z linkami. Linki są automatycznie usuwane jeśli w całości zostanie pobrany pakiet. JDownloader "nadpisuje" pliki, które istnieją już w folderze o tej samej nazwie.
Przykład: Rok temu zapisałem jakieś pliki. Po ileś tam miesiącach ponownie chcę zapisac pliki w tym samym folderze z uwzględnieniem, że JDownloader pominie istniejące już pliki a zapisze tylko te nowe z innymi nazwami ale niestety tak jak mówie JDownloader "nadpisuje" istniejące pliki zamiast ich poprostu pominąć. Czy naprawde tak trudno wprowadzić taką możliwość pominięcia, czy JDownloader nie może sprawdzić, ze taki plik istnieje i już go NIE POBIERAĆ!???
Po co program nadpisuje te same pliki(z tą samą nazwą), które już istnieją?




Jiaz - It has nothing to do with links(in my case List of links has been deleted!). JDownloader "overwrite" the files that already exist in the folder with the same name. Example: A year ago I download some files. When you write a number of months back there I want to save files in the same folder on the understanding that JDownloader skip existing files and saves only the new one with different names but unfortunately, as I say JDownloader "overwrite" existing files instead of simply omitted. Is that really so hard to make this possible omissions, or JDownloader can not verify that such a file exists? JDownloder Why downlod the same files that already exist?

Last edited by djmakinera; 23.07.2012 at 16:23.
Reply With Quote
  #35  
Old 23.07.2012, 16:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

jd only overwrites files when the link in jd is unchecked (online status unknown) because then the *file exists* check does not work!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #36  
Old 23.07.2012, 16:45
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

Jiaz - I understand. The program correctly only skip files if MEMBER Vkontakte has a duplicate image in your albums (and I do not set then check the file name)
Reply With Quote
  #37  
Old 23.07.2012, 16:48
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

jd is only able to check for duplicate on this, when the link has onlinestatus "online", after a restart, they are in state "unknown" and then the bug happens
__________________
JD-Dev & Server-Admin
Reply With Quote
  #38  
Old 23.07.2012, 16:52
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,387
Default

I do not check that a file has a name and its status online, because for many image files takes a very long time and it may happen that it will block the server.


Window7:
new file is saved as 123.jpg.part

and overwrite the old file 123.jpg

Last edited by djmakinera; 23.07.2012 at 16:57.
Reply With Quote
  #39  
Old 23.07.2012, 16:57
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,532
Default

I know this issue is known and will be fixed soon
__________________
JD-Dev & Server-Admin
Reply With Quote
  #40  
Old 31.07.2012, 00:10
asdfjkl
Guest
 
Posts: n/a
Default

yes please fix this. It's quite annoying and can't be to hard since it's only a check for a local file
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 06:38.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.