JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 28.11.2023, 02:18
lamole234 lamole234 is offline
Fibre Channel User
 
Join Date: Sep 2010
Location: En el infierno
Posts: 119
Default checksum mismatch retries are infinite and problems with CRC + archive.org

Currently JD will retry the file infinitely when the hash check fails on one file, in this particular case (it has happened before with other hosts and wrong checksum given by the uploaders) archive.org has a SHA1 that doesn't match the file's hash because of this JD keeps retrying forever the same file over and over writing to disk non-stop, would be great to have an option to customize the times it will retry the download, there is no point in failing 10 times and re-downloading again because either the host's servers are failing or the available checksum is incorrect and retrying more than 10 times (which is already a big number) wont fix that, and if it's a bad drive then writing 10 times the same file and getting a corrupted sector is an indication the drive can die any second so the less writes the better in such cases.


EDIT: The problem I was having with the archive.org file was that when I added the package some days ago the files had a particular checksum (which I guess JD only downloads the meta file once when the package is added), resuming the download some days later (today) downloaded a file that was updated in between the days I had paused the download making the meta file JD had outdated making it redownload one of the files over and over. I guess what should be added is if the host is archive.org (which can have its content modified) and a file fails then re-download the meta file and recheck before re-downloading the "failed" file again.

Last edited by lamole234; 28.11.2023 at 02:25.
Reply With Quote
  #2  
Old 28.11.2023, 12:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,607
Default

Please provide example URLs and a log.

JD should not retry such files infinitely - that would be a bug.

Please post your log-ID here
If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.

Bitte poste deine Log-ID hier.
Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 06.12.2023, 01:52
lamole234 lamole234 is offline
Fibre Channel User
 
Join Date: Sep 2010
Location: En el infierno
Posts: 119
Default

Sorry for the late reply.
It's been days now and I don't have the necessary info, but it should be easy to reproduce using this package **External links are only visible to Support Staff****External links are only visible to Support Staff** some files are updated kinda constantly.
Make JD scan the url, add it to the queue (I don't know if it actually needs to sit in the queue but that's what I did) then check the archive website and wait for a file to be updated, if there is one then start the download (don't scan the url again, just start the download of the updated file) and you will see JD retrying again and again.
When I posted this the file that had been retrying over and over was this file: **External links are only visible to Support Staff****External links are only visible to Support Staff** from my ff history.
Reply With Quote
  #4  
Old 06.12.2023, 12:49
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,607
Default

Was able to reproduce the problem by simply adding a file and setting a random CRC hash on it.
Ticket:

Solutions/Workarounds for both of your problems for now:
Go to Settings -> General -> Section "File Writing"
SFV / CRC check -> Disable
Restart Download when SFV/CRC check fails -> Disable

Regarding the specific archive.org problem that files change and thus the expected CRC hash is invalid:
I will look into this.

In the worst case, I will disable CRC check for all archive.org items.
The problem here is that I currently do not know about any non-time-consuming way to obtain the current hash of an archive.org file.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 06.12.2023 at 15:30. Reason: Fixed typo
Reply With Quote
  #5  
Old 06.12.2023, 16:05
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,607
Default

I've just fixed the underlying issue:
The archive.org API does indeed return a "Last-Modified" file timestamp as value "mtime".
The new plugin is now comparing this against the "Last-Modified" timestamp which gets returned in the response-header.
If those values do not match or the API did not provide a timestamp, the previously set hash information will be deleted and the hash check will be skipped.
For reference - their own Python software project is kinda also doing it like that, see:
archive.org/developers/internetarchive/quickstart.html
--> Search for "mtime"

This will automatically happen for all items added with older plugin revisions.
Essentially this means that once you got the update, you can enable the CRC check again and you should not run into the issue "Error CRC check failed but file is okay" again for archive.org items.

However the 2nd problem with the infinite retry remains so I recommend keeping the retry option disabled for now.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 13:02.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.