[shinji2009]

hi. when hashcat is working, new download is not startning. i sometimes observe a situation when downloading stopped completly because packages from all hosts are waiting for hashcat. maybe it is possible to make jd start new downloads while hashcating/waiting for hashcat? i think modern ssd will handle this.

[Jiaz]

@shinji2009: I guess you mean hashing? Hashing is only limited by IO and CPU and SSD means that hashing should be very fast. Also meaning that when this takes very long for you, it's a sign that system cannot handle IO and/or CPU and just putting more pressure with concurrent hash checks and/or more concurrent downloads/IO stuff going on would be no good. Next downloads cannot start because the result of the hash check also has influence on the result of a download. hash fails -> retry or try other mirror. download okay, mark file and mirrors as done.Changes to this also means major internal changes as the link/file still can be active/required despite the slot being marked as *finished*.

[shinji2009]

isn't this crc check? i set 2 downloads per host, when one file is crc checked, other downloaded file have status "waiting for hashcat", and no files are downloading from this host, because as i thinking waiting and checking is handled as not finished download.

well i don't use any mirrors and not set to auto redownload if crc fails so i was thinking there will be no problem. and my download speed is limited, so download time sometimes can be crucial. it'll be nice to give maximum i/o bandwith and time to downloading, and left what left for crc.

also i don't quite understand what these two tables mean.

[Jiaz]

@shinji2009: does it really say "hashcat" and not "hashing"? if so I have to check/fix translation. Yes, it can be crc32, sha256 or any other hash check and the "waiting for" means that some other download is still doing this check as only one is concurrently running. and when this takes long, that's already a sign that the system cannot handle CPU/IO load as this normally only takes a few secs, even for large files when IO is limiting factor.

The two tickets are ideas to allow concurrent hash checks, so links do not have to wait for others and the other one is for allowing new downloads when link is still waiting for/doing hash checking.

how large are the files in question? are you downloading on local ssd/hdd/usb or network drive?

[shinji2009]

maybe i am wrong about "hashcat".

files are different, but this is most noticeable on large files, 1-30gb. i am using ssd via usb3.0 adapter now.

maybe it is possible to give all crc operations low priority? downloading is writing and i can download only at 10mb/s maximum, but crc is reading, and ssd read speed is basically usb3.0 limit. so if crc will have low i/o priority, it will still have most of the bandwith. and maybe it is limited not by interface, but by cpu. in this case limiting crc cpu usage will help.

[Jiaz]

@shinji2009: as explained, currently it's not possible, best I can do is to allow concurrent hashchecks to have several links do this at the same time

what read speed do you get on that exernal drive? at slow 100mbyte/s, a 30gb file would take 300 secs/5 mins. when 300mbyte/s, only takes 1.5 minute.

[pspzockerscene]

Originally Posted by Jiaz

@shinji2009: as explained, currently it's not possible, best I can do is to allow concurrent hashchecks to have several links do this at the same time

That was added about a week ago, see this advanced setting:

GeneralSettings.maxconcurrenthashchecks

[shinji2009]

ok thank you
it is 150mb/s