JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 12.01.2021, 10:42
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default [MEGA.NZ] Start Decryptying Files After Download Queue Is Completed

Hi, I started to download a bunch of big sized files and just noticed that once each one of them is finished, the consequent decryption process takes place right away, even if there are files still pending to download. So, I'm wondering if is there a way of making JD2 leaves the whole decryption process starts only after the download queue has been finished? Or even let it be started manually by the user. That way, I could let JD2 keeps downloading files and/or even decompressing at least another one at the same time, without hogging the whole disk's bandwidth.

Otherwise, making the "decryption" process doesn't count as another "download" task to the queuing system, will let at least the files' amount set in the "Max. simultaneous Downloads" option be respected, and the rest of files keep being downloaded, while just "one at a time" is being decrypted.

Recently I had an active queue, which had an active download and a decryption taking place at the same time, and I wanted to decompress a little file (~ 3 GB) using WinRAR but couldn't do it, since once I stated it the decryption speed at JD2 was slowed down severely and the decompressing itself hanged (I had to shut it down, and it was closed a few minutes later).

Thanks.

Last edited by ElCho; 12.01.2021 at 10:51.
Reply With Quote
  #2  
Old 12.01.2021, 11:19
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

The decryption process is required to have a final file, part of the download processed.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 12.01.2021, 11:23
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

Quote:
Originally Posted by ElCho View Post
That way, I could let JD2 keeps downloading files and/or even decompressing at least another one at the same time, without hogging the whole disk's bandwidth.

Otherwise, making the "decryption" process doesn't count as another "download" task to the queuing system, will let at least the files' amount set in the "Max. simultaneous Downloads" option be respected, and the rest of files keep being downloaded, while just "one at a time" is being decrypted.
The "decryption" process is part of the download and required so we know that the file is complete and can be used by post processing, eg extraction and other stuff like Mirror-Handling or *File already exists*. "one at a time" decryption is to reduce IO pressure on disk

Either your download speed is much much faster than your disk IO throughput or you have configuration issues on your setup.
It won't matter if the "decryption" process is causing IO issues or "hash check" after each download. The disk must be able to support IO for download AND hashcheck and others, eg extraction. You cannot download with 100Mbyte/s when your disk only holds up to 50Mbyte/s.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 12.01.2021, 11:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

Quote:
Originally Posted by ElCho View Post
Eryption taking place at the same time, and I wanted to decompress a little file (~ 3 GB) using WinRAR but couldn't do it, since once I stated it the decryption speed at JD2 was slowed down severely and the decompressing itself hanged (I had to shut it down, and it was closed a few minutes later).
This sounds like hardware/configuration issues. Decryption/hashcheck and extraction can and must work all fine in parallel.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 12.01.2021, 11:26
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

Please check your disk with crystalmark.info/en/software/crystaldiskmark/ and show a screenshot of the results. I know that for example there are laptop disks that fail badly with random IO and you will find plenty of reports via search engine of your choice
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 12.01.2021, 11:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

wait for next plugin update, then Settings->Plugins->mega will provide an option to allow multiple decryption at the same time
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 12.01.2021, 15:10
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default

I'm downloading at ~30 MB/s max., but the download speed wasn't that high when the decompressing hanged; I think it was below half that speed. I was using the next disk:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Connected directly to a "SATA 600" port. Test results:



I think more than having one decryption process at the same time, will come handy, mostly for those cases when there is one "big" file being downloaded and a rest of tiny ones blocking further downloads. For example, let's say, I have "16" simultaneous downloads to be running; one "20 GB" file is being decrypted, while "15" tiny (a few KBs each) remnant ones are pending to be decrypted.

Last edited by ElCho; 12.01.2021 at 15:13.
Reply With Quote
  #8  
Old 12.01.2021, 15:31
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

Google for "ST2000DM008 slow" and you will find plenty of user complains for this drive and screenshot shows that disk slows down to nearly 280kb/s random IO.
Wait for next update and enable the given option. Also try to increase Settings->Advanced Settings->search for buffer and increase to 4096
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 13.01.2021, 09:46
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default

It seems you are right regarding the Seagate disks being the culprit of this problem. I just read about them, and although
the manufacturer doesn't mention it in the disks' documentation neither labels, "Barracuda" models (ST2000DM008, ST4000DM004 and others) are Shingled Magnetic Recording drives, instead regular PMR ones. So, that should explain the slowdowns and speed's fluctuations. Next time I buy another drive I will make sure it isn't an "SMR" one.

Anyway, the "parallel" files' decryption for MEGA is a nice feature to have, so thanks.
Reply With Quote
  #10  
Old 13.01.2021, 13:09
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

Thanks for the feedback and sorry for the bad news. Yes SMR is bad if you are not aware of it and its disadvantages.
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:16.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.