JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 06.04.2023, 03:06
SMS SMS is offline
Mega Loader
 
Join Date: Jun 2019
Posts: 66
Default Decryption too tied to downloading?

What's the difference between doReconnect() and requestReconnect()?

It seems that requestReconnect() waits for decryption to finish, even though decryption seems to have nothing to do with the internet connection.

Another example where decryption is too tied to the downloading process is that decryption of a downloaded file doesn't seem to happen after JDownloader crashes. Instead, it has to re-download the file in order to decrypt it.
Reply With Quote
  #2  
Old 06.04.2023, 12:15
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

@SMS: requestReconnect informs the controller that a reconnect is requested and reconnect method should be executed when there are no active download plugins in process.
Also see Settings->Advanced Settings->Reconnect.downloadcontrollerprefersreconnectenabled
and Settings->Advanced Settings->Reconnect.reconnectallowedtointerruptresumabledownloads
requestReconnect does NOT WAIT for execution/result of reconnect.
doReconnect does WAIT for execution/result of reconnect. please be aware that by using doReconnect might result in deadlock when your script is actively waiting for result of reconnect while for example blocking a download plugin process.

I guess with decryption you mean mega file decryption after download? or do you mean something different?
Reconnect is executed once no there are no active download plugins. It doesn't matter if the plugin is currently waiting for user input or doing some non network stuff (like decryption), as long as the plugin is running, it blocks the reconnect. This will not be changed as it would require major changes to controller and the plugins to provide support for plugins to tell controller that reconnect is okay/not okay and/or actively wait for reconnect to be finished.

Last edited by notice; 06.04.2023 at 12:19.
Reply With Quote
  #3  
Old 06.04.2023, 12:21
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

Quote:
Originally Posted by SMS View Post
Another example where decryption is too tied to the downloading process is that decryption of a downloaded file doesn't seem to happen after JDownloader crashes. Instead, it has to re-download the file in order to decrypt it.
In case you're talking about mega, this should not happen. JDownloader should detect the downloaded encrypted file to be complete and restart the decryption process without having to redowload the file. Of course this requires the last written state on disk is when the download is marked as finished.
I already explained in your other thread that you should find out the cause of crash and fix/solve this. JDownloader does not simply crash, that's always a sign for more deeper Java/OS/xy issues. See https://board.jdownloader.org/showpo...04&postcount=6
Reply With Quote
  #4  
Old 06.04.2023, 13:53
SMS SMS is offline
Mega Loader
 
Join Date: Jun 2019
Posts: 66
Default

Yes, I mean mega file decryption after download.

Quote:
Originally Posted by notice View Post
Of course this requires the last written state on disk is when the download is marked as finished.
Maybe that is not being written to disk when the UI freezes. Solving this is not so important.

Reconnecting without waiting for decryption is important though, because decryption is slow:

Settings->Advanced Settings->Reconnect.reconnectallowedtointerruptresumabledownloads is enabled. Then why does reconnect wait for decryption? Does it consider decryption to be non-resumable? Maybe this can be changed?

Last edited by SMS; 10.04.2023 at 10:35.
Reply With Quote
  #5  
Old 06.04.2023, 14:02
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

Quote:
Originally Posted by SMS View Post
because decryption is slow
Can you provide more details? Decryption speed is primary limited by disk IO.
Maybe some numbers? takes x secs for filesize y?
or maybe provide example links for testing?

Last edited by notice; 06.04.2023 at 14:06.
Reply With Quote
  #6  
Old 06.04.2023, 14:05
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

Quote:
Originally Posted by SMS View Post
Does it consider decryption to be non-resumable? Maybe this can be changed?
What host is asking for reconnect? If the reconnect is requested by same host, in this case mega, then this is blocked to avoid reconnect loops. Imagine you download from mega, 2nd download is asking for reconnect, resume is supported. now the download is aborted, reconnect takes place, download starts, the 2nd download again requests reconnect, and then we have reconnect loop without any progress in download.
Reply With Quote
  #7  
Old 06.04.2023, 20:23
SMS SMS is offline
Mega Loader
 
Join Date: Jun 2019
Posts: 66
Default

Quote:
Originally Posted by notice View Post
Can you provide more details? Decryption speed is primary limited by disk IO.
Maybe some numbers? takes x secs for filesize y?
Decryption on a HDD takes several minutes for a gigabyte. That's slower than some downloads. Decryption blocks reconnects, often causing very low download speed during decryption. So overall, downloads take two or more times longer to finish than without this bug.

Quote:
Originally Posted by notice View Post
What host is asking for reconnect?
My EventScripter script asks for reconnects. Without these reconnects, the average download speed would be even lower, because some servers stick to the initial download speed (even though they could increase it after my other downloads finish and free up my bandwidth).

Why does resumable decryption (with reconnectallowedtointerruptresumabledownloads enabled) prevent reconnects?
Reply With Quote
  #8  
Old 14.04.2023, 17:50
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

Quote:
Originally Posted by SMS View Post
Decryption on a HDD takes several minutes for a gigabyte. That's slower than some downloads.
Can you please provide some numbers for your setup? Decryption is only limited by CPU and DISK IO. For example if a file is 100 mbyte in size and your disk can read/write at 100mbyte/s, then this process should only take a few seconds. 1 gigabyte file should only take about 10 (eg 100mbyte/s)-max 30(30mbyte/s) seconds.


Quote:
Originally Posted by SMS View Post
So overall, downloads take two or more times longer to finish than without this bug.
I'm sorry but this is no bug. It's a design decision that a plugin may do several tasks, for example decryption or hash check and others and only at the end of this process the download is finished with a result (eg error, ok, hash failed, ip waiting, incomplete..)

Quote:
Originally Posted by SMS View Post
My EventScripter script asks for reconnects. Without these reconnects, the average download speed would be even lower, because some servers stick to the initial download speed (even though they could increase it after my other downloads finish and free up my bandwidth).
Why does resumable decryption (with reconnectallowedtointerruptresumabledownloads enabled) prevent reconnects?
I will check if the decryption process can be interrupted , so reconnect can be done earlier.

Would it be possible to provide 1,2 larger example links from mega for testing? you can mail them to support@jdownloader.org

Last edited by notice; 14.04.2023 at 17:56.
Reply With Quote
  #9  
Old 16.04.2023, 15:31
SMS SMS is offline
Mega Loader
 
Join Date: Jun 2019
Posts: 66
Default

Dercyption is much slower for me than 100MB/s. CPU usage is 100% in total, with JDownloader using about 70%. So decryption is probably slow due to the CPU.

If other programs used the CPU or disk intensely, then decryption would be even slower - arbitrarily slow.

The problem is not that decryption takes long by itself, or that it is defined as part of the download process. The problem is that decryption prevents reconnects. Thus, a lot of time is spent decrypting, with very low or zero download speed, waiting for the reconnect. Resumable downloads don't prevent reconnects, and neither should the decryption of a (resumable) download.

Here are some random example links from a Google search:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by SMS; 16.04.2023 at 15:36.
Reply With Quote
  #10  
Old 17.04.2023, 13:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,935
Default

Quote:
Originally Posted by SMS View Post
The problem is that decryption prevents reconnects. Thus, a lot of time is spent decrypting, with very low or zero download speed, waiting for the reconnect. Resumable downloads don't prevent reconnects, and neither should the decryption of a (resumable) download.
We've understood that.

Please wait for us to look into this...
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 20:36.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.