JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 24.06.2021, 22:03
fuknuckle fuknuckle is offline
Black Hole
 
Join Date: Aug 2015
Posts: 253
Default JDownloader runs like ass

so JD has been running like absolute ass for several weeks now. it is so slow and laggy. everything I do it has these delays and momentary freezes. scrolling, switching tabs, right clicking, every action has a delay.

I'm getting very annoyed because it also seems to be affecting download speeds

I have been using JD2 since late 2014 and my linklist has nearly 1.3 million links in it. my linklist (downloadList998393.zip) file is nearly at its millionth iteration and is almost 800MB in size so I am sure this plays a role but it has almost suddenly gotten really bad the last month. I have found that killing JD with the taskmanager while its running can sometimes make it much more stable the next time I start it but it doesn't always work.

JD2 is typically having high CPU usage and uses 4.5GB of RAM

Is there something wrong with my instance of JD2, if not any plans to make it more stable? I think JD2 needs an overhaul of how it saves the linklist. Most of my linklist never changes and is there for archival and history purposes. JD doesn't need to read and load most of this data or constantly resave it. Maybe JD needs an archive tab where old links can be saved to its own separate file and only loaded when needed. it would be nice to have different labeled archives

24.06.21 11.28.56 <--> 24.06.21 11.43.27 jdlog://5693825302851/
Reply With Quote
  #2  
Old 25.06.2021, 19:48
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

@fuknuckle: JDownloader internas are not designed for JDownloader being used as an archive/history.
That's the simple answer to this.
There is no dedicated effort to make it lightweight use/handling for archived/history links.
For JDownloader it's just a normal link that you might enable/download at any time.
And it highly depends on the link/site if a link requires more or less memory for internal representation.

You can set higher delays to write modified list to disk, see Settings-Advanced Settings
https://board.jdownloader.org/showpo...09&postcount=4
eg, set
DownloadController.minimumsavedelay to 300000
DownloadController.maximumsavedelay to 900000
So JDownloader will wait minimum of 5 minutes after last modifiation to write to disk, but max 15 mins.
It will write the changes to disk independent from those settings.

Your JDownloader instance is limited to 3.6gb of heap memory
MaxMemory=3806855168bytes (3630Megabytes)
You can always check memory usage/allocation in about dialog of JDownloader, see
https://support.jdownloader.org/Know...tion-directory
When the first(used) and second(allocated) number are *near* the third(max), then it's time to remove links from list or incease available max heap.
You should modify the .vmoptions file and provide more memory, eg, remove any -Xmx and -Xms line and add
-Xmx6g
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 25.06.2021 at 19:52.
Reply With Quote
  #3  
Old 25.06.2021, 19:57
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

Quote:
Originally Posted by fuknuckle View Post
I think JD2 needs an overhaul of how it saves the linklist. Most of my linklist never changes and is there for archival and history purposes. JD doesn't need to read and load most of this data or constantly resave it. Maybe JD needs an archive tab where old links can be saved to its own separate file and only loaded when needed. it would be nice to have different labeled archives/
I absolutely agree and this is long demanded power feature, see


But it's not that simple because most ppl want this *archive/history* to work like the current system, so JDownloader informs/shows
that you've already a file or not. So this implementation needs to be *fast* and consume less memory than current *workaround* with keeping
links in list. Of course you could just dump those into a database, but this would just create another dependency that might not be available in all supported environments and ppl easily forget that databases are only *fast* if the index mostly fits into memory.
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 25.06.2021 at 20:05.
Reply With Quote
  #4  
Old 25.06.2021, 19:58
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

Quote:
Originally Posted by fuknuckle View Post
I think JD2 needs an overhaul of how it saves the linklist
I also have ideas to reduce write pressure and file size when writing lists to disk but that work is not yet done. Just some basic proof of concept.
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 25.06.2021 at 20:06.
Reply With Quote
  #5  
Old 25.06.2021, 20:07
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

It would help me to get hands on such a large/long list for working/testing on real life data. Maybe you can upload it to mega and send me the link to support@jdownloader.org ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 03.09.2021, 05:58
fuknuckle fuknuckle is offline
Black Hole
 
Join Date: Aug 2015
Posts: 253
Default

hey so its been a while since I replied. I solved my issue by doing what you suggested but I was confused for a long time because that vmoptions file didn't exist but after some more searching I got it figured out

It would be really nice if in the future you can increase JD's heap memory in advanced settings

you also mentioned that archiving links is a feature that others have requested and that you are working on a proof-of-concept for reducing linklist (and hopefully memory) size. I would like to help anyway I can. I am not a developer but I would be willing to test these features and considering I am a heavy/power user, that might be beneficial to you

as for you last request. I have not forgotten but I do have reservations.

my linklist contains years worth of download history from many files hosters. Is there any data in the linklist that can identify me by the accounts used to download or any other information that could hurt my privacy?
Reply With Quote
  #7  
Old 03.09.2021, 19:34
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

@fuknuckle: didn't find time yet, will read/answer by next week
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 16:25.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.