#1
|
|||
|
|||
Memory Leak?
I have been pushing JD a tad hard, but by no means harder than heavy users. For example, right now I have 5500 links queued. I am running Max Conn 5 and Max Dls 5. I have all kinds of links in it.
My system is a tad old, but still OK: Windows 2000 SP4 Server RAM: 2 Gb Swap File: 1.5 Gb (fixed size) There are no other active apps running in the background and and JD is the only java app. What I noticed is that at start-up JD consumes RAM and Virtual Memory at the same rate and then it goes ballistic with the VM and it swallows it all (according to Windows 1.7 Gb which is strange). At that point, MemTurbo kicks in trying to release RAM and JD (presumably) runs out of VM and crashes. What is strange is that JD is not releasing VM. If it would do so, there would be no problems. Any ideas? Last edited by Jiaz; 05.09.2011 at 10:19. |
#2
|
||||
|
||||
What is your Java version? I think Java should handle this.
|
#3
|
||||
|
||||
You can try starting jdownloader with more head room
default setting 'java -Xmx512m -jar JDownloader.jar' try 'java -Xmx1024m -jar JDownloader.jar' or even up to 2048 depending on your system. (but since your system only has 2GB maybe not hehe) This allows JD to be able to work larger download queues. Each entry in the queue takes up memory. So remove any completed packages/downloads from the queue, as this also reduces memory requirements. Maybe save half of your queue (as DLC), remove it and add it back later once you finished the first half. This will also reduce your memory requirement.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#4
|
|||
|
|||
Hi, Java version:
Version 6 Update 22 build 1.6.0 22-b04 |
#5
|
|||
|
|||
Quote:
will try expanding ram usage see how it goes. Wrt removal: I do so religiously but the VM does not shrink no matter how many completed files I remove. Wrt DLCing, it's a pain in the neck cos it ruins my "downloading style", besides, It was working OK up to a few weeks ago with a larger queue. |
#6
|
||||
|
||||
Get faster internet and or buy premium accounts :>
zoom zoom
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#7
|
|||
|
|||
Huh?
What's got bigger bandwidth and/or premium accounts got to do with it? It is the number of active downloads (in parallel OR sequence) that's killin JD. So, let's assume I get a larger bandwidth.... what's the end result? I still need to dl 5500 links. So, it does nothing to solve my problem. Now, let's assume I buy 10 or 15 premium account (assuming I don't want to eat for a month or so), I still need to dl 5500 links. Which, again, does bupkus to solve my problem. |
#8
|
||||
|
||||
The faster your link is, or the ability to download (say with premium accounts) the quicker you download list clears. That was the only correlation I was trying to make with large queue.
You stated previously that you were using multiple chunks + concurrent downloads, but those settings to me would not necessarily cause any harm if your connection can handle (5 5 0) = 25 connections. try logging to file by running 'windows_createlog.bat' from within jD install folder. upload your logs to http://jdownloader.org/pastebin/ and then post the urls here.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#9
|
|||
|
|||
Quote:
Quote:
Quote:
Tx! for all the help. |
#10
|
||||
|
||||
Sorry this is were I'll disagree with you. Unless your adding links to JD as faster than they can download, at most times the queue should finish faster than you can add (That's what I find, though I hardly download these days, and I only have 3Mbit). The correlation is, if the link is faster than you can queue data to download, the download queue should never be 5000+ links in length to start with, or for a very short period. Since your computer very old maybe you should look at 'settings > basic > download and connection :: remove finished downloads: immediately' thus freeing up memory as the download queue progresses. JD has the same memory requirements of a queued list VS completed list, so the sooner you clear out completed downloads the better. Coming version (v1) will address memory issues within the table view better, along with a new database storage system.
Now I run a old system also 2048 MB (PC3200 DDR SDRAM) RAM also, that said I haven't experienced any troubles with memory (once when linkgrabbing 5000 youtube videos). My current download tab has 519 packages 2027 links all complete, using 113MB RAM. Maybe your issues are around connections, I've seen the logger fill up memory specially when associated with socket connection problems. When you upload your logfile this will give me a better indication. Now to the p2p, its highly depending on what bandwidth peer/seeders can supply you per connection. Generally as this is done from range of connection speeds, on average they do not necessarily provide the same download speed per connection than say ftp/http (given that you have premium accounts). JD shouldn't need anywhere near the amount of connections as p2p to produce same bandwidth transfer KiB/sec as ISPs tend to QoS P2P more than HTTP. But that said I'm sure it's possible to transfers faster than JD assuming you are connected to only fast seed/peers.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] Last edited by raztoki; 04.09.2011 at 08:43. |
#11
|
|||
|
|||
@globus999
Update 22 is not the best version of java. I would remove it and replace it by JRE (1.)6 update 21. See "**External links are only visible to Support Staff**. Also use JavaRa before installing the new version to be sure that all remaining bits of java are removed. Note also that the JRE will only reclaim memory when it's needed for other programs. If you don't run any other programs then the memory will increase until it reaches the maximum memory that you set at jD's startup. |
#12
|
||||
|
||||
Hi, I guess will just have to disagree. Pls see notes below.
Quote:
Quote:
Quote:
Quote:
Oh, btw, I am running JD 24x7 for several days at the time (sometimes more than a week without reboot). I don't think there are too many people doing this. This may also be a contributing factor to the VM issue. |
#13
|
|||
|
|||
Log files, as requested:
http://jdownloader.org/pastebin/51781 http://jdownloader.org/pastebin/51782 Not sure if they will be of any use since I am testing different memory settings. FYI, it would seem that setting ram to 1024 makes no difference. For example, in the latest test I was running 4600 links - 750 Gb and JD's original mem consumption was: RAM: 115 Mb VM: 125 VM After one day of running, whithout adding new links and cleaning up the old ones, JD was using: RAM: 310 Mb VM: 325 Mb (the numbers in RAM and VM consumption fluctuate a little bit, but they always increse). It takes a few days of downloading for JD to consume all VM, so I'll wait until it crashes to submit the latest log. |
#14
|
|||
|
|||
Quote:
So 21 is better? OK, I guess I can give it a try asap I figure out how not to loose all my links. It may take a while to complete all started dlds. What about 27? Any recommendations? Wrt mem utilization, yes, I understand. The problem is that it goes well beyond the set up memory. |
#15
|
|||
|
|||
You've experienced a memory leak and it might be caused by update 22. Update 21 is stable and your links are safe.
If you upgrade to a version higher than 25 you might lose your links. First make backups or use jD's versioning feature (Ctrl-B). |
#16
|
||||
|
||||
I understood all of what you stated previously just disagree with most of it, regarding correlation between bandwidth/queue, and even some of the memory related statements.
100Mbit connection at conservative ~10MiB/sec * 86400(second/day) = 864000 MiB/Day or 843.75GB/day, your queue would be cleared in less than one day. I get about 1GB/hour on 3mbit so you must be ~2 .:. flat out your current queue would take ~12.5 days to download (without adding any more content). JD should be releasing memory once removed from the download list. As long as JD removes data from its memory allocations (regardless where its been stored, ram or disk), it should be cleared. As I understand JD currently stores info for said link in memory regardless 0% 5% or 100% only way to clear those from memory is to remove it from list. Hence my comments. The coming version of JD has new format and handles memory allocations differently, you should be able to store many more packages/links without the overheads. If it was your normal practice to store that many in your queue, and never came apon these problems until recently you have to wonder what changed. We have not pushed any updates into stable or beta since ~December last year, other than plugin updates, maybe theirs something running in loop? I see in one of the logs hr_error which I assume was caused by your memory tool? Try the different JRE see if it makes any difference, as remi recommended. Maybe might be worth while starting with clean configs by: finishing your already commenced downloads create a dlc of all other files queued. rename jdownloader/config/ to jdownloader/config.old/ run jdupdate.jar (creates new configs and make sure all JD matches CRC). Reconfigure JD. Import the DLC. See how you go from there. You can always revert back by renaming your old folder back again.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] Last edited by raztoki; 04.09.2011 at 19:12. |
#17
|
||||
|
||||
wait for next major update which uses a lot less cpu/ram. many memleaks fixed and so on.
you will notice the difference for sure
__________________
JD-Dev & Server-Admin |
#18
|
|||
|
|||
Thank you kindly to all.
Will try different suggestions and see what happens. Looking forward to the next version |
#19
|
|||
|
|||
FYI, well, it's official. Nothing works.
Any combination / permuation of the following yields the same result: memory leak. Java 21 or 27 Brand new config 512 or 1024 RAM And yes, I did take the precaution to run JavaRa before re-installing 21 or 27. Just for the heck of it, I also did refresh all W2K server critical files (sfc /purgecache). The only thing that works is to shut-down and restart JD every 24 hrs or so |
#20
|
||||
|
||||
maybe provide one of these logfiles? you have yet to provide anything useful as the previous logs where only from testing?. If you start windows_createlog.bat , it will log to file you can then upload via http://jdownloader.org/pastebin/
cheers
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#21
|
||||
|
||||
what memleak? jd is set to use max 512mb heap. in case your java process uses more than ca. 600mb then its a memleak inside firewall/av. nothing unusual and has happened before aswell
__________________
JD-Dev & Server-Admin |
#22
|
|||
|
|||
I am puzzled, what firewall? I am using W2K SP4 Server and no third party firewall. Wrt av, I have been using *the same* McAfee (up to date, of course) for almost 5 years now and it is rock solid.... albeit a tad overzealous.
|
#23
|
|||
|
|||
Quote:
1 - Kept growing 2 - Did not diminish upon removing (a *significant*) amount of files from queue. |
#24
|
||||
|
||||
Well without real indication (useful logs) of what might be causing the problem, we may as well close this thread. You haven't provided us with the necessary information to diagnose. We have already given you the possible causes/solutions, which are educated guesses at best based on conversation within this thread. The only other aid we can use is teamviewer but that's probably pointless also as it takes a long time to occur.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#25
|
|||
|
|||
Quote:
|
Thread Tools | |
Display Modes | |
|
|