[djmakinera]

VMMap is a process virtual and physical memory analysis utility:

Screenshot:
https://i.postimg.cc/k9h2RsXh/Screen...t-05-30-PM.jpg

[djmakinera]

When update, for example, "Event Scripter" or "Core Update" or plug-ins, some error during the restart, but during the second attempt always starts correctly.

------------------------Thread: 365:org.jdownloader.updatev2.restart.WindowsRestarter.log-----------------------
--ID:365TS:1563567340243-19.07.19 22:15:40 -  [org.appwork.utils.logging2.InputStreamLogger(readInputStreamToString)] -> Error occurred during initialization of VM
--ID:365TS:1563567340244-19.07.19 22:15:40 -  [org.appwork.utils.logging2.InputStreamLogger(readInputStreamToString)] -> Unable to allocate 163840KB bitmaps for parallel garbage collection for the requested 5242880KB heap.

5242880KB = 5GB ???


My Parameters to run:
"C:\Program Files\Java\jre1.8.0_191\bin\javaw.exe" -Xms2G -Xmx5G -jar C:\JD\JDownloader.jar

Physical memory still available, so I do not know what the problem is?
screen: https://i.postimg.cc/HWyGt2Jd/Screen...t-10-16-PM.jpg

Still need to increase -Xmx?

Download List: 70K
LinkGrabber List: 63K

19.07.19 22.17.14 <--> 19.07.19 22.32.09 jdlog://0992186935451/

[djmakinera]

I had a similar problem with some recent updates.
If the problem persists, I will publish the logs.


https://i.postimg.cc/CM9XzvmT/Screen...t-12-11-AM.jpg

[Jiaz]

That error is not caused by JDownloader. Java fails to start. Most likely because your system is out of memory

[djmakinera]

This is not an error of out of memory, because the last pid with this type of error is from July 19.

JVM initialization fails with parallel GC

probably should use parameter start???

-XX:-UseParallelGC

https://board.jdownloader.org/showthread.php?t=81097

[djmakinera]

Expected results: The JVM should not attempt to allocate memory beyond that required to accommodate the specified maximum heap size when using the parallel garbage collector.

[Jiaz]

something made your JVM fail to start and it's not JDownloader. It's up to you what parameters/JVM settings you set.

[Jiaz]

Originally Posted by djmakinera

This is not an error of out of memory, because the last pid with this type of error is from July 19.

JVM does NOT create hs_err files when not started Invalid combination of parameters or failing to start does NOT create hs_err files

[djmakinera]

Closing or running the application works correctly. Only in some specific conditions during the upgrade with the required restart it encounters an error.
It's just that the error does not appear during the second attempt - running with still the same parameters.

The process may also be busy or working in the background, which may prevent a proper restart when trying to update. The first attempt may fail, but the second may not.

[Jiaz]

Originally Posted by djmakinera

The process may also be busy or working in the background, which may prevent a proper restart when trying to update. The first attempt may fail, but the second may not.

That's not the case.

Originally Posted by djmakinera

The first attempt may fail, but the second may not.

Because your system runs out of memory and JVM fails to start

Originally Posted by djmakinera

Closing or running the application works correctly. Only in some specific conditions during the upgrade with the required restart it encounters an error.
It's just that the error does not appear during the second attempt - running with still the same parameters.

Because your system runs out of memory and JVM fails to start


The answer is very simply. You specify the parameter -Xms2g which makes JVM allocate minimum 2Gbyte of Memory.
During update JD starts another instance and then you have up to 3 instances running, 3*2Gbyte Memory -> the new processes
fail to start because out of memory.
Removing this parameter or reduce it should solve it for your setup

[thecoder2012]

Originally Posted by Jiaz

fail to start because out of memory.
Removing this parameter or reduce it should solve it for your setup

Or buy more memory for your computer. I have 64 GB.

[djmakinera]

Originally Posted by thecoder2012

Or buy more memory for your computer. I have 64 GB.

Really, do not need a lot of physical memory, my 12GB, it seems to be optimal. Many people have even less physical memory, 32-64GB is for advanced and hardcore people who will not actually use that much physical memory. Who is it for?
Physical memory can be replaced with virtual memory, and this does not slow down the operation of the computer and programs when you have a very fast SOLID STATE DRIVE (Tested!)
Is something changed in the long run? I've been using JD2 for a long time and I have not had any problems for a long time, now it seems that JD2 needs more and more memory to run JD2, but it does not need a lot RAM while downloading.

[Jiaz]

Originally Posted by djmakinera

Download List: 70K
LinkGrabber List: 63K

The more links you've got in list, the more memory is required.
It highly depends on link/host type if a link requires less/more memory.

[Jiaz]

Originally Posted by djmakinera

Physical memory can be replaced with virtual memory, and this does not slow down the operation of the computer and programs when you have a very fast SOLID STATE DRIVE (Tested!)

I'm sorry but NO it can not! But I already gave up about this

[raztoki]

my thoughts, under Jiaz explanation.
why does the update module need to start with the same vm parameters as main program? Surely it should not not have the same memory requirements. You would only have 2*3GB and say 64MB for the update requirements.

and if the original instance is still running at end of updating, maybe it should wait until the original pid is cleared? or at least an option to do so. Doing so would only ever mean one instance of the higher memory allocation running at any time.

raztoki

[Jiaz]

@raztoki: for sure we could *filter out* some parameters but in that case the parameters are lost for the *after update* process.
and the first process will shutdown once the 2nd proccess has validated the pending update.

[djmakinera]

Originally Posted by Jiaz

I'm sorry but NO it can not! But I already gave up about this

Yes and no.

It depends on the type of program / application you use.
WEB browsers, movie and music players, text editor, simple programs do not require the need of having large physical memory. Maybe for professionals who render the film or nobody knows what else.
There is an editor that can open 248 GB of text data, but still uses a small amount of RAM, and uses disk storage, if a very large amount of memory is required.

I have the most soundcloud.com host links
I do not know if it matters.
It's not much, anyway ... especially when in the distant past I had 10 times more links, and it was very good (And the usage was in a similar range of RAM)

Decrease the value of -Xms0,5G to see the effects.

[Jiaz]

Originally Posted by djmakinera

Decrease the value of -Xms0,5G to see the effects.

You can leave out the -Xms parameter

[Jiaz]

Originally Posted by djmakinera

There is an editor that can open 248 GB of text data, but still uses a small amount of RAM, and uses disk storage, if a very large amount of memory is required.

I don't know why a filesize should be connected to application? video player also playback gbyte of video without using that much memory.

[Jiaz]

Originally Posted by djmakinera

Yes and no.

It depends on the type of program / application you use.

correct, and when you tell Java to start with 2 Gbyte of memory (-Xms2g) then it will allocate 2gbyte of memory and either the os is able to provide that much memory or not. In your case, it doesn't have enough memory left.

[Jiaz]

Originally Posted by djmakinera

I have the most soundcloud.com host links
I do not know if it matters.
It's not much, anyway ... especially when in the distant past I had 10 times more links, and it was very good (And the usage was in a similar range of RAM)

Yes, this matters as different links require different amount of memory.
and of course we add more features to plugins and that will increase memory usage. For example plugins with customizable filename pattern do require more memory because the link needs to hold all possible meta information.

[Jiaz]

you may send me a link to your downloadList/linkcollector zip and I can check for possible optimizations in memory requirements

[djmakinera]

Originally Posted by Jiaz

I don't know why a filesize should be connected to application?

Example. How a program designed that does not need a lot of RAM, even with huge multi operations, and performance, still high.

Read FEATURE:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Screen Feature:
https://i.postimg.cc/xCK63knX/Screen...t-01-46-PM.jpg

[djmakinera]

Originally Posted by Jiaz

you may send me a link to your downloadList/linkcollector zip and I can check for possible optimizations in memory requirements

Respect... I want to keep privacy, just like most users.

I can provide links only in this way:
50000 links:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

Originally Posted by djmakinera

Example. How a program designed that does not need a lot of RAM, even with huge multi operations, and performance, still high.

That's what I meant. Large filesize doesn't mean large memory usage.

[Jiaz]

Originally Posted by djmakinera

Respect... I want to keep privacy, just like most users.

I'm sorry but I have more important things to do than to recreate a situation like yours and adding/waiting for 130k links to be added/processed.

[Jiaz]

There's no need to continue here. The issue is caused by system memory and your jvm settings.

[djmakinera]

JD2 after starting the use of pagefile.sys increases significantly from 1GB to 4GB, but after closing JD2 it still uses as much disk space all the time. Release memory can only take place when the system is restarted, otherwise the maximum size will be constant all the time.
How do you force JD2 to use only physical memory?

"C:\Program Files\Java\jre1.8.0_231\bin\javaw.exe" -Xms512M -Xmx5G -XX:-UseParallelGC -jar C:\JD\JDownloader.jar

[Jiaz]

Originally Posted by djmakinera

How do you force JD2 to use only physical memory?

I'm not aware that this is possible for user space applications at all and even kernel space
"Because contiguous physical memory is usually in short supply, it should be used sparingly and only when necessary." so even drivers should only allocate small portions of such memory.

[djmakinera]

My SSD drive is not large.
That's why I try to manage space as much as I can.
But after running JD2 or performing what works, the space for it just immediately drops to 0 bytes, even though there was, say, 5-7 GB free space.

Another problem is that the logs will be created indefinitely if there are errors, and it is not interrupted automatically, but it is possible manually.
In the advanced JD2 settings there is an option - the number of attempts after which stop the download attempt or the infinite error loop?
I asked about it many years ago, any new information?

[Jiaz]

Normal applications cannot allocate *non pagable* memory because that way you could easily *make your os non functional*
Please provide more information about "infinite error loop" and the logs are limited in size and can even be disabled (see advanced settings)

[djmakinera]

The repeated error indefinitely generates a log record all the time. And no automatic interruption of the download process.
Please fix this unhandled error

Log:
24.10.19 17.57.43 <--> 24.10.19 18.00.13 jdlog://4895130900751/

[Jiaz]

the loop is caused by connection issue and JDownloader does auto retry because this error can be temp. or IP dependant.

[djmakinera]

This link will also repeat the unknown error indefinitely and generate a log.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

And? JDownloader retries. You can easily set limits for logs in Advanced Settings. Without a log, my guess is that the error that has happened doesn't lead to a *don't try again*

[djmakinera]

And?

You can limit the log limit, but it depends on all hosts. How to exclude logs only for specific hosts?

[Jiaz]

You can't and I don't see any need for this. Just reduce log size.

[djmakinera]

There is a conflict with memory management between the Chrome browser and JD2. I don't know how to solve this problem.

Error: Out Of Memory. JD2 Auto-Terminate

RAM: 12 GB


28.12.19 17.08.39 <--> 28.12.19 17.12.40 jdlog://9553330900751/

"C:\Program Files\Java\jre1.8.0_231\bin\javaw.exe" -Xms512M -Xmx5G -XX:-UseParallelGC -jar C:\JD\JDownloader.jar