[In progress] DJ performance issues

[Jiaz]

Originally Posted by djmakinera

ISSUE 2:

Jiaz -> pspzoccerscene long time ago "suggested I reduced Chunk", but how to reduce Chunk 1 to ZERO ???

How do I reduce the Chunk 1 to 0?

its hardcoded to 1 and will stay that way

[djmakinera]

Jiaz or pspzockerscene or coalado used to fix the developers (more often if this error occurred UncaughtExceptionHandler - I had to wait a long time for an update of the NEW DOWNLOAD SYSTEM and UPDATE CORE)
Of course - now very, very rarely occurs error :D

[djmakinera]

Originally Posted by Jiaz

its hardcoded to 1 and will stay that way

Excuse me, who is right?

https://board.jdownloader.org/showpo...16&postcount=4

[raztoki]

1 is the least chunks possible, 0 would mean no/nill/zip connection

raztoki

[Jiaz]

@dj: that post is from 2014, meanwhile it is limited to 1

[Jiaz]

Originally Posted by djmakinera

Jiaz or pspzockerscene or coalado used to fix the developers (more often if this error occurred UncaughtExceptionHandler - I had to wait a long time for an update of the NEW DOWNLOAD SYSTEM and UPDATE CORE)
Of course - now very, very rarely occurs error :D

please provide logs, the one you've provided is okay, nothing to do about it

[djmakinera]

Jiaz - a)You use Win 7
b)You have UAC enabled ? (if so, what level ?)
c)JDownloader possible cause problems if it is installed on the System C:\Jdownloader2 Partition?
Other installations have a Non-System-on drives example K:\Jdownloader2 Z:\Jdownloader2

[Jiaz]

a.) yes, b.) default (never touched any settings, it comes up when needed) c.) JDownloader folder is not important, the folder where you download too counts

[djmakinera]

b)

Level 0 (The default is "never notify" (not recommended))
Level 1 (Medium)
Level 2 (High)
Level 3 (Very High) My setting

[Jiaz]

much fun...It doesnt happen to me as I don't use any restricted areas of my windows.

[djmakinera]

You have to remember that UAC (Very High) efficiently protects Reansomware (File Encryption !!).
Really IT WORKS because the few times in my life I had such a situation and never to break my strong security !!!

[Jiaz]

As I said....much fun. If you use UAC that way, then you will have to use JDownloader as admin all the time. UAC support does not help anything as UAC simply restarts application as admin

[djmakinera]

My friend said his application always works:

"(...) Uses Task Manager to bypass the UAC prompt. It always worked."

<setting name="UacBypassEnabled" serializeAs="String">

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<assemblyIdentity version="1.0.0.0" name="MyApplication.app"/>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
<security>
<requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
<requestedExecutionLevel level="asInvoker" uiAccess="false"/>
</requestedPrivileges>
</security>
</trustInfo>
</assembly>

No need to run as Administrator, and without any confirmation action!
How to do the same for javaw.exe? You do not know?

[Jiaz]

How about using a search engine yourself?!

[djmakinera]

Ja good, but I see only the old topics 2009-2014
Is it still valid?

[Jiaz]

I don't know, you have to check. I don't have issues with UAC

[djmakinera]

I don't have issues with UAC

Because you are not using UAC high OR very high, so you can not 100% confirm that always downloading works perfectly.

I have a choice:
UAC DISABLE and lose the better security of the system or UAC ENABLE to see from time to time some rare errors downloads.

[djmakinera]

An error Occurd FileExistsException jpg.part Invalid download directory

[djmakinera]

Too many links ??? 155K links (155.000) / 1236 packages = EXCEPTION_ACCESS_VIOLATION (0xc0000005)

uses 89% RAM (13 -14 GB) + 100% (8096 MB Virtual Memory).
Please tell me how to set it up properly "Virtual memory"?


#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000667e3777, pid=5636, tid=0x00000000000012a8
#
# JRE version: Java(TM) SE Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.102-b14 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# V [jvm.dll+0x213777]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp


#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1232464 bytes for Chunk::new
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (allocation.cpp:390), pid=1036, tid=0x0000000000000e40
#
# JRE version: Java(TM) SE Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.102-b14 mixed mode windows-amd64 compressed oops)
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
------------------------------------------

[djmakinera]

The number of links / packages are automatically removed when the download is complete, so the heap/memory should release ??? But on the contrary does not release! Why ?;)


Long record of active I/O small file on the disk causes a rapid increase in the use of RAM.
How to free up physical memory and virtual memory without restart JD2 eg. Every 20-30 minutes?

I start with the default settings JD2 (without parameters)

As shown in 3.88 GB / 3.88 GB / 3.88GB
it uses 80-90% of RAM and GUI slow to respond.
62K links
458 packages

I tried with different parameters, garbage collection - but its objective causes exhaustion of the cache/overflow/error Out Of Memory.
Parameters Garbage Collection is for advanced users.
I know nothing about it!

[Jiaz]

Originally Posted by djmakinera

The number of links / packages are automatically removed when the download is complete, so the heap/memory should release ??? But on the contrary does not release! Why ?;)

To find that out we need to do a teamviewer session and analyse a heapdump!

[djmakinera]

Maybe it caches thumbnails in the memory by downloading only the images?
How to disable caching?
can not reproduce the issue by TeamViewer, only in real-time while long-time downloading memory usage is high.

[Jiaz]

JDownloader does not create any thumbs or uses any caches.
I'm doing some tests now. Still could not reproduce the issue.
When you encounter this again, lets do Teamviewer and create a heap dump with jvisualvm, we can open it later and analyse it

[Jiaz]

Does this happen during Download or does memory even stay high when you stop the downloads?

[Jiaz]

Found one memleak, will fix it

[djmakinera]

Strange chars: and Internal exceptions


Deoptimization events (10 events):
Event: 361.509 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ab469c method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/DeserializatÝ@¬¶%˜v@
Event: 361.509 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ea3eb4 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/Deserializatio
Event: 361.510 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ab4688 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/Deserializat£Â{_'˜v@
Event: 361.510 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ab4688 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/Deserializatøl¾'˜v@
Event: 361.510 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ea3eb4 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/Deserializatio“rï'˜v@
Event: 361.510 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ea3eb4 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/Deserializatio*â€**e(˜v@
Event: 361.510 Thread 0x0000000028477000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ea3eb4 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/DeserializatiofÃu
(˜v@
Event: 361.510 Thread 0x0000000028474000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003ea3eb4 method=com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(Lcom/fasterxml/jackson/core/JsonParser;Lcom/fasterxml/jackson/databind/Deserializatioî
!ÖK˜v@
Event: 361.519 Thread 0x0000000028475800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003b87500 method=com.fasterxml.jackson.core.json.ReaderBasedJsonParser._decodeEscaped()C @ 8
Event: 361.530 Thread 0x0000000028475800 Uncommon trap: reason=uninitialized action=reinterpret pc=0x000000000323d994 method=jd.plugins.DownloadLink.getDownloadCurrent()J @ 21

Internal exceptions (10 events):
Event: 361.667 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x0000000545700878) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.667 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x0000000545700b30) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.684 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x00000005458a26b0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.684 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x00000005458a2968) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.688 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x00000005458f4718) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.688 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x00000005458f49d0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.689 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x00000005458f5a60) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.689 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x00000005458f5d18) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.689 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x0000000545b1d038) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 361.689 Thread 0x000000002bee3000 Exception <a 'java/io/IOException'> (0x0000000545b1d2f0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u102\7268\hotspot\src\share\vm\prims\jni.cpp, line 709]

[Jiaz]

Nothing to do with JDownloader.

[Jiaz]

Please test again with next core update
In case you still suffer from memleak, the easiest way is to do teamviewer and we check with jvisualvm

[djmakinera]

Jiaz - Jaz how to adjust to javaw.exe used only C:pagefile.sys (only fast disk SOLID STATE DRIVE - Eliminate temporary no response JDownloader 2)
and avoid error in the allocation of heap (Multiple physical disks)

SSD X: pagefile 1 javaw.exe
HDD X: pagefile 2 other applications for the needs of biggest use PAGEFILE 20-50 GB !

[raztoki]

Originally Posted by djmakinera

Jiaz - Jaz how to adjust to javaw.exe used only C:pagefile.sys (only fast disk SOLID STATE DRIVE - Eliminate temporary no response JDownloader 2)
and avoid error in the allocation of heap (Multiple physical disks)

SSD X: pagefile 1 javaw.exe
HDD X: pagefile 2 other applications for the needs of biggest use PAGEFILE 20-50 GB !

operating system has total control over 'swap/page' file, applications have ZERO control.

general rule with swap files is to have same size as ram, 1:1 ratio. eg. 8gb ram, 8gb swap. You can have more but it wont make it 'faster' as swapping from ram to hdd/ssd is generally slow compared to just having more ram. You can increase swap file ratio out to 1:3 but normally not required.

I see no need for 20-50gb swap file, that's rediculious.
Please get yourself informed... google is your friend! top search result _https://support.microsoft.com/en-au/kb/2860880

raztoki

[djmakinera]

raztoki - You do not know how to set the page java for the SSD so I suggest GOOGLE - I still do not understand the complicated settings and edit the registry !!!

Originally Posted by raztoki

I see no need for 20-50gb swap file, that's rediculious.

You're wrong !!!

Developer Emeditor:

You should be able to search or copy very large files.

Do you have enough available memory in the system temporary folder, you can check the folder in the Advanced tab of the Customize dialog box, and you will see the available memory size in the Large File Controller.

EmEditor also requires a lot of memory stronicownaia operations, "UNDO" !!!

UltraSearch requires 40 GB (!!!) of RAM, so I have to Increase the memory in the paging because they do not have enough RAM !!

Powerful graphics applications also require a lot of pagefile !!!

It is recommended to use a large paging file for many "heavy operation" or Out of Memory "Fatal System Error: 0x00000093" Error Message

[Jiaz]

Pagefiles larger than your memory size makes no sense...But we let you in believe this is a great idea

[Jiaz]

A pagefiles does not replace memory When an application requires 32GB of RAM, you will not be happy/work at all when you have 24GB of RAM and enough pagefile. But we have learned that you will ignore us so there is nothing to do here.

[Jiaz]

Originally Posted by djmakinera

UltraSearch requires 40 GB (!!!) of RAM, so I have to Increase the memory in the paging because they do not have enough RAM !!

Well, then its no good application The only reason a texteditor would really require that much memory when you search for 40gb string There is no reason for a texteditor to keep complete file in memory...

[raztoki]

I'm not wrong, neither is Microsoft. But of course dj knows everything/best!

[Jiaz]

Do you have enough available memory in the system temporary folder,
-> he speaks about temp folder and NOT pagefile

[djmakinera]

Preventing "JAVA: Stack Overflow"
How to Set Dynamic Heap Size?

[Jiaz]

Stack Overflow has nothing to do with heap!
How about provide logs?

[djmakinera]

Example:
Set -Xmx 5GB
If Heap exceeds the limit of 5GB is JDownloader crash and shows an error Stack Overflow

[Jiaz]

Heap cannot exceed the limit You will get a Out-of-Memory
Stack Overflow is complete different type of error. Again no logs!