[raztoki]

you are going in circles again, please read this thread in relation to directories with lot of content.

raztoki

[djmakinera]

08.10.17 16.53.38 <--> 08.10.17 16.42.20 jdlog://6642714015941/
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Can not download files from ONLY mega.co.nz (several concurrent downloads)
causes JD2 failures and freezes and logs also can not be sent


ISSUE 1:
Create logs (not working)

java.lang.RuntimeException: jd.http.Browser$BrowserException
at org.jdownloader.jdserv.JDServUtils.upload(JDServUtils.java:47)
at org.jdownloader.jdserv.JDServUtils.uploadLog(JDServUtils.java:26)
at org.jdownloader.jdserv.JDServUtils.uploadLog(JDServUtils.java:22)
at jd.gui.swing.jdgui.menu.actions.sendlogs.LogAction.onNewPackage(LogAction.java:69)
at org.appwork.utils.logging2.sendlogs.AbstractLogAction.createPackage(AbstractLogAction.java:233)
at jd.gui.swing.jdgui.menu.actions.sendlogs.LogAction.createPackage(LogAction.java:53)
at org.appwork.utils.logging2.sendlogs.AbstractLogAction$2.run(AbstractLogAction.java:157)
at org.appwork.utils.swing.dialog.ProgressDialog$4.run(ProgressDialog.java:254)
Caused by: jd.http.Browser$BrowserException
at jd.http.Browser.openRequestConnection(Browser.java:1649)
at jd.http.Browser.openRequestConnection(Browser.java:1466)
at org.jdownloader.jdserv.JDServUtils.upload(JDServUtils.java:39)
... 7 more
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at org.appwork.utils.net.CountingOutputStream.write(CountingOutputStream.java:73)
at org.appwork.utils.net.CountingOutputStream.write(CountingOutputStream.java:73)
at jd.http.requests.PostFormDataRequest.write(PostFormDataRequest.java:156)
at jd.http.requests.PostFormDataRequest.postContent(PostFormDataRequest.java:96)
at jd.http.requests.PostFormDataRequest.postRequest(PostFormDataRequest.java:121)
at jd.http.Request.connect(Request.java:332)
at jd.http.Request.connect(Request.java:320)
at jd.http.Browser.openRequestConnection(Browser.java:1606)
... 9 more



Download mega.co.nz = LEAK MEMORY + HIGH CPU !!!
VERY LOW PERFORMANCE AND FREEZING OF APPLICATIONS DURING MEGA.CO.NZ <===(ONLY HOST)

[raztoki]

post download mega decrypts
this takes cpu cycles
since you always downloading entire internet / using JDownloader outside of normal use it will take time. In this case your system can not keep up, sockets have issue connecting/writing

raztoki

[djmakinera]

I only download a few files at the same time and JD2 crash!
This is a bug in mega.co.nz. It uses too many HeapSize or buffer
With other hosting I have no problems.
How to solve the problem?

Only one file downloaded simultaneously = works correctly.
Several files at the same time = High HeapSize & crash jd2

So JD2 only supports one file at a time ( max 1 simultaneous) ???

Originally Posted by raztoki

since you always downloading entire internet

40 files.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
40 files is the entire internet. ?
Of course not, so why are you suggesting that? See how high the stack is, it's a bug !!!

Probably because JD2 currently uses too little physical memory.
so there is a stack overflow.
developers set to permanent a lower value 247.50MB, so I have to start with other values in the parameter, because dynamic stacking does not work.
For some unspecified reason mega.co.nz requires a larger buffer/heap size/more physical memory etc..

1.8.0.144(32bit) (247.50MB/247.50MB/247.50MB)

[Jiaz]

You should know that 256MB heap is NOT enough memory for
404 packages/19176 links! There is no mem leak and there is no issue. You cannot expect JDownloader to work properly if you don't provide enough memory.

[djmakinera]

I can not listen to these absurdities.
Why for vk.com 10000+ (and even more - seriously) works perfectly.
But for mega.co.nz 40 files (YES 40 FILES !!!!) (but CONDITION IS ONE MULTIPLE simultaneous downloads) uses a great GB of memory???


NOTE:Just restarting JD2 will save you from a big leaked memory while downloading - but only for a moment)

Set: 10 chunkt
Set 10 simultaneous
= slow performance mega.co.nz and crash

[Jiaz]

JDownloader only can use max 256MB of memory (you can tell from your logs).
But I guess it's time for a new ban. I don't need you to tell me stuff you don't know about.

[Jiaz]

Originally Posted by djmakinera

NOTE:Just restarting JD2 will save you from a big leaked memory while downloading - but only for a moment)

This sentence alone shows that you STILL have no idea what you are talking about. Sad to see since we've explained you several times about memory/heap memory.

[djmakinera]

Okay, first issue.
Why is JD2 crashing when downloading files from mega.co.nz?
08.10.17 13.03.13 <--> 08.10.17 16.42.20 jdlog://5413714015941/
**External links are only visible to Support Staff****External links are only visible to Support Staff**


------------------------Thread: 23:UncaughtExceptionHandler.log-----------------------
--ID:23TS:1507473726675-10/8/17 4:42:06 PM - [jd.SecondLevelLaunch$7(uncaughtException)] -> Uncaught Exception in: 23=AWT-EventQueue-0|Java heap space
--ID:23TS:1507473728347-10/8/17 4:42:08 PM - [] -> java.lang.OutOfMemoryError: Java heap space

------------------------Thread: 812:UncaughtExceptionHandler.log-----------------------
--ID:812TS:1507473740837-10/8/17 4:42:20 PM - [jd.SecondLevelLaunch$7(uncaughtException)] -> Uncaught Exception in: 812=DownloadChunkRAF:[A1] Dj Xtorsion - Just Be you.mp3|Java heap space
--ID:812TS:1507473740837-10/8/17 4:42:20 PM - [] -> java.lang.OutOfMemoryError: Java heap space

[Jiaz]

Java is out of memory with 256MB and 19138 links. There is barley memory available to work with. Java does not have enough free heap left for garbage collection and that's why it fails with OutOfMemoryError.
Restart your JDownloader and provide a screenshot of your About dialog. It shows how much memory is left to work with

[djmakinera]

nEW CORE JD2 Java:/MB / /MB /2gb HEAP MAX(DEFAULT SETTING)Heap size=Crash JD2
Work Download




OLD JD CORE Java:/MB /MB /256mb HEAP MAX.
Failure (DEFAULT SETTING)

You do not need a screen because the window is all black.

[djmakinera]

Originally Posted by Jiaz

Java is out of memory with 256MB and 19138 links. There is barley memory available to work with. Java does not have enough free heap left for garbage collection and that's why it fails with OutOfMemoryError.
Restart your JDownloader and provide a screenshot of your About dialog. It shows how much memory is left to work with

OutOfMemoryError is a known bug in Windows and even released patches (hotfix) for some systems.
Alternatively PROGRAM WHEN REPORTS ABOUT LACK OF MEMORY NP. When I use 83% of physical memory, I've reported it to some developers, but it's an unbelievably difficult error to fix.

https://support.microsoft.com/en-us/...ge-number-of-p

[Jiaz]

OMG. you still haven't understood what this OutOfMemoryError is and that it does NOT come from windows...I'm giving up.

[djmakinera]

I use 23% of physical memory in my memory (12 GB RAM), and I'm not interested in using 100% memory (out of memory) because it's crap jd2.
The heap size in JD2 is high when download mega.co.nz? so it crashes.
Well, it looks like mega.co.nz uses more heap than my favorite VK server.

My suggestion:
tion:Return to old JD2 settings: Default is ~2048 MB (~2GB) Max. Heap

[Jiaz]

Please tell me where you check heap size? I'm very curious how you check that value ?
Do you even know what java heap is? How it works?
It has nothing to do with mega or vk or anything else. 256MB max heap is NOT ENOUGH for thousands of links + gui + enough room for java to garbage collection.
Of course it works fine when you provide more heap, that is no surprise.

[djmakinera]

Download Master resumes without any problem, without using a lot of RAM.
Zero problems for multiple connections.

Check stack/heap size in JDK application by downloading multiple files simultaneously (not just one at a time)
Generating too many errors in milliseconds.

[djmakinera]

Generating large logs with BUGS error, and error and error and error and error and error and error and error and error in a second causes a large CPU usage and a stack OVErflow (High Hepa Size use)

[Jiaz]

Originally Posted by djmakinera

Download Master resumes without any problem, without using a lot of RAM.
Zero problems for multiple connections.

Fine, then we can finally stop supporting you. I will talk to team about no longer providing support.

[djmakinera]

03.11.17 19.15.06 <--> 03.11.17 19.17.24 jdlog://1831914015941/

Please wait: Parsing Clipboard content

I added only 2343 links and it takes a long time (!) And nothing adds...
Probably freeze.

[Jiaz]

It was just a gui bug, already fixed

[djmakinera]

sharing memory for jd2 and, for example: Browsers, because there is not enough memory in the system message

My current setting
RAM:12GB
Virtual Memory: 2000MB min - 4000MB max <== how much to increase pagefile

[Jiaz]

Your message shows that you doesn't even understand/know what virtual memory is and how it works.
I'm sorry but please use search engine of your choice and inform yourself. Pagefile is NO memory!

[djmakinera]

Virtual memory or swap, exhaustion of the heap - whatever it is called, does it matter.

Simply the system lacks memory even if it is not fully exploited !!!, and the JD2 application crashes or you close java yourself to avoid system problems. I tried to tune the virtual memory, but either there is too little or too much, each tuning does not bring positive results. So what can you do about it.

[djmakinera]

@Jiaz - After some time, the application closes automatically.

https://i.postimg.cc/sxqymnJC/Screen...t-10-16-AM.jpg

[djmakinera]

The application can not be started, all the time a critical error.

---------------  T H R E A D  ---------------

Current thread (0x000000002f034800):  JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=6088, stack(0x000000002f4d0000,0x000000002f5d0000)]

Stack: [0x000000002f4d0000,0x000000002f5d0000]
[error occurred during error reporting (printing stack bounds), id 0xc0000005]

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)


Current CompileTask:
C2:37741887 16654 %     4       org.appwork.utils.swing.Graph::paintComponent @ 90 (985 bytes)


---------------  S Y S T E M  ---------------

OS: Windows 7 , 64 bit Build 7601 (6.1.7601.24444)

CPU:total 4 (initial active 4) (4 cores per cpu, 1 threads per core) family 6 model 60 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, lzcnt, tsc, tscinvbit, bmi1, bmi2

Memory: 4k page, physical 12519556k(1704384k free), swap 17637656k(5104k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.191-b12) for windows-amd64 JRE (1.8.0_191-b12), built on Oct  6 2018 09:29:03 by "java_re" with MS VC++ 10.0 (VS2010)

time: Thu Jun 13 10:16:31 2019
elapsed time: 37741 seconds (0d 10h 29m 1s)

[Jiaz]

No Problem with JDownloader but with your setup/system/installation

[Jiaz]

Originally Posted by djmakinera

swap 17637656k(5104k free)

Nearly 18Gbyte in Swap, that's DEFINITELY a sign that you use your system more than it can handle

[djmakinera]

Only that the "swap" is wrongly called, this is pagefile.sys (
that is, paging memory). But see that there is 1GB physical memory of free RAM, but JD2 does not use it, it only uses paging memory.

physical 12519556k(1704384k free) ~ 1GB

[Jiaz]

You know that pagefile IS swap, right?
And you know that it's not up to JDownloader/application what memory to use. This discussion/topic sounds familiar to me....

[djmakinera]

If the setting is "Automatically manage the paging file" - it creates a very large pagefile.sys file each time. I have SSD and you realize that it is very harmful, save a lot of data every time. Of course, the file automatically reduces the size when you restart the computer, but after some time the re-size is big. Such a continuous high-write operation will "kill" my SSD.

I use only Browser and JD2, I do not use other programs that could consume all my memory resources.

[Jiaz]

Having a system that requires nearly 18GB of pagefile is definitely NOT okay.
Your system is using nearly 30GB of memory of which only 12 are real memory. that is NOT normal

[djmakinera]

You misread the size of the paging file, I can not have such a large paging file, I have

4,57 GB (bajtów: 4*913*201*152)

MemoryExhaustionInfo use pagefile:

See why, as many as two javaw.exe processes, it should not be. One instance and two processes?

https://postimg.cc/MM5cWn57

[Jiaz]

Your jdownloader may use max 3.56GB of heap memory and therefore much much less than your 12GB of system memory and yet your system had much more memory in use. you should find the cause of this

The application doesn't close, it dies/gets killed

[Jiaz]

Originally Posted by djmakinera

You misread the size of the paging file, I can not have such a large paging file

No I don't

12519556k(1704384k free), swap 17637656k(5104k free)
12519556k = 12.519.556k = 12GB Memory
17637656k = 17.637.656k = 17GB Page/Swap-File

[Jiaz]

Originally Posted by djmakinera

See why, as many as two javaw.exe processes, it should not be. One instance and two processes?

Your system, not mine.

[djmakinera]

1. I can not have such a huge swap file because I do not have enough disk space. So it is unreal!
2. 17GB Page/Swap-File ??? Your size calculator is completely wrong.

I checked the appropriate calculator. Real conversion:
17 637 656 kilobits = 2.204707 gigabytes = ~ 2GB :D That is exactly how many of my values, for example, 2000 MB - 5000 MB

[Jiaz]

Originally Posted by djmakinera

Memory: 4k page, physical 12519556k(1704384k free), swap 17637656k(5104k free)

It's from your crashlog

[Jiaz]

Originally Posted by djmakinera

kilobits

those values are in kilobytes

[djmakinera]

There is a small letter "k" there, what next? Do I have to guess?
It is impossible to be sure that 17 GB is set, and that whatever uses so much memory, because you can check it in an advanced tool: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Either this log pretends false data, or you incorrectly convert values.


https://i.postimg.cc/rmXjKND8/Screen...t-04-54-PM.jpg

[Jiaz]

@djmakinera: Your System has 12GB of memory/RAM, yes? then the physical memory value is correct. so why should tbe swap with 17GB be wrong them.
this topic is finished. when you start arguing about your pagefile size, then it's offtopic