JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 22.09.2021, 03:05
spectre64 spectre64 is offline
Modem User
 
Join Date: Sep 2021
Posts: 3
Default Possible memory leak in HCaptcha handling

Hi, wondering if you are aware of any memory leaks in HCaptcha handling? I'm currently bisecting the problem.

JDownloader 2 (Build: Fri Sep 17 15:19:08 CEST 2021)
Java 1.8.0_282(64bit) Windows 10

Have premium filejoker, and some credits on imagetyperz

I have a heap dump (hprof) file, not sure how to securely share that. Here's a screenshot of some of them expanded where I suspect the leak is: https://imgur.com/a/dFXSRft (this was from Java 11 I believe)

I've set the heap as big as 16gb and leaks about 2gb/hour (post GC) and eventually will run out of memory / slow to a crawl.

~1000 links in downloads

Next, I downgraded to Java 8 (via the provided JDownloader packages), no change.

Next, I disabled all captcha handling and the problem went away.
Next, I enabled imagetyperz [no leaks]
Next, I enabled Dialog(window) in JDownloader, Dialog(window) in Browser [no leeks]

Previously, I believe i also had JAntiCatpcha and 9kw.eu enabled.

Will continue my investigation, but for now letting my downloads run for a bit...
Reply With Quote
  #2  
Old 22.09.2021, 09:45
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@spectre64: thanks for sharing your findings! I'm not aware of any memleak but of course in combination with a solver, there might be a situation where this can happen. I will look into it.
Can you share the dump? eg upload to mega and then send link to support@jdownloader.org ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 22.09.2021, 10:07
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

Quote:
Originally Posted by spectre64 View Post
I've set the heap as big as 16gb and leaks about 2gb/hour (post GC) and eventually will run out of memory / slow to a crawl.
Just out of interest, because the dump/screenshot doesn't show any leaking reference. Without pressure and manual GC, java doesn't cleanup memory. have you checked memory usage after manual GC?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 22.09.2021, 15:10
spectre64 spectre64 is offline
Modem User
 
Join Date: Sep 2021
Posts: 3
Default

Yes it leaks roughly 2GB/hour after manual GC, I don't recall the specifics, but I think the heap dump was taken in a state such that roughly 6-7gb of memory was consumed even after manual GC.

Will investigate sharing the heap dump...
Reply With Quote
  #5  
Old 22.09.2021, 15:45
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@spectre64: We can also do a teamviewer session and check it out together live, contact me via support@jdownloader.org
that way you don't have to share such large file and still easy/fast can do some testings
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 22.09.2021, 18:47
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

I can reproduce the issue...but not yet sure where the leak comes from
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 22.09.2021 at 19:07.
Reply With Quote
  #7  
Old 22.09.2021, 19:20
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@spectre64: great CATCH! Update is on the way. I was no longer able to reproduce the issue. Issue was happening for all browser captchas, so very GOOD CATCH

There was a reference leak in the class handling the browser captcha stuff. when captcha handling was done, the browser handler was not stopped/dereferenced and thus the browser handler did hold a hard reference to everything else (plugin, browser,....) from that download.
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 22.09.2021 at 19:38.
Reply With Quote
  #8  
Old 23.09.2021, 03:28
spectre64 spectre64 is offline
Modem User
 
Join Date: Sep 2021
Posts: 3
Default

/hug that's great! I thought it was just me!

Thanks again for such a fast response.
Reply With Quote
  #9  
Old 23.09.2021, 04:09
xvertigo xvertigo is offline
DSL User
 
Join Date: Mar 2016
Posts: 35
Default

I just came to comment the same thing. I wait for the update. Thanks.

One question. The update will be an program update or an browser addon update? I ask because i use Firefox and i wanna be alert on that.
Reply With Quote
  #10  
Old 23.09.2021, 09:56
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

The update is already live, please update and test

@xvertigo: just a normal JDownloader update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 23.09.2021, 13:16
xvertigo xvertigo is offline
DSL User
 
Join Date: Mar 2016
Posts: 35
Default

Thanks, How many hours ago was realeased the update. I don't received yet. The program says that i am up to date. The last update came like 12 hours ago.

With my current version when a browser captcha tab is opened to solve a captcha and i cancelled o expired for time jdownloader go to the next file in line but the captcha tab still open. If i go to sleep i ended with 20 captcha tab open in line with only the last one still valid.

Last edited by xvertigo; 23.09.2021 at 13:18.
Reply With Quote
  #12  
Old 23.09.2021, 13:38
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@xvertigo yes, the update is already live and you have it. 12 hours is about the time the update has been released. I will check the *tab don't close* thing, but the memleak should no longer exist, so please check this
__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 23.09.2021, 14:07
xvertigo xvertigo is offline
DSL User
 
Join Date: Mar 2016
Posts: 35
Default

Quote:
Originally Posted by Jiaz View Post
@xvertigo yes, the update is already live and you have it. 12 hours is about the time the update has been released. I will check the *tab don't close* thing, but the memleak should no longer exist, so please check this
You are right with the memlick issue. Is no longer happening. The tab issue starts with the last update (like 12 hours ago) so i suppose is something related to it and is just matter of a minor repair.

Thanks for your efforts and specially for your fast answer to the users problems. I am still sure that jdownloader has the best suport team in all the web.
Reply With Quote
  #14  
Old 23.09.2021, 14:11
xvertigo xvertigo is offline
DSL User
 
Join Date: Mar 2016
Posts: 35
Default

Hi, just installed the last update (minutes ago and with no knowing if is related to this issue) and the tab problem still happening. Every don't attended or cancelled captcha tab still open when JD pass to the next file in line. (sorry for the bad english)

I am using Firefox 92
Reply With Quote
  #15  
Old 23.09.2021, 14:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@xvertigo: the tab not closing issue is not yet fixed Thanks about the feedback of the memleak. Working on the tab issue now
__________________
JD-Dev & Server-Admin
Reply With Quote
  #16  
Old 23.09.2021, 17:05
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@xvertigo: next core update (available in about 5 mins) will fix the tab closing issue
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 23.09.2021, 18:10
xvertigo xvertigo is offline
DSL User
 
Join Date: Mar 2016
Posts: 35
Default

Quote:
Originally Posted by Jiaz View Post
@xvertigo: next core update (available in about 5 mins) will fix the tab closing issue

Checked and working fine. You're the best.
Reply With Quote
  #18  
Old 23.09.2021, 18:13
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

Thanks for the feedback
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 05:45.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.