JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 04.04.2020, 16:02
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default [rapidgator] jd2 prone to timeouts relative to browser

so in a previous thread https://board.jdownloader.org/showthread.php?t=83565 I discovered a problem with rapidgator (not jd2) that jd2 couldn't resume a download because of something on rapidgator side.

I had come to depend on resuming as my downloads would regularly time out (we are talking 12+ hour downloads). At first I thought it was an issue with my proxies, but it turns out even if I don't use a proxy, it happens just as often. So, I thought it might be rapidgator, but to test, I first tried downloading the same file I had difficulty with in firefox and it completed on its first try.

I dont know what's going on, but to me it seems that jd2 is prone to timing out in a way that firefox is not. While this was going on, I had a 30 hour rclone push to google drive going on and none of those connections broke either.

any good way of debugging this? if I had to throw a dart blindly, I'd wonder if it has anything to do with garbage collection causing hiccups and freezing things long enough for connections to die?
Reply With Quote
  #2  
Old 06.04.2020, 14:44
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 53,612
Default

A good way would be to post a log.
Please keep in mind that the initial resume issues were NOT caused by any timeout issue but I understood that your report is ment differently - it is you suspecting downloads to stop e.g. in the middle because of a too small timeout value in JD.

Please post your log-ID here | bitte poste deine Log-ID hier.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 08.04.2020, 22:48
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

just happened again

08.04.20 00.45.33 <--> 08.04.20 23.47.57 jdlog://6644815302851/
Reply With Quote
  #4  
Old 09.04.2020, 10:47
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 53,612
Default

In your log I can see that the read timeout of 60 seconds was reached.

I've increased it to 120 seconds for the next update.
This should really be enough ...

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #5  
Old 09.04.2020, 11:22
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,175
Default

I'm not sure of firefox defaults (or your proxies), putting that aside you can adjust timeout settings yourself globally in advances settings which might help your setup.
InternetConnectionSettings.httpconnecttimeout
InternetConnectionSettings.httpreadtimeout

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #6  
Old 09.04.2020, 11:29
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 53,612
Default

Aside from that rapidgator has had a lot of issues in the past so a higher timeout in our plugin might be a good idea but to be honest I've never seen any website needing > 60 seconds to continue responding ...

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #7  
Old 09.04.2020, 11:40
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,175
Default

rapidshare did burst rate stuff, then nothing for a longer period which timed out most ppls downloads. That is the worst one that comes to memory.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #8  
Old 09.04.2020, 13:01
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

Quote:
Originally Posted by raztoki View Post
I'm not sure of firefox defaults (or your proxies), putting that aside you can adjust timeout settings yourself globally in advances settings which might help your setup.
InternetConnectionSettings.httpconnecttimeout
InternetConnectionSettings.httpreadtimeout

raztoki
i already had httpreadtimeout set to 600000 (i.e. max number) which should be 10 minutes. so my quesiton would be, is it not repecting that value (hence 60s timeout) or is the comment of 60s timeout bad assumption and it really a 10 minute timeout? (which would indicate some other problem?)

Last edited by thetoad; 09.04.2020 at 13:11.
Reply With Quote
  #9  
Old 09.04.2020, 13:14
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 53,612
Default

Your log shows:
Code:
Connection-Timeout: 60000ms
Read-Timeout: 60000ms
-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #10  
Old 09.04.2020, 13:19
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,175
Default

the timestamps should indicate when request was requested vs received.
on the data side of thing its not logged the last time a data packet was received, and how long the transmission between packetless causing the timeout. I guess you can assume via the http request preheader (its present) there, and work backwards from the timeout error exception.

from one of my logs with default settings
Code:
----------------Request Information-------------
...
Connection-Timeout: 20000ms
Read-Timeout: 60000ms
----------------Request-------------------------
please note that I do remember some operating systems at the TCP/IP stack prevent sockets from been open without data transfer for extended periods, regardless of settings within application/client.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #11  
Old 09.04.2020, 16:18
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

my log might show that, but this is what the settings are in the app when I look at them

https://imgur.com/a/thZ6iQ0
Reply With Quote
  #12  
Old 09.04.2020, 18:38
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

so I changed the value and restarted and now I see it updated.

Connection-Timeout: 60000ms
Read-Timeout: 120000ms

don't know why it showed a different thing in the UI before (still show a different value for connection timeout, but I didn't change that one)
Reply With Quote
  #13  
Old 10.04.2020, 11:49
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

still getting occasional timeouts on my proxies, but seemed to work a little better.
Reply With Quote
  #14  
Old 11.04.2020, 13:41
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

so experience shows 2 minutes works better, but not perfect. hard to determine if this is because connection is actually dying or just braindeadness on their side.

recommendation: every connection should keep track of the longest delta between bytes, when a connection dies after a timeout, can print the longest delta that didn't cause a timeout. if said delta is nowhere near the timeout length, indicative that its actually a problem, if the longest delta recorded is somewhat near the timeout length, perhaps value in increasing timeout.

thoughts?
Reply With Quote
  #15  
Old 14.04.2020, 13:19
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 53,612
Default

It is very unlikely that this will be changed soon.
Please keep in mind that we also cannopt update all of our default timeouts just for you.
Please either stick to what's there or use EventScripter Scripts to modify your JD's behavior however you'd like.
There is also a big "proxy helper" script linked which may help you.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #16  
Old 25.04.2020, 12:57
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

I think there was confusion. I wasn't saying you should increase the timeout just for my experience. I was trying to make a 2 prong proposal.

1) the ability for users to set per hoster timeouts

2) the ability for users to judge if there's a value to increase per hoster timeout (i.e.on timeout can tell if this hoster has long deltas per packets, as rapidgator seems to have)

but mostly working for me now, so not a huge deal
Reply With Quote
  #17  
Old 26.04.2020, 11:16
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 67,320
Default

What timeout have you set/increased it to at the moment? Also you should switch to newer Java version as you are still using heavily old Java 1.7 (seems to be very old JDownloader installation) Just rename the java/jre folder within your JDownloader folder and launcher should use the system installed java instead. Due to the age of used Java version and ongoing kernel/network changes, this might also be caused by combination of old java/modern kernel
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 26.04.2020 at 11:22.
Reply With Quote
  #18  
Old 26.04.2020, 11:24
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

120K (i.e. 2 minutes) as mentioned above for changing rapidgator and as I said, its working better. Though recaptcha has gotten really annoying of late making it difficult to use but that's not a jd2 isuse. I see the annoyance even in web browser (not logged into google) on my jd2 box)
Reply With Quote
  #19  
Old 26.04.2020, 11:36
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 67,320
Default

Thanks for the feedback!
Please try a newer Java version, eg latest 1.8 from adoptopenjdk.net
or from your OS repositories
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 01.05.2020, 20:56
thetoad thetoad is offline
Storm
 
Join Date: Jan 2014
Posts: 221
Default

ok, didn't even realize running on java8. updated all java alternatives to openjdk11, so next time I restart it should be running on that.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 14:57.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.