JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 06.09.2021, 21:53
lezboyd lezboyd is offline
JD Beta
 
Join Date: Nov 2011
Posts: 58
Default vipr.im dropping connections while downloading

I am using JDownloader2 (JD2) to download imagesets from vipr.im.

Things worked smoothly in the past, but recently I have noticed a high level of dropped connections while downloading (links load into JD2 just fine, but errors occur after starting download). Errors thrown include "Final Destination Link is not an image" , "Try Restarting this link again" , "Unknown Host error" (sic). All of these are usually solved by resetting the download link. However, if you are downloading a set of 120 pics and this keeps occurring, you are basically playing whack-a-mole within JD2, instead of an unattended download.

I am sure this is server-side, not because of me or JD2, since (a) all other hosts work fine, (b) this started happening recently with vipr.im, which was usually pretty stable in the past.

Anyone know why this may be happening, or facing the same issue?
Reply With Quote
  #2  
Old 07.09.2021, 10:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,531
Default

Please provide example links and a log, see
https://support.jdownloader.org/Know...d-session-logs
and post shown logID here. Wait for error to happen and then create the log
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 07.09.2021, 17:18
lezboyd lezboyd is offline
JD Beta
 
Join Date: Nov 2011
Posts: 58
Default

Here is the log:

07.09.21 20.30.10 <--> 07.09.21 20.47.00 jdlog://1457825302851/

Example Links: Downloaded this whole set:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #4  
Old 07.09.2021, 17:33
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,531
Default

@lezboyd: Cloudflare protection kicks in and causes the issue
see https://board.jdownloader.org/showthread.php?t=83712
There is not much you can do about. I will set a request limit in plugin, maybe that helps
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 12.09.2021, 14:00
lezboyd lezboyd is offline
JD Beta
 
Join Date: Nov 2011
Posts: 58
Default

Update: I am no longer getting errors while downloading from vipr.im. I suppose cloudflare was kicking in and dropping concurrent connections. Now that you have limited request limit to one at a time, at least the downloads are now unattended and without errors.

Thanks. Issue Solved indeed.
Reply With Quote
  #6  
Old 13.09.2021, 14:19
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 59,235
Default

Thanks for your feedback!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #7  
Old 25.09.2021, 16:58
lezboyd lezboyd is offline
JD Beta
 
Join Date: Nov 2011
Posts: 58
Default

Update 2 : The "one file at a time" change has got rid of the connection dropping while downloading, for sure, but it has also had the side-effect of loading links into LinkGrabber excruciatingly slow.

Is there any way the links can be made to load faster into Linkgrabber, while downloading can still happen "one at a time"? There was never any connection dropping while loading into LinkGrabber.
Reply With Quote
  #8  
Old 27.09.2021, 15:05
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,531
Default

@lezboyd: can you provide some example links/galleries with vipr.im links? then we can check/update the plugin
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 28.09.2021, 12:40
lezboyd lezboyd is offline
JD Beta
 
Join Date: Nov 2011
Posts: 58
Default

Test case: I loaded one gallery into the Linkgrabber, then started downloading it. While that gallery was being downloaded in the Downloads tab, I loaded 5 more of these galleries in LinkGrabber tab, using the Add New Links option, with about a min in between each gallery (the time taken to copy the links from the browser and then using the add new links option in JD2, etc.)

It took 23 minutes to load all the links from the first gallery of these 5 galleries into Linkgrabber while a download was ongoing in the downloads tab, and multiple galleries were loaded in Linkgrabber.

Gallery Links:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #10  
Old 28.09.2021, 15:22
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 59,235
Default

I don't see a way to check them without running into rate-limits again.
Usually such XFS websites have a "mass linkchecking" feature which can be used to check up to 50 links at the same time but theirs seems to be broken:
vipr.im/?op=checkfiles

You could go ahead and disable linkchecking globally whenever you want to add a lot of those URLs:
Settings -> Advanced Settings ->
Code:
LinkCollector: dolinkcheck
--> Disable this

You should not permanently leave this disabled as JD will not look for filename/size/status as long as this is enabled.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #11  
Old 28.09.2021, 15:29
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,531
Default

@pspzockerscene: I was able to speedup and reduce the number of requests by half
@lezboyd: wait for next plugin update, you should notice a significant/time by half speed up
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 28.09.2021, 15:44
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 59,235
Default

Quote:
Originally Posted by Jiaz View Post
@pspzockerscene: I was able to speedup and reduce the number of requests by half
Thanks indeed I've overlooked that...
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 02:38.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.