JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 14.06.2024, 05:37
TGU TGU is offline
JD Fan
 
Join Date: Jun 2024
Location: International Waters where DRM/DMCA protections are ignored
Posts: 72
Default Multihoster on missing/dead files fallback to Dedicated Plugin

The multihoster should not be the source of truth for if a file is missing or link is dead, there should be a fallback option to handle the link with the next+different available multihoster and/or retry without a multihoster as a optional general application setting which is applied to all multihosters.

Thread reference: https://board.jdownloader.org/showthread.php?p=535250

Last edited by TGU; 14.06.2024 at 05:42.
Reply With Quote
  #2  
Old 14.06.2024, 12:48
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

Hi TGU,
this is actually a topic that I had discussed in detail with another (german) user just recently.

I will answer this in a short way - I recommend reading the german thread for a more detailed answer:
https://board.jdownloader.org/showthread.php?t=94812

TL;DR
- We used not to trust the offline status of multihosters
- We've changed that just some months ago and yes we are aware of the fact that this is not the best idea but there is no better solution at this moment
- It's not as straight forward as you think: There are e.g. situations in which the read status of the original host can't be determined

Suggestion are welcome.
EDIT

Ticket:
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 14.06.2024 at 13:08. Reason: EDIT
Reply With Quote
  #3  
Old 14.06.2024, 21:51
TGU TGU is offline
JD Fan
 
Join Date: Jun 2024
Location: International Waters where DRM/DMCA protections are ignored
Posts: 72
Default

Hmm yes that's quite the situation, and I imagine that's mostly dealing directly with APIs (I mostly handle extraction via html, so I really have to handle every situation that can occur).

When I scrape html for links sometimes I get an optimized image URL, **External links are only visible to Support Staff****External links are only visible to Support Staff**. Internally I will convert this to a known original format **External links are only visible to Support Staff****External links are only visible to Support Staff** and attempt to download this 2 times, and on the 3rd time it will then try the original, optimized image URL or fail.

Maybe the solution could be somewhat similar keeping track of attempts with plugin tried, or for every attempt try switching it up until x attempts (? number of supported multihosters for URL +1); If they all have the same failing result then we should just accept it.

Last edited by TGU; 14.06.2024 at 22:04.
Reply With Quote
  #4  
Old 14.06.2024, 22:13
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

Thanks for your input.

Initially the plan was to simply never trust the multihoster offline status except if the original filehost status could not be determined (e.g. because Cloudflare is blocking or it is currently offline).
We will see - this is not a high prio thing for me atm but I want to have it done too since last time I refactored existing multihoster plugins to signal the offline status but that was only the first half of the changes needed to make this work in a good way.
If you can't wait, you are more than welcome to temporarily modify the source code yourself, see:
https://support.jdownloader.org/Know...up-ide-eclipse
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 14.06.2024 at 22:24. Reason: Fixed typo
Reply With Quote
  #5  
Old 28.08.2024, 14:21
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

My idea to solve this so far would be to solve this as follows:
If a "file is offline" error happens within a multihoster, JDownloader will instead display error "Multihoster XY claims that this file is offline" with a wait time.
Screenshot:
Spoiler:



As always, there are exceptions:
If the file status was unknown before, JDownloader may trust the status of the multihoster.
EDIT
If you ask me though, JDownloader should never trust the multihosters' offline-status.
Let's say JDownloader was unable to check the file via the real multihoster due to DNS issues or even due to an ISP DNS block, then a multihoster claims that it is offline -> It would still be wrong to trust the multihosters' offline status.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 28.08.2024 at 21:20. Reason: EDIT
Reply With Quote
  #6  
Old 24.09.2024, 20:21
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 25.09.2024, 17:17
cremor cremor is offline
Linkgrabbing Monster
 
Join Date: Feb 2013
Posts: 82
Default

Quote:
Originally Posted by pspzockerscene View Post
If a "file is offline" error happens within a multihoster, JDownloader will instead display error "Multihoster XY claims that this file is offline" with a wait time.
Should this wait time also be displayed in the My JDownloader web UI? Because I can't find it. There is no seperate "ETA" column like in the client, and it isn't directly in the error message (like some other errors that have the wait time in it).
Reply With Quote
  #8  
Old 25.09.2024, 17:26
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

Afaik yes, at least if you did setup an account usage rule for the main host before.

If you are in doubt about what happened, please provide logs.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #9  
Old 25.09.2024, 17:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

Addendum:
Here is a link you can use to trigger an error with wait time:

Code:
d i r e c t h t t p : / / h t t p s : / / httpbin.dev/status/503
Remove all spaces then add it to JD and yes, it is supposed to contain "http" two times.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #10  
Old 25.09.2024, 17:44
cremor cremor is offline
Linkgrabbing Monster
 
Join Date: Feb 2013
Posts: 82
Default

I do have account usage rules set up. If I disable the multi hoster account then JDownloader shows "Skipped - account is missing".

Here is a screenshot of the error:
Screenshot 2024-09-25 164018.png

Your test link shows a completely different error.
Screenshot 2024-09-25 163904.png
Reply With Quote
  #11  
Old 25.09.2024, 20:50
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,459
Default

Looks fine

Seems like our my.jdownloader.org webinterface isn't displaying the wait time (so just use a JD with GUI to verify) but you can clearly see that while this error is present, JDownloader stays in "play" mode (= play button is still pressed so downloads are still running).
Errors with such an exclamation mark symbol in myjd are "Temporarily unavailavle -> Retry later"-errors.
This means that in such cases, a retry will happen once the wait time is over which is just want is supposed to happen in that case.

Addendum:
How did you trigger the error with multihoster high-way.me?
Do you have an example link which can be used to trigger it?
Is it, like before, a false positive so the file you were trying to download is actually online?
If so: Did you report that to the high-way.me support too?
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 16:16.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.