JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 07.08.2018, 06:59
plip plip is offline
Wind Gust
 
Join Date: Jun 2016
Posts: 40
Default Twitch - in acount manager says "plugin outdated", multi-part download questions

I noticed today that in the Twitch plugin settings, under Account Manager it has my account enabled, but right underneath that it says "Status Plugin Outdated".

I'm not seeing any actual problems from this as far as I can tell so this is very minor. But it does continue to say plugin outdated in that spot even right after updating JD2, so it seems that's showing up erroneously.

I also have a question about the way Twitch downloading recently changed. Now it seems to download in multiple parts and then has to merge when everything finishes. That's despite having "max chunks per download" set to 1.

(Unless that was changed back in the latest update. I haven't had time for a download to complete since updating and I need to sleep soon.)

This multiple-part change to twitch downloading worries me. (Sorry for the long explanation that follows).

My net was unreliable for an extended period of time recently, which means I have lost net for periods of time during long a twitch download a bunch of times recently. But if the net wasn't down too long the download would resume and end up completing.

When that happened, JD2 would often think the download had stopped at close to complete, but not fully complete.

The result of this would be a completely playable file, however. All I had to do was make sure JD2 wouldn't retry to download the whole thing again right away after it stopped (by moving other downloads higher in the list). Then I could disable the net-hiccup download after it stopped, so it wouldn't get retried at all. Then just rename the file to get rid of ".part" at the end.

Doing these steps when there's a net hiccup during a twitch download will almost always result in a completely playable file, which seems to include the entire stream, as long as the net wasn't down for a long time.

What I'm worried about is that this all relies on there being one part to the download.

I haven't had any net issues since the recent change to multi-part-downloading and merging at the end.

But I don't think I would be able to salvage a download after a net hiccup anymore because of the multi-part downloading that's done now. I mean, I don't think that JD2 would merge a download that it thought stopped at not quite incomplete. Correct?

So I'm wondering a few things.

Would the multiple part files be usable if a net hiccup caused what I described (a download which JD2 thought was almost-but-not-quite complete when it stopped)? I mean, would renaming them to mp4 result in a playable file? Or do twitch downloads now need to be merged to be playable?

I'm also wondering, what is the particular reason for this multi-part download change? I don't know what the advantage is supposed to be.

From what I can see, it seems to make things more complicated, and add a gap of time where nothing is downloading until the merge finishes (if you have max downloads at a time at 1 because of slow net). And when the merging is going on it's very intensive.

I'm having trouble thinking of an advantage in that. So the multi part thing confuses me.

Could there, please, be a setting added to the twitch plugin to let you choose not to do multi-part downloading? I would strongly like to avoid that high-intensity merging, and it would simplify salvaging a download when a net hiccup happens too.

Last edited by plip; 07.08.2018 at 07:03.
Reply With Quote
  #2  
Old 07.08.2018, 09:54
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Twitch Login is broken at the moment, and
That means that Streams that require being subscriber or an account won't work at the moment.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 07.08.2018, 09:59
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Quote:
Originally Posted by plip View Post
I also have a question about the way Twitch downloading recently changed. Now it seems to download in multiple parts and then has to merge when everything finishes. That's despite having "max chunks per download" set to 1.

(Unless that was changed back in the latest update. I haven't had time for a download to complete since updating and I need to sleep soon.)

This multiple-part change to twitch downloading worries me. (Sorry for the long explanation that follows).
Twitch uses HLS(video/stream is split in many many small parts) since the beginning and JDownloader uses ffmpeg to merge those parts into one file. Resume has never been supported for HLS downloads and it always has been 1 part at a time.
So nothing has changed at all for Twitch support.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 07.08.2018, 10:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Quote:
Originally Posted by plip View Post
My net was unreliable for an extended period of time recently, which means I have lost net for periods of time during long a twitch download a bunch of times recently. But if the net wasn't down too long the download would resume and end up completing.

When that happened, JD2 would often think the download had stopped at close to complete, but not fully complete.

The result of this would be a completely playable file, however.
JDownloader checks if all parts has been downloaded and only in that case the download will be marked as finished.
I need logs to check what exactly has happened, see https://support.jdownloader.org/Know...d-session-logs
Some routers/modems responds with *fake* http responses and those could be interpreted as *broken* segments by ffmpeg.
I need logs to tell more.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 07.08.2018, 10:05
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Quote:
Originally Posted by plip View Post
I haven't had any net issues since the recent change to multi-part-downloading and merging at the end.
Some streams are split into *multiple* playlists (eg streamer restarts the stream, changes settings...) and in this case JDownloader
downloads each playlist on its own (still HLS) and in the end merges all playlists back into one large file.
Maybe you're talking about this? Please provide example links, then I can tell if the stream has one or more playlists.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 07.08.2018, 10:07
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Quote:
Originally Posted by plip View Post
But I don't think I would be able to salvage a download after a net hiccup anymore because of the multi-part downloading that's done now. I mean, I don't think that JD2 would merge a download that it thought stopped at not quite incomplete. Correct?
I need logs. Then I can check what has happened and why JDownloader *thinks* that the download is finished
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 07.08.2018, 10:10
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Quote:
Originally Posted by plip View Post
I'm also wondering, what is the particular reason for this multi-part download change? I don't know what the advantage is supposed to be.
From what I can see, it seems to make things more complicated, and add a gap of time where nothing is downloading until the merge finishes (if you have max downloads at a time at 1 because of slow net). And when the merging is going on it's very intensive.
In case we're talking about the *multiple* playlist support, that one exists since LONG time and not every twitch stream has them. As explained above, it may happen if the streamer changes settings /restart streams/ reconnects....

Quote:
Originally Posted by plip View Post
Could there, please, be a setting added to the twitch plugin to let you choose not to do multi-part downloading? I would strongly like to avoid that high-intensity merging, and it would simplify salvaging a download when a net hiccup happens too.
I'm sorry but without that support, you cannot download those streams.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 07.08.2018, 10:11
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Please provide example links and logs, see https://support.jdownloader.org/Know...d-session-logs
and post the shown logID here. Create log after you could reproduce the issue.

Please provide example links and I can do some testing. Do you know if the network was just unreliable (short offline time) or longer offline time?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 09.08.2018, 05:52
plip plip is offline
Wind Gust
 
Join Date: Jun 2016
Posts: 40
Default

Quote:
Originally Posted by Jiaz View Post
Some streams are split into *multiple* playlists (eg streamer restarts the stream, changes settings...) and in this case JDownloader
downloads each playlist on its own (still HLS) and in the end merges all playlists back into one large file.
Maybe you're talking about this? Please provide example links, then I can tell if the stream has one or more playlists.
Oh! I'll bet that is what was happening. I was seeing some weird things like a part1 more than a GB, then a part2 a few MB, then the next part more than a GB again. After seeing what you said there, that was probably the streamer messing with something mid-stream.

I haven't been seeing the multi-part-download and merging thing again since I posted. And there are two streamers I watch who were having streaming issues for some days lately, so it very well could have been from them trying to figure things out mid-stream. I just had no idea that can cause a single stream download to have multiple parts which then get merged. That's something I had never seen before.

And then suddenly seeing it happen a number of times, I mistakenly got the idea something about twitch downloading in JD2 had been changed so every twitch download would be multi-part and merging at the end. Very glad to realize that was completely wrong

Here's a link I'm 99% sure was one of the ones I saw do the merging at the end, if you still want a link to check:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

That vod will probably get auto-deleted in a little over a week, btw, so you don't try to check it after it's gone.

Quote:
Originally Posted by Jiaz View Post
Do you know if the network was just unreliable (short offline time) or longer offline time?
My ISP was having net outages in my area (not problems specifically at my place) several times within a little over a week, and then for several days the net was stalling occasionally (for much shorter periods of time - maybe roughly five to ten minutes on average I'd say).

I didn't think to make logs from any of the downloads affected by that, because I thought (due to unreliable internet causing the downloads to behave strange) that it wasn't really a JD2 issue.

But now I realize that I should have made logs for those. It might be a while before anything like that happens (hopefully, cause it would mean my net's having issues again). But I'll post logs here if I see net issues cause download completion to act strange again.
Reply With Quote
  #10  
Old 09.08.2018, 11:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

Yes, those streams contain "#EXT-X-DISCONTINUITY" (in case you want to searchengine for it) and JDownloader will download the multiple parts and later join them to one file.

Thanks for the feedback and example link. I will do some more testing and make sure that issues are detected/handled correctly.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 09.08.2018, 11:31
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,918
Default

In case you encounter this issue again, please provide logs Thanks in advance!
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 06:13.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.