JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 05.01.2020, 15:25
sfadschm sfadschm is offline
Baby Loader
 
Join Date: Jan 2020
Posts: 5
Default m3u8 size increases inifinitely during download

Hey guys,
I'm trying to download some files from an m3u8 playlist, which are correctly recognized by JD2 (size ~500 MB). However, during the download, the displayed file size keeps increasing and seems to never end (I stopped the download after ~20 GB).

Example Link:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

I found an old bug report which describes the same issue but seems to have been resolved:
https://board.jdownloader.org/showthread.php?t=71750

Edit: Streaming with VLC from the m3u8 seems to be working fine.

Would be glad if you can figure this out.
Thanks and Best

Last edited by sfadschm; 05.01.2020 at 15:27.
Reply With Quote
  #2  
Old 06.01.2020, 11:51
sfadschm sfadschm is offline
Baby Loader
 
Join Date: Jan 2020
Posts: 5
Default Addition

I tried some other links again, where the same video is present in different qualities. However, downloads always continued until ~4.5 GB (although the correct sizes were displayed before starting the download).
When the download finished, the mp4 however were not playable.
Reply With Quote
  #3  
Old 06.01.2020, 14:21
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,284
Default

the size within linkgrabber is based on the the master m3u file (provided) or from ffprobe analyse of the stream which can be estimate based on bitrate or average segment size.

If you provide the website where you found master file and along with the stream quality you tested with. We can do some further testing, and provide feedback on the matter.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #4  
Old 06.01.2020, 14:46
sfadschm sfadschm is offline
Baby Loader
 
Join Date: Jan 2020
Posts: 5
Default

Thanks for the feedback!

This goes basically with every Stream from:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

I tried the same m3u8's with m3u8 and figured the following:

The playlist containing all qualities is e.g.:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

In there, every quality is basically just condensed as e.g.:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

The syntax is always: Host + /drive/hls/**id**/**id**.m3u8
If you add ".playlist" before ".m3u8" you will end up with the quality list again.

Now apparently, all stream are divided into ~30 chunks which are addressed e.g by:
/drive/hls/bf890eea38043f175ba44eb534cc024d/bf890eea38043f175ba44eb534cc024d0.html?ch=bf890eea38043f175ba44eb534cc024d-chunk-0.txt&s=21&l=1519416
Here, the ID is not included anymore. However, each chunk gets called multiple times, with different "s" (Start) and "l" (Length) parameters, hence all chunks seem to further be divided into smaller parts for download.

Downloading with m3u8x also resulted in a folder of ~4.5 GB size. The reason seems to be, that anytime one of the small parts is requested, the whole chunk was downloaded.
However, m3u8x was able to correctly only merge each chunk one time into the mp4 file (resulting in total 400 MB), while JD2 seems to have merged every small part (hence every chunk multiple times) which corrupts the mp4 file.

I guess this is the issue here.
Solution might be to tell JD2 to only download each chunk one time, so as to ignore the "s" and "l" parameters.

Hope this helps
Alex

Last edited by sfadschm; 06.01.2020 at 14:48.
Reply With Quote
  #5  
Old 06.01.2020, 15:15
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,284
Default

may I also ask which version of ffmpeg you have within your jd install path ? since JD uses ffmpeg to download HLS, the issue most likely lies within this program.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #6  
Old 06.01.2020, 15:55
sfadschm sfadschm is offline
Baby Loader
 
Join Date: Jan 2020
Posts: 5
Default

ffmpeg from the installation directory returns:

"ffmpeg version N-92112-g03123e4053 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8.2.1 (GCC) 20180813"

So seems to be an older version.
Reply With Quote
  #7  
Old 06.01.2020, 16:29
sfadschm sfadschm is offline
Baby Loader
 
Join Date: Jan 2020
Posts: 5
Default

I just retried with a newer ffmpeg version by changing the binary's path in the settings, but got the same result.

Tested version:
"ffmpeg version git-2019-11-08-e700038 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.2.1 (GCC) 20191010"

The mp4 file remains ~4 GB, but is now playable. However, it is 7 h long, because every chunk is repeated several times.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 01:07.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.