JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 21.01.2015, 05:43
notnot
Guest
 
Posts: n/a
Default new twitch.tv vod system

so recently twitch changed the vods of some channels to a hls system, meaning vods are stored in 3-4 second segments instead of 30 minute ones. is it possible for jd to download these and combine them? youtube-dl can.

here's a sample link: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2  
Old 21.01.2015, 06:36
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

Hi, we can support this but only in JDownloader 2.0, because it requires ffmpeg (same as youtube-dl) which provide the HLS library.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 21.01.2015, 13:52
Anjin Anjin is offline
Super Loader
 
Join Date: Jul 2013
Posts: 25
Default Twitch.tv plugin update?

The video links from Twitch.tv aren't grabbed anymore by my JD2. As it seems, reading this post from here http://board.jdownloader.org/showthread.php?t=61755, they changed their packing system, but that user can download something and i can't even get any links.
Test links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #4  
Old 21.01.2015, 13:56
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Merged twitch tickets and added a ticket:


GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #5  
Old 22.01.2015, 18:28
ivanpowa
Guest
 
Posts: n/a
Default

Yeps i have the same problem,

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

I copy & paste the links but nothing happens
Reply With Quote
  #6  
Old 23.01.2015, 11:35
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

support added!

NOTE: you must use JDownloader 2.0 for /v/ support

I've committed source changes, the plugin update will be available within JD2 shortly after, older versions of JDownloader you will need to wait for the next update cycle to take place. No ETA provided.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #7  
Old 25.01.2015, 06:25
Tom Bombadildo
Guest
 
Posts: n/a
Default

Figured I might as well post this here instead of making a new thread...

The updated Twitch plugin seems to be working fine for grabbing videos, however it seems it only grabs the absolute lowest quality version of the VOD. I've tested a couple of VODs so far, with each being available in up to 1080p, but each resulted in the lowest quality version. Any way we can get it to capture higher qualities?
Reply With Quote
  #8  
Old 25.01.2015, 06:47
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@Tom Bombadildo

a) decrypt returns links without knowing what they are
b) online checking determines quality, by downloading and analysing.

here is an example of all links returned from a given vod in this thread, I then downloaded them all at the same time. keep note that the filesize is hard to estimate due to variance in piece sizes and JD keeps adjusting this whilst downloading.



The only way I can see you doing this is, that you never edited your custom filename in the plugin. There are new filename tags for VOD (when vod isn't used it will be removed from your custom tag). The new variable filename tags so you can determine which video added is the one you want! without doing this you will have trouble identifying.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]

Last edited by raztoki; 25.01.2015 at 06:50.
Reply With Quote
  #9  
Old 25.01.2015, 15:21
ivanpowa
Guest
 
Posts: n/a
Default

>_< Cant download

Reply With Quote
  #10  
Old 25.01.2015, 15:47
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Log please.

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #11  
Old 25.01.2015, 16:14
ivanpowa
Guest
 
Posts: n/a
Default

Trying...

25.01.15 15.08.41to25.01.15 14.17.01 jdlog://9296733212241/
Reply With Quote
  #12  
Old 25.01.2015, 16:15
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

log and example url.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #13  
Old 25.01.2015, 16:45
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

ok created different filename settings for /v/ vod in plugin settings. this will provide defaults and clear up filenames with all the same name.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #14  
Old 25.01.2015, 17:07
Tom Bombadildo
Guest
 
Posts: n/a
Default

Looks like you were right, I did change the filename tags before trying to download but I had a typo so it didn't go through lol...

I checked for an update, since you changed the plugin just now, and now I'm getting an error when trying to download the 1080p version of videos. Log file is below:

25.01.15 10.00.12to25.01.15 10.00.29 jdlog://3407733212241/

I also got an error code as well:


And this is the link I tried: **External links are only visible to Support Staff****External links are only visible to Support Staff**

720p seems to work fine though.
Reply With Quote
  #15  
Old 25.01.2015, 17:30
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

Sorry I cant reproduce using 1080.
raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #16  
Old 26.01.2015, 03:18
Anjin Anjin is offline
Super Loader
 
Join Date: Jul 2013
Posts: 25
Default

After these changes were made i notices that:
1. for me, the download speed had decreased comparing to the old system - that's because JD2 uses ffmpeg codec while downloading to convert the file?
2. now the downloads aren't resumable anymore. Is something that you guys can do to make them resumable again?
Reply With Quote
  #17  
Old 26.01.2015, 03:29
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@Anjin
/v/ (vods. as they are calling it) videos are provided by a format called HLS, this is segment downloads it saves video streamer money because they are small parts and not much traffic is wasted. the down side to it is lots of http requests are needed to be sent vs the traditional 1.

We are using ffmpeg for HLS library via local proxy. HLS downloads under our current implementation are not resumable. Though I think that could be changed easy enough, just keep record of the segment that has been downloaded and continue.

Download speeds will vary because they can be rate limited by the hoster (speed slightly faster than time within segment.)
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #18  
Old 26.01.2015, 03:54
migrena
Guest
 
Posts: n/a
Default

There seems to be still something wrong with filename syntax. In current version it looks like the plugin ignores the settings for hls videos and always uses something like:
Code:
*videoname* _ *videoQuality*_*videoCodec*-*audioBitrate*_*audioCodec**ext*
Previous version also had some problems with naming syntax, eg using *date* would brake *videoname* tag resulting in inserting "videoname" string instead of actual video name.
Also I don't see much point in using *partnumber* in syntax for hls videos because there will always be only one part, right?

Lastly - sometimes the resolution, codec and bitrate are not enough to let user determine which file is the one he wants to download. The information provided might be outright misleading eg parsing that link
**External links are only visible to Support Staff****External links are only visible to Support Staff**
yields two files at 720p with two different audio bitrates, and surprisingly the one with higher bitrate is the one with lower quality. Would it be possible to add video bitrate tag as well?

PS
Parsing and downloading 1080p videos from twitch seems to be working fine for me, but I encountered cases where some of the qualities were missed or marked as faulty (it could not detect resolution etc) by decrypter.
Reply With Quote
  #19  
Old 26.01.2015, 11:54
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

I don't use partname for this either, and not having any issues. Partnumber was left in because it will continue the same pattern as previous videos even if there is just 1 part, which was the behaviour with the other format. I didn't try with date, as I don't use it either, but I made sure the data reference was stored.

I'll see about video bitrate, it's most likely possible.

re: could not detect resolution, can please paste links for those. Decrypter isn't actually doing any of the variable tags for /v/, at this given time! Decrypter only gets the required HLS component and returns the many results. Download linkcheck routine returns information and returns filename (Downloads a part of the file and ffprobe gives feedback), so under that proviso it would be ffprobe which is returning incorrect feedback.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #20  
Old 26.01.2015, 21:05
migrena
Guest
 
Posts: n/a
Default

I'm not sure if this will have the correct data but thats the newest log jd shows. hmm...
Code:
26.01.15 19.31.17to26.01.15 19.40.04 jdlog://8489733212241/
Anyway here is the screenshot of some of the faulty links
http://www.imagebam.com/image/642092384788562
It seems that generating too many request at the same time causes problems - I have feeded clipboard with 16 links in a few seconds and that's the result.

And about that naming syntax - this time I was using
Code:
*date* - *partnumber* - *videoname* -*videoQuality*_*videoCodec*-*audioBitrate*_*audioCodec**ext*
so filenames should include date. But they don't. But with the newest version of the plugin it seems that "date" tag doesn't brake anything. Maybe because it does nothing.
Reply With Quote
  #21  
Old 27.01.2015, 01:08
GreenGT
Guest
 
Posts: n/a
Default

Hello. Installed JD 2, but cant download video from Twitch.TV with HLS system. What I'm doing wrong.

My log - 27.01.15 00.33.46to27.01.15 00.42.51 jdlog://2630833212241/

Screenshot

Reply With Quote
  #22  
Old 27.01.2015, 03:58
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@migrena
I might need to add synchronized lock to linkchecking to only allow one at once.

@GreenGT
did you get a prompt to install ffmpeg?
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #23  
Old 27.01.2015, 12:05
GreenGT
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by raztoki View Post
@GreenGT
did you get a prompt to install ffmpeg?
Dont know How to install ffmpeg?
Reply With Quote
  #24  
Old 27.01.2015, 13:38
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

as I said, it should 'prompt' you to install as in provide you with confirmation dialog and JDownloader will install for you!
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #25  
Old 28.01.2015, 13:09
GreenGT
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by raztoki View Post
as I said, it should 'prompt' you to install as in provide you with confirmation dialog and JDownloader will install for you!
Well. In jD 2 i have ffmpeg plugin, but I havent special 'prompt' to install this. Just installed programm and all
Reply With Quote
  #26  
Old 29.01.2015, 01:25
migrena
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by raztoki View Post
@migrena
I might need to add synchronized lock to linkchecking to only allow one at once.
Well, now that I know what exactly triggers that problem I can just work around it. But yes, preventive measures might be nice.

I'm still more interested in being able to set my own naming pattern.

Also - do I need to start new thread for adding videobitrate to the list of available tags or requesting in this thread is enough?
Reply With Quote
  #27  
Old 29.01.2015, 14:35
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@migrena
linkchecking in hoster plugins only does one at a time, so synchronized lock wont be needed.
also bitrate of video is not possible, I just checked ffprobe value in eclipse variable the value is null. I'd say that is common due to most video codecs having variable video bitrates (allows stills been less, and fast action scene having more). You would need to download the entire video to determine true bitrate (or an average). So new thread isn't needed.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #28  
Old 01.02.2015, 21:28
ToastMan
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by GreenGT View Post
Hello. Installed JD 2, but cant download video from Twitch.TV with HLS system. What I'm doing wrong.

My log - 27.01.15 00.33.46to27.01.15 00.42.51 jdlog://2630833212241/

Screenshot

I've had the same problem!

I fixed it by downloading ffmpeg manually from the ffmpeg webite (can't insert links, just google it)

Just extract its contents anywhere you like.

Then in JD:
Go to Settings -> Advanced Settings.
Look for FFmpegSetup: Binary Path and FFmpegSetup: Binary Path Probe.
For each of those replace 'null' with the correct path to the files ffmpeg.exe and ffprobe.exe respectively. You have to add the file names to the path.

Last edited by ToastMan; 01.02.2015 at 21:31.
Reply With Quote
  #29  
Old 14.02.2015, 11:07
velffula
Guest
 
Posts: n/a
Default

Quote:
I had the same issue and found the best way is to have avconv/ffmpeg write the necessary header information right away during recording thus avoiding currupt files. This is done by the -f ismv flag, also consider adding -movflags frag_keyframe. This way, if the recording is interrupted, the file will not be corrupt.
Twitch.tv often disconnects and makes the video file corrupted. So I found this maybe to fix the problem? Put where do I add these flags?

Also can there be an option to split video in segments or choose time range? Because there is often very much muted parts and unmute doesn't work e.g. if the start of the video is muted, then the whole video will be muted.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 00:48.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.