#1
|
|||
|
|||
missav.com - download HLS URLs with referer
I got a link to stream from Video DownloadHelper but jDownloader doesn't recognize it. For example
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#2
|
||||
|
||||
Working on it
__________________
JD-Dev & Server-Admin |
#3
|
||||
|
||||
@UriF: you will be able to add the m3u8 with next core update, but you also have to add
Quote:
__________________
JD-Dev & Server-Admin |
#4
|
|||
|
|||
Sorry, what would I have to encode or decode? bcdn_token? token_path? Could write a original string and converted string, for example... Thank you in advance
P.S. I downloaded using Ant Downloader but I got a strange effect. The length of resulted video were 2-4 min greater than original (for 4 hrs video - 4 min, for 2 hrs video - 2 min) Last edited by Jiaz; 30.06.2022 at 10:02. |
#5
|
||||
|
||||
@UriF: You have to wait for next core update (not yet released) first, else the m3u8 link won't work at all. But as this site also checks for correct referer, you have to add https://board.jdownloader.org/showpo...91&postcount=3 at the end of the url before copying to JDownloader to tell JDownloader what referer to use.
__________________
JD-Dev & Server-Admin |
#6
|
||||
|
||||
Quote:
Quote:
__________________
JD-Dev & Server-Admin |
#7
|
|||
|
|||
I tried this referrer for a video
**External links are only visible to Support Staff****External links are only visible to Support Staff** but it doesn't work. How did you get a referrer for any video at this site? Thanks. I just want to try to get it for myself. |
#8
|
||||
|
||||
@UriF: Those m3u8 DO NOT work right now. You will have to wait for next core update.
AFTER next core update, just append https://board.jdownloader.org/showpo...91&postcount=3 at the end of the url and copy to clipboard
__________________
JD-Dev & Server-Admin |
#9
|
|||
|
|||
Thanks
P.S. I got a solution: ffmpeg -headers "Referer: **External links are only visible to Support Staff** -i "**External links are only visible to Support Staff** -c copy -bsf:a aac_adtstoasc test.mp4 Last edited by UriF; 01.07.2022 at 04:14. |
#10
|
||||
|
||||
@UriF: yes, that's what JDownloader would do as well but the & within the URL is triggering a bug and therefore you'll have to wait for core update. the Referer is set via forcedreferer and hex encoded
__________________
JD-Dev & Server-Admin |
#11
|
||||
|
||||
Bitte auf das nächste CORE-Update warten!
Please wait for the next CORE-Update! Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#12
|
|||
|
|||
Thank you anyway. I downloaded everything that I wanted from this site using this ffmpeg script with referrer
|
#13
|
||||
|
||||
Nothing to thank us for! You helped to find/fix a bug in JDownloader
__________________
JD-Dev & Server-Admin |
#14
|
|||
|
|||
Thank you to give me a hint (referrer) where to search. There are some web sites which follow the same rules. I will use this method FOR now until Ant Downloader would fix their bug which I mentioned above (if jDownloader of course would be able to implement for all of sites which have the same restrictions).
|
#15
|
||||
|
||||
Please first wait for the update and then provide more example URLs if you find more similar websites which JD fails to download streams from.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#16
|
|||
|
|||
1. **External links are only visible to Support Staff****External links are only visible to Support Staff**
I can get a link to stream but I get only first 3mb (ts files misnumbered???) I tried using Google development tools but I was able to see a lot ts file with the same name sample: **External links are only visible to Support Staff****External links are only visible to Support Staff** 2. **External links are only visible to Support Staff****External links are only visible to Support Staff** Video are not recognized neither VDH no Ant Downloader sample: **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#17
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin |
#18
|
|||
|
|||
Re. 2. It interesting that some browsers let you a download button. When I clicked on it I got
**External links are only visible to Support Staff****External links are only visible to Support Staff** I clicked on it and it asked me to register. After I registered they asked me to pay something to download. I tried Firefox Development tool but I has been kick off from this web page. By the way I found out how to download from hellabyte but I can get only 1080p resolution file P.S. There were this button on Dissenter and PaleMoon. Now this button disappeared on Dissenter. Palemoon stopped to play this file at all (hourglass forever) P.P.s. I believe that jable.tv and javrip.net can be used with the same technology like missav.com. If you are interested with yt-dlp (like ffmpeg) I can post here a sample. Last edited by UriF; 24.07.2022 at 22:26. |
#19
|
|||
|
|||
By the way I wanted to update ffmpeg version (or check if it is current). I opened a directory
C:\Users\MyUserName\AppData\Local\JDownloader 2.0\tools\Windows\ffmpeg\x64 I found out that files are TOO old. Moreover you use a download link to **External links are only visible to Support Staff****External links are only visible to Support Staff** This site is dead. Therefore you are not able to update old files and user is not able to get all of bug fixes |
#20
|
||||
|
||||
Quote:
You can use any FFmpeg build you like from any source you like as long as it's not super outdated nor some kind of special fork of FFmpeg.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#21
|
|||
|
|||
I apologize but the latest ffmpeg build from this site - 4.3. The current version of ffmpeg - 5.1
|
#22
|
||||
|
||||
I'm sorry but in this thread Jiaz has explained why there is no need to constantly update FFmpeg.
As explained, if you want to use the latest FFmpeg version or any other version apart from the one we deliver with JD, you are free to do so.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#23
|
|||
|
|||
Sorry, l don't know how to access pro settings. I see a standard (???) GUI with following icons: General, Reconnect, Connection Manager and so on. Now I don't see a lot of jDownloader options which were available in previous versions.
P.S. Google translate translated from German "Pro Settings" instead of "Advanced Settings". By the way which version of ffmpeg version to be used - ffmpeg-release-full-shared or ffmpeg-release-full? I didn't create a path to ffmpeg in Environmental Variables. Could I create adirectore somewhere where I will copy the newest version of ffmpeg and just replace a path in jDownloader Advanced Settings with mine or I will have to create a new Environmental variable? Last edited by UriF; 27.07.2022 at 02:20. |
#24
|
||||
|
||||
Quote:
Quote:
You don't need to do that. Quote:
Either place FFmpeg in the JDownloader folder where the current version also sits (overwrite it) or manually change the location in the advanced settings (not recommended), see the following advanced settings: FFmpegSetup.binarypathprobe and: FFmpegSetup.binarypath
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#25
|
|||
|
|||
Thanks
|
#26
|
||||
|
||||
@UriF:
I will update bundled ffmpeg version as soon as I find some time for it
__________________
JD-Dev & Server-Admin |
#27
|
||||
|
||||
Doesn't really matter. Shared just requires less memory because both binaries share the same library files while the full ones are larger but *normally* don't depend on additional library files
__________________
JD-Dev & Server-Admin |
#28
|
|||
|
|||
Sorry. I tried but it doesn't work. I'm able to download using ffmpeg directly using referrer. I don't know how to add referrer
|
#29
|
||||
|
||||
You mean you're adding m3u8 and want to specify the referer? Or do you mean something else?
__________________
JD-Dev & Server-Admin |
#30
|
|||
|
|||
Yes, you are right. I tried missav by adding link has been taken from Video Download Helper or The Stream Detector but I got in jDownloader link grabber something like "file off-line" t.e. typical j Downloader message. I tried the same link with referrer at ffmpeg and single threaded (one connection) yt-dlp and downloaded video without any problems. Therefore the problem in jDownloader that I didn't pass a referrer. The Stream Detector is a more simple tool than VDH but it generate a script for ffmpeg and yt-dlp.
Last edited by UriF; 04.09.2022 at 19:55. |
#31
|
|||
|
|||
Quote:
|
#32
|
||||
|
||||
@UriF: thanks for clarification. can you please provide example links and referer to support@jdownloader.org then I will look into it and add a way to set custom referer
__________________
JD-Dev & Server-Admin |
#33
|
|||
|
|||
That's samples for ffmpeg (for yt-dlp the similar code):
ffmpeg -headers "Referer: **External links are only visible to Support Staff** -i "**External links are only visible to Support Staff** -c copy -bsf:a aac_adtstoasc test.mp4 ffmpeg -headers "Referer: **External links are only visible to Support Staff** -i "**External links are only visible to Support Staff** -c copy -bsf:a aac_adtstoasc test.mp4 yt-dlp --add-header "User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" --add-header "Referer:**External links are only visible to Support Staff** "**External links are only visible to Support Staff** As a rule I never put a reference to browser user-agent but maybe it is necessary when it req'd browser's cookies (get session variables) P.S. The Stream Detector generate link with referrer and user-agent for ffmpeg and yt-dlp. I believe that the easiest way to implement it in GUI just add a text box into Analyze and Add Screen in Link Grabber Last edited by UriF; 05.09.2022 at 15:52. |
#34
|
||||
|
||||
Quote:
Anyways depending on how you are copying URLs JD can also already auto find the referer e.g. if you copy a URL on a page. I guess when copying it using browser addons that reference won't be there so exactly such cases are hard/impossible to auto handle in JD atm.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#35
|
|||
|
|||
Yes, you are right but if you get a link from Video Download Helper it contains just a link to stream/file. If I copy/paste it in jDownloader link grabber it will not contain a referrer. As a rule a referrer is a link to video hoster site or link to a web page where a video is located. That's why I gave you sample links are generated by Stream Detector. You have 2 choices:
1. Link grabber will accept all of link is generated by Stream Detector including referrer or (if it is necessary) user-agent. 2. Accept direct link generated by Visual Download Helper and add a text box for referrer. 3. There is an opportunity to create jDownloader plug-in which automatically recognize a link to stream/file and copy this link with a link web page to video as a referrer but I am afraid it is too much job for jDownloader. This application don't intercept a stream but use like ffmpeg or yt-dlp because it is possibly to recognize a video when user start to watch a video. P.S. By the way I got an interesting response from Ant Downloader developers r.e. javhdworld.com | soe-681 Quote:
|
#36
|
||||
|
||||
@UriF: Can you provide complete examples of 1 and 2 ? Maybe we can add support via simple parser
It's a wrong answer, the site uses normal hls streaming but the urls are missing known file extension and most likely therefore their detection fails to find it. Same reason you need to look more accurate in developer mode of browser to find the stream
__________________
JD-Dev & Server-Admin |
#37
|
||||
|
||||
@Jiaz
Depending on the case he could also use a LinkCrawler REWRITE rule to make that link get picked up by JD.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#38
|
|||
|
|||
1. **External links are only visible to Support Staff****External links are only visible to Support Staff**
or **External links are only visible to Support Staff****External links are only visible to Support Staff** 2. Under a large text box add a standard text box for referrer. You can also add a text box for an user-agent Last edited by UriF; 06.09.2022 at 18:03. |
#39
|
||||
|
||||
@UriF: the first url is missing the text referer? did you remove it or why first one is missing it while second one does have it?
__________________
JD-Dev & Server-Admin |
#40
|
|||
|
|||
No. It is the same link. I just joined the link to stream/file and referral through character "|" as a separator
https//"link to stream"|https"link referral" or https//"link to stream"|Referral:https"link referral" or https//"link to stream"|https"link referral" or https//"link to stream"|Referral:https"link referral"|user-agent:"user agent" Whatever you will chose a format it is up to you. If you don't want to change GUI and add a text box(s) the "|" as a separator would be the best chice |
Thread Tools | |
Display Modes | |
|
|