|
#1
07.04.2019, 13:40
|
|||
|
|||
request plugin for reddit.com (video download is missing audio)
There are a lot of subreddits and search results from reddit.com that I would like to capture, such as:
subreddit **External links are only visible to Support Staff****External links are only visible to Support Staff** search result **External links are only visible to Support Staff****External links are only visible to Support Staff** user posts **External links are only visible to Support Staff****External links are only visible to Support Staff** Can you add reddit.com account support so it will allow for showing/capturing nsfw by default, and also add scrolling support? Right now, if I add a link it captures 54 images which appear to be mostly related to the website theme, and then about 5 images which are the content I want, but they show as being "offline". Last edited by asdpoi2k; 09.04.2019 at 13:43. Reason: extra clarification 
|
|
#2
09.04.2019, 13:44
|
|||
|
|||
|
bump, did this one get missed or did I miss something? I don't think reddit is supported anywhere yet?
|
|
#3
10.04.2019, 19:32
|
||||
|
||||
|
I can think of supporting /user/ and maybe /r/ but unlikely search urls as we don't often add support for those as those can cause high cpu load and are only good/usefull for mass crawling.
We first have to check how the site works and maybe they provide a lightweight api that could be used
__________________
JD-Dev & Server-Admin
|
|
#4
12.07.2020, 16:39
|
|||
|
|||
Is there a reason Reddit videos aren't supported?
Whether I copy a v.redd.it link or a Reddit post's link, JDownloader doesn't do anything. I found a thread from 2 years ago with the progress still at 0%, but that person at least had the video and audio files even if they weren't merged.
Am I doing something wrong? I don't even see a Reddit plugin within JDownloader.
|
|
#5
13.07.2020, 12:02
|
||||
|
||||
|
@Kaegel
Merged reddit threads. The reasons are simple: - We cannot add support for every website of the Internet - A reddit plugin has been requested by serveral users but so far, not yet been created My "workaround" suggestion: Please use free browser addons such as "Video DownloadHelper" to download reddit content - you can use this addon as a helper to find the video downloadurls and download them with JD (well, in most cases). If not, you can still manually download via browser. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#6
15.07.2020, 20:57
|
||||
|
||||
|
Was planning on making a thread but noticed this is becoming the thread for Reddit, so I'll just add a reply here.
Not sure if supporting Reddit has ever been in your actual backlog but maybe this could help. A couple software that I used/use to archive Reddit /users and /r are: RipMe: **External links are only visible to Support Staff**... and BDFR: **External links are only visible to Support Staff**... Since the introduction of RedGIFs, RipMe has essentially stopped working for almost all hosts (imgur, gfycat, v.redd.it, etc) and BDFR has some troubles of its own with very poor dev support. Maybe a Reddit plugin support can be built from these two to be utilised in JD2? Not saying exactly copying them but going back to Jiaz's comment from last year about checking how the site works, I figured these applications could help provide some insight. Just spitballing here as I'm out of options when it comes to software for archiving Reddit content and was kind of hoping the people behind JD2 would be interested in filling this gap. Thank you.
|
|
#7
16.07.2020, 12:30
|
||||
|
||||
|
Hi DukeM,
we already know these other tools  Also there are still plenty of online "reddit downloaders" up and running fine. Also, you can easily add URLs of ANY video site to JD by using simple helper browser addons such as e.g. "Video DownloadHelper". Regarding reddit: Do they provide an official API? Do they mind ppl. downloading their content or do they not care? I've even seen public reddit downloader bots on reddit itself so it seems like they at least don't try to stop ppl. from downloading the content ... after all it's all user-generated anyways. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#8
16.07.2020, 22:43
|
||||
|
||||
|
Hey there, psp!
 Ah, I do have VDH but I don't think their format really works for archiving entire users or subreddits. Although, I'm not aware of any other proper full archiving software out there that works as efficiently as these two (or as these two did/do, lol). The most important bit, as far as I'm concerned, is the proper filenaming system among other things and imo, BDFR making it customisable like {DATE}{USERNAME}{POSTID} etc is a pretty neat feature. If anything though, VDH is really good at fetching HLS links. Haha. Quote:
There is also a feature where you can connect an "app" to your reddit account if you go over to **External links are only visible to Support Staff****External links are only visible to Support Staff**. Not sure what other benefits it gives yet aside from maybe accessing private/restricted subreddits your account is a member in and maybe bypassing captchas(?). Quote:
If you go over at r/datahoarder, you can even sometimes see discussions about archiving some users or communities. There are also entire reddit archivers that exist like ceddit.com, unreddit.com, removeddit.com, etc. that works like WayBack Machine and can even show you mod-deleted comments. So, I guess it'd be pretty safe to assume that Reddit doesn't care about this stuff, at least for now. And even then, most of the content are uploaded on third-party hosts (imgur, gfycat, redgifs, etc.) which is out of their jurisdiction in the first place anyway. Really hope you lot would take an interest in this. 
|
|
#9
17.07.2020, 14:22
|
||||
|
||||
|
Indeed you're right and they're really developer-friendly - seems like they do not even mind crawlers - I actually read their developer.
I've started working on it but please do not ask me for any ETA. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#11
21.07.2020, 15:40
|
||||
|
||||
|
The next update will contain the very first version of our reddit crawler
What it can do: Crawl single reddit posts: - External embedded (video) content such as e.g. gfycat - External embedded URLs e.g. "embedded" imgur.com images What it can't do (yet): - Crawl reddit-selfhosted videos and images (Yes I know a lot of content is hosted by reddit ... will work on it once I find the time ...) What it might never be able to do (unsure): - Crawling complete user profiles - Crawling complete subreddits Reasons: Requires a LOT of http requests What it will definitely never be able to do as we do not provide such crawlers: - Crawling "search query URLs" e.g.: Code:
reddit.com/search?q=nature%20images We are open source - you can contribute code at any time Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#12
22.07.2020, 16:14
|
||||
|
||||
|
Added simple support for downloading reddit selfhosted content.
Now it is up to you to test our "single posts crawler" and e.g. find posts for which JD does not find anything although you might expect it to. Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#13
23.07.2020, 11:58
|
|||
|
|||
|
how about adding TITLE from single reddit post to foldername?
that make all files much more interesting to have a name itself Quote:
|
|
#14
23.07.2020, 12:01
|
||||
|
||||
|
Quote:
As you can imagine, what I've done until now was just "quick and dirty" - it'll get better every day -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#15
23.07.2020, 12:28
|
|||
|
|||
|
"pattern" : "**External links are only visible to Support Staff**,
"rule" : "DEEPDECRYPT", "packageNamePattern" : "<title>(.+)</title>", Quote:
|
|
#16
23.07.2020, 12:45
|
||||
|
||||
|
@verheiratet1952
Your LinkCrawler rule will not work anymore once our crawler is released which it already is ... Also, we are not using the reddit mainpage e.g. Code:
reddit.com/r/de/comments/hvzlf7/falschparken_in_stuttgart/ Code:
reddit.com/comments/hvzlf7/.json -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#17
23.07.2020, 14:36
|
||||
|
||||
|
For the next update:
- Added package name including date when the content was posted and name of the subreddit - Added ability to parse multiple reddit hosted pictures of present - Added parser for post text itself: If the text of post contains downloadable URLs which JD can handle, it will find them as well now I did not yet modify the filenames for reddit selfhosted content. Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#18
23.07.2020, 15:20
|
|||
|
|||
|
great changes!
is there any chance to edit/change settings? to customize for its own needs? Quote:
|
|
#19
23.07.2020, 15:33
|
||||
|
||||
|
Quote:
Quote:
If you're talking about filenames/packagenames: You can always modify those via Packagizer rules. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
|
#20
24.07.2020, 15:43
|
||||
|
||||
|
For the next update:
- v.redd.it and i.redd.it content will now also get displayed as host "reddit.com" - v.redd.it: Always only grab the BEST video quality available - Improved offline detection Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
Quote:
|
 |
| Thread Tools | |
| Display Modes | |
|
|
Linear Mode
