#1
|
|||
|
|||
How to AUTOMATICALLY remove the remaining files showing in the LinkGrabber tab?
How to AUTOMATICALLY remove the remaining files showing in the LinkGrabber tab for a particular URL, ONCE the desired file has been added to the Downloads tab?
For instance, when pasting a Youtube link into jDownloader, it shows many files in LinkGrabber. I Select the video (.mp4) to download and would like that ONCE I do that, the remaining files get AUTOMATICALLY removed from the LinkGrabber tab (together with their parent folder containing such files; I mean, to automatically delete the full URL containing all such remaining files that were not selected). I know that I can delete such files manually, but an automatic feature would be great. I am using the latest version of jDownloader on macOS 12.5.1 (21G83) Monterey on Intel x86 Mac. Thanks! Last edited by jDoX; 26.08.2022 at 13:36. |
#2
|
||||
|
||||
@jDoX: You can setup the yt plugin in Settings->Plugins->youtube.com and there you can disable the other types (images, subtitle...)Then you don't have to clean them up.
right now there is no support to auto remove the remaining files
__________________
JD-Dev & Server-Admin |
#3
|
|||
|
|||
Many thanks!
|
#4
|
|||
|
|||
Further to my previous question above, how to do it (AUTOMATICALLY remove the remaining files showing in the LinkGrabber tab) when the link does not belong o any of the servers shown at "jDownloader - Preferences - Plugins - Plugins - Please choose a plugin"? It is a blogspot.com server which has embedded videos.
Thanks! Last edited by jDoX; 19.09.2022 at 11:43. |
#5
|
||||
|
||||
@jDoX: I'm sorry but I don't understand. Maybe you can provide screenshot/example links and explain what exactly you want to achieve?
__________________
JD-Dev & Server-Admin |
#6
|
|||
|
|||
Thanks for the quick reply. For instance, the following blogspot page
MACOS ON WINDOWS **External links are only visible to Support Staff****External links are only visible to Support Staff** contains two videos at the bottom of the page. Yet, pasting such blogspot link in jDownloader shows many files not wanted. I only want the ".mp4" movies in such pages. I have tried the procedure above, but many different files are shown in the LinkGrabber tab of jDownloader. |
#7
|
||||
|
||||
@jDoX: for example you can setup a Settings->Linkgrabber Filter
Quote:
You can either use Linkgrabber Filter rules or setup LinkCrawler Rules, see https://support.jdownloader.org/Know...kcrawler-rules and *teach* JDownloader how to handle unsupported websites and what content you're interested in
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 21.09.2022 at 20:15. |
#8
|
|||
|
|||
Thanks for the information. It would be great if jDownloader could support such site in the future.
|
#9
|
||||
|
||||
@jDoX: *such site* Blogspot is a blog, how should JDownloader know what content you are interested in
Your example link contains several images and videos, so you still would have to setup a filter anyway You can easily setup your own Linkcrawler Rules for such sites/cases and *tell* JDownloader what content you want to download from them
__________________
JD-Dev & Server-Admin |
#10
|
|||
|
|||
Thanks. What I meant is that in the same way that jDownloader works with Youtube to download all content, or just ".mp4" videos as I want (or whatever selected in Plugins), it would be great if jDownloader had also a plugin for blogspot. Then, it should be possible to do what I described.
Amazingly, jDownloader has 3799 plugins already, which is remarkable. Adding another one for blogspot would make jDownloader even more fantastic. |
#11
|
||||
|
||||
@jDoX: I don't think that there will be a out-of-the-box blogspot plugin because blogspot is too generic as it is free to the writer how to include content and where this content is located. there is no real way to differ between the actual content /eg images you want to download or images that just part of the website.
If you just want to download videos from blogspot, the linkgrabber filter from here https://board.jdownloader.org/showpo...62&postcount=7 should do just fine
__________________
JD-Dev & Server-Admin |
#12
|
|||
|
|||
Thanks, although I am lost when code is involved. I wonder if someone could do a GUI for it.
|
#13
|
||||
|
||||
@jDoX: I'm sorry but I don't understand? I already provided the solution for generic blogspot. the rule will filter out any non video files when you add blogsport links.
You don't have to code anything, just setup the rule from here https://board.jdownloader.org/showpo...62&postcount=7
__________________
JD-Dev & Server-Admin |
#14
|
|||
|
|||
Thanks. By code I meant programming language, script or similar.
Such link goes to Knowledgebase : LinkCrawler Rules https://support.jdownloader.org/Know...kcrawler-rules And there I do not know what to do. Sorry for my ignorance. I do not know if that is possible, but a 1-2-3 step procedure for newbies would be most useful. |
#15
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 21.09.2022 at 20:15. |
#16
|
|||
|
|||
Wow! Many thanks! Sorry that I did not understand before. Now it works (almost):
1. I understand that I can use file type - is not - drop down(video file) or sourceURL - contains - blogspot.com or both. Is that correct? 2. After using that, jDownloader shows videos in LinkGrabber (great!), but also many other spurious files that are not videos (which I do not want), even after selecting File Name contains FirstPartOfNameHere*.mp4 File Size is between 1.00 MB 20 MB All videos start with the same name, and then have extra characters. As far as I know, they are about 7 MB each, but I set a wider range, just in case. How to force jDownloader to show only videos, and in particular .mp4 files only in LinkGrabber (as said, I only want them)? 3. jDownloader only shows up to four video files in LinkGrabber, but some pages contain more (one showing and others as links which redirect to other pages). Is it possible to force jDownloader to search for them and show all videos in a particular page and in the links in such page? All links belong to the same server. In other words, there is an index page which has links for the pages (many; about 400) in which the videos are shown. What I would like is to paste the URL of such index page so that jDownloader can show all videos in the linked pages instead of just the one on such page and three more corresponding to the linked pages (1 + 3 = 4 in total). Is that possible? I guess that it should be possible, since, as said, jDownloader shows three of such linked videos (albeit not all 400 ones). I can download all 400 videos manually, but it would be tedious and error-prone. Thanks again for all. Last edited by jDoX; 21.09.2022 at 20:23. |
#17
|
||||
|
||||
Quote:
It is AND I've updated my post Quote:
Quote:
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 21.09.2022 at 20:17. |
#18
|
|||
|
|||
Thanks. In relation to topic 1 above, I have tested using both settings at the same time, but the results are the same as just with the first one, as reported above.
Maybe I am doing something wrong, but I do not know what. The link for the index page containing about 400 links for pages with videos (about 7 MB each) is: **External links are only visible to Support Staff****External links are only visible to Support Staff** Thanks for all. |
#19
|
||||
|
||||
Quote:
And please provide example links! Maybe the video is embedded via different method and what you get are the individual segments of the stream.
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 22.09.2022 at 16:16. |
#20
|
|||
|
|||
Thanks. The link containing all links (for about 400 videos) in a matrix of numbers on the top is
**External links are only visible to Support Staff****External links are only visible to Support Staff** BTW, if possible, I would like to download each video with its web page (each video in a folder containing the video and its web page), with all of them showing the video name (folder, web page and video). For instance, the video 1 corresponds to this link which corresponds to the 1 in the top matrix of the above web page: **External links are only visible to Support Staff****External links are only visible to Support Staff** I am attaching five screen captures that can be downloaded from: **External links are only visible to Support Staff****External links are only visible to Support Staff** Since I think that it is not possible to embed them here. Thanks! Last edited by jDoX; 22.09.2022 at 17:22. |
#21
|
||||
|
||||
@jDoX: Your rules are wrong.
DownloadURL -> this is the final URL that leads to the downloaded file/content but the videos/content are NOT hosted on this domain! That's why I use sourceURL in my example rule. Example: you posted mediafire link here. DownloadURL is mediafire BUT sourceURL is board.jdownloader.org because that's the website where this link is posted. Also please know that you specify filter/rules that tell JDownloader what to filter/block. So for example your first rule blocks all Videos with name BLOGGER-video*.mp4 with size bewteen 1 and 980 mb and are hosted on the given domain. But those video are hosted on complete different domain! Also please just specify the domain and leave alone the procotol else you will either have to use regex to support http and https or create two rules to support both. leave alone the procotol and it will work as well. Just create a rule like explained Quote:
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 22.09.2022 at 18:03. |
#22
|
||||
|
||||
Quote:
1.) add the html website as directhttp, so JDownloader downloads the html sourcecode. This is NO mirror. JDownloader does not support mirror/download of websites (including all files and rewriting the html) It can only be used to download that actual html file. 2.) deep decrypt or linkcrawler rule to find the embedded videos on that website
__________________
JD-Dev & Server-Admin |
#23
|
||||
|
||||
@jDoX: before we further trying to help you, maybe it would be better to make sure that it's possible to do what you want to achieve
__________________
JD-Dev & Server-Admin |
#24
|
|||
|
|||
Thanks. I have done as shown in the four screen captures at
**External links are only visible to Support Staff****External links are only visible to Support Staff** Please, note that I have merged the two previous Linkgrabber filters into a single one, removed some previous settings, including the ones that you said (sorry that I did not properly understood that before!) and also selected Condition - is true - Link is already in Downloadlist. I do not know if all that is correct, but the fact is that it still does not work as I want (explained below). The first link is **External links are only visible to Support Staff****External links are only visible to Support Staff** and corresponds to the web page shown when clicking the number 1 in the matrix on top of such web pages. It is the first video out of about 400 (done just for testing). In such a case, the video shows in LinkGrabber, which is what I want. But that can be also accomplished just pasting the link in jDownloader using the standard jDownloader factory-default settings. No need to create any Linkgrabber filter. I just did that as a testing example, as said above. What do I want, then? I do not know if I explained it properly in my previous messages. What I want is to download all 400 videos at once, just using the index page containing their links as a number matrix on top: **External links are only visible to Support Staff****External links are only visible to Support Staff** But when using such link, only four videos are shown. And that is because such web page contains them embedded below. So, the 400 videos corresponding to the top matrix of numbers are not shown. And those are the videos that I want to download, but not on a one-by-one basis, which would be utterly tedious and error-prone. In other words, I do not want to click each one of the 400 links in the matrix, but just download them all at once using the index web page indicated above that contains such matrix. Is that possible? How? Additionally, in the two examples of the screen captures indicated above, a lot a spurious unwanted files are shown. I only want the videos (and if possible the web pages that contain them). Is it possible to show only the desired files in LinkGrabber? How? Last edited by jDoX; 23.09.2022 at 00:16. |
#25
|
||||
|
||||
Hi again,
Quote:
You will need to setup one or multiple LinkCrawler Rules to crawl the videos of all sub-pages and depending on how the website is made you might need to manually collect the URLs to all sub-pages in beforehand. Some useful links: https://support.jdownloader.org/Know...kcrawler-rules and: https://support.jdownloader.org/Know...orted-websites Quote:
2. Your rule looks wrong as you've checked Condition -> is true -> Link is already in downloadlist In the bottom toolbar in your screenshots 4 and 5 I can see that 39 links were filtered. These are probably items that are already in your downloadlist but all others got not filtered. Change the rule accordingly and try again.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#26
|
|||
|
|||
Thanks. I am including expanded screen captures at
**External links are only visible to Support Staff****External links are only visible to Support Staff** As before, they are sorted by size and only the top selected ones are videos (two and four, respectively). The others are spurious files that I do not want, yet they show in the LinkGrabber tab. As said, link **External links are only visible to Support Staff****External links are only visible to Support Staff** was just the first video page out of the about 400 available. I can download it and all the rest of 400 videos just pasting their web pages in jDownloader. No special setting required. That works with the jDownloader factory-default settings, albeit they show in LinkGrabber tab with unwanted spurious files. Multiply about 400 web pages with those 37 unwanted spurious files for each and you get 14,800 unwanted spurious files. I just indicated such link for testing purposes. The main problem is when I try to use jDownloader to download at once (automatically; not manually on a one-by-one basis) all about 400 videos without unwanted spurious files in the links of the top matrix of numbers shown at **External links are only visible to Support Staff****External links are only visible to Support Staff** Is that possible? Could you accomplish that? I mean, could you get all such 400 video links to show in the jDownloader LinkGrabber tab without all the other spurious files that I do not want? How? I mean, which settings or procedure should be used for that? Having said that, I want to download such about 400 videos, but it possible I would also like to download each of them with its web page as well (.html or similar like .webarchive, as Safari does). Ideally, each video in a subfolder, or at least all in the same folder with all videos and web pages named with the specific video names, for easier identification, if possible. Thanks again for all, and sorry for my ignorance on this. Last edited by jDoX; 24.09.2022 at 11:59. |
#27
|
||||
|
||||
Okay let's start from the beginning.
This is a blog-style website. It contains a mainpage which you can scrill down to view more post. You want to have all videos of all posts. It will be hard/impossible to auto find all posts but you can tech JD to only find videos when adding single post links. It looks like all their videos are embedded via "blogger.com/video..." which is then hosting the videos on google servers. Please disable the filter rule you've created before and continue reading: Let's drop the filter approach for now and instead of filtering, just grab only what we want straight away using a LinkCrawler Rule: Code:
[ { "enabled": true, "logging": false, "maxDecryptDepth": 1, "name": "jdownloader.org example rule grab all video urls from 1000manerasdemorirx.blogspot.com", "pattern": "https?://1000manerasdemorirx\\.blogspot\\.com/[0-9]{4}/[0-9]{2}/[a-z0-9\\-_]+\\.html", "rule": "DEEPDECRYPT", "packageNamePattern": null, "deepPattern": "\"(https?://(www\\.)?blogger\\.com/video\\.g?[^\"]+)" } ] pastebin.com/raw/fUaDP2QV Once you got this working you can start thinking about how to collect all links to all individual posts... Now I know you want to exclude stuff that is already in your downloadlist. To simplify this, simply use the "Already in downloadlist" view in the linkgrabber and hide those items. Quote:
Automated no but you already need to collect all individual post links first so you can do it in 3 total steps: 1. Collect all links to individual posts. 2. Add them to JD to add- and download all video items. 3. Now modify each link and change the ending from ".html" to ".html.jdeatme" (basically just add ".jdeatme"). This will force JD to download that link/html without deep-analyzing it for non-html content. You can easily modify all links by doing a "search & replace" action using the text-editor of your choice. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 27.09.2022 at 13:59. Reason: Updated LinkCrawler rule to allow underscores in blog post URLs |
#28
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin |
#29
|
||||
|
||||
Quote:
There are tools specifically designed to download/copy all needed references. I will link one old tool that I know from 2017 here though I'm pretty sure there are newer/better alternatives. Old tool: httrack.com
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#30
|
|||
|
|||
Many thanks for the comprehensive reply! When I do it for instance for
**External links are only visible to Support Staff****External links are only visible to Support Staff** I rename it as **External links are only visible to Support Staff****External links are only visible to Support Staff** and then paste it in jDownloader. Yet, then LinkGrabber only shows: blog-post_448.html No videos are shown. I must be doing something wrong, but I do not know what. |
#31
|
||||
|
||||
Please re-read my last post.
I'll quote myself and add some more information: Quote:
All other parts of my last reply cover the question "how to download only videos when adding such single blog-post URLs (without ".jdeatme" ending!!!).
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#32
|
||||
|
||||
Addition:
I've just updated the LinkCrawler rule in my reply #27 as it didn't allow underscores in blog-post-URLs like the one you've used as an example in your last reply.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#33
|
|||
|
|||
Wow! Amazing! Many thanks!
Following your directions, jDownloader downloaded 302 vdeos after pasting **External links are only visible to Support Staff****External links are only visible to Support Staff** and 389 ".html" web pages after pasting the 389 URL which I previously extracted from the source of the web page above (replacing .html by .html.jdeatme). The html web pages have the video names, but not the videos, so I will need to check them to place in the same subfolder each video file with its web page. What will take a while of manual work. BTW, your post 27 above shows the code updated as you said, but the link pastebin.com/raw/fUaDP2QV shows the previous version. Many thanks for all. If technically possible, it would be great if all the functionality required in this thread (or a similar one) could be built-in jDownloader for anyone to use. Thanks again for all! |
#34
|
||||
|
||||
Quote:
- Items are offline - URLs contain characters which are not allowed by the rule I've created - Items embed other types of videos which the rule does not pick up You will need to analyze this on your own. Quote:
Quote:
I've corrected that in the existing paste so the link stays the same. Quote:
The generic tools JD comes with are there to be used by our users to automate downloading from simple websites like this one without the need to write any code. I know it's not the most user friendly way but you probably won't find a software which provides user-friendlier ways to perform such tasks Thanks for your feedback.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#35
|
|||
|
|||
jDownloader is awesome. But I could not have accomplished the tasks of this thread without your help. That is why I said that it would be great if jDownloader could have it built-in.
Thanks again for making such awesome tool and for all your great help in this forum. You rock! |
#36
|
||||
|
||||
Quote:
Just as a slight overview: I'd estimate that at this moment we got at least 100+ open plugin request tickets which will probably never be done. At the same time we have to maintain the existing plugins which takes several hours per day just check our forums daily updated threads and you will see what I mean. Quote:
Thanks for your awesome feedback!
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|