#1
|
|||
|
|||
Add links from Tampermonkey script?
Some gallery sites are tricky and I have my own Tampermonkey scripts that essentially go through all the pages, make a bunch of requests and fetch all the full-size image links.
When finished the scripts add a textarea element to the page, containing all the links and a text input element containing whatever title I have chosen to give that gallery. I then need to copy the links to Jdownloader and set the package name as the title. ------- Can this be automated? In other words, instead of copying the links and title to jdownloader, once my script is finished, it contacts Jdownloader with that information automatically. ------- Note that the package name is extremely important here. In jdownloader I have a custom renaming rule, which names all the images based on the package name, followed by a number. This is obviously so that everything is organized and the original order of the gallery is preserved. |
#2
|
|||
|
|||
The feature FolderWatch can be a possible solution for your request. Your script create crawljob files with Links, Packagename, ...
JD will look into the defined Folder and import the crawljobs automatic. Info: https://support.jdownloader.org/Know...ch-basic-usage |
#3
|
|||
|
|||
Quote:
|
#4
|
|||
|
|||
IGNORE POSTS BELOW ABOUT THE PROBLEMS I HAD. IT WAS RESOLVED (THE SCRIPTS BELOW WERE FIXED)
Folder Watch works great. Here is the function I have btw for saving the .crawljob file if anyone finds this and needs it for some reason: Code:
function save_as_crawljob_file(linksArr, galleryName) { let obj = [ { //autoStart: "TRUE", text: linksArr.join("<br>"), packageName: galleryName } ]; let base64Encoded = btoa(JSON.stringify(obj)); GM_download({ url:`data:text/plain;base64,${base64Encoded}`, name:`${galleryName}.crawljob`, saveAs:false, onerror:()=>{alert(`FAILED SAVING FILE`)}, onload:()=>{alert(`FILE SAVED with ${linksArr.length} links`)} }); } ---------------------------------------------------------------------------- Also if you want to use Click n Load instead, here is the function for that as well: Code:
function contact_jdownloader(linksArr, galleryName) { // used so that the form doesn't load a new page or open a new window try {document.querySelector("#jdFrame").remove()} catch(e) {} let frameHtml = `<iframe id="jdFrame" name="jdownloader-frame" style="width:100%;" hidden></iframe>`; document.querySelector("body").appendChild(html_to_node(frameHtml)); try {document.querySelector("#jdForm").remove()} catch(e) {} // since jdownloader site doesn't allow links, set the form action below to 'http{colon,slash.slah}127.0.0.1:9666/flash/add' let formHtml = `<form id='jdForm' action='' method='POST' target='jdownloader-frame'><input type="hidden" name="source" value="tampermonkey"/><input type="hidden" name="package" value="${galleryName}"/><input id="jdUrlsInput" type="hidden" name="urls" value="${linksArr.join("<br>")}"/></form>`; document.querySelector("body").appendChild(html_to_node(formHtml)); let formElem = document.querySelector("#jdForm"); formElem.submit(); } function html_to_node(code) { let tempWrapper = document.createElement("div"); tempWrapper.innerHTML = code; if (tempWrapper.childElementCount == 1) tempWrapper = tempWrapper.firstChild; return tempWrapper; } Last edited by hishtnik011; 02.11.2022 at 18:25. |
#5
|
|||
|
|||
Perfect, that it works for you
|
#6
|
||||
|
||||
Thanks for helping out @tarkett.
@hishtnik011 What you could also use for this: 1. Click and load API. 2. Myjd API.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#7
|
|||
|
|||
Ok thank you! Will look at those too when I get the time.
|
#8
|
||||
|
||||
@hishtnik011: In case you've got questions about api usage, just contact us via support@jdownloader.org
I can also provide a basic pure js MyJDownloader api client for further testing/development, that way your script would also work from remote
__________________
JD-Dev & Server-Admin |
#9
|
|||
|
|||
Quote:
Despite of having the "LinkCollector: Do Link Check" option disabled in Jdownloader Advanced Settings, both Folder Watch and Click n Load seem to crawl the links I provide. This can take a long time if I feed it hundreds/thousands of links and especially with some hosts. Can that be fixed? |
#10
|
||||
|
||||
Please add more details e.g. which links are you adding.
The "do link check" option is for single links that go into host plugins. Example where this will have an effect: uploaded.net, rapidgator.net and most of all other websites for which you're typically adding links that lead to single files. Examples where this will have no effect: youtube, vimeo and other websites where links first get processed by a crawler as they will/can contain multiple results --> That setting does not have any effect for links which need to be processed by a crawler plugin first. If "fast linkcheck" disabled that too, JD just wouldn't do anything at all anymore when adding such links.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
|||
|
|||
Quote:
I have custom scripts for all the sites I download from that get me the image links, combined with some tricks to trigger my custom renaming rule in packagizer (#custnum= added at the end of each link). This is why when I download images I do not need the link check thing. Anyway, by having the link check option disabled, if I were to add links manually through the Jdownloader interface, it does not check their online status, it just adds them and that's it. For example, I click on "add links" (or Ctrl+L), enter a package name and provide however many links I want. It accepts them with a question mark beside each link and I can begin downloading right away. Meanwhile with Folder Watch (.crawljob files) and Click n Load, in both cases it analyzes (crawls) the links before I can begin downloading them (that spinning thing at the bottom right corner). It appears to me that both Click n Load and Folder Watch do not respect/know that I have disabled link check in advanced settings. I looked through the APIs of both of them and they don't seem to have anything related to this. |
#12
|
||||
|
||||
Quote:
That would indeed be a bug. We'll check this once we find the time. Do you have any CNL/example URLs which can be used to reproduce this? Otherwise we'll try with random example URLs.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#13
|
||||
|
||||
I was unable to confirm your theory.
Here is the LinkCrawler rule I've used for checking: Code:
extractPasswords=["Password1","Password2"] downloadPassword=123456Test enabled=TRUE text=http://cdn8.appwork.org/speed.zip packageName=MyPackageName filename=NewFilename.zip comment=SuperUsefulComment autoConfirm=TRUE autoStart=TRUE extractAfterDownload=FALSE forcedStart=FALSE downloadFolder=C:\Users\test\Downloads overwritePackagizerEnabled=false Code:
cdn8.appwork.org/speed.zip My theory is that you're adding links which JD needs to deep crawl e.g. direct-links without file-extension. Without such testlinks I won't be able to provide help with this.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#14
|
|||
|
|||
Honestly though I'm not really that concerned about this since I can set up my scripts so that i add the links from the few hosts that take a long time to process manually, and everything else is automatic.
-------------- Although, I think I'm done using Click n Load and Folder Watch due to another issue I encountered. It appears to be caused by my custom renaming rule and it happens regardless of whether I have link check on or off. My custom renaming packagizer rule is as follows: if Sourceurl(s) contains *#custnum=* then set Filename <jd:packagename> - <jd:source:2>.<jd:orgfiletype> Since it's hard to explain, I will provide you with a sample .crawljob file. So save it and load it in Jdownloader (you will need folder watch installed and enabled to be able to load the JSON-format) WARNING: links are NSFW Code:
[{"text":"**External links are only visible to Support Staff**,"packageName":"example"}] What happens after I add this file is I get this: example - 22_.jpg example - 22_.jpg example - 22_.jpg .... AKA the same thing repeating over and over again. It also adds that weird underscore before .jpg which is not supposed to be there. If I disable my custom renaming rule it works just fine, but the links are no longer named the way I want them to. -------------- Obviously, when I add this package manually through the interface (Ctrl+L, copy paste) (links separated by new line), everything, including my custom renaming rule works: package name: example Code:
**External links are only visible to Support Staff** Last edited by hishtnik011; 02.11.2022 at 08:42. |
#15
|
||||
|
||||
Jiaz might be able to add a better description but that's all I can provide you with now:
Just in case you're already auto-generating those rules via a script: If wou were to create one rule per URL, you'd be able to work around this problem. The background roughly is this: The complete content inside that "text" field of your crawljob is available for every single item as "source" this is why you will get the same "custnum" result for all items with that Packagizer rule even though each link is different. Jiaz will correct me / edit this post if any of that information is wrong.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#16
|
|||
|
|||
Quote:
Last edited by hishtnik011; 02.11.2022 at 18:09. |
#17
|
||||
|
||||
As said Jiaz might be able to provide more/better information than me but for now that's all I can tell you about this topic^^
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#18
|
|||
|
|||
Quote:
Thanks for your help! Last edited by hishtnik011; 03.11.2022 at 04:56. |
#19
|
||||
|
||||
Thanks for your feedback.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
|
|