|
[See Bugtracker] Skript oder sonst eine automatisierte Möglichkeit: Links nicht 1 weiteres Mal laden? |
|
Thread Tools | Display Modes |
#81
|
|||
|
|||
I may need to run some tests. Can you contact me again in chat?
|
#82
|
|||
|
|||
Yes, OK.
__________________
Aktuelles Windows |
#83
|
|||
|
|||
Just added three zippy links, the message appeared again
all three download links marked as duplicate: One of the files has a combination of numbers and letters (as it often is) instead of file.html. I added three more zippy downloads without a problem. Not being marked as duplicates. All of their file names were a combination of numbers and letters, without extension. Unfortunately I cannot find script_errors.txt file, may be there is not one created after to log that error? But, it seems (like it was yesterday) or may be the error only appears once in a JD session (from start to close of JD) and never again in the sesson, so one could leave it as it is.
__________________
Aktuelles Windows Last edited by Dockel; 07.07.2020 at 08:48. |
#84
|
|||
|
|||
This seems to be from a different script. Not from the one modified yesterday. You can also see that the error messages are different. Contact me in chat when you are free.
|
#85
|
||||
|
||||
Quote:
Fixed! Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#86
|
|||
|
|||
I am sorry, but the same problem again: "html.file" links are marked as duplicates (also before chainging the last script yesterday). May be there is anything....
__________________
Aktuelles Windows Last edited by Dockel; 08.07.2020 at 08:16. |
#87
|
|||
|
|||
Please contact me in chat.
|
#88
|
|||
|
|||
Suddenly (since the updated script(s)) the duplicates are not disabled anymore (https://board.jdownloader.org/showth...led#post457412) but shown as duplicates with a message for each link. How could I turn showing the messages off and let the script(s) disable the links?
__________________
Aktuelles Windows |
#89
|
|||
|
|||
I only added a code to display message for troubleshooting purpose. It should not affect any other function of the script. The script which you have linked appears to be different. You will have to again contact me in chat to check it.
|
#90
|
|||
|
|||
Links are added to the Linkgrabber and JD marks them in red as dups (so they are already in the download list). When I then delete the dup links in the download list the dups in the grabber are not marked / indicated anymore as dups (by JD) and will be downloaded? So the script does not indicate them as dups (although they have been added to history.txt?
__________________
Aktuelles Windows |
#91
|
||||
|
||||
@Dockel: dupe is no property but just a runtime evaluated information when required, eg during rendering of table or when filtering the links.....
Or do you mean something different?
__________________
JD-Dev & Server-Admin |
#92
|
|||
|
|||
Sorry, I do not understand what that means, but I meant the script mgpai made. To not to download dups.
__________________
Aktuelles Windows |
#93
|
||||
|
||||
That's something the script cannot do! I tried to explain that this is no property to a link (being a dupe or not) but just evaluated in real time when this information is required.
The script for example could modify the link comment text and add a marker like DUPE. in combination with custom view filter you can easily filter those entries in linkgrabber list. @mgpai: script just need to modify the comment field
__________________
JD-Dev & Server-Admin |
#94
|
|||
|
|||
Quote:
Quote:
__________________
Aktuelles Windows |
#95
|
||||
|
||||
@Dockel: Can you link/show the script?
Onces you've removed the link from downloadlist, JDownloader will no longer highlight the link in linkgrabber as dupe (different colour), the script cannot do this
__________________
JD-Dev & Server-Admin |
#96
|
|||
|
|||
I am not sure which one this does:
Code:
// Skip link if it present in download history (has "#duplicatelink" in comment) // To download the file (prevent skipping), remove "#duplicatelink" from comment // Trigger required: A Download Started var comment = link.getComment() || ""; if (comment.indexOf("#duplicatelink") > -1) { // var url = link.getContentURL() || link.getPluginURL(); // Nur tatsächlich runtergeladene Varianten von YouTube werden als Duplikate behandelt var url = link.getHost() == "youtube.com" ? link.getContainerURL() : link.getContentURL() || link.getPluginURL(); // alle Varianten auf YouTube werden als Duplikate behandelt link.setEnabled(false); //To disable the link instead of skipping it, replace link.setSkipped(true); with link.setEnabled(false); in both the scripts alert("Download Skipped: \"" + url + "\" is present in history file."); } Quote:
Quote:
__________________
Aktuelles Windows Last edited by Dockel; 03.12.2020 at 15:21. |
#97
|
||||
|
||||
@Dockel: When I understant the scripts right, the following happens
you add the link to download list, then add again to linkgrabber list. Now the dupe support of JDownloader highlights the link via color. then you use script to mark as dup and remove from download list. At that moment the *mark as duplicate* script in Linkgrabber is already done. the link was not part of history.txt at time of link adding, no comment has changed. When you remove the link from linkgrabber and add again, then it should contain the comment. For this to work @mgpai has to modify the scripts.
__________________
JD-Dev & Server-Admin |
#98
|
|||||
|
|||||
Quote:
Quote:
Quote:
Quote:
Quote:
OK, so we will wait for what mgpai will say.
__________________
Aktuelles Windows |
#99
|
|||
|
|||
@Jiaz has correctly explained it. The whole point of the script was to check for dupes which are not already in the download list. There doesn't seem to be much point in the script again doing what JD has already done (highlighted as dupe). You can use the sidebar to filter and disable/delete the highlighted links.
|
#100
|
|||
|
|||
Quote:
Obviously I explained completly bad, very sorry: Links are added to the Linkgrabber and JD marks them in red as dups (because they are already in the download list). When I then delete the dup links in the download list the dups in the grabber correctly are not marked / indicated anymore as dups (by JD). These links are not marked / disabled as dups by the script (although they are dups snf already available in history.txt. So the script obviously does not indicate them as dups (although they have already been added to history.txt. Could those links be marked / disabled as dups anyway?
__________________
Aktuelles Windows |
#101
|
|||
|
|||
If the urls were already present in the history file at the time of adding them to LG list, it will be (should be) marked as duplicate by the script. Only time they would not be marked as such is, if they were not an exact match. Check both urls and see if there is any difference. Also see if you can reproduce the issue with same or any other urls.
|
#102
|
|||
|
|||
Sorry, I made a mistake. The links in the link grabber are shown as dups by JD (red), they are in the download list already. When I then add the links in the download list to history.txt and press "Start all downloads" they are not marked as dups by the script. Is there a way the script could mark it es dups?
__________________
Aktuelles Windows |
#103
|
||||
|
||||
@Dockel
@mgpai The script "move to history" needs to add the TAG to comment field, so the "Start Download" script can work as intended.
__________________
JD-Dev & Server-Admin |
#104
|
|||
|
|||
I know there is a tag added to the comment, but I do not know which of the scripts it does.
__________________
Aktuelles Windows |
#105
|
|||
|
|||
They are not mapped to corresponding highlighted links in linkgrabber, so it will require another dupe check to match the selected links against each link present in linkgrabber and mark them as duplicate.
|
#106
|
||||
|
||||
@mgpai: maybe another script to *recheck/match* against history.txt ?
__________________
JD-Dev & Server-Admin |
#107
|
|||
|
|||
One more is one too many.
Implemented in existing script with slight changes. Also added availablestate check to the script. Code:
/* Add finished download links from selection to history, and remove them from the list Tirgger required: Downloadlist Contextmenu Button Pressed Customize downloadlist context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional) */ if (name == "Move to history") { var file = getPath(JD_HOME + "/cfg/history.txt"), urls = file.exists() ? readFile(file) : "", newUrls = "", url, data, text, comment; dlSelection.links.forEach(function(link) { if (link.finished) { url = link.contentURL || link.pluginURL; if (urls.indexOf(url) == -1) { newUrls += url + "\r\n"; link.remove(); } } }); if (newUrls.length) { data = "Added from the downloadlist list on " + Date().toString().substring(4, 24) + "\r\n"; data += newUrls + "\r\n"; writeFile(file, data, true); getAllCrawledLinks().forEach(function(link) { if (link.availableState == "ONLINE") { url = link.contentURL || link.pluginURL; text = "#duplicatelink"; comment = link.comment || ""; if (comment.indexOf(text) == -1 && (urls + newUrls).indexOf(url) > -1) { link.comment = comment ? text + " " + comment : text; } } }); } } |
#108
|
|||
|
|||
Great, thank you very much!
__________________
Aktuelles Windows |
#109
|
|||
|
|||
@Dockel: You're welcome. Fee free to let me know if you need any new scripts, or make any changes to the existing ones.
|
#110
|
|||
|
|||
Yes, I will do so, very happy about that, very many thanks! And very, very needful the history script.
__________________
Aktuelles Windows |
#111
|
||||
|
||||
@mgpai: Thanks for your help with the scripts
__________________
JD-Dev & Server-Admin |
#112
|
|||
|
|||
@Dockel: Please provide in a single post, the current versions of ALL your 'history' scripts. Use 'code' tag for scripts when composing the post.
|
#113
|
|||
|
|||
Code:
// Add selected linkgrabber links to history, and remove them from the list // Tirgger required: Linkgrabber Contextmenu Button Pressed // Customize linkgrabber context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional) if (name == "Move to history") { var historyFile = getPath(JD_HOME + "/cfg/history.txt"); if (!historyFile.exists()) writeFile(historyFile, "", true); var history = readFile(historyFile); var linkUrls = []; var linkIds = []; var removeLinks = function(linkIds) { callAPI("linkgrabberv2", "removeLinks", linkIds, []); } lgSelection.getLinks().forEach(function(link) { var url = link.getUrl(); var id = link.getUUID(); if (history.indexOf(url) == -1) linkUrls.push(url); linkIds.push(id); }) if (linkUrls.length) { data = "Added from the lingkrabber list on " + Date().toString().substring(4, 24) + "\r\n"; data += linkUrls.join("\r\n") + "\r\n\r\n"; writeFile(historyFile, data, true); } if (linkIds.length) removeLinks(linkIds); } Code:
/* Add finished download links from selection to history, and remove them from the list Tirgger required: Downloadlist Contextmenu Button Pressed Customize downloadlist context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional) */ if (name == "Move to history") { var file = getPath(JD_HOME + "/cfg/history.txt"), urls = file.exists() ? readFile(file) : "", newUrls = "", url, data, text, comment; dlSelection.links.forEach(function(link) { if (link.finished) { url = link.contentURL || link.pluginURL; if (urls.indexOf(url) == -1) { newUrls += url + "\r\n"; link.remove(); } } }); if (newUrls.length) { data = "Added from the downloadlist list on " + Date().toString().substring(4, 24) + "\r\n"; data += newUrls + "\r\n"; writeFile(file, data, true); getAllCrawledLinks().forEach(function(link) { if (link.availableState == "ONLINE") { url = link.contentURL || link.pluginURL; text = "#duplicatelink"; comment = link.comment || ""; if (comment.indexOf(text) == -1 && (urls + newUrls).indexOf(url) > -1) { link.comment = comment ? text + " " + comment : text; } } }); } } Code:
// Remove #duplicatelink and #dupliatefile from comment field for selected links // Trigger: Downloadlist Contextmenu Button Pressed if (name = "Clear dupe comment") { dlSelection.getLinks().forEach(function(link) { var myComments = ["#duplicatelink", "#duplicatefile"]; myComments.forEach(function(myComment) { var comment = link.getComment() || ""; if (comment.indexOf(myComment) > -1) link.setComment(comment.replace("#duplicatelink", "").replace("#duplicatefile", "")); }) }) } Code:
// Remove/Re-add filecrypt.cc links with wrong captcha // Trigger: Interval (Recommended 60000 or more) // IMPORTANT: Enable ''Synchronous execution of script' in top panel var delay = 15; // Set delay (in seconds) between retries links = getAllCrawledLinks().filter(function(crawledLink) { var link = crawledLink.contentURL || ""; return link.indexOf("filecrypt.cc") > -1 && crawledLink.name.indexOf("Wrong Captcha!") > -1; }).forEach(function(crawledLink) { sleep(delay * 1000); callAPI("linkgrabberv2", "addLinks", { "links": crawledLink.contentURL }); crawledLink.remove(); }) Code:
/* Disable duplicate link in Lingrabber Trigger required: A new link has been added */ var link = crawledLink; if (link.availableState == "ONLINE") { var name = "history"; var lock = getModifyLock(name); lock.writeLock(); var urls = getProperty(name, true); if (urls == null) { try { var file = JD_HOME + "/cfg/history.txt"; var urls = readFile(file); setProperty(name, urls, true); } catch (e) {} } lock.writeUnlock(); if (urls) { var url = link.contentURL || link.pluginURL; if (link.host == "youtube.com") { url = link.containerURL; } if (urls.indexOf(url) > -1) { var text = "#duplicatelink"; var comment = link.comment; if (comment) { text = text + " " + comment; } link.comment = text; link.enabled = false; } } } Code:
// Skip download link, if filename exists in the user-spcified list // To download the file (prevent skipping), remove "#duplicatefile" from comment // Trigger required: A Download Started var fileNamesList = "I:\\jD-Downloads\\jD-Dummies von jD-Downloads - alle Dateien, Dummies aus dem Ordner - alle ca. 95.000 Dateien als Liste.txt"; // < Set path to text file which contain the file names. Use "\\" or "/" as path separators. var dupeFilecheck = link.getProperty("dupeFileCheck"); var linkName = link.getName(); var comment = link.getComment() || ""; var skipLink = function() { link.setSkipped(true); //To disable the link instead of skipping it, replace link.setSkipped(true); with link.setEnabled(false); in both the scripts. alert("Download Skipped: File \"" + linkName + "\" is present in files list."); } if (dupeFilecheck) { if (comment.indexOf("#duplicatefile") > -1) link.setSkipped(true); } else { var fileNames = readFile(getPath(fileNamesList)).trim().split("\r\n"); fileNames.some(function(fileName) { if (linkName == fileName.trim()) { var text = "#duplicatefile"; comment = comment ? text + " " + comment : text; link.setComment(comment); // link.setEnabled(false); falls Duplikate im Download-Fenster enabled werden sollen link.setSkipped(true); return true; } }) link.setProperty("dupeFileCheck", true); } Code:
// Skip download link, if filename exists in the user-spcified list // To download the file (prevent skipping), remove "#duplicatefile" from comment // Trigger required: A Download Started var fileNamesList = "I:\\jD-Downloads\\jD-Dummies von jD-Downloads - alle Dateien, Dummies aus dem Ordner - alle ca. 95.000 Dateien als Liste.txt"; // < Set path to text file which contain the file names. Use "\\" or "/" as path separators. var dupeFilecheck = link.getProperty("dupeFileCheck"); var linkName = link.getName(); var comment = link.getComment() || ""; var skipLink = function() { link.setSkipped(true); //To disable the link instead of skipping it, replace link.setSkipped(true); with link.setEnabled(false); in both the scripts. alert("Download Skipped: File \"" + linkName + "\" is present in files list."); } if (dupeFilecheck) { if (comment.indexOf("#duplicatefile") > -1) link.setSkipped(true); } else { var fileNames = readFile(getPath(fileNamesList)).trim().split("\r\n"); fileNames.some(function(fileName) { if (linkName == fileName.trim()) { var text = "#duplicatefile"; comment = comment ? text + " " + comment : text; link.setComment(comment); // link.setEnabled(false); falls Duplikate im Download-Fenster enabled werden sollen link.setSkipped(true); return true; } }) link.setProperty("dupeFileCheck", true); }
__________________
Aktuelles Windows |
#114
|
|||
|
|||
I needed only 'history' scripts. I am less sure than you as to which scripts are needed. I'll try to figure out.
|
#115
|
|||
|
|||
Sorry, so I see I am not sure what the history scripts are, I thought the ones working together somehow, needed for the entire task. I have marked them as belonging to the history scripts.
__________________
Aktuelles Windows |
#116
|
|||
|
|||
It seems the script(s) does not disable / mark the dups anymore. And there was a finished download I could not move to the history.txt (edit: when it already is in the history, it stays in the download window when trying to move it to history.txt). Two others I could move. May be I am missing anything. Is there anything I could do?
__________________
Aktuelles Windows Last edited by Dockel; 08.12.2020 at 20:21. |
#117
|
||||
|
||||
@Dockel: mgpai has to update all other *history* scripts as well to work with optimized inmemory history
__________________
JD-Dev & Server-Admin |
#118
|
|||
|
|||
OK, thank you.
__________________
Aktuelles Windows |
#119
|
|||
|
|||
Quote:
I will optimize these scripts, but, I do not want to use global properties in new scripts, if it needs additional code to handle collision. It doesn't really make sense having to repeat the same code every time we need to read/write them. I think that should be the default behavior for global properties, or at least there should be a flag/switch to make it so. |
#120
|
|||
|
|||
When adding new links to linkgrabber? New script will only process links which are ONLINE, unlike the old script, which did not check for availability, which would result in wrong type of links (e.g. filecrypt container) being added to the history file.
Quote:
Code:
/* Add finished download links from selection to history, and remove them from the list Tirgger required: Downloadlist Contextmenu Button Pressed Customize downloadlist context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional) */ if (name == "Move to history") { var file = getPath(JD_HOME + "/cfg/history.txt"), urls = file.exists() ? readFile(file) : "", newUrls = "", url, data, text, comment; dlSelection.links.forEach(function(link) { if (link.finished) { url = link.contentURL || link.pluginURL; if (urls.indexOf(url) == -1) { newUrls += url + "\r\n"; } link.remove(); } }); if (newUrls.length) { data = "Added from the downloadlist list on " + Date().toString().substring(4, 24) + "\r\n"; data += newUrls + "\r\n"; writeFile(file, data, true); getAllCrawledLinks().forEach(function(link) { if (link.availableState == "ONLINE") { url = link.contentURL || link.pluginURL; text = "#duplicatelink"; comment = link.comment || ""; if (comment.indexOf(text) == -1 && (urls + newUrls).indexOf(url) > -1) { link.comment = comment ? text + " " + comment : text; } } }); } } |
|
|