JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #101  
Old 03.12.2020, 21:14
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by Dockel View Post
... the script obviously does not indicate them as dups (although they have already been added to history.txt. Could those links be marked / disabled as dups anyway?
If the urls were already present in the history file at the time of adding them to LG list, it will be (should be) marked as duplicate by the script. Only time they would not be marked as such is, if they were not an exact match. Check both urls and see if there is any difference. Also see if you can reproduce the issue with same or any other urls.
Reply With Quote
  #102  
Old 03.12.2020, 21:26
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

Sorry, I made a mistake. The links in the link grabber are shown as dups by JD (red), they are in the download list already. When I then add the links in the download list to history.txt and press "Start all downloads" they are not marked as dups by the script. Is there a way the script could mark it es dups?
__________________
Aktuelles Windows
Reply With Quote
  #103  
Old 04.12.2020, 10:54
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

@Dockel
@mgpai
The script "move to history" needs to add the TAG to comment field, so the "Start Download" script can work as intended.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #104  
Old 04.12.2020, 11:24
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

I know there is a tag added to the comment, but I do not know which of the scripts it does.
__________________
Aktuelles Windows
Reply With Quote
  #105  
Old 04.12.2020, 11:55
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by Jiaz View Post
@mgpai: The script "move to history" needs to add the TAG to comment field, so the "Start Download" script can work as intended.
They are not mapped to corresponding highlighted links in linkgrabber, so it will require another dupe check to match the selected links against each link present in linkgrabber and mark them as duplicate.
Reply With Quote
  #106  
Old 04.12.2020, 13:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

@mgpai: maybe another script to *recheck/match* against history.txt ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #107  
Old 04.12.2020, 16:31
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by Jiaz View Post
maybe another script to *recheck/match* against history.txt ?
One more is one too many.

Implemented in existing script with slight changes. Also added availablestate check to the script.

Code:
/*
    Add finished download links from selection to history, and remove them from the list
    Tirgger required: Downloadlist Contextmenu Button Pressed
    Customize downloadlist context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional)
*/

if (name == "Move to history") {
    var file = getPath(JD_HOME + "/cfg/history.txt"),
        urls = file.exists() ? readFile(file) : "",
        newUrls = "",
        url, data, text, comment;

    dlSelection.links.forEach(function(link) {
        if (link.finished) {
            url = link.contentURL || link.pluginURL;

            if (urls.indexOf(url) == -1) {
                newUrls += url + "\r\n";
                link.remove();
            }
        }
    });

    if (newUrls.length) {
        data = "Added from the downloadlist list on " + Date().toString().substring(4, 24) + "\r\n";
        data += newUrls + "\r\n";
        writeFile(file, data, true);

        getAllCrawledLinks().forEach(function(link) {
            if (link.availableState == "ONLINE") {
                url = link.contentURL || link.pluginURL;
                text = "#duplicatelink";
                comment = link.comment || "";
                
                if (comment.indexOf(text) == -1 && (urls + newUrls).indexOf(url) > -1) {
                    link.comment = comment ? text + " " + comment : text;
                }
            }
        });
    }
}
Reply With Quote
  #108  
Old 04.12.2020, 17:03
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

Great, thank you very much!
__________________
Aktuelles Windows
Reply With Quote
  #109  
Old 04.12.2020, 17:21
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

@Dockel: You're welcome. Fee free to let me know if you need any new scripts, or make any changes to the existing ones.
Reply With Quote
  #110  
Old 04.12.2020, 20:11
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

Yes, I will do so, very happy about that, very many thanks! And very, very needful the history script.
__________________
Aktuelles Windows
Reply With Quote
  #111  
Old 07.12.2020, 18:08
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

@mgpai: Thanks for your help with the scripts
__________________
JD-Dev & Server-Admin
Reply With Quote
  #112  
Old 08.12.2020, 17:28
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

@Dockel: Please provide in a single post, the current versions of ALL your 'history' scripts. Use 'code' tag for scripts when composing the post.
Reply With Quote
  #113  
Old 08.12.2020, 19:42
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

Code:
// Add selected linkgrabber links to history, and remove them from the list
// Tirgger required: Linkgrabber Contextmenu Button Pressed
// Customize linkgrabber context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional)

if (name == "Move to history") {
    var historyFile = getPath(JD_HOME + "/cfg/history.txt");
    if (!historyFile.exists()) writeFile(historyFile, "", true);
    var history = readFile(historyFile);

    var linkUrls = [];
    var linkIds = [];

    var removeLinks = function(linkIds) {
        callAPI("linkgrabberv2", "removeLinks", linkIds, []);
    }

    lgSelection.getLinks().forEach(function(link) {
        var url = link.getUrl();
        var id = link.getUUID();

        if (history.indexOf(url) == -1) linkUrls.push(url);
        linkIds.push(id);
    })

    if (linkUrls.length) {
        data = "Added from the lingkrabber list on " + Date().toString().substring(4, 24) + "\r\n";
        data += linkUrls.join("\r\n") + "\r\n\r\n";
        writeFile(historyFile, data, true);
    }

    if (linkIds.length) removeLinks(linkIds);
}
Code:
/*
    Add finished download links from selection to history, and remove them from the list
    Tirgger required: Downloadlist Contextmenu Button Pressed
    Customize downloadlist context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional)
*/

if (name == "Move to history") {
    var file = getPath(JD_HOME + "/cfg/history.txt"),
        urls = file.exists() ? readFile(file) : "",
        newUrls = "",
        url, data, text, comment;

    dlSelection.links.forEach(function(link) {
        if (link.finished) {
            url = link.contentURL || link.pluginURL;

            if (urls.indexOf(url) == -1) {
                newUrls += url + "\r\n";
                link.remove();
            }
        }
    });

    if (newUrls.length) {
        data = "Added from the downloadlist list on " + Date().toString().substring(4, 24) + "\r\n";
        data += newUrls + "\r\n";
        writeFile(file, data, true);

        getAllCrawledLinks().forEach(function(link) {
            if (link.availableState == "ONLINE") {
                url = link.contentURL || link.pluginURL;
                text = "#duplicatelink";
                comment = link.comment || "";
                
                if (comment.indexOf(text) == -1 && (urls + newUrls).indexOf(url) > -1) {
                    link.comment = comment ? text + " " + comment : text;
                }
            }
        });
    }
}
Code:
// Remove #duplicatelink and #dupliatefile from comment field for selected links
// Trigger: Downloadlist Contextmenu Button Pressed

if (name = "Clear dupe comment") {
    dlSelection.getLinks().forEach(function(link) {
        var myComments = ["#duplicatelink", "#duplicatefile"];
        myComments.forEach(function(myComment) {
            var comment = link.getComment() || "";
            if (comment.indexOf(myComment) > -1) link.setComment(comment.replace("#duplicatelink", "").replace("#duplicatefile", ""));
        })
    })
}
Code:
// Remove/Re-add filecrypt.cc links with wrong captcha
// Trigger: Interval (Recommended 60000 or more)
// IMPORTANT: Enable ''Synchronous execution of script' in top panel

var delay = 15; // Set delay (in seconds) between retries

links = getAllCrawledLinks().filter(function(crawledLink) {
    var link = crawledLink.contentURL || "";
    return link.indexOf("filecrypt.cc") > -1 && crawledLink.name.indexOf("Wrong Captcha!") > -1;
}).forEach(function(crawledLink) {
    sleep(delay * 1000);
    callAPI("linkgrabberv2", "addLinks", {
        "links": crawledLink.contentURL
    });
    crawledLink.remove();
})
Code:
/* 
    Disable duplicate link in Lingrabber
    Trigger required: A new link has been added
*/

var link = crawledLink;

if (link.availableState == "ONLINE") {
    var name = "history";
    var lock = getModifyLock(name);

    lock.writeLock();

    var urls = getProperty(name, true);

    if (urls == null) {
        try {
            var file = JD_HOME + "/cfg/history.txt";
            var urls = readFile(file);

            setProperty(name, urls, true);
        } catch (e) {}
    }

    lock.writeUnlock();

    if (urls) {
        var url = link.contentURL || link.pluginURL;

        if (link.host == "youtube.com") {
            url = link.containerURL;
        }

        if (urls.indexOf(url) > -1) {
            var text = "#duplicatelink";
            var comment = link.comment;

            if (comment) {
                text = text + " " + comment;
            }

            link.comment = text;
            link.enabled = false;
        }
    }
}
This one does not seem not to be needed anymore, it is deactivated, but who knows:

Code:
// Skip download link, if filename exists in the user-spcified list
// To download the file (prevent skipping), remove "#duplicatefile" from comment
// Trigger required: A Download Started

var fileNamesList = "I:\\jD-Downloads\\jD-Dummies von jD-Downloads - alle Dateien, Dummies aus dem Ordner - alle ca. 95.000 Dateien als Liste.txt"; // < Set path to text file which contain the file names. Use "\\" or "/" as path separators.
var dupeFilecheck = link.getProperty("dupeFileCheck");
var linkName = link.getName();
var comment = link.getComment() || "";
var skipLink = function() {
    link.setSkipped(true); //To disable the link instead of skipping it, replace link.setSkipped(true); with link.setEnabled(false); in both the scripts.
    alert("Download Skipped: File \"" + linkName + "\" is present in files list.");
}

if (dupeFilecheck) {
    if (comment.indexOf("#duplicatefile") > -1) link.setSkipped(true);
} else {
    var fileNames = readFile(getPath(fileNamesList)).trim().split("\r\n");

    fileNames.some(function(fileName) {
        if (linkName == fileName.trim()) {
            var text = "#duplicatefile";
            comment = comment ? text + " " + comment : text;
            link.setComment(comment);
            //            link.setEnabled(false); falls Duplikate im Download-Fenster enabled werden sollen
            link.setSkipped(true);
            return true;
        }
    })

    link.setProperty("dupeFileCheck", true);
}
And this also seems not be needed anymore, deactivated also:

Code:
// Skip download link, if filename exists in the user-spcified list
// To download the file (prevent skipping), remove "#duplicatefile" from comment
// Trigger required: A Download Started

var fileNamesList = "I:\\jD-Downloads\\jD-Dummies von jD-Downloads - alle Dateien, Dummies aus dem Ordner - alle ca. 95.000 Dateien als Liste.txt"; // < Set path to text file which contain the file names. Use "\\" or "/" as path separators.
var dupeFilecheck = link.getProperty("dupeFileCheck");
var linkName = link.getName();
var comment = link.getComment() || "";
var skipLink = function() {
    link.setSkipped(true); //To disable the link instead of skipping it, replace link.setSkipped(true); with link.setEnabled(false); in both the scripts.
    alert("Download Skipped: File \"" + linkName + "\" is present in files list.");
}

if (dupeFilecheck) {
    if (comment.indexOf("#duplicatefile") > -1) link.setSkipped(true);
} else {
    var fileNames = readFile(getPath(fileNamesList)).trim().split("\r\n");

    fileNames.some(function(fileName) {
        if (linkName == fileName.trim()) {
            var text = "#duplicatefile";
            comment = comment ? text + " " + comment : text;
            link.setComment(comment);
            //            link.setEnabled(false); falls Duplikate im Download-Fenster enabled werden sollen
            link.setSkipped(true);
            return true;
        }
    })

    link.setProperty("dupeFileCheck", true);
}
__________________
Aktuelles Windows
Reply With Quote
  #114  
Old 08.12.2020, 19:52
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

I needed only 'history' scripts. I am less sure than you as to which scripts are needed. I'll try to figure out.
Reply With Quote
  #115  
Old 08.12.2020, 20:05
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

Sorry, so I see I am not sure what the history scripts are, I thought the ones working together somehow, needed for the entire task. I have marked them as belonging to the history scripts.
__________________
Aktuelles Windows
Reply With Quote
  #116  
Old 08.12.2020, 20:44
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

It seems the script(s) does not disable / mark the dups anymore. And there was a finished download I could not move to the history.txt (edit: when it already is in the history, it stays in the download window when trying to move it to history.txt). Two others I could move. May be I am missing anything. Is there anything I could do?
__________________
Aktuelles Windows

Last edited by Dockel; 08.12.2020 at 21:21.
Reply With Quote
  #117  
Old 09.12.2020, 12:35
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

@Dockel: mgpai has to update all other *history* scripts as well to work with optimized inmemory history
__________________
JD-Dev & Server-Admin
Reply With Quote
  #118  
Old 09.12.2020, 12:50
Dockel Dockel is offline
JD Legend
 
Join Date: Feb 2020
Posts: 664
Default

OK, thank you.
__________________
Aktuelles Windows
Reply With Quote
  #119  
Old 09.12.2020, 14:38
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by Jiaz View Post
@Dockel: mgpai has to update all other *history* scripts as well to work with optimized inmemory history
The problem reported does not seem to be related to memory optimization or lack thereof.

I will optimize these scripts, but, I do not want to use global properties in new scripts, if it needs additional code to handle collision. It doesn't really make sense having to repeat the same code every time we need to read/write them. I think that should be the default behavior for global properties, or at least there should be a flag/switch to make it so.
Reply With Quote
  #120  
Old 09.12.2020, 15:06
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by Dockel View Post
It seems the script(s) does not disable / mark the dups anymore.
When adding new links to linkgrabber? New script will only process links which are ONLINE, unlike the old script, which did not check for availability, which would result in wrong type of links (e.g. filecrypt container) being added to the history file.

Quote:
Originally Posted by Dockel View Post
And there was a finished download I could not move to the history.txt (edit: when it already is in the history, it stays in the download window when trying to move it to history.txt). Two others I could move. May be I am missing anything. Is there anything I could do?
I think is being caused by wrong placement of 'remove' method in current version of the script. Try with this script:

Code:
/*
    Add finished download links from selection to history, and remove them from the list
    Tirgger required: Downloadlist Contextmenu Button Pressed
    Customize downloadlist context menu > Add new "Eventscripter Trigger" button > Rename it to "Move to history" (without quotes) > Change Icon (optional)
*/

if (name == "Move to history") {
    var file = getPath(JD_HOME + "/cfg/history.txt"),
        urls = file.exists() ? readFile(file) : "",
        newUrls = "",
        url, data, text, comment;

    dlSelection.links.forEach(function(link) {
        if (link.finished) {
            url = link.contentURL || link.pluginURL;

            if (urls.indexOf(url) == -1) {
                newUrls += url + "\r\n";
            }
            
            link.remove();
        }
    });

    if (newUrls.length) {
        data = "Added from the downloadlist list on " + Date().toString().substring(4, 24) + "\r\n";
        data += newUrls + "\r\n";
        writeFile(file, data, true);

        getAllCrawledLinks().forEach(function(link) {
            if (link.availableState == "ONLINE") {
                url = link.contentURL || link.pluginURL;
                text = "#duplicatelink";
                comment = link.comment || "";
                
                if (comment.indexOf(text) == -1 && (urls + newUrls).indexOf(url) > -1) {
                    link.comment = comment ? text + " " + comment : text;
                }
            }
        });
    }
}
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:40.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.