[xefeg]

@mgpai
REQ Clipboard Observer catch links only from sites(.)com

Pretty much clear.
1. Turn on Clipboard Observer
2. Add to Link Grabber tab only if the clipboard contains sites from asd(.)com, zyx(.).com only

[mgpai]

Originally Posted by xefeg

REQ Clipboard Observer catch links only from sites(.)com

Script not required. You can create/use linkgrabber filter rule:

if > sourceurl(s) > contains not > mysite.com
link origin > is > Clipboard

[xefeg]

Originally Posted by mgpai

Script not required. You can create/use linkgrabber filter rule:

if > sourceurl(s) > contains not > mysite.com
link origin > is > Clipboard

It worked Thanks!
But how to add multiple URLs? adding 3 rules does not work. But 1 rule is working with 1 url

[mgpai]

Originally Posted by xefeg

But how to add multiple URLs?

Use regex (enable regex checbox fo the field):

(one\.com|two\.com|three\.com)

[xefeg]

Perfect and Thank you.

[Jiaz]

@mgpai: thanks for your quick help

[booyabeetz]

Hi There.

I have been experimenting with event scripter and have run into some complications in trying to get it to do what I want.

Ultimately my goal is to make a script that when a downloaded finishes & extracts the script will check if an .iso file was extracted. If an .iso file is detected then the script should run a Unix Executable against this .iso file. This executable (called "extract-xiso") is designed to further extract the contents of the .iso into a folder of the same name as the .iso. Lastly the script should move the final extracted contents to a new directory.

At the moment however I am unable to get event scripter to run my unix executable. Here is the basic test script I have at the moment with the Unix executable located in the /output folder for simplicity. For reference I am using the Jdownloader docker container:

var extract = "/output/extract-xiso";
var file = "/output/test.iso";
callSync(extract, file);

When I test run this script I get the following error: "java.io.IOException: Cannot run program "/output/extract-xiso": error=2, No such file or directory"

Can you please advise what I am doing wrong? I am able to run this executable successfully from command line as follows: extract-xiso test.iso

For reference this unix executable has been compiled to run on a 64-bit linux OS. It does not have a file extension. Any help would be much appreciated. Thanks in advance.

[Jiaz]

Originally Posted by booyabeetz

When I test run this script I get the following error: "java.io.IOException: Cannot run program "/output/extract-xiso": error=2, No such file or directory".

Sure the path to the executable is correct and it is executable? (flag is set? chmod +x /output/extract-xiso)

[booyabeetz]

File paths seem to be correct. I confirmed that extract-xiso is executable. It is also showing as bright green in the docker console. I have this docker container installed on my Unraid system so as a test I executed the file from Unraid terminal. Running "file extract-xiso" from Unraid terminal gives me the following: "extract-xiso: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=7857e5a9ca7d61be7223fba2d6918be2a3fcc6af, not stripped"

For reference this is executable I am trying to use although I had to build from source: **External links are only visible to Support Staff**...

I am not an expert on any of this stuff so maybe there is something basic that I am missing here...

[StefanM]

Request: Script for exporting Adcanced Settings

@mpgai
Hi again!
You already helped me before. So, maybe you can/will help once again.

I discussed with Jiaz a feature request to export Advanced Settings to a file. And he told me that this could be done by using a script.

I would like to export all lines and all 4 (5) columns in TSV format - if possible:

Key  Description  Value  Actions (parts 1 and 2)

Following is just a suggestion. Maybe there are other/better ways or ways that are easier to implement.

My suggestion:
(Contents of that TSV file)

1. Column Key should contain the original text as you can see it in Advanced Settings.
2. Column Description should contain the original text as you can see it in Advanced Settings.
3. Column Value: please see detailed description below
4. Column Actions, part 1 [?] should contain the message you get when clicking on the Question Mark (if that is possible at all!).
5. Column Actions, part 2 should read Default for default setting (gray arrow) and Custom for custom setting (yellow arrow).

Details for Columns Value, the most complicated part, I guess
I don't want to write any too specific requirements, because I don't know, what is possible.

Examples:

Of course it would be great, to have that what you see here in example 1 in a column:

Spoiler:

In below case, example 2, the word checked or unchecked would describe it best, I guess

Spoiler:

And in the following two cases, examples 3 and 4, just writing SYSTEM_DEFAULT or When the actual Download starts would be just fine.

Spoiler:

Spoiler:

Do you think, you could write such a script?
Thanks for any efforts taken in advance!

PS: What do I need that TSV file for?
I frequently modify many of those Advanced Settings and I would like to comment, why I made this or that setting.

At the moment I have to create a single document for every setting. And I cannot even copy everything from Advanced Settings list. :(

[Jiaz]

@mgpai: I recommend to store the (default)values JSON encoded as strings for easier storage as key/value will be strings then.

[StefanM]

Originally Posted by Jiaz

@mgpai: I recommend to store the (default)values JSON encoded as strings for easier storage as key/value will be strings then.

Thanks a lot for your assistance and support!
It sounds that there really is a (not too complicated) way to export those data.

Hope that mgpai will see it the same way...

[mgpai]

Originally Posted by theskyishigh

... remove all finished packages/files that have been deleted/moved.

/*
    Remove links
    Trigger : JDownloader Started
*/

getAllFilePackages().forEach(function(package) {
    package.finished && package.downloadLinks.forEach(function(link) {
        !getPath(link.downloadPath).exists() && link.remove();
    })
})

[mgpai]

Originally Posted by StefanM

Script for exporting Adcanced Settings

The script will export the settings in JSON format. You can use excel to import/format it.

/*
    Export settings
    Trigger: None
*/

var file = JD_HOME + "/settings.json";
var settings = callAPI("config", "list", ".+", true, true, true, true);

writeFile(file, JSON.stringify(settings, null, 2), false);

[StefanM]

Originally Posted by mgpai

The script will export the settings in JSON format. You can use excel to import/format it.

/*
    Export settings
    Trigger: None
*/

var file = JD_HOME + "/settings.json";
var settings = callAPI("config", "list", ".+", true, true, true, true);

writeFile(file, JSON.stringify(settings, null, 2), false);

First of all: Thanks for the prompt reply. If I may, I have some questions for you. And I hope I'm not too intrusive or impudent...

After Excel import I get this (example)



which is this line in Advanced Settings:



Question 1:
Is there any way, to create something like a tab separated file instead?
This would solve almost all below problems/questions.

Question 2:
How do I search for a specific key in Excel? I consider myself quite experienced with Excel, but never let Excel handle any *.json files for me.

Question 3:
I notice that the key name is shown in two parts: In my example, I will find part 1 (GeneralSettings) in InterfaceName and I will find the rest of the key in (key): DeleteEmptySub...
The key, however, in Advanced Settings is: GeneralSettings: Delete Empty Sub Folders...
So, even the naming is different:
DeleteEmptySubFolders... (in Excel import) vs. GeneralSettings: Delete Empty Sub Folders... (in Advanced Settings)

Question 4:
Any chance to show the key as it is displayed the same way as in Advanced Settings (green instead of red)?

Question 5:
How do I sort those keys in Excel? How can I display the key names?
At the moment each and every line is just called 'Record'.

Question 6:
I would also like to be able to use Excel filters, e.g. just showing me all keys where there is a custom value.

Question 7:
I want to use the exported file (in Excel) to write comments for some of those keys. How do I do that in this format? This was/is my main intention

EDIT

In the meantime I found a tutorial here: howtogeek.com/775651/how-to-convert-a-json-file-to-microsoft-excel/

Anyway, is it possible to create a tab separated file directly?

[Jiaz]

Originally Posted by StefanM

Question 1:
Is there any way, to create something like a tab separated file instead?
This would solve almost all below problems/questions.

@mgpai: instead write out the whole json at once, loop/interate through the entries and write out
the information in CSV format, make sure to JSON.stringify the value

Originally Posted by StefanM

Question 3:
I notice that the key name is shown in two parts: In my example, I will find part 1 (GeneralSettings) in InterfaceName and I will find the rest of the key in (key): DeleteEmptySub...
The key, however, in Advanced Settings is: GeneralSettings: Delete Empty Sub Folders...
So, even the naming is different:
DeleteEmptySubFolders... (in Excel import) vs. GeneralSettings: Delete Empty Sub Folders... (in Advanced Settings)

Question 4:
Any chance to show the key as it is displayed the same way as in Advanced Settings (green instead of red)?

I don't recommend doing this as the names display in Advanced Settings are *beautified*. You should stick with the *real* keys , especially if you're having plans to be able to import/load back those settings. you need interfacename/storage and key to correctly identify a setting

[StefanM]

@mgpai, @Jiaz:

Originally Posted by Jiaz

@mgpai:
I don't recommend doing this [as quoted below] as the names display in Advanced Settings are *beautified*. You should stick with the *real* keys , especially if you're having plans to be able to import/load back those settings. you need interfacename/storage and key to correctly identify a setting

Originally Posted by StefanM

Question 3:
I notice that the key name is shown in two parts: In my example, I will find part 1 (GeneralSettings) in InterfaceName and I will find the rest of the key in (key): DeleteEmptySub...
The key, however, in Advanced Settings is: GeneralSettings: Delete Empty Sub Folders...
So, even the naming is different:
DeleteEmptySubFolders... (in Excel import) vs. GeneralSettings: Delete Empty Sub Folders... (in Advanced Settings)

Question 4:
Any chance to show the key as it is displayed the same way as in Advanced Settings (green instead of red)?

@mgpai::
I follow Jiaz's recommendation here. So, please disregard my requests as per Q3 and Q4.
It sounds really good, that there even might be a chance to import/restore those exported settings later.

@mgpai, @Jiaz:
Thanks to both of you for all your efforts!

[StefanM]

@mgpai:

Just found out, that when using Excel, the limit is 1,000 rows.
Message from Excel 2019

Limit of 1000scanned rows reached

So, even when I would use Excel, I would never get the complete list.
That's why I would need a TSV file, please!

Originally Posted by StefanM

In the meantime I found a tutorial here: howtogeek.com/775651/how-to-convert-a-json-file-to-microsoft-excel/

Anyway, is it possible to create a tab separated file directly?

[Jiaz]

Originally Posted by StefanM

So, even when I would use Excel, I would never get the complete list.
That's why I would need a TSV file, please!

I'm sure mgpai will find time to modify script to write TSV file instead. just give him time and a weekend

[StefanM]

Originally Posted by Jiaz

I'm sure mgpai will find time to modify script to write TSV file instead. just give him time and a weekend

Of course, just wanted to share this.
As this was completely new to me: limitations of only 1,000 rows, while on the other hand Excel can handle sheets with 1,048,576 rows and 16,384 columns...

Guess, the 'json-module' in Excel is pretty old, as it is also extremely slow...

[Jiaz]

Originally Posted by StefanM

Guess, the 'json-module' in Excel is pretty old, as it is also extremely slow...

see limitation here, support.microsoft.com/en-us/office/power-query-specifications-and-limits-in-excel-5fb2807c-1b16-4257-aa5b-6793f051a9f4
One *workaround* woud be to split the exported config into max 1000 rows per file
then you can import them one by one. Issue might also happen when the json object changes and new columns show up, see
stackoverflow.com/questions/50230358/llimitation-while-converting-big-json-into-csv-in-power-query-editor

[mgpai]

Originally Posted by Jiaz

instead write out the whole json at once, loop/interate through the entries and write out
the information in CSV format, make sure to JSON.stringify the value

I only wish it were that simple. The db uses the following keys, but not all keys are used in all objects. The number of keys used in an object varies between 4 and 10.

[
 "abstractType",
 "defaultValue",
 "docs",
 "enumLabel",
 "enumOptions",
 "interfaceName",
 "key",
 "storage",
 "type",
 "value"
]

While it is possible, it will take more than a simple iteration. The 'tsv' file is ultimately intended for use in excel, which is better equipped to handle db structure like this one. When imported correctly, it will contain a table with above 10 keys as column headers, with rows empty or filled with data, depending on whether it is present in the original object.

Originally Posted by StefanM

Of course, just wanted to share this.
As this was completely new to me: limitations of only 1,000 rows, while on the other hand Excel can handle sheets with 1,048,576 rows and 16,384 columns...

Guess, the 'json-module' in Excel is pretty old, as it is also extremely slow...

Check the page linked by Jiaz. It is a navigation items limitation which will be applicable to all db types, not only JSON. From what I understand, by default it will be limited to the number of column headers (keys) in the database or 1000, whichever is less, with the option of manually adding more as required.

In this case it should be equal to 10, not 1000, which is the number of keys listed above.

[StefanM]

Originally Posted by Jiaz

see limitation here, ...

@Jiaz

Funny thing is, that I see all 1,178 rows. But I don't know, whether or not everything is complete.

Another things that puzzles me is, that every time I e.g. filter a column, and then reset all filters, Excel will only load (once again) only a small part of those 1,178 rows and I have to scroll down several times, until the last row has been reached once again.

And all of this is slow: I takes a couple of seconds.

And I also cannot adjust the column widths to any size I want. The last column, I can only set to some 30 or 50 % of the width needed.


I know, all this is kind of off topic, but maybe you are familiar with handling *.json files with Excel. And you can answer this anyway:

As I'm always curious and eager to learn new things (in this case handling *.json files and Excel)...

How do I copy/save those rows and columns from the attached screenshot to a 'normal' Excel sheet?
Any other opened Excel window is blocked, as long as the attached window is open.

What I did, was copy it to another Excel sheet on another PC, connected via Remote Desktop. And also copying it to a txt file and later back to Excel would work.

But I guess there must be another way.

EDIT

Got 'Close & Load' finally working, but the result is not correct. Because now I have 2,618 rows instead of 1,178. Maybe because of that 1,000 rows limitation...

[StefanM]

Originally Posted by mgpai

Check the page linked by Jiaz. It is a navigation items limitation which will be applicable to all db types, not only JSON. From what I understand, by default it will be limited to the number of column headers (keys) in the database or 1000, whichever is less, with the option of manually adding more as required.

In this case it should be equal to 10, not 1000, which is the number of keys listed above.

@mpgai:
Well, for me it's the first time that I got that warning. Never saw it before with other databases.

If you read my post addressed to Jiaz, you will see some other 'peculiar' behavior of Excel I experienced here.

Anyway, do you think you can find a way to create a tsv file? Or is it that complicated or even impossible, that you have to give up?

[mgpai]

Originally Posted by StefanM

Or is it that complicated or even impossible, that you have to give up?

Really? That is what you took from what I wrote?

[StefanM]

Originally Posted by mgpai

Really? That is what you took from what I wrote?

@mgpai:
Please accept my apologies!

I guess, this is what you are referring to?

Originally Posted by mgpai

While it is possible, it will take more than a simple iteration. The 'tsv' file is ultimately intended for use in excel, which is better equipped to handle db structure like this one. When imported correctly, it will contain a table with above 10 keys as column headers, with rows empty or filled with data, depending on whether it is present in the original object.

But I don't know how to interpret it:

• More than a simple iteration -> I understand it is more complicated
• 'tsv' file ultimately intended for use in Excel -> Yes, when I could use a tsv file, in particular there wouldn't be all those problems I mentioned here.

But I still don't know, whether or not you are prepared to create a tsv generating script.

And I have no idea, how much time that would take, and if it would be OK or if it even would be impudent or outrageous to ask for such a script.

Once again, I apologize, in case my reaction sounded kind of rude, which for sure wasn't my intention.

But if you decide to create that tsv script (with those 10 columns) I can only say once again thanks a lot for your efforts!

[mgpai]

Originally Posted by StefanM

Please accept my apologies!

No worries.

Originally Posted by StefanM

But I still don't know, whether or not you are prepared to create a tsv generating script.

The JSON file which the scirpt provides should have been enough for you to see things through in excel, especially since it is very well equipped to perform such tasks. So the question really is not "whether I am prepared to" or "whether I can", but "do I need to". Since I have provided a viable solution, the answer should have been obvious. That being said, here is the script:

/*
    Export settings
    Trigger : none
*/    

var rows = [];
var file = JD_HOME + "/settings.tsv";
var settings = callAPI("config", "list", ".+", true, true, true, true);
var keys = [
    "abstractType",
    "defaultValue",
    "docs",
    "enumLabel",
    "enumOptions",
    "interfaceName",
    "key",
    "storage",
    "type",
    "value"
];

rows.push(keys.join("\t"));

settings.forEach(function(item) {
    var row = [];

    keys.forEach(function(key) {
        row.push(JSON.stringify(item[key]));
    })

    rows.push(row.join("\t"));
})

writeFile(file, rows.join("\n"), false);

Let me know if you need any modifications.

[StefanM]

Originally Posted by mgpai

...That being said, here is the script:

Haven't started testing, yet. But nevertheless already now I have to say:

Thank you so much for your time and efforts. I really appreciate it.
Will get back to you with my report later.

Will also try to verify the result/columns by comparing the Excel-result (importing the *json file to Excel that is) with the result of the script you just sent me.

Once again: Thank you very much.

[StefanM]

Originally Posted by mgpai

Let me know if you need any modifications.

Now, here are my test results:

While the *.json export misses some information, the *.tsv export seems to be complete.

Examples (first line is from tsv file, second line is from json file)

["-i","%audio","-vn","-f","ogg","-acodec","libvorbis","-aq","4","%out","-y"]
[List]

{"mp4":"mp4"}
[Record]

If I may ask for a minor modification:
Would it be possible to change the name of the output (tsv) file to something like this?
2022-06-20-11-35-10.tsv
YYYY-MM-DD-hh-mm-ss.tsv

And a far more complex request would be the possibility to restore the settings from the saved tsv file, writing them back to JD that is.

But I'm afraid that this would probably really be too much work load.
But if I'm wrong and it's not that complicated, I thought I might as well ask...

And once more: Thank you very much for what you already did.
It's a great and very useful script!

[mgpai]

Originally Posted by StefanM

... change the name of the output

var file = JD_HOME + "/" + new Date().toISOString().substr(0, 19).replace(/[^\d]/g, "-") + ".tsv";

Originally Posted by StefanM

.. restore the settings from the saved tsv file, writing them back to JD that is.

We'll see. No ETA though.

Originally Posted by StefanM

Thank you ..

You're welcome.

[Jiaz]

Originally Posted by StefanM

While the *.json export misses some information, the *.tsv export seems to be complete.
Examples (first line is from tsv file, second line is from json file)

["-i","%audio","-vn","-f","ogg","-acodec","libvorbis","-aq","4","%out","-y"]
[List]

{"mp4":"mp4"}
[Record]

What makes you think that there is something missing here? Those are two different settings.

[StefanM]

Originally Posted by Jiaz

What makes you think that there is something missing here? Those are two different settings.

Some misunderstanding:

Example 1:

in tsv it reads: ["-i","%audio","-vn","-f","ogg","-acodec","libvorbis","-aq","4","%out","-y"]
In json the same reads only:[List]

Same applies to example 2

Here instead of: {"mp4":"mp4"}
in json it only reads: [Record]

Please have a look at the attached screenshot:
https://i.imgur.com/9oIEVFh.jpg

Black is tsv export and blue is json export.

[StefanM]

Originally Posted by mgpai

var file = JD_HOME + "/" + new Date().toISOString().substr(0, 19).replace(/[^\d]/g, "-") + ".tsv";

We'll see. No ETA though.

You're welcome.

Even without any ETA...
... thanks a lot for trying, maybe, some time later...

[TomNguyen]

Originally Posted by mgpai

/*
    Random delay before download
    Trigger : A Download Started
    IMPORTANT : Enable "Synchronous execution" checkbox in the top panel
*/

var seconds = Math.floor(Math.random() * 5) + 1 // 1 to 5 seconds

while (seconds && !link.aborting) {
    sleep(seconds * 1000);
    seconds--;
}

Dear mgpai,
Today I got this error when running this script

net.sourceforge.htmlunit.corejs.javascript.EcmaError: ReferenceError: "link" is not defined. (#9)

I am not sure why. I have run it for a few days and this is the first time I got this error. I have created a log incase Jdownloader Team can take a look at it.

20.06.22 03.34.21 <--> 20.06.22 17.19.40 jdlog://6301035302851/

Thank you!

[StefanM]

Originally Posted by mgpai

Let me know if you need any modifications.

@mpgai:
Sorry to bother you once again. There is one thing I forgot from my initial spec:

A column which indicates whether or not the default settings for a row have been changed. (e.g. DEFAULT / CUSTOM)

Could you add such a column, please?

I'm referring to the yellow arrow in Advanced Settings, indicating that default settings have been changed:



Thanks once again!

[Jiaz]

@TomNguyen: You've set wrong Trigger

Trigger=ON_DOWNLOADS_RUNNING

But must set to "A Download started"

[Jiaz]

Originally Posted by StefanM

A column which indicates whether or not the default settings for a row have been changed. (e.g. DEFAULT / CUSTOM)

@mgpai: you could JSON.stringify default and value and compare to check if value is customized or not

[TomNguyen]

Originally Posted by Jiaz

@TomNguyen: You've set wrong Trigger
But must set to "A Download started"

Dear Jiaz. The trigger was set as: "Download controller started". I just set it to "A Download started" as your post. I will update the result after a few days. Thank you!

Update: It works well for the last week. Thanks again!

[Jiaz]

Originally Posted by TomNguyen

Thank you!

Nothing to thank me for and you're welcome!

[Jiaz]

Originally Posted by StefanM

in tsv it reads: ["-i","%audio","-vn","-f","ogg","-acodec","libvorbis","-aq","4","%out","-y"]
In json the same reads only:[List]
Here instead of: {"mp4":"mp4"}
in json it only reads: [Record]

Excel informs you that JSON value is a List or Record, the type and you have to query which information of that value you're intersted in
see marcusrauhut.com/importing-json-data-into-excel-power-query/
A single value may contain multiple other values and that's why a single value to column representation doesn't work. In TSV the value is stored as JSON String and that's why value <-> column assignment works fine.