[Viga562]

Hello everyone, I hope to find you in good health.

I'm running into a small difficulty, which I don't know if it's an issue or just a limitation.
With Instagram or YouTube plugins, when I set a custom filename, if there are placeholders that wouldn’t be relevant based on the downloaded file, the corresponding field won’t appear, which is right, but the punctuation marks related to the placeholder will, for their part, remain.


E.g. the Instagram plugin’s default filename scheme is (order-ID and shortcode included) :
uploader - main_content_id - orderID_shortcode.ext

For a given URL with 1 media, and another with multiple media, we get :
account.name - ABcDefGhIJk.jpg
account.name - LmnOPqrstUv - 1.jpg
account.name - hiJKlmNOpQr - 2.jpg

( or, specifically for a Profile URL crawl with the Website method, you could get :
account.name - LmnOPqrstUv - 1_WxYZabcDEFg.jpg
account.name - hiJKlmNOpQr - 2_sTuvwxYZabC.jpg )

In all these cases, we see that the filename adapts to the circumstances, and takes only the part of the scheme that is relevant to it.

Now, if you set a custom filename, say :
*uploader* - *main_content_id* - *orderid*_of_*orderid_max* - *shortcode**ext*

You get, for the same URLs :
account.name - ABcDefGhIJk - 1_of_1 - -.jpg
account.name - LmnOPqrstUv - 1_of_2 - -.jpg
account.name - hiJKlmNOpQr - 1_of_2 - -.jpg

( or, specifically for a Profile URL crawl with the Website method, you could get :
account.name - ABcDefGhIJk - 1_of_1 - ABcDefGhIJk.jpg
account.name - LmnOPqrstUv - 1_of_2 - WxYZabcDEFg.jpg
account.name - hiJKlmNOpQr - 1_of_2 - sTuvwxYZabC.jpg )

In all that cases, we see that, unlike before, the filename doesn't adapt itself to the circumstances, and generically integrate all the scheme.

We can see that for URLs with one or more media, *shortcode* will be replaced by a dash " - ", and the surrounding punctuation (dashes and spaces) remains.

Except for the URL/Website Profile method. For an URL with only 1 media, it will put 1_of_1 although it isn’t necessary, and will repeat *main_content_id* twice, using it as placeholder *shortcut*. For an URL with multiple media, you’ll get the right scheme.

[ Examples of usable URLs: **External links are only visible to Support Staff****External links are only visible to Support Staff** ; **External links are only visible to Support Staff****External links are only visible to Support Staff** ; **External links are only visible to Support Staff****External links are only visible to Support Staff** ]




The same happens with the YouTube plugin, but a missing part is replaced by a space.

E.g. there will be no problem with the default scheme :
*3D* *360* *VIDEO_NAME* (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*

You’ll get :
The video title (720p_30fps_H264-192kbit_AAC).mp4

But if you want to add the placeholder *PLAYLIST_NAME* between dashes "-" surrounded by spaces :
*3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - *H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*

For each video that isn’t part of a playlist or crawled with its playlist, you’ll get :
The video title - - (720p_30fps_H264-192kbit_AAC).mp4

*PLAYLIST_NAME* doesn’t appear at all, rightly so, but the surrounding punctuation (dashes and spaces) remains.

If the video is part of a playlist and crawled with it, this will be correct:
The video title - The playlist name - (720p_30fps_H264-192kbit_AAC).mp4

[ Examples of usable URLs : **External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff** ]

Etc. etc.

I already know that this can be handled with the packagizer, and I've already made the rules for it. But there are limitations, e.g. the packagizer rules don't apply to the variant of a YouTube video added via the LinkGrabber context menu.

As for Instagram, if you want to include mention of something like “ story “ or “ highlights “ or “ name of highlights “ or whatever, good luck (too many variables). It gets a little complex in my opinion compared to the original goal.

I've searched a lot on the forum, but I couldn't find anything on the subject, except this post that proposes a script, but that doesn't seem to work, for me at least (https://board.jdownloader.org/showpost.php?p=395430 ; showpost.php?p=395430). But couldn't we avoid going through a script? As with the default filename schemes the adaptation is correct, I thought it should work with custom schemes too ?

Thank you for your answers!

[pspzockerscene]

Hi,
I wouldn't consider this a bug.
Indeed these are the "limitations" of our current naming schemes.
...but there is more to it:
If you e.g. use the following scheme:

*uploader* - *main_content_id* - *orderid*_of_*orderid_max* - *shortcode**ext*

...how would you expect a software to auto detect that the following part as a whole is not necessary if *orderid_max* == 1?

 - *orderid*_of_*orderid_max*

I don't see how.
Also, you could use anything in between which would make it even more complicated for an automated logic to see "spaceholder XY makes no sense now so we can remove it and the space before":

 - *orderid*_of random text here __ bla _*orderid_max*

Also, how should the application know what you're using as a "separator" between used tags?
In your example it is mostly " - "but it could be anything.

Without an extended "search and replace" or "custom filenames editor" I don't see this happening while I do understand what you mean.

-psp-
EDIT

For such "complex" things, you could always use an EventScripter script to add more logic to apply custoim packagenames.
More information:
https://support.jdownloader.org/Know...event-scripter

[Jiaz]

@Viga562: I've got an idea and will test it...at least for dashes when the property/field is empty. will do some trials

[Viga562]

Thanks for the answers. First, sorry for the size of the previous post, I wanted to be as clear and precise as possible.

@pspzockerscene : I totally agree with you and understand the logic. Maybe it's just me who has reached my limits of software understanding. What was puzzling me was that with the default filenames schemes it works.

With YouTube, the *3D* *360* placeholders don't leave spaces.

With Instagram, for a URL with one media, let alone orderID_max, it still manages to not place the orderID at all, or else a shortcode (or a " - ") since there is none, or for URLs with multiple media, a shortcode (or a " - ") if the method does not find any.

@Jiaz : That's nice of you



Edits to original post

Spoiler:

Originally Posted by Viga562

Except for the URL/Website Profile method.

Should've said : Except for the Profile URL/Website method.

Originally Posted by Viga562

But if you want to add the placeholder *PLAYLIST_NAME* between dashes "-" surrounded by spaces :
*3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - *H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*

Forgot : *3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*

[Jiaz]

@Viga562: I have some ideas but of course they also will have their limits. When I understand right, you are looking for a way to define a name scheme, for example

Bla-placeholder-x.filetype

to result in

blax.filetype

but currently would result in

bla--x.filetype

so you want to be able to *tell JDownloader* that certain part of the scheme can be removed if the value of the placeholder is empty

Just asking to be sure we mean the same at least for the *start* of this feature/wish

[Viga562]

Hi Jiaz, thanks for your idea, really appreciated. Sorry for the delay of my reply, I'm in the middle of my exams, so you know...

And yes, that's it. So that in the end we have the most adaptive filename possible, although I know we can't do miracles either.

[pspzockerscene]

Originally Posted by Viga562

With YouTube, the *3D* *360* placeholders don't leave spaces.

With Instagram, for a URL with one media, let alone orderID_max, it still manages to not place the orderID at all, or else a shortcode (or a " - ") since there is none, or for URLs with multiple media, a shortcode (or a " - ") if the method does not find any.

All plugins have this "custom filename system" implemented in a different way because s far we do not have a generic system for that.
We got tickets for that though e.g.:


As explained I think atm. the easiest way to do this in a more "intelligent" way would be to use the EventScripter.

-psp-

[Jiaz]

@Viga562: first idea I would like to share and gather some thoughts on it

A syntax similiar to bbcode. some example

[*video_id*]_this is the *video_id*[/*video_id*]

if the video_id property is available then evaluate the content and output would be

_this is the 3213123

in case the video_is proprty is not set, then it won't be evaluated and output is empty

another example

[*video_id*]videoID yes[/*video_id*][!*video_id*]videoID no[/*video_id*!]

then output would be either "videoID yes" or "videoID no" depending if property is available or not

that way we could also provide some more sophisticated features like

[*video_id*][UPPERCASE]yes[/UPPERCASE][/*video_id*]

would result in "YES"

with such a syntax you can define what name should look like if property xy is available or not available. Please feel free to share comments/thoughts/ideas

[mgpai]

@Jiaz: Implementation in foobar audio player:

Filename:

song name.mp3

Titleformat:

%filename%[- %artist%].%filename_ext%

Output if artist tag is present:

song name - artist name.mp3

Output if artist tag is missing (all the contents inside the bracket are discarded):

song name.mp3

In JD, instead of brackets, maybe a different character which is normally not found/allowed in filenames can be used.

Attachment > other > syntax > [ ... ]

[Jiaz]

@mgpai:
thanks for the first feedback with my idea it is possible to create if/else situation, like

[*is_video*]This is a video file *file_id*[/*is_video*][!*is_video*]This is a NOT video file *file_id*[/*is_video*!]

or

*filename*[*artist*]-*artist*[/*artist*][!*artist*]_unknown artist[/*artist*!].*filename_ext*

results in

filename-artist.mp3

or

filename_unknown artist.mp3

[mgpai]

@Jiaz: Remembering syntax or tag names corresponding to each placeholder is a tough ask. Perhaps a uniform syntax for all placeholders, without using tag names?

FYI for if/else, foobar uses:

%filename% - $if(%artist%,%artist%,unknown artist).%filename_ext%

OR

%filename% - $if2(%artist%,unknowm artist).%filename_ext%

Output, ff artist tag is present:

song name - artist name.mp3

Output, if artist tag is absent/missing:

song name - unknown artist.mp3

Not that is wish to see it implemented in any particular way. Similar implementation in foolbar came to mind, which I though will share.

[Jiaz]

Originally Posted by mgpai

@Jiaz: Remembering syntax or tag names corresponding to each placeholder is a tough ask. Perhaps a uniform syntax for all placeholders, without using tag names?

hmm, what exactly do you mean by*without using tag names*?
filename and artist is also a tag name ? or do you mean *speaking* syntax like

$if(*video_id*...

instead of

[*video_id*]-*video_id*[/*video_id*]

I like the other(your example) syntax more but it's more complicated to parse/evaluate. Bot on the other side, foobar syntax is well documented
wiki.hydrogenaud.io/index.php?title=Foobar2000:Title_Formatting_Reference
so we just need to write parser/evaluator for this syntax

[mgpai]

Originally Posted by Jiaz

hmm, what exactly do you mean by*without using tag names*?
filename and artist is also a tag name ?

Meant without having to repeat/typing the placeholder names multiple times.

[*video_id*]-*video_id*[/*video_id*]

Something which remains same across all placholders?

Example:

[jd]-*video_id*[/jd]
[jd]-*date_addded*[/jd]

For if/else:

[JD]-(date_added|date_published|none)[/jd]

Just brainstorming. I understand implementing it is an entirely different beast.

Due to similar features, I cannot help but draw parallels between JD and foobar.

[Jiaz]

@mgpai: The more I read about foobar syntax, the more I like it I will try to create a poc for parser/evaluator

[pspzockerscene]

Originally Posted by Jiaz

so we just need to write parser/evaluator for this syntax

That's going to be a separate project :D

But I like that syntax too.
I've never really used applications with custom naming schemes so I'm unable to provide input here.

I'd like it to be as short as possible while still being readable and it should provide the ability to:
- Remove "gaps" if no data for a tag is available
- Provide fallback for "no data available" but I guess with those conditions that's already possible
- Format strings and especially timestamps to whatever the user prefers to use

Then plugins (also crawlers) could return a list of possible tags + description and those tags could just be used in our Packagizer.

...so far that would be my dream result.

-psp-

[pspzockerscene]

Some more thoughts on this:
Basides the replace system itself, this could potentially lead to the following more useful features:

• Generic filenames for e.g. all video websites: Define them once in one place and get the same filenames no matter what the source is
• More sophisticated Packagizer condition handling

[pspzockerscene]

Moved thread to Suggestions forum as the initial post is not a bugreport (if you ask me). It simply points out the limits of our current naming system(s).

[pspzockerscene]

Just for collecting input:

The yt-dlp syntax is also nice and has a lot of fields for common data:
mankier.com/1/yt-dlp#Output_Template

[JDmurphy]

Not sure what the current state on this is, but from all I can see or know, nothing has changed since the last post in 2022 - and I was sent here by a pspzs link from only a couple months ago, hence I'm assuming this to be still relevant.

So, as soon as I saw this:

Originally Posted by Jiaz

[*video_id*]videoID yes[/*video_id*][!*video_id*]videoID no[/*video_id*!]

I also immediately had to think of something extremely similar to this:

Originally Posted by mgpai

%filename%[- %artist%].%filename_ext%

... except - applying to both proposals - it wouldn't be exactly smart to use brackets as reserved characters for this since they're allowed for filenames in Windows as well as in Linux/Unix. You would then have to escape them, which unnecessarily causes a mess - idk about others but personally I use [...] in almost every single filename pattern, especially for ids, counters, etc.

That said, how about this:

single use

pattern:

foo< - *video_id*> - bar

case video_id defined:

foo - 6avJHaC3C2U - bar

case video_id undefined:

foo - bar

pattern:

foo<!*video_id* weirdly has no video id> - bar

video_id defined:

foo - bar

video_id undefined:

foo weirdly has no video id - bar

combined use

pattern:

foo< - *video_id*><!*video_id* weirdly has no video id> - bar

case video_id defined:

foo - 6avJHaC3C2U - bar

case video_id undefined:

foo weirdly has no video id - bar

shortening/simplifying this pattern:

[*video_id*]videoID *video_id*[/*video_id*][!*video_id*]no videoID[/*video_id*!]

to this pattern:

<videoID *video_id*><!*video_id* no videoID>

for combined use we could optionally think of the ternary operator
( utilizing a pipe symbol instead of a colon for mentioned reasons )

pattern:

foo<*video_id*? - *video_id*| weirdly has no video id> - bar

case video_id defined:

foo - 6avJHaC3C2U - bar

case video_id undefined:

foo weirdly has no video id - bar

shortening/simplifying this pattern:

[*video_id*]videoID *video_id*[/*video_id*][!*video_id*]no videoID[/*video_id*!]

to this pattern:

<*video_id*?videoID *video_id*|no videoID>

If someone really really wants to use chevrons in their *nix filenames, you could of course also use /.../ instead of <...> - but anyone who does that also eats babies, so I guess angle brackets are good enough.

I didn't read every single word above but most of it, so I guess this covers all of the essentials :]

[pspzockerscene]

Originally Posted by JDmurphy

Not sure what the current state on this is, but from all I can see or know, nothing has changed since the last post in 2022

That's right.

So far this thread is only a collection of ideas.

[pspzockerscene]

Todays update did add a more sophisticated global regex character replacement feature, see:
https://board.jdownloader.org/showthread.php?t=95411

[dbkarm]

To avoid unnecessary spaces I think the space separator could be stablished inside the expression.

Instead of:

*DATE[yyyyMMdd]* *3D* *360* *VIDEO_NAME*

Use this:

*DATE[yyyyMMdd] **3D **360 **VIDEO_NAME*

So if one of the fields does not exist or is empty, there are no additional spaces, because the asterisk touch each other.

[pspzockerscene]

That is actually even simpler than what I had in my mind lol
...but it will only work in a good way for spaces or some other 'special' characters.
Simply use the * as the 'tag end' character.
Problem with this:
If you want to add other chars to it, it will make the internal parsing reall hard to impossible e.g.:

*_bla_DATE[yyyyMMdd]__ **3D **360 **VIDEO_NAME*

Somethinf like this might be better machine-readable:

[jd:tag]_bla_*DATE[yyyyMMdd] [/jd:tag]**3D **360 **VIDEO_NAME*

--> Like just something that wraps around the existing tags if you want to add any other character as a part of them.
If no tag inside "[jd:tag]" is available, the whole thing just gets eliminated.

[Jiaz]

@dbkarm: yt plugin will support your idea with next core update

[JDmurphy]

Apparently our bug_reporter_girl didn't bother to follow the advice in #5692 so I'll invest the 2 mins of effort in her place:

2. An option to replace or remove emojis from file names would be nice, as they cause trouble with some file managers. For instance, when using "*CHANNELNAME*" and/or "*VIDEO_NAME*" as parameters for package/file name.

... because I think that's an extremely good suggestion.

Emoji [sic] (it's already the plural) in filenames are the literal plague of the 21st century and constantly having to remove them manually - or even worse, the utterly useless substitute characters - cost me at least 3 days combined over my whole time of using JD(2). Also in order to make myself look like I have no clue what I'm talking about: "Can't you just define emoji unicode blocks and yeet them entirely?"

I don't really know how any of that is actually on topic here (this is about defining placeholders, not about what can happen *if* you use certain placeholders) ... but hey, psp told us so :D

[JDmurphy]

Oh wow, this thread has a second page now ... I didn't even notice. Since [] still appear to be a hot contender for tag delimiters here (as if my post on the previous page never happened ) I'm not entirely sure what wasn't intelligible about this:

Originally Posted by JDmurphy

... except - applying to both proposals - it wouldn't be exactly smart to use brackets as reserved characters for this since they're allowed for filenames in Windows as well as in Linux/Unix. You would then have to escape them, which unnecessarily causes a mess - idk about others but personally I use [...] in almost every single filename pattern, especially for ids, counters, etc.

(followed by structured, concise proposals of a vastly simplified tag syntax using <> instead …)

*?*

[pspzockerscene]

@JDmurphy
You are on topic, all good.

Please just keep in mind that this topic is just for collecting ideas - we are currently not actively working on this one so more complex replacements will need to be done either outside JDownloader or using EventScripter scripts.