JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 01.06.2022, 15:43
Viga562 Viga562 is offline
I will play nice!
 
Join Date: Apr 2022
Posts: 4
Default Placeholders and inaccurate punctuation with custom name scheme of some plugin

Hello everyone, I hope to find you in good health.

I'm running into a small difficulty, which I don't know if it's an issue or just a limitation.
With Instagram or YouTube plugins, when I set a custom filename, if there are placeholders that wouldn’t be relevant based on the downloaded file, the corresponding field won’t appear, which is right, but the punctuation marks related to the placeholder will, for their part, remain.


E.g. the Instagram plugin’s default filename scheme is (order-ID and shortcode included) :
uploader - main_content_id - orderID_shortcode.ext

For a given URL with 1 media, and another with multiple media, we get :
account.name - ABcDefGhIJk.jpg
account.name - LmnOPqrstUv - 1.jpg
account.name - hiJKlmNOpQr - 2.jpg


( or, specifically for a Profile URL crawl with the Website method, you could get :
account.name - LmnOPqrstUv - 1_WxYZabcDEFg.jpg
account.name - hiJKlmNOpQr - 2_sTuvwxYZabC.jpg
)

In all these cases, we see that the filename adapts to the circumstances, and takes only the part of the scheme that is relevant to it.

Now, if you set a custom filename, say :
*uploader* - *main_content_id* - *orderid*_of_*orderid_max* - *shortcode**ext*

You get, for the same URLs :
account.name - ABcDefGhIJk - 1_of_1 - -.jpg
account.name - LmnOPqrstUv - 1_of_2 - -.jpg
account.name - hiJKlmNOpQr - 1_of_2 - -.jpg


( or, specifically for a Profile URL crawl with the Website method, you could get :
account.name - ABcDefGhIJk - 1_of_1 - ABcDefGhIJk.jpg
account.name - LmnOPqrstUv - 1_of_2 - WxYZabcDEFg.jpg
account.name - hiJKlmNOpQr - 1_of_2 - sTuvwxYZabC.jpg
)

In all that cases, we see that, unlike before, the filename doesn't adapt itself to the circumstances, and generically integrate all the scheme.

We can see that for URLs with one or more media, *shortcode* will be replaced by a dash " - ", and the surrounding punctuation (dashes and spaces) remains.

Except for the URL/Website Profile method. For an URL with only 1 media, it will put 1_of_1 although it isn’t necessary, and will repeat *main_content_id* twice, using it as placeholder *shortcut*. For an URL with multiple media, you’ll get the right scheme.

[ Examples of usable URLs: **External links are only visible to Support Staff****External links are only visible to Support Staff** ; **External links are only visible to Support Staff****External links are only visible to Support Staff** ; **External links are only visible to Support Staff****External links are only visible to Support Staff** ]




The same happens with the YouTube plugin, but a missing part is replaced by a space.

E.g. there will be no problem with the default scheme :
*3D* *360* *VIDEO_NAME* (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*

You’ll get :
The video title (720p_30fps_H264-192kbit_AAC).mp4

But if you want to add the placeholder *PLAYLIST_NAME* between dashes "-" surrounded by spaces :
*3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - *H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*

For each video that isn’t part of a playlist or crawled with its playlist, you’ll get :
The video title - - (720p_30fps_H264-192kbit_AAC).mp4

*PLAYLIST_NAME* doesn’t appear at all, rightly so, but the surrounding punctuation (dashes and spaces) remains.

If the video is part of a playlist and crawled with it, this will be correct:
The video title - The playlist name - (720p_30fps_H264-192kbit_AAC).mp4

[ Examples of usable URLs : **External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff** ]

Etc. etc.

I already know that this can be handled with the packagizer, and I've already made the rules for it. But there are limitations, e.g. the packagizer rules don't apply to the variant of a YouTube video added via the LinkGrabber context menu.

As for Instagram, if you want to include mention of something like “ story “ or “ highlights “ or “ name of highlights “ or whatever, good luck (too many variables). It gets a little complex in my opinion compared to the original goal.

I've searched a lot on the forum, but I couldn't find anything on the subject, except this post that proposes a script, but that doesn't seem to work, for me at least (https://board.jdownloader.org/showpost.php?p=395430 ; showpost.php?p=395430). But couldn't we avoid going through a script? As with the default filename schemes the adaptation is correct, I thought it should work with custom schemes too ?

Thank you for your answers!
Reply With Quote
  #2  
Old 01.06.2022, 18:40
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Hi,
I wouldn't consider this a bug.
Indeed these are the "limitations" of our current naming schemes.
...but there is more to it:
If you e.g. use the following scheme:
Code:
*uploader* - *main_content_id* - *orderid*_of_*orderid_max* - *shortcode**ext*
...how would you expect a software to auto detect that the following part as a whole is not necessary if *orderid_max* == 1?
Code:
 - *orderid*_of_*orderid_max*
I don't see how.
Also, you could use anything in between which would make it even more complicated for an automated logic to see "spaceholder XY makes no sense now so we can remove it and the space before":
Code:
 - *orderid*_of random text here __ bla _*orderid_max*
Also, how should the application know what you're using as a "separator" between used tags?
In your example it is mostly " - "but it could be anything.

Without an extended "search and replace" or "custom filenames editor" I don't see this happening while I do understand what you mean.

-psp-
EDIT

For such "complex" things, you could always use an EventScripter script to add more logic to apply custoim packagenames.
More information:
https://support.jdownloader.org/Know...event-scripter
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 01.06.2022, 20:45
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,339
Default

@Viga562: I've got an idea and will test it...at least for dashes when the property/field is empty. will do some trials
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 03.06.2022, 17:34
Viga562 Viga562 is offline
I will play nice!
 
Join Date: Apr 2022
Posts: 4
Default

Thanks for the answers. First, sorry for the size of the previous post, I wanted to be as clear and precise as possible.

@pspzockerscene : I totally agree with you and understand the logic. Maybe it's just me who has reached my limits of software understanding. What was puzzling me was that with the default filenames schemes it works.

With YouTube, the *3D* *360* placeholders don't leave spaces.

With Instagram, for a URL with one media, let alone orderID_max, it still manages to not place the orderID at all, or else a shortcode (or a " - ") since there is none, or for URLs with multiple media, a shortcode (or a " - ") if the method does not find any.

@Jiaz : That's nice of you



Edits to original post
Spoiler:
Quote:
Originally Posted by Viga562 View Post
Except for the URL/Website Profile method.
Should've said : Except for the Profile URL/Website method.
Quote:
Originally Posted by Viga562 View Post
But if you want to add the placeholder *PLAYLIST_NAME* between dashes "-" surrounded by spaces :
*3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - *H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*
Forgot : *3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT*
Reply With Quote
  #5  
Old 03.06.2022, 20:22
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,339
Default

@Viga562: I have some ideas but of course they also will have their limits. When I understand right, you are looking for a way to define a name scheme, for example
Quote:
Bla-placeholder-x.filetype
to result in
Quote:
blax.filetype
but currently would result in
Quote:
bla--x.filetype
so you want to be able to *tell JDownloader* that certain part of the scheme can be removed if the value of the placeholder is empty

Just asking to be sure we mean the same at least for the *start* of this feature/wish
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 12.06.2022, 12:22
Viga562 Viga562 is offline
I will play nice!
 
Join Date: Apr 2022
Posts: 4
Default

Hi Jiaz, thanks for your idea, really appreciated. Sorry for the delay of my reply, I'm in the middle of my exams, so you know...

And yes, that's it. So that in the end we have the most adaptive filename possible, although I know we can't do miracles either.

Reply With Quote
  #7  
Old 14.06.2022, 13:21
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Viga562 View Post
With YouTube, the *3D* *360* placeholders don't leave spaces.

With Instagram, for a URL with one media, let alone orderID_max, it still manages to not place the orderID at all, or else a shortcode (or a " - ") since there is none, or for URLs with multiple media, a shortcode (or a " - ") if the method does not find any.
All plugins have this "custom filename system" implemented in a different way because s far we do not have a generic system for that.
We got tickets for that though e.g.:


As explained I think atm. the easiest way to do this in a more "intelligent" way would be to use the EventScripter.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #8  
Old 22.06.2022, 18:16
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,339
Default

@Viga562: first idea I would like to share and gather some thoughts on it

A syntax similiar to bbcode. some example
Quote:
[*video_id*]_this is the *video_id*[/*video_id*]
if the video_id property is available then evaluate the content and output would be
Quote:
_this is the 3213123
in case the video_is proprty is not set, then it won't be evaluated and output is empty

another example
Quote:
[*video_id*]videoID yes[/*video_id*][!*video_id*]videoID no[/*video_id*!]
then output would be either "videoID yes" or "videoID no" depending if property is available or not

that way we could also provide some more sophisticated features like
Quote:
[*video_id*][UPPERCASE]yes[/UPPERCASE][/*video_id*]
would result in "YES"

with such a syntax you can define what name should look like if property xy is available or not available. Please feel free to share comments/thoughts/ideas
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 22.06.2022 at 18:22.
Reply With Quote
  #9  
Old 22.06.2022, 19:17
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

@Jiaz: Implementation in foobar audio player:

Filename:
Code:
song name.mp3

Titleformat:
Code:
%filename%[- %artist%].%filename_ext%

Output if artist tag is present:
Code:
song name - artist name.mp3

Output if artist tag is missing (all the contents inside the bracket are discarded):
Code:
song name.mp3

In JD, instead of brackets, maybe a different character which is normally not found/allowed in filenames can be used.

Attachment > other > syntax > [ ... ]
Attached Files
File Type: html Title Formatting Syntax Reference.html (21.2 KB, 0 views)
Reply With Quote
  #10  
Old 22.06.2022, 19:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,339
Default

@mgpai:
thanks for the first feedback with my idea it is possible to create if/else situation, like
Quote:
[*is_video*]This is a video file *file_id*[/*is_video*][!*is_video*]This is a NOT video file *file_id*[/*is_video*!]
or

Quote:
*filename*[*artist*]-*artist*[/*artist*][!*artist*]_unknown artist[/*artist*!].*filename_ext*
results in
Quote:
filename-artist.mp3
or
Quote:
filename_unknown artist.mp3
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 22.06.2022, 20:00
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

@Jiaz: Remembering syntax or tag names corresponding to each placeholder is a tough ask. Perhaps a uniform syntax for all placeholders, without using tag names?

FYI for if/else, foobar uses:

Code:
%filename% - $if(%artist%,%artist%,unknown artist).%filename_ext%

OR

Code:
%filename% - $if2(%artist%,unknowm artist).%filename_ext%

Output, ff artist tag is present:
Code:
song name - artist name.mp3

Output, if artist tag is absent/missing:
Code:
song name - unknown artist.mp3

Not that is wish to see it implemented in any particular way. Similar implementation in foolbar came to mind, which I though will share.
Reply With Quote
  #12  
Old 22.06.2022, 20:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,339
Default

Quote:
Originally Posted by mgpai View Post
@Jiaz: Remembering syntax or tag names corresponding to each placeholder is a tough ask. Perhaps a uniform syntax for all placeholders, without using tag names?
hmm, what exactly do you mean by*without using tag names*?
filename and artist is also a tag name ? or do you mean *speaking* syntax like
Quote:
$if(*video_id*...
instead of
Quote:
[*video_id*]-*video_id*[/*video_id*]
I like the other(your example) syntax more but it's more complicated to parse/evaluate. Bot on the other side, foobar syntax is well documented
wiki.hydrogenaud.io/index.php?title=Foobar2000:Title_Formatting_Reference
so we just need to write parser/evaluator for this syntax
__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 22.06.2022, 20:58
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by Jiaz View Post
hmm, what exactly do you mean by*without using tag names*?
filename and artist is also a tag name ?
Meant without having to repeat/typing the placeholder names multiple times.
Code:
[*video_id*]-*video_id*[/*video_id*]

Something which remains same across all placholders?

Example:
Code:
[jd]-*video_id*[/jd]
[jd]-*date_addded*[/jd]

For if/else:
Code:
[JD]-(date_added|date_published|none)[/jd]

Just brainstorming. I understand implementing it is an entirely different beast.

Due to similar features, I cannot help but draw parallels between JD and foobar.

Last edited by mgpai; 22.06.2022 at 21:01.
Reply With Quote
  #14  
Old 23.06.2022, 12:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,339
Default

@mgpai: The more I read about foobar syntax, the more I like it I will try to create a poc for parser/evaluator
__________________
JD-Dev & Server-Admin
Reply With Quote
  #15  
Old 23.06.2022, 15:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Jiaz View Post
so we just need to write parser/evaluator for this syntax
That's going to be a separate project :D

But I like that syntax too.
I've never really used applications with custom naming schemes so I'm unable to provide input here.

I'd like it to be as short as possible while still being readable and it should provide the ability to:
- Remove "gaps" if no data for a tag is available
- Provide fallback for "no data available" but I guess with those conditions that's already possible
- Format strings and especially timestamps to whatever the user prefers to use

Then plugins (also crawlers) could return a list of possible tags + description and those tags could just be used in our Packagizer.

...so far that would be my dream result.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #16  
Old 24.06.2022, 13:02
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Some more thoughts on this:
Basides the replace system itself, this could potentially lead to the following more useful features:
  • Generic filenames for e.g. all video websites: Define them once in one place and get the same filenames no matter what the source is
  • More sophisticated Packagizer condition handling
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #17  
Old 31.08.2022, 14:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Moved thread to Suggestions forum as the initial post is not a bugreport (if you ask me). It simply points out the limits of our current naming system(s).
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #18  
Old 20.10.2022, 12:20
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Just for collecting input:

The yt-dlp syntax is also nice and has a lot of fields for common data:
mankier.com/1/yt-dlp#Output_Template
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #19  
Old 17.03.2024, 16:48
JDmurphy JDmurphy is offline
Fibre Channel User
 
Join Date: Mar 2012
Posts: 121
Lightbulb Simple Filename Placeholder Suggestion

Not sure what the current state on this is, but from all I can see or know, nothing has changed since the last post in 2022 - and I was sent here by a pspzs link from only a couple months ago, hence I'm assuming this to be still relevant.

So, as soon as I saw this:

Quote:
Originally Posted by Jiaz View Post
Code:
[*video_id*]videoID yes[/*video_id*][!*video_id*]videoID no[/*video_id*!]
I also immediately had to think of something extremely similar to this:

Quote:
Originally Posted by mgpai View Post
Code:
%filename%[- %artist%].%filename_ext%
... except - applying to both proposals - it wouldn't be exactly smart to use brackets as reserved characters for this since they're allowed for filenames in Windows as well as in Linux/Unix. You would then have to escape them, which unnecessarily causes a mess - idk about others but personally I use [...] in almost every single filename pattern, especially for ids, counters, etc.

That said, how about this:

single use

pattern:
Code:
foo< - *video_id*> - bar
case video_id defined:
Code:
foo - 6avJHaC3C2U - bar
case video_id undefined:
Code:
foo - bar

pattern:
Code:
foo<!*video_id* weirdly has no video id> - bar
video_id defined:
Code:
foo - bar
video_id undefined:
Code:
foo weirdly has no video id - bar

combined use

pattern:
Code:
foo< - *video_id*><!*video_id* weirdly has no video id> - bar
case video_id defined:
Code:
foo - 6avJHaC3C2U - bar
case video_id undefined:
Code:
foo weirdly has no video id - bar

shortening/simplifying this pattern:
Code:
[*video_id*]videoID *video_id*[/*video_id*][!*video_id*]no videoID[/*video_id*!]
to this pattern:
Code:
<videoID *video_id*><!*video_id* no videoID>



for combined use we could optionally think of the ternary operator
( utilizing a pipe symbol instead of a colon for mentioned reasons )

pattern:
Code:
foo<*video_id*? - *video_id*| weirdly has no video id> - bar
case video_id defined:
Code:
foo - 6avJHaC3C2U - bar
case video_id undefined:
Code:
foo weirdly has no video id - bar

shortening/simplifying this pattern:
Code:
[*video_id*]videoID *video_id*[/*video_id*][!*video_id*]no videoID[/*video_id*!]
to this pattern:
Code:
<*video_id*?videoID *video_id*|no videoID>



If someone really really wants to use chevrons in their *nix filenames, you could of course also use /.../ instead of <...> - but anyone who does that also eats babies, so I guess angle brackets are good enough.

I didn't read every single word above but most of it, so I guess this covers all of the essentials :]

Last edited by JDmurphy; 17.03.2024 at 17:01.
Reply With Quote
  #20  
Old 18.03.2024, 13:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by JDmurphy View Post
Not sure what the current state on this is, but from all I can see or know, nothing has changed since the last post in 2022
That's right.

So far this thread is only a collection of ideas.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 10:14.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.