#1
|
|||
|
|||
Placeholders and inaccurate punctuation with custom name scheme of some plugin
Hello everyone, I hope to find you in good health.
I'm running into a small difficulty, which I don't know if it's an issue or just a limitation. With Instagram or YouTube plugins, when I set a custom filename, if there are placeholders that wouldn’t be relevant based on the downloaded file, the corresponding field won’t appear, which is right, but the punctuation marks related to the placeholder will, for their part, remain. E.g. the Instagram plugin’s default filename scheme is (order-ID and shortcode included) : uploader - main_content_id - orderID_shortcode.ext For a given URL with 1 media, and another with multiple media, we get : account.name - ABcDefGhIJk.jpg account.name - LmnOPqrstUv - 1.jpg account.name - hiJKlmNOpQr - 2.jpg ( or, specifically for a Profile URL crawl with the Website method, you could get : account.name - LmnOPqrstUv - 1_WxYZabcDEFg.jpg account.name - hiJKlmNOpQr - 2_sTuvwxYZabC.jpg ) In all these cases, we see that the filename adapts to the circumstances, and takes only the part of the scheme that is relevant to it. Now, if you set a custom filename, say : *uploader* - *main_content_id* - *orderid*_of_*orderid_max* - *shortcode**ext* You get, for the same URLs : account.name - ABcDefGhIJk - 1_of_1 - -.jpg account.name - LmnOPqrstUv - 1_of_2 - -.jpg account.name - hiJKlmNOpQr - 1_of_2 - -.jpg ( or, specifically for a Profile URL crawl with the Website method, you could get : account.name - ABcDefGhIJk - 1_of_1 - ABcDefGhIJk.jpg account.name - LmnOPqrstUv - 1_of_2 - WxYZabcDEFg.jpg account.name - hiJKlmNOpQr - 1_of_2 - sTuvwxYZabC.jpg ) In all that cases, we see that, unlike before, the filename doesn't adapt itself to the circumstances, and generically integrate all the scheme. We can see that for URLs with one or more media, *shortcode* will be replaced by a dash " - ", and the surrounding punctuation (dashes and spaces) remains. Except for the URL/Website Profile method. For an URL with only 1 media, it will put 1_of_1 although it isn’t necessary, and will repeat *main_content_id* twice, using it as placeholder *shortcut*. For an URL with multiple media, you’ll get the right scheme. [ Examples of usable URLs: **External links are only visible to Support Staff****External links are only visible to Support Staff** ; **External links are only visible to Support Staff****External links are only visible to Support Staff** ; **External links are only visible to Support Staff****External links are only visible to Support Staff** ] The same happens with the YouTube plugin, but a missing part is replaced by a space. E.g. there will be no problem with the default scheme : *3D* *360* *VIDEO_NAME* (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT* You’ll get : The video title (720p_30fps_H264-192kbit_AAC).mp4 But if you want to add the placeholder *PLAYLIST_NAME* between dashes "-" surrounded by spaces : *3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - *H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT* For each video that isn’t part of a playlist or crawled with its playlist, you’ll get : The video title - - (720p_30fps_H264-192kbit_AAC).mp4 *PLAYLIST_NAME* doesn’t appear at all, rightly so, but the surrounding punctuation (dashes and spaces) remains. If the video is part of a playlist and crawled with it, this will be correct: The video title - The playlist name - (720p_30fps_H264-192kbit_AAC).mp4 [ Examples of usable URLs : **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** ] Etc. etc. I already know that this can be handled with the packagizer, and I've already made the rules for it. But there are limitations, e.g. the packagizer rules don't apply to the variant of a YouTube video added via the LinkGrabber context menu. As for Instagram, if you want to include mention of something like “ story “ or “ highlights “ or “ name of highlights “ or whatever, good luck (too many variables). It gets a little complex in my opinion compared to the original goal. I've searched a lot on the forum, but I couldn't find anything on the subject, except this post that proposes a script, but that doesn't seem to work, for me at least (https://board.jdownloader.org/showpost.php?p=395430 ; showpost.php?p=395430). But couldn't we avoid going through a script? As with the default filename schemes the adaptation is correct, I thought it should work with custom schemes too ? Thank you for your answers! |
#2
|
||||
|
||||
Hi,
I wouldn't consider this a bug. Indeed these are the "limitations" of our current naming schemes. ...but there is more to it: If you e.g. use the following scheme: Code:
*uploader* - *main_content_id* - *orderid*_of_*orderid_max* - *shortcode**ext* Code:
- *orderid*_of_*orderid_max* Also, you could use anything in between which would make it even more complicated for an automated logic to see "spaceholder XY makes no sense now so we can remove it and the space before": Code:
- *orderid*_of random text here __ bla _*orderid_max* In your example it is mostly " - "but it could be anything. Without an extended "search and replace" or "custom filenames editor" I don't see this happening while I do understand what you mean. -psp- EDIT For such "complex" things, you could always use an EventScripter script to add more logic to apply custoim packagenames. More information: https://support.jdownloader.org/Know...event-scripter
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
||||
|
||||
@Viga562: I've got an idea and will test it...at least for dashes when the property/field is empty. will do some trials
__________________
JD-Dev & Server-Admin |
#4
|
|||
|
|||
Thanks for the answers. First, sorry for the size of the previous post, I wanted to be as clear and precise as possible.
@pspzockerscene : I totally agree with you and understand the logic. Maybe it's just me who has reached my limits of software understanding. What was puzzling me was that with the default filenames schemes it works. With YouTube, the *3D* *360* placeholders don't leave spaces. With Instagram, for a URL with one media, let alone orderID_max, it still manages to not place the orderID at all, or else a shortcode (or a " - ") since there is none, or for URLs with multiple media, a shortcode (or a " - ") if the method does not find any. @Jiaz : That's nice of you Edits to original post
Spoiler:
Should've said : Except for the Profile URL/Website method. Quote:
Forgot : *3D* *360* *VIDEO_NAME* - *PLAYLIST_NAME* - (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*).*EXT* |
#5
|
||||
|
||||
@Viga562: I have some ideas but of course they also will have their limits. When I understand right, you are looking for a way to define a name scheme, for example
Quote:
Quote:
Quote:
Just asking to be sure we mean the same at least for the *start* of this feature/wish
__________________
JD-Dev & Server-Admin |
#6
|
|||
|
|||
Hi Jiaz, thanks for your idea, really appreciated. Sorry for the delay of my reply, I'm in the middle of my exams, so you know...
And yes, that's it. So that in the end we have the most adaptive filename possible, although I know we can't do miracles either. |
#7
|
||||
|
||||
Quote:
We got tickets for that though e.g.: As explained I think atm. the easiest way to do this in a more "intelligent" way would be to use the EventScripter. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#8
|
||||
|
||||
@Viga562: first idea I would like to share and gather some thoughts on it
A syntax similiar to bbcode. some example Quote:
Quote:
another example Quote:
that way we could also provide some more sophisticated features like Quote:
with such a syntax you can define what name should look like if property xy is available or not available. Please feel free to share comments/thoughts/ideas
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 22.06.2022 at 18:22. |
#9
|
|||
|
|||
@Jiaz: Implementation in foobar audio player:
Filename: Code:
song name.mp3 Titleformat: Code:
%filename%[- %artist%].%filename_ext% Output if artist tag is present: Code:
song name - artist name.mp3 Output if artist tag is missing (all the contents inside the bracket are discarded): Code:
song name.mp3 In JD, instead of brackets, maybe a different character which is normally not found/allowed in filenames can be used. Attachment > other > syntax > [ ... ] |
#10
|
||||
|
||||
@mgpai:
thanks for the first feedback with my idea it is possible to create if/else situation, like Quote:
Quote:
Quote:
Quote:
__________________
JD-Dev & Server-Admin |
#11
|
|||
|
|||
@Jiaz: Remembering syntax or tag names corresponding to each placeholder is a tough ask. Perhaps a uniform syntax for all placeholders, without using tag names?
FYI for if/else, foobar uses: Code:
%filename% - $if(%artist%,%artist%,unknown artist).%filename_ext% OR Code:
%filename% - $if2(%artist%,unknowm artist).%filename_ext% Output, ff artist tag is present: Code:
song name - artist name.mp3 Output, if artist tag is absent/missing: Code:
song name - unknown artist.mp3 Not that is wish to see it implemented in any particular way. Similar implementation in foolbar came to mind, which I though will share. |
#12
|
||||
|
||||
Quote:
filename and artist is also a tag name ? or do you mean *speaking* syntax like Quote:
Quote:
wiki.hydrogenaud.io/index.php?title=Foobar2000:Title_Formatting_Reference so we just need to write parser/evaluator for this syntax
__________________
JD-Dev & Server-Admin |
#13
|
|||
|
|||
Quote:
Code:
[*video_id*]-*video_id*[/*video_id*] Something which remains same across all placholders? Example: Code:
[jd]-*video_id*[/jd] [jd]-*date_addded*[/jd] For if/else: Code:
[JD]-(date_added|date_published|none)[/jd] Just brainstorming. I understand implementing it is an entirely different beast. Due to similar features, I cannot help but draw parallels between JD and foobar. Last edited by mgpai; 22.06.2022 at 21:01. |
#14
|
||||
|
||||
@mgpai: The more I read about foobar syntax, the more I like it I will try to create a poc for parser/evaluator
__________________
JD-Dev & Server-Admin |
#15
|
||||
|
||||
That's going to be a separate project :D
But I like that syntax too. I've never really used applications with custom naming schemes so I'm unable to provide input here. I'd like it to be as short as possible while still being readable and it should provide the ability to: - Remove "gaps" if no data for a tag is available - Provide fallback for "no data available" but I guess with those conditions that's already possible - Format strings and especially timestamps to whatever the user prefers to use Then plugins (also crawlers) could return a list of possible tags + description and those tags could just be used in our Packagizer. ...so far that would be my dream result. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#16
|
||||
|
||||
Some more thoughts on this:
Basides the replace system itself, this could potentially lead to the following more useful features:
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#17
|
||||
|
||||
Moved thread to Suggestions forum as the initial post is not a bugreport (if you ask me). It simply points out the limits of our current naming system(s).
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#18
|
||||
|
||||
Just for collecting input:
The yt-dlp syntax is also nice and has a lot of fields for common data: mankier.com/1/yt-dlp#Output_Template
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#19
|
|||
|
|||
Simple Filename Placeholder Suggestion
Not sure what the current state on this is, but from all I can see or know, nothing has changed since the last post in 2022 - and I was sent here by a pspzs link from only a couple months ago, hence I'm assuming this to be still relevant.
So, as soon as I saw this: Quote:
... except - applying to both proposals - it wouldn't be exactly smart to use brackets as reserved characters for this since they're allowed for filenames in Windows as well as in Linux/Unix. You would then have to escape them, which unnecessarily causes a mess - idk about others but personally I use [...] in almost every single filename pattern, especially for ids, counters, etc. That said, how about this: single use pattern: Code:
foo< - *video_id*> - bar Code:
foo - 6avJHaC3C2U - bar Code:
foo - bar pattern: Code:
foo<!*video_id* weirdly has no video id> - bar Code:
foo - bar Code:
foo weirdly has no video id - bar combined use pattern: Code:
foo< - *video_id*><!*video_id* weirdly has no video id> - bar Code:
foo - 6avJHaC3C2U - bar Code:
foo weirdly has no video id - bar shortening/simplifying this pattern: Code:
[*video_id*]videoID *video_id*[/*video_id*][!*video_id*]no videoID[/*video_id*!] Code:
<videoID *video_id*><!*video_id* no videoID> for combined use we could optionally think of the ternary operator ( utilizing a pipe symbol instead of a colon for mentioned reasons ) pattern: Code:
foo<*video_id*? - *video_id*| weirdly has no video id> - bar Code:
foo - 6avJHaC3C2U - bar Code:
foo weirdly has no video id - bar shortening/simplifying this pattern: Code:
[*video_id*]videoID *video_id*[/*video_id*][!*video_id*]no videoID[/*video_id*!] Code:
<*video_id*?videoID *video_id*|no videoID> If someone really really wants to use chevrons in their *nix filenames, you could of course also use /.../ instead of <...> - but anyone who does that also eats babies, so I guess angle brackets are good enough. I didn't read every single word above but most of it, so I guess this covers all of the essentials :] Last edited by JDmurphy; 17.03.2024 at 17:01. |
#20
|
||||
|
||||
Quote:
So far this thread is only a collection of ideas.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#21
|
||||
|
||||
Todays update did add a more sophisticated global regex character replacement feature, see:
https://board.jdownloader.org/showthread.php?t=95411
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#22
|
|||
|
|||
To avoid unnecessary spaces I think the space separator could be stablished inside the expression.
Instead of: Code:
*DATE[yyyyMMdd]* *3D* *360* *VIDEO_NAME* Code:
*DATE[yyyyMMdd] **3D **360 **VIDEO_NAME* |
#23
|
||||
|
||||
That is actually even simpler than what I had in my mind lol
...but it will only work in a good way for spaces or some other 'special' characters. Simply use the * as the 'tag end' character. Problem with this: If you want to add other chars to it, it will make the internal parsing reall hard to impossible e.g.: Code:
*_bla_DATE[yyyyMMdd]__ **3D **360 **VIDEO_NAME* Code:
[jd:tag]_bla_*DATE[yyyyMMdd] [/jd:tag]**3D **360 **VIDEO_NAME* If no tag inside "[jd:tag]" is available, the whole thing just gets eliminated.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#24
|
||||
|
||||
@dbkarm: yt plugin will support your idea with next core update
__________________
JD-Dev & Server-Admin |
#25
|
|||
|
|||
Apparently our bug_reporter_girl didn't bother to follow the advice in #5692 so I'll invest the 2 mins of effort in her place:
Quote:
Emoji [sic] (it's already the plural) in filenames are the literal plague of the 21st century and constantly having to remove them manually - or even worse, the utterly useless substitute characters - cost me at least 3 days combined over my whole time of using JD(2). Also in order to make myself look like I have no clue what I'm talking about: "Can't you just define emoji unicode blocks and yeet them entirely?" I don't really know how any of that is actually on topic here (this is about defining placeholders, not about what can happen *if* you use certain placeholders) ... but hey, psp told us so :D |
#26
|
|||
|
|||
Oh wow, this thread has a second page now ... I didn't even notice. Since [] still appear to be a hot contender for tag delimiters here (as if my post on the previous page never happened ) I'm not entirely sure what wasn't intelligible about this:
Quote:
Last edited by JDmurphy; 07.09.2024 at 05:44. |
#27
|
||||
|
||||
@JDmurphy
You are on topic, all good. Please just keep in mind that this topic is just for collecting ideas - we are currently not actively working on this one so more complex replacements will need to be done either outside JDownloader or using EventScripter scripts.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|