JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 07.07.2023, 21:40
JTK JTK is offline
Super Loader
 
Join Date: Jun 2021
Posts: 26
Default Twitter - text filenames need fixing

log : 08.07.23 00.38.25 <--> 08.07.23 01.12.11 jdlog://8036311370661/
I set naming to auto and it's work fine couple of days ago.
After today update that add more naming option for twitter, the new name is changed from

[date][tweet_id][originalFilenameWithoutExt][ext]
to
[date][originalFilenameWithoutExt][some random string][ext]

I tried changing the option from auto to Original+ and delete file and recrawling again but the name is still the same without tweetid.


edit: I try changing it to Original+2 and Plugin and it's show the result of Original+ setting. Here's the result of select setting.

Auto, Original+
[date][originalFilenameWithoutExt][some random string][ext]

Original+2
[date][tweet_id][originalFilenameWithoutExt][some random string][ext]

Plugin
[date][tweet_id][originalFilenameWithoutExt][ext]

So it's work out for me at the moment to choose "Plugin" for "original+" option.

Btw there're [some random string] add at the end of every files now after the [originalFilename] for some option. Is it a bug or twitter changed how they naming their files?

Last edited by JTK; 07.07.2023 at 23:12.
Reply With Quote
  #2  
Old 10.07.2023, 06:33
Marco86 Marco86 is offline
DSL Light User
 
Join Date: Aug 2022
Posts: 34
Default

Everything is working for me, check if you have a packagizer rule enabled. Try downloading and installing the new jdownloader version in a new folder and see if the fresh installation has the same problem.
Reply With Quote
  #3  
Old 10.07.2023, 06:47
Marco86 Marco86 is offline
DSL Light User
 
Join Date: Aug 2022
Posts: 34
Default Twitter - text filenames need fixing

All tweets without media have only the tweetID as the text filename, this is true for profile links and single status links, no matter which API or filename scheme is chosen (except the plugin filename).
The filename scheme Plugin uses the text filename format date_username_tweetID.ext, this is also how the text filenames were handled in all previous versions with original and plugin filenames.
I think the text filename format should stay the same for filename scheme original and plugin, and be changed for filename scheme Original+ and Original+2.
The text filename format for Original+ should be date_tweetID.ext.
The text filename format for Original+2 should be date_username_tweetID.ext, same as the plugin filename scheme.

Example links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Also the text filenames have been changed, the textfiles always took the name of the last image or video in a tweet, now it's random.
This was a recent change since May 2023 or sooner.
Please change this back, as I'm skipping files based on already existing filenames.

Example 1:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
text filename is: FvOaW4yaYAAbZ3k.txt
should be: 5IQArLM9GvBLz9eA.txt

Example 2:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
text filename is: E2nK9eJe8dQFMMx_.txt
should be: FuzOo3FaIAAHM2g.txt
Reply With Quote
  #4  
Old 10.07.2023, 09:52
RedViper RedViper is offline
Bandwidth Beast
 
Join Date: Apr 2016
Location: Mexico
Posts: 138
Default

I don't know if it's the same but I was going to create a new topic that the date is repeated twice, I don't know if it's my fault.

For example:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

The name of the file comes out like:

2023-06-30-2023-06-30_1674810336149381121_OpHLMpClNOB2P31h.mp4.mp4


:confused:

It had been a few months since I updated JDownloader 2, that's why I hadn't reported it before. I don't know if it's the same problem as this topic.

Log:

09.07.23 18.51.47 <--> 10.07.23 00.51.20 jdlog://7636311370661/
__________________
- Ramón V M
Reply With Quote
  #5  
Old 11.07.2023, 16:22
Marco86 Marco86 is offline
DSL Light User
 
Join Date: Aug 2022
Posts: 34
Default

@RedViper I get the name 2023-07-01_1674810336149381121_OpHLMpClNOB2P31h.mp4, I think there is something wrong with your jdownloader installation, check if you have any packagizer rules enabled, if not try a fresh jdownloader installation in another folder.
I'm still using the Jdownloader version #47140 from November 26, 2022 if you are able to use packagizer rules you can get the filenames you want. If you want that version as a backup write me a pm, even extmatrix plugin is working with that old version while not working in the new one.
Reply With Quote
  #6  
Old 11.07.2023, 18:00
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

Merged related twitter threads.

@RedViper
The last update changed the filename behavior once again.
Go to Settings -> Plugins -> twitter.com -> Filename scheme and change it to your desired value.

Changes of next twitter plugin-update:
- fixed fallback-name (= only tweet_id) being used for single .txt downloads even when FilenameScheme is set to AUTO
- fixed "original filename without ext" in .txt files coming from the first crawled media item and not the last

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 12.07.2023, 08:23
RedViper RedViper is offline
Bandwidth Beast
 
Join Date: Apr 2016
Location: Mexico
Posts: 138
Default

Thank you pspzockerscene!

I didn't know this, I always named the images with the original file name and the videos with the tweet ID.

I hope I don't make a mess with files that I have already downloaded, but I will start downloading the images and videos with the original name
__________________
- Ramón V M
Reply With Quote
  #8  
Old 12.07.2023, 14:11
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

You can choose your preferred filenaming scheme.
By selecting the one which was used before, the filenames will be identical to the ones of files you've downloaded before.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #9  
Old 12.07.2023, 17:19
RedViper RedViper is offline
Bandwidth Beast
 
Join Date: Apr 2016
Location: Mexico
Posts: 138
Default

Quote:
Originally Posted by pspzockerscene View Post
You can choose your preferred filenaming scheme.
By selecting the one which was used before, the filenames will be identical to the ones of files you've downloaded before.

Thanks again pspzockerscene!

But I think it will be for the better like this because when there was more than one video in a Tweet I always had to list them because they all ended up having the same name. Apparently putting the original file name in the videos as well will be better, I'll just have to be careful not to download a video I already have
__________________
- Ramón V M
Reply With Quote
  #10  
Old 12.07.2023, 21:14
Marco86 Marco86 is offline
DSL Light User
 
Join Date: Aug 2022
Posts: 34
Default

Please do not mark this thread as solved, as the issues have not been solved yet.

Textfilenames:
There are still a lot of tweets remaining where the textfile name is not the last picture or video name.
Here are examples:

4 pictures, taking name of 1. (should be: FLr0B5UVkAAyrlK.txt, is: FLr0B5OVQAoQFZ4.txt)
**External links are only visible to Support Staff**...

4 pictures, taking name of 2. (should be: FKYVmlrWQAUrNrJ.txt, is: FKYU5nSXwAErv6x.txt)
**External links are only visible to Support Staff**...

4 pictures, taking name of 3. (should be: FJT96m3XMAgENaW.txt, is: FJT96m0WYAMqxE7.txt)
**External links are only visible to Support Staff**...

3 pictures, taking name of 1. (should be: FSayaraXEAEyFNR.txt, is: FSayUzMWUAAcQP1.txt)
**External links are only visible to Support Staff**...

3 pictures, taking name of 2. (should be: FSHjm39WUAEf7Po.txt, is: FSHjm3-WYAU_OGA.txt)
**External links are only visible to Support Staff**...

2 pictures, taking name of 1. (should be: FGCXhvAXwAYy91B.txt, is: FGCXhujXIAEm6ei.txt)
**External links are only visible to Support Staff**...

3 videos, taking name of 1. (should be: JNypQxxZJUIgxCmO.txt, is: O1Xgw7-M51klh19u.txt)
**External links are only visible to Support Staff**...

1 video and 2 pictures, taking name of 2. (should be: Fve_1TtWwAEHUUD.txt, is: Fve_1TzXwAECw7Y.txt)
**External links are only visible to Support Staff**...

4 gifs, taking name of 1. (works with New API only when parsing single link, profile link will work with Old and New API) (should be: Fw76AyVWIAA3P-1.txt, is: Fw75_bIWAAAzomq.txt)
**External links are only visible to Support Staff**...

4 gifs, taking name of 3. (should be: Fw7-uXKX0AAdvTb.txt, is: Fw7-uW2XsAEsapA.txt)
**External links are only visible to Support Staff**...

3 gifs, taking name of 1. (should be: Fw2kkYYXgAE5aF6.txt, is: Fw2kjlWXwAIjfLR.txt)
**External links are only visible to Support Staff**...

2 gifs, taking name of 1. (should be: FxlHcwIWcAEwo4r.txt, is: FxlHcNBXsAEPL_G.txt)
**External links are only visible to Support Staff**...

1 gif and 1 picture, taking name of 1. (should be: FyOiGq7XsAE_oJ5.txt, is: FyOiF2OWYAAo31M.txt)
**External links are only visible to Support Staff**...

These tweets' textfiles are taking the name of the last media when using previous JD2 versions, so this should be fixable.

Textfile format:
The text filename format was now changed to date_username_tweetid.ext for the AUTO setting, which is Original+, but Original+ has the format date_tweetid_originalfilename.ext, which does not include the username, so if someone would chose to download with the Original+ setting in the format date_tweetid_originalfilename.ext, they likely do not want to have the username in the filenames. But the current change would be perfect for the Original+2 setting.
And this is what I was asking for, to have the Original+2 textfiles (of tweets without media) in the same format as the current Plugin filename scheme textfiles.
But choosing Original+2 still gives only the filename 1678098028849143809.txt for the following tweet **External links are only visible to Support Staff**..., while giving the correct filename 2023-07-10_elonmusk_1678098028849143809.txt when choosing Plugin filename scheme, but I do not use the Plugin filename scheme as the original filenames are not included.

I also think the AUTO settings should not include some hidden features. If the AUTO setting is the Original+ setting, then it should be exactly the same as choosing Original+ manually.

Also I think it is very important to have the accountname in the filename, because if you load a profile link, jdownloader will also (rightfully so) download the tweets from other accounts that were quoted, without having the accountname in the filename you wouldn't know.
Reply With Quote
  #11  
Old 13.07.2023, 07:45
RedViper RedViper is offline
Bandwidth Beast
 
Join Date: Apr 2016
Location: Mexico
Posts: 138
Default

Marco86 you should definitely add the Log as well, so they can catch those issues.
__________________
- Ramón V M
Reply With Quote
  #12  
Old 13.07.2023, 14:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

Quote:
Originally Posted by Marco86 View Post
Please do not mark this thread as solved, as the issues have not been solved yet.
We usually leave such threads open just in case so that users can still answer.
I've changed that prefix once again.

Quote:
Originally Posted by Marco86 View Post
There are still a lot of tweets remaining where the textfile name is not the last picture or video name.
Here are examples:...
Unfurtunately due to a rare forum bug, all of your URLs got lost.
Please either post them again (1-2 examples should be enough) or put them into a .txt file and attach that to your next reply.

Quote:
Originally Posted by Marco86 View Post
The text filename format was now changed to date_username_tweetid.ext for the AUTO setting
No.
Also there is no such thing as a "text-filename format".
The text filename format orientates itself on the other given filenames.
Example:
twitter.com/50cent/status/1679127118603509762
Filenames in AUTO mode:
Code:
2023-07-12_1679127118603509762_F011NYSWAAUXNgt.jpg
Text:
Code:
2023-07-12_1679127118603509762_F011NYSWAAUXNgt.txt
How FilenameScheme.AUTO works atm:
- Prefer FilenameScheme.ORIGINAL_PLUS if possible
- Use FilenameScheme.PLUGIN if ORIGINAL_PLUS is not possible

EDIT
Now I get it - tweets without media but only text. I will re-check this but please also provide separate example URLs so I got your test-cases.
I want to mention again that we're usually not adding extra filename settings for single plugins - I urge you to read the following thread and be part of the discussion about the new filename replacer feature:
https://board.jdownloader.org/showthread.php?t=90744

Quote:
Originally Posted by Marco86 View Post
And this is what I was asking for, to have the Original+2 textfiles (of tweets without media) in the same format as the current Plugin filename scheme textfiles.
So effectively you'd like to have: <date>_username_<tweetID>_<originalFilenameWithoutExt>.<ext>
--> But this is exactly what FilenameScheme.ORIGINAL_PLUS_2 does or am I missing something?

Quote:
Originally Posted by Marco86 View Post
But choosing Original+2 still gives only the filename 1678098028849143809.txt for the following tweet...
That link also got lost so please re-provide that.

Quote:
Originally Posted by Marco86 View Post
I also think the AUTO settings should not include some hidden features. If the AUTO setting is the Original+ setting, then it should be exactly the same as choosing Original+ manually.
I've already explained the purpose of the AUTO setting in a post in another twitter related thread.
Also I've added further explanation of it in this post - AUTO does not simply select any of the other pre-selectable FilenameSchemes.

Quote:
Originally Posted by Marco86 View Post
Also I think it is very important to have the accountname in the filename...
Good that filenames are customizable so users who do not want this can disable it.

Quote:
Originally Posted by Marco86 View Post
...because if you load a profile link, jdownloader will also (rightfully so) download the tweets from other accounts that were quoted, without having the accountname in the filename you wouldn't know.
Are you talking about quoted threads or retweets?
I'm asking because re-tweets are not crawled by default.

Quote:
Originally Posted by RedViper View Post
Marco86 you should definitely add the Log as well, so they can catch those issues.
If an issue is 100% reproducable with example links, I do not need any logs.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 13.07.2023 at 14:39.
Reply With Quote
  #13  
Old 14.07.2023, 09:45
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

[Internal] Changelog for the next update:
TwitterComCrawler:
- updated filename handling: for schemes with 'relatedOriginalFilename': allow usage of them even if relatedOriginalFilename is not given RE forum 93871
- updated text crawl setting to ENUM based setting: Adds ability to completely disable tweet text results RE forum 93874
- fixed 'related_original_filename' not necessarily being the name of the last media item due to usage of HashMap vs LinkedHashMap RE forum 93871

Btw. we're open source - you can view the code here: jd.plugins.decrypter.TwitterComCrawler -> Function 'setFormattedFilename' (line 817 and below)

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 14.07.2023 at 12:33. Reason: Updated text-block regarding CORE-updates
Reply With Quote
  #14  
Old 15.07.2023, 16:37
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #15  
Old 19.07.2023, 14:56
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

@Marco86
Just another example why a generic filename formatting system is better than e.g. what is implemented in the twitter plugin now:
Different users will want different filename schemes which may result in a lot of pre-sets: Click

Again: In this case that's fine as the twitter plugin is a very popular plugin and we do not have a good generic filename formatting system yet.

Either way I hope it's finally working as intended and in the way you wanted it to work.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #16  
Old 20.07.2023, 19:00
Akasen Akasen is offline
DSL Light User
 
Join Date: Jul 2017
Posts: 34
Default

Quote:
Originally Posted by pspzockerscene View Post
@Marco86
Just another example why a **External links are only visible to Support Staff**... is better than e.g. what is implemented in the twitter plugin now:
Different users will want different filename schemes which may result in a lot of pre-sets: **External links are only visible to Support Staff**...

Again: In this case that's fine as the twitter plugin is a very popular plugin and we do not have a good generic filename formatting system yet.

Either way I hope it's finally working as intended and in the way you wanted it to work.
I'm gonna chime in on this on that note to say that unfortunately, it is very clearly a tricky thing.

So much so that when the initial update to the filename scheme hit, I was really besides myself on whether to just pop in here and inquire about getting the old naming scheme back.

I'm throwing my cents here on the matter more than anything, since I think I have my problem handled as is (hopefully)

Because as a long time user, I have a terabyte of scrapes with the convention of <tweet_id>_<filename.ext>, which I have problems with, but this is one of those things when scraping for archival purposes I have to bear the burden of at this point.

So any changes down the line in an attempt to improve the naming scheme is a bother, especially since when these updates hit, I never know what is in them because I just hope and assume the software will function as it did yesterday, just as the sun rises in the morning as it did the day before.

Heck, even the introduction of the text files with the tweets was enough to cause a pause for concern. Cause on one hand, yeah that's a really useful thing to have and is certainly something I wish I'd had maybe four or five years ago. On the other, it seems really weird to suddenly scrape. But also on the other, its naming scheme is close to that of the images/videos and they are text files in the kilobytes.

I think personally down the line, it may just be better to do something where a user has the ability to declare in the plugin itself the filenaming scheme like they would with the packagizer.

Right now, I have the plugin set to the "original" scheme in the plugin and then in the packagizer I have it so that the files are named with this scheme in the packagizer rules I have for twitter "<jd:prop:tweetid>_<jd:orgfilename>" to somewhat match how the plugin handled scrapes. I say somewhat because I don't know if there's an edge case I'm unaware of.
Reply With Quote
  #17  
Old 21.07.2023, 08:22
Marco86 Marco86 is offline
DSL Light User
 
Join Date: Aug 2022
Posts: 34
Default

@pspzockerscene Thank you very much, the issues have been fixed now.

I have found another issue: When loading single links with the New API, the link is not recognized if the uploader has restricted who can reply.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

@Akasen You should really use one of the new formats, why wouldn't you want to have the date included, especially if you are a scraper. I also have a lot of files in the useless format <tweet_id>_<filename.ext>, but only because JD2 forced this format without me knowing that packagizer rules or plugin settings exist. Maybe some notifications can be added to the program.
Reply With Quote
  #18  
Old 21.07.2023, 08:44
Akasen Akasen is offline
DSL Light User
 
Join Date: Jul 2017
Posts: 34
Default

Quote:
Originally Posted by Marco86 View Post
@pspzockerscene Thank you very much, the issues have been fixed now.

I have found another issue: When loading single links with the New API, the link is not recognized if the uploader has restricted who can reply.
**External links are only visible to Support Staff**...
**External links are only visible to Support Staff**...

@Akasen You should really use one of the new formats, why wouldn't you want to have the date included, especially if you are a scraper. I also have a lot of files in the useless format <tweet_id>_<filename.ext>, but only because JD2 forced this format without me knowing that packagizer rules or plugin settings exist. Maybe some notifications can be added to the program.
Simply put, because I've been using the software for years now, so when I need it to scrape a profile and not duplicate a file it has already downloaded, but is no longer in the downloaded list, it's useful to have the program see that there's already a file already in the folder it's trying to download to and skip it.

So what I don't need is to have a folder for one account now have duplicates files between <post-date>_<tweetid>_<imageid> and <tweetid>_<imageid>

Were I to go back in time and setup my own rules, I probably would have done <post-date>_<tweetid>_<imageid>_<mediaindex>.

But I'm multiple thousands of folders and accounts in too deep now.

I'm just lucky that the tweet ID works well enough to organize items by when they were posted
Reply With Quote
  #19  
Old 21.07.2023, 10:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

Quote:
Originally Posted by Akasen View Post
Heck, even the introduction of the text files with the tweets was enough to cause a pause for concern. Cause on one hand, yeah that's a really useful thing to have and is certainly something I wish I'd had maybe four or five years ago. On the other, it seems really weird to suddenly scrape. But also on the other, its naming scheme is close to that of the images/videos and they are text files in the kilobytes.
No worries I can absolutely understand you but please keep in mind:
  • JD is an allround tool and not a specific tool for scraping from twitter; other tools may be suited better for that purpose. And yes I know that pointing to other tools is absolutely not the answer you'd expect of me but it is the only right one in this context as there are complete open source projects available dedicated to archiving specific websites, those ones may be harder to use but ultimately they will do their job better than JD.
  • We're trying not to touch default/existing filenames but in some cases it is necessary
  • We do not have the time to do loads of tests for plugin updates so mistakes happen, especially with complex plugins such as the one for twitter. Feel free to test changes before release in our dev version: https://support.jdownloader.org/Know...up-ide-eclipse
    --> Even at this moment trhere are unreleased twitter changes available.
  • If any of your previously used "schemes" is missing, please let me know and I'll make sure to bring it back


Quote:
Originally Posted by Akasen View Post
I think personally down the line, it may just be better to do something where a user has the ability to declare in the plugin itself the filenaming scheme like they would with the packagizer.
This topic is not that easy. I urge you to read the following thread:
https://board.jdownloader.org/showthread.php?t=90744
Short summary:
  • The Packagizer is already the place to alter filenames
  • Its' rename-capabilities are currently quite limited but a lot of very good ideas are floating around in the linked thread
  • Moving file naming into each websites' plugin may seem like the way to go for our users but from the programmers point of view it doesn't make sense as it is basically a copy & paste each time and each scheme may work differently - just e.g. look at the filename scheme stuff of our utube plugin
    -> There is no standards, everything is a mess, some renaming-functionalities can do more stuff, some less

Quote:
Originally Posted by Akasen View Post
Right now, I have the plugin set to the "original" scheme in the plugin and then in the packagizer I have it so that the files are named with this scheme in the packagizer rules I have for twitter "<jd:prop:tweetid>_<jd:orgfilename>" to somewhat match how the plugin handled scrapes. I say somewhat because I don't know if there's an edge case I'm unaware of.
I think the scheme you wanted is the one a user asked for here and yes it was changed by mistake [explained in the following thread]:
https://board.jdownloader.org/showthread.php?t=93929
Please keep in mind that the placeholder hich is titled "Original Filename" is simply using the last filename set by the plugin as in that context, that is the original filename.
Here is a simplified explanation of how filenames in JD work atm:
  1. Plugin (hoster/crawler) sets filenames. If the plugin itself has some kind of settings, those filenames will be set here, else default names.
  2. Now Packagizer rules are processed. Filenames can change be changed by those rules at this stage.
  3. [Optional] The user can change filenames inside JDownloader. If done so, those ones will be the "forced filenames", overriding any previously set names and any names the plugin tries to set later.
  4. When the download is started, filenames can change again [as long as they were not touched by the user in GUI]:
    - Too long filenames can be truncated
    - File-extension can be corrected if it turns out to be wrong
    - Previously truncated filenames may be changed or filenames may be changed in any way by the plugin which handles the download. Mostly they can change due to metadata/information which is not available before the download is attempted.
    - Some plugins got settings which result in filenames changing when download is started e.g. google drive stream download

@Marco86
I will reply to your post #17 and Akasen to your post #18 soon in a separate reply.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #20  
Old 21.07.2023, 11:36
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

Now responding to posts #17 and #18:

Quote:
Originally Posted by Marco86 View Post
@pspzockerscene Thank you very much, the issues have been fixed now.
Thanks for your feedback.

Quote:
Originally Posted by Marco86 View Post
I have found another issue: When loading single links with the New API, the link is not recognized if the uploader has restricted who can reply.
Fixed.
The fix will be released with our next set of CORE-updates.
As explained to Akasen, feel free to test changes before official release using the dev version:
https://support.jdownloader.org/Know...up-ide-eclipse
--> While some general software knowledge is good to be able to follow these instructions, you do not need to be a developer to be able to follow them!

Quote:
Originally Posted by Marco86 View Post
Maybe some notifications can be added to the program.
I [partially] disagree.
I'm counting this as an advanced set of features which the user will usually have to find on his own.
JDownloader is a quite advanced application so in order to find such functionality, I do expect the user to browser the applications' settings in order to find such features.
At least that's the way I'm doing it with software I'm using.
I'm even going through all the settings of a new software when I'm not searching for something in particular though that's just my way of handling it.

Quite often if you think "there must be a setting for this", investing these 5 minutes to find it will reward you.
Just a sidenote: We do have a support category dedicated to plugin settings:
https://support.jdownloader.org/Know...dex/24/plugins
While modern apps often contain some kind of instructions when using them the first time, I do not consider JDownloader a 'modern' application and I also do not see us as the current team ever finding the time to modernize it.

Quote:
Originally Posted by Akasen View Post
Simply put, because I've been using the software for years now, so when I need it to scrape a profile and not duplicate a file it has already downloaded, but is no longer in the downloaded list, it's useful to have the program see that there's already a file already in the folder it's trying to download to and skip it.
Ideally, JD would write as much metadata as possible in the downloaded files.
This way you could use external applications to change filenames to your needs afterwards - at any point of time.
We do have tickets/threads about putting tags in files e.g.:
https://board.jdownloader.org/showthread.php?t=84246
--> They're mostly about audio files but the concept for others is the same.
That is a feature which could be addressed whenever someone will be working on the other feature - the generic filename builder/replacer.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #21  
Old 21.07.2023, 20:08
Akasen Akasen is offline
DSL Light User
 
Join Date: Jul 2017
Posts: 34
Default

Yeah I get that overall this is very much a tricky situation here having had to come back and forth to the forums either to read over various posts or to report on something.

Quote:
This topic is not that easy. I urge you to read the following thread:
https://board.jdownloader.org/showthread.php?t=90744
Short summary:

The Packagizer is already the place to alter filenames
Its' rename-capabilities are currently quite limited but a lot of very good ideas are floating around in the linked thread
Moving file naming into each websites' plugin may seem like the way to go for our users but from the programmers point of view it doesn't make sense as it is basically a copy & paste each time and each scheme may work differently - just e.g. look at the filename scheme stuff of our utube plugin
-> There is no standards, everything is a mess, some renaming-functionalities can do more stuff, some less
To this point, I know this isn't an easy topic, but it was also the most logical one to come to mind since I've been in the middle of a naming scheme change twice and I felt like I'd just be a bother to go "Hey, the naming scheme changed, can we get that back again?" every time some update changes it without

So it came to me as the overall more logical answer to that once I found myself coming to.

I can't really respond to most other points here, mostly because the tagging thing I'd have to look more into with relation to images and video.

I'm fairly married to Jdownloader at this point as a tool. I do use various other specialized tools, and it probably would just be in my better interest to use a CLI program and save a script to run through and do the same job I have Jdownloader do. Maybe this is also a good time for me to consider doing that.
Reply With Quote
  #22  
Old 24.07.2023, 09:44
Marco86 Marco86 is offline
DSL Light User
 
Join Date: Aug 2022
Posts: 34
Default

I tested the Original with tweetID option (#48037), the names for textfiles without media are tweetID.ext only, this should be date_user_tweetID.ext, at least this is how these names were always handled.
Reply With Quote
  #23  
Old 24.07.2023, 14:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,827
Default

Quote:
Originally Posted by Akasen View Post
Yeah I get that overall this is very much a tricky situation here having had to come back and forth to the forums either to read over various posts or to report on something.
I'm not complaining about that - I'm absolutely fine with users who are taking care about such details and stick to the topic until it is resolved 100% [for them].


Quote:
Originally Posted by Akasen View Post
I tested the Original with tweetID option (#48037), the names for textfiles without media are tweetID.ext only, this should be date_user_tweetID.ext, at least this is how these names were always handled.
I disagree as this is the exact opposite handling we agreed one / it's the one I used before: If selected scheme is not possible, fallback to some default scheme.
The new logic for all schemes with original filename parts is: Build filename segment by segment with underscore in between. If the "original filename part" is not given, just don't include it.

If you need a solution for that, I may add an option like "Fallback filename Scheme" though at this moment I don't know what would be the best logic here because maybe you'd want to have a different fallback scheme for each item or each target-scheme.
...or call it "preferred filename scheme for single textfiles witout media-items in related tweeet"?
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 24.07.2023 at 14:35. Reason: Fixed typo
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 19:23.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.