Im a long time IDM user, and it has a option to download file types to a specific folder, movies, compressed , docs, music etc.

Can we get a option to do this on JDownloader?:angel:

I use jD's packages as categories (name and download folder) and that works fine.

Your categories would correspond to different media types.

Just drag & drop your links in the proper packages.

Great Idea Remi!

I was trying to figure out exactly how to do that. Documentaries to Array A, PDF's to Array B, MP3's to Array C, etc.

Your idea is perfect. Slapping myself on the fore-head for not thinking of it ;(

RockinJD

Originally Posted by RockinJD

Great Idea Remi!

I was trying to figure out exactly how to do that. Documentaries to Array A, PDF's to Array B, MP3's to Array C, etc.

Your idea is perfect. Slapping myself on the fore-head for not thinking of it ;(

RockinJD

Yeah it would be a excellent feature , i just don't want all my downloads to be shoved into one folder.

Originally Posted by remi

I use jD's packages as categories (name and download folder) and that works fine.

Your categories would correspond to different media types.

Just drag & drop your links in the proper packages.

This sounds complicated , care to explain how to do that? I think it would be a lot easier if we could just specify a file extension to go to a certain folder.

=======================

So...to the masterminds behind JD, what say you?

A package can have a download folder (Save to) and a name. Go to its Properties and change the default values. Always keep at least one dummy download link in your category package to avoid it disappears.

When you add links to the Download pane, select them and then drag & drop them on the package that corresponds with their category. If this isn't sufficiently clear, please ask.

Note that media files are often stored in archives (zip, rar, etc.) and jD won't recognise the media types in those archives before the files are extracted. An archive often contains more than one file that have different media types and you wouldn't like to disperse the files over your disk.

DBM this is what Remi said but slightly more verbose,

Collect a few links/packages in Linkgrabber (or some not downloading links in the download tab). Select your movies, then right click and choose move to new package, Call it MOVIES. Now you have a Movies package that you can drag and drop all your movie related links/packages into..

Repeat this for Music, Videos, PDF's, etc. Just remember you need to disable or put a dummy link inside each package, or else if you remove ALL links from the MOVIES package, it goes away.

Now that you have packages defined, right click on the package, and change the save to folder.

Start your downloads, all your packages (aka categories) will now go to the locations you want.

RockinJD

[drbits]

One addition about JD's abilities. JD gets the table of contents for an archive before opening any files in the archive. So it would seem like JD could "Decide" which category to assign. However, there are two big problems with doing it this way:
1) As remi said, archives usually contain more than one kind of file:
a) An application is normally stored as one or more Zip archives inside a RAR archive. This is especially true if there is hidden information (such as a serial number, which would usually be an a RAR at a lower level). So, obviously, this is category ZIP (but that isn't allowed, it is still an archive). So JD would have to get the table of contents of the ZIP, find the EXE, and then assign it to APPLICATIONS (I don't know about you, but I have around 20 categories for applications).
b) Similarly movies are sometimes packaged in self extracting archives inside a RAR (but only by people who don't understand the "System" setup by the "Scene".
c) Audio books now usually come with a text copy (PDF) and images of the cover as well.
d) The Scene standard is to package anything under around 700MB in 15 000 000 byte RAR volumes with a signature (SFV) file and an NFO file that are packaged separately. Larger archives use 50 000 000 byte volumes. When somebody wants to upload this into a file storage host like Hotfile, they create a post file (containing links to images, most of the text from the NFO, and links to sites like IMDB). They then take everything (usually including the image files) and create a RAR archive with 200MiB volumes for Hotfile (if they do it correctly). Some sites (like ugotfile) automatically split these files into 100MiB volumes, while other sites allow free users to download 1000MB or 1000MiB volumes. Ugotfile just splits these into volumes aa through jj. A couple of hosts see the aa and bb files and recombine them, but don't know what to do with cc-jj.
e) For heavy downloaders, one level of categories is not enough. I personally have separate disk drives for different major categories. I would say that once you exceed 100 Assemblies, you need categories and once you exceed 3000 Assemblies you need at least two levels of categories. Once you exceed 100 000 assemblies, you need a librarian to help.
f) To get an idea of the amazing amount of data involved: the total output of Binary releases on the web averages somewhere between 1.5TB and 3TB. Eliminating duplication where it is easy to eliminate, makes the 1.5TB number about right. This includes alt.binaries, Scene, Privately assembled collections, self published books and so on. It does not include personal posts, e-mail, social sites, viruses, advertising, happy birthday programs, blog text or boards (this thread is not an assembly for this discussion, but a package of photos from OnlyTeasing.com is). Assuming that the average assembly is 200MB, there are around 7000 assemblies added to the web daily. Again, this does not count reuploading and other simple changes. You could probably cut this to 1/4 by elimination of different versions (no sense keeping the 320x180 copy of a movie if you have the 1080P it was made from). That is still over 500 000 new assemblies per year. (assuming some collector wanted to download all of this that would be over 100TB/year - after elimination of everything but the newest version of software or the best version of each movie).

-----------------------------------------

A better way to automatically divide files into categories is to ignore most the file extensions for categorizing assemblies, unless there is only one significant file in the assembly. However, if a file is found with an IMDB, Hulu, NFOHUMP, katzforums.com, downloads.cnet.com, amazon.com, katz.cd, pornbb.com, homepage (or similar) link, the page linked to usually indicates the 2-level category tags (for example, Movie, drama, comedy and so on). More information than one ever needs.

JD would look
A) in the comment for the Assembly and/or Package
B) NFO file (often contains the category information)
C) DIZ file
D) Search the major sites.
i) A search of site:downloads.cnet.com (title with spaces [or OR] instead of punctuation) would usually find application categories.
ii) A similar search of each of the resources listed in the first paragraph in this section can lead to the category.

This way is harder, but it usually provides a two level category and/or a list of tags that can be used to organize one's library. The 1-5% of downloads not identified this way can be put into an "Unknown" category. Even just including the referrer page and the home page of an item as tags can provide significant information for manual sorting (and later more automation).

Audio files often come with a Tag record inside the file. JPEG files usually contain an EXIF record. Even some movie formats use an EXIF based format for tagging. When downloading audio and movie information, don't forget the Language tag!
-------------------------------
OK,
1) What did I miss? Did I get the most important web sites? I know in Music, there are open catalogs, the covers project, and so on.
2) How crazy an idea is this:
a) Search for an information source link with the download
b) Search the web for information sources
c) Search Tag records
d) Have a list of major categories (use Hulu for TV and movie categories, CNET for software categories, and so on), to select the top two levels of tags, and use them to organize the downloads.
e) Along the way, links or tags will be found that might be helpful.
f) Are two levels of categories enough if you also have tags?

If we use GoogleAPI or IxQuick API, we can get the search from the title almost for free (little code). We could even use the title search first, then check for reference sites that would have the information we need.

Your turn. Your ideas? Your criticism?

[Think3r]

There already is such a ticket: http://svn.jdownloader.org/issues/show/1234
Greetz

All nice ideas and hints, but why? As I wrote in another thread, jD shouldn't become a cataloguing tool. There are other tools for that. They can provide all the technical and other details for your media files.

I recommend jD focuses on download management. There's still a lot to do and we should avoid further scope creep.

Concerning "Feature #1234 - Add categories to Packages & Download Units" http://svn.jdownloader.org/issues/show/1234

This originally came from remi, but I am embellishing it based on various discussions.

Every time I read this I wonder how bad my memory must be. I don't remember I ever asked for categories. I stress again that jD already has categories if you use the packages. I think there's no need for more. IMO, building hierarchies is good for determining plants and animals, but not for media or other types of files. Tagging and searching with keywords (in a semantic way) is a lot easier.

1) remi is uncomfortable with the name Archive unite (see issue 1203). Download Unit may be more descriptive.

I prefer the term "Download Set". I believe that term "Download Unit" was coined by rafi.

I agree with Remi with regards to scope creep. JD needs to focus on downloading, and downloading well, which is already does.

However, I don't see where adding some basic category functions blows the scope out of the water, and makes JD even more oganized. It can't be too hard to implement some basic categories or at a minimum not blow away the category names when adding to another package (or maintain sub directories of the packages when added).

JD should not be your media catalogging tool, but at the same time needs to facilitate the organization of what it downloads a bit better.

If I'm downloading 20 made for internet movies I don't need JD to have Movies, Movies-Asian, Movies-Western, etc, just Movies, but I would like my "Movies" download directory to have the folder names of the package, so as not to have 200 rars in the same directory.

I think the new data structure that is on the roadmap will enhance a lot of things.

Keep in mind these are my opinions, and I think JD is the best program by far. I'm not a typical user. I have all premium accounts, so I don't even use 90% of what JD has under the hood

RockinJD

I agree, only small changes are needed to make jD a better link cataloguer, not media cataloguer.

I just discovered that FDM has a smart way of dealing with download directories. When you change a download directory for a particular file it asks whether you want to change the directory for other files of the same media type. Nice. Note that this program has no packages. Its categories are based on file extensions, which makes no sense if most of your downloads are archives.

[drbits]

The problem is that the original topic of this thread was automatic distribution of files based on file type. I submitted Feature request 1234 with that assumption.

As we discussed this, I realized that determination of file type might not be easy. It will require deep extraction (and merging). It will require a list of extensions - In Priority Order - to use to identify the type of an assembly (as indicated earlier, the better downloads will include NFO, JPG, and possibly other support files).

This is preliminary, but categories could be:
Applications (EXE, DLL, CLASS, JAVA, JAR and so on)
Video (20-30 extensions, but only about 4 common ones)
Audio (another 20-30 extensions, bu only about 6 common ones)
Office/OpenOffice documents
Application addons (for example, Adobe data types)
Images
DLC
Text documents

The first match would define the Assembly type and all of the files in the assembly would be moved to the directory defined for that file type.

Is this approach acceptable?
----------------------------------
How flexible does this have to be?
1) Can we predefine the list of types, or does it have to be an editable grid?
2) Do we predefine the list of extensions for a type? or does that have to be editable?
3) How flexible do we need the directory specification to be (either a path for each type or just another variable in the final save path).

Can this be implemented in two phases? Phase 1 would be a predefined list and providing a variable for the save path. Phase 2 would incorporate the edit-ability that users request.

Note that I don't use jD Unrar, because of all its limitations. I mostly download archives, so my jD will never be able to determine automatically what it's downloading.

I think categories should be customer definable. It would be nice if the customer can choose the package (designated as a category) where a number of selected links should go to.

The simple implementation steps I see :-

1) upgrade a package to a category
2) assign a category (selected from a menu) to selected links

I 2nd Remi's post. I don't use JD's unrar either.
The reason, is the file extentions mean different things on different systems, and can mean different things on the same system.

in .BIN a Movie, or Audio rip, or is it a ROM? I think you just added bloatware

My implementation idea is similar to remi's.

Let me define my categories to put my packages into on the download tab.
Let all link/packages not assigned to a cat got to "Unassigned"
On the link grabber tab, "Continue with selected packages to -> Insert cats" so that I can ctrl-click on my packages, and send them to the correct category on the download tab.
Then allow me to drag and drop any unassigned links to my categories.

RockinJD

There is already some kind of your idea implemented in the jd... Try the "Package Customizer" addon... You can add a setting, that files, which name matches the regex "(.*?)\\.mp3", will be downloaded in a specific download folder... if you add a setting for all of your wished extensions, this should be exactly what you wanted...

Thanks for the info about the Package Customiser.

If I look at the links I usually download there is no way I can use regex or extensions to automate this assignment process.

I think the assignment of links to packages/categories cannot be automated.

@RockinJD

If I understand your proposal, then your assignments are "manual" and not automated.

Correct. Manual is sufficient for me. Automated seems to be complicated, and riddled with logic flaws

RockinJD

Originally Posted by Greeny

There is already some kind of your idea implemented in the jd... Try the "Package Customizer" addon... You can add a setting, that files, which name matches the regex "(.*?)\\.mp3", will be downloaded in a specific download folder... if you add a setting for all of your wished extensions, this should be exactly what you wanted...

Greeny

how do I use this add-on ?

Tky.

Originally Posted by remi

A package can have a download folder (Save to) and a name. Go to its Properties and change the default values. Always keep at least one dummy download link in your category package to avoid it disappears.

When you add links to the Download pane, select them and then drag & drop them on the package that corresponds with their category. If this isn't sufficiently clear, please ask.

Remi

I did not understand how to do this. Did it may for example in the youtube?
How to create packages specific.?

Grateful

[drbits]

First you enable the Package Customizer addon. Then,
1) Click on Add Filter
2) A dialog will appear asking for the name of the filter (for example, Music). The defaults for other fields will be filled in.
3) You must supply a regular expression in *nix enhanced regex format, but with every backslash doubled. Search the wiki for "regex" or "regular expression syntax".
3a) Greeny's example "(.*?)\\.mp3" is a good one. It searches the link address (URI) for any sequence ending in .mp3 (case sensitive). I would use "\\i.*?/(.*?)\\.mp3".
3b) The quotes in the examples are just for this text, they are not part of the expression.
3c) The \\i means the search is case insensitive.
3d) The .*? means any character (the .), 0 or more times (the *), but be careful to leave characters for the rest of the expression (the ?). Without the ?, .* would match the entire string and the expression would not match anything.
3e) The / is to match the slash before the file name.
3f) Then there is (.*?) - The parentheses mean that this is a useful piece that can be extracted in using the expression.
3g) The \\. is because a period character is special. If you want an actual period, you have to put a backslash in front of it (remember, backslashes are doubled here).
4) The rest of the parameters are the same as in package properties. The Package name can be entered here or the default would be used.
5) Download folder is whatever you want it to be (for example N:\downloads\music).
6) I think "Match-count from start" is how many times this rule has been used, but I haven't seen that part of the source code.

Originally Posted by Mcnammara

How to create packages specific.?

1) Double click on a package (this is a shortcut for selecting its Properties).

2) Change its name in the Package Name field. Its name will be one of your categories.

3) Change its download folder by changing the "Save to" field. There is a menu of already used folders and you can also use the Select button to browse to or create your download folder.

Oh...

Thanked for the help.
But this is awful.
I have to write to each new download ... even so not always the names of the archives are friendly, for future recognition.
Out of question.

Grateful

[drbits]

Not for each new download. I do not know if it is implemented yet, but the package name should default as it usually does in Link Grabber. You can set your download folder for each rule as in c:\downloads\music. You then check the use subdir to use the package name.

In addition, I would have all (.*?)\.rar and all (.*?)\.r\d\d go to c:\downloads\extract. Then, use a subdirectory during extraction. The problem is that the Package Customizer does not define any constants that are substituted in unrar, so it cannot be used to categorize assemblies after extraction (which is when it is most useful).

Originally Posted by RockinJD

DBM this is what Remi said but slightly more verbose,

Collect a few links/packages in Linkgrabber (or some not downloading links in the download tab). Select your movies, then right click and choose move to new package, Call it MOVIES. Now you have a Movies package that you can drag and drop all your movie related links/packages into..

Repeat this for Music, Videos, PDF's, etc. Just remember you need to disable or put a dummy link inside each package, or else if you remove ALL links from the MOVIES package, it goes away.

Now that you have packages defined, right click on the package, and change the save to folder.

Start your downloads, all your packages (aka categories) will now go to the locations you want.

RockinJD

thanks for the info,:)

but can some1 help this part to a noob:biggrin:

[Think3r]

Just leave a link inside the package and disable it? Then it won't get removed from the download's list automatically.
Greetz

If a file is compressed its in the comprssed folder, if the link you click is a video then it goes to video

if its a mp3 file then it goes to music

so, you download rar file that has a song in it, it doesnt go to music, it goes to compressed because thats the file extension ,

Hi there,

this would be my wish:

I would like to mark (tag) my downloads with categories (ie: music, movies, documetary, learn, books...) and then the downloads that would have this tag would be saved into different directory that you would configure before.

The way I see it. You would add some configuration window that anyone could add their name tags and where to save the files. In the properties in LinkGrabber you could than select the tags on the files (or packages) you wish. If you don't put the tag, than the download would be saved into default download location.

I really hope this is implemented (someday)

Thanks

[Think3r]

@kixs:
1. Read the rules before posting - a new request means a new thread.
2. Read the rules before posting - search, next time. The feature request exists, already. In addition, you will find the answer + a workaround in THIS THREAD.
Ticket #1234:

Greetz