A users' JD crashes and the first thing to ask is:
Quote:
|
Originally Posted by Jiaz
Do you have Nero installed?
|
|
#1
12.07.2010, 08:15
|
|||
|
|||
Multiple 'Downloads' tabs in single instance of JD
I would like to have more control of packages at the filehost level.
In particular, I would like to manually create separate tabs for filehosts. Thus keeping those long list of packages per filehost separate from each other. I can then sort packages per filehost without mixing into the other packages of other filehosts. Each 'downloads' tab should essentially emulate an instance of JD. For example, it should be possible to have tabs share a proxy list, thus allowing parallel use of proxies. In regard to parallel use of proxies, it would be very convenient to manually create dedicated tabs within a filehost tab for which packages could be evenly split and the links placed into those tabs but still associated with the original package somehow (maybe a package name column). Then each tab would use a different proxy from the proxy list. Thus allowing multiple simultaneous downloads from the same filehost within a single instance of JD. And since a tab can be created for each filehost, you can have simultaneous downloads of multiple files for every filehost within a single JD instance! For example, a filehost tab can have, say, 10 packages, each with, say, 7 or 14 links. I can create 7 'downloads' tabs within the filehost tab. Each tab can use a different proxy from the rotating proxy list. I can then select, say, 2 packages in the filehost tab and drag'n'drop them in any one of the 7 tabs and JD will automatically split the links in the packages such that they are evenly distributed across the 7 tabs but still remain associated with the original package. So if package 1 (p1) has 7 links p2 has 14 links, then tab 1 (T1) will have link 1 of p1 (p1l1) followed by p2l1 and p2l2. T2 will have links: p1l2, p2l3, p2l4. Etc... How difficult would it be to implement those multiple 'downloads' tabs? BTW, what framework is used to develop JDownloader, assuming the very basic GUI functionality isn't written from scratch? 
|
|
#2
12.07.2010, 08:39
|
||||
|
||||
Gui is written from scratch, no gui builder used.
as dev-ide the most devs use eclipse. multitab downloadview dont offer any advantages and only many disadvantages 1.) you no longer can see a complete list of downloads 2.) you have to check multiple tabs to see whats going on, eg wonder why a download does not start..ah right..on tab7 there are 2 downloads going 3.) manage downloads? manage download order? how across multiple tabs? 4.) mirror management? how? now you have 1 package with all mirror links in it....with your multiple tab you have 7 tabs with single package and only one host.... 5.) would be very confusing for the user to view/control/use download view is a gui thing! proxy usage is a controlling thing. 2 complete different things  why mix them? proxy has nothing to do with the downloadlist you can see.
__________________
JD-Dev & Server-Admin
|
|
#3
12.07.2010, 10:02
|
|||||||
|
|||||||
|
Thanks for the quick reply. I must say I have been loosely following the development of JD and the JD devs are quite active in hearing the suggestions of its users, so great job so far.
 BTW, whatever happened to the survey you all did some time ago? Quote:
Quote:
As a free user with, say, over 1000 packages (with 7-14 links within each of them) from various filehosts including RS, HF, MU, etc... I need a multi-level view of links! HF has 30mins wait time between downloads, RS has 15mins and MU none. I need a way to bypass the wait times and keep my bandwidth at full throttle. That requires using lots of proxies simultaneously. I can currently do so by having separate instances of JD in separate folders. However, that would require me to have a different instances for parallel links (each instance using a different proxy) for each filehost, i.e a new instance per download tab - eating endless resources and making link management very difficult. Quote:
Each tab within the filehost tab can only use a rotating proxy list, i.e not parallel proxies. The purpose of multiple tabs is to use multiple proxies - 1 proxy per tab. Quote:
Have you ever used Azureus in classic view? You have a global view with over 20+ columns and with global download rules for all torrents. However, it was possible to override those rules for individual torrents. Having the ability to override download rules per package may be a nice feature, but for me, isn't as necessary as download rules per filehost. For example, how do I limit my bandwidth to 20% for MU, 30% for RS and 50% for HF to download more HF files faster (of course, the allocated bandwidth would dynamically change if a filehost has completed)? Quote:
Quote:
Quote:
Proxies will only know about tabs (1 proxy per tab using rotating proxies). Those 'proxy-tabs' will only be created within 'filehost' tabs (think of it as parallel threads in a process). I would like to view packages in different ways - flat view and the user-customizable multi-tab 'filehost' view.
|
|
#5
12.07.2010, 10:26
|
|||
|
|||
|
Quote:
|
|
#6
12.07.2010, 10:56
|
|||
|
|||
@Bill
I like reading posts like yours. Most of it is exactly what I feel for a long time, based on customer comments and requests but I fully understand Jiaz' reply as well. At the moment jD is as it is. It's simply the best multi-platform file host download program on this planet (as long as we don't receive contradictory data from other (exo)planets or star systems) but it isn't perfect. Who or what's perfect? Even a diamond can be smashed when you apply enough force. If you want, you can also comment on my Concept Diagram I published last year. It's probably full of mistakes, short-cuts and other types of misfortune. In other words, there are many ways in which that "one page" model can be enhanced. All comments are welcome. I prefer diagrams over long texts, because a picture is often worth a thousand words. Jiaz has written several times that he wants to redesign the download engine. The download engine is the kernel of jD. I think he should take into account these requirements. They shouldn't be implemented overnight but the engine's architecture should be able to cope with such features. I know your simultaneous downloads using proxy servers is not included in the diagram, because I feel it's still open for debate. As to the views you would like to see, there have been similar (probably easier to understand) proposals, without using tabs. Some propose sub-packages and others propose filters based on criteria such as host or customer defined categories. Please, try to understand and comment on the diagram. Tell me if you don't understand it.
Last edited by remi; 12.07.2010 at 10:58.
|
|
#7
12.07.2010, 11:59
|
|||
|
|||
|
Your Concept Diagram is flat. It isn't modular. There is no workflow. I see it as a branching brainstorm.
Consider the following: A user/client adds links to JD and clicks Start. JD downloads the files. The user is happy. That is your use-case. JD keeps track of links and downloads files pointed-to by those links. Link is the main conceptual object. Properties of Link, their cardinality, etc. will unfold as the workflow (object collaboration) becomes more rigorously defined overtime. The functionality of JD should always relate to the way in which links are intended to be used. This is because link management (including both the link's properties (URL, File, etc.) and behaviour (downloading)) is what the user is most concerned about. As to the views I would like to see, they are management views with the potential to offer group-level functionality that is a level lower than the current flat view of all packages, but yet is a level above packages. Such group-level functionality should be able to exhibit behaviour that is different from other packages or groups by allowing the user to specify different download rules (and possibly other rules like extraction rules, etc.). There would have to be strict global rules/limits such that group rules/limits must not collectively break/exceed. In any case, I want a downloader that can download multiple files from multiple filehosts simultaneously for free users by using proxies. The idea I was trying to get at is to have tabs offer the functionality of individual JD instances, but JD would still manage (i.e. know about, extract, etc.) all links as usual because there would be only a single instance of JD.
|
|
#8
12.07.2010, 12:49
|
|||||
|
|||||
|
Quote:
Quote:
Quote:
Quote:
Quote:
|
|
#9
12.07.2010, 13:17
|
|||
|
|||
|
I meant if you were going to have a use case then that is what it should be as a starting point. I can see your diagram is a brainstorm to get the ideas on paper.
My grouping is at a higher level and is more flexible as it is user-defined/created. I can have a tab/group that has links from more than one filehost and I can download those links as though the tab was a separate instance of JD, with different JD setting. But because it is in a single instance of JD I can switch views, move the links to other tabs, modify the grouping or its settings, etc. I would still like to know how difficult it would be to implement such a multilevel view. How modular is the JD source code? How modular is the JD kernel? Would the JD kernel have to be re-written from scratch? The parallel proxies feature is really the only thing that matters to me right now because I still have to manually enter the captcha for HF and RS has no captcha.
|
|
#10
12.07.2010, 13:40
|
|||
|
|||
|
I think the different views are easier to implement than kernel changes. In order to get them accepted by the devs you'll need a lot more ideas to get that proposal passed. Maybe, some screen dumps of mock-ups might help to illustrate your ideas.
If you want to assess the modularity of the code, please have a look at jD's developers page and "First Developer Steps". You can contact the developers with support@jdownloader.org I'm sorry but I don't understand the link between parallel proxies and captchas. Can you please explain that in more detail?
|
|
#11
12.07.2010, 14:29
|
|||
|
|||
|
RS not having any captcha means the bottleneck is its waiting time of 15mins between downloads, not to mention the slow download speed. Both bottlenecks (waiting time + slow speed) can be solved with parallel proxies. HF has ?unbreakable? captcha, so that bottleneck remains, but its other bottlenecks - 30mins waiting time between downloads and slow speed - can be solved with parallel proxies.
If I was going to write a downloading program to automate downloads (using proxies) from other sites, I would have to steal the anti-captcha code (among other things) because I can't implement that myself. If JDownloader uses a modular architecture like one that's imposed by the NetBeans Platform through its Modules API, then it shouldn't be overly complicated to place multiple tabs in JD with each tab having the functionality of individual JD instances, but would still share properties/functionality that is global to JD (i.e. not particularly specific to each JD instance). The devs won't implement this multi-level link management feature anytime soon as this thread has been marked unsolvable. Also, this feature will require each Link to know (or be associated with) almost every setting in JD as it relates to link management, similar to individual torrent entries in Azureus/Vuze (classic view) and uTorrent. I'll have to look at the code and add my own hacks to it, assuming it's not spaghetti code. I'll call it JDMod. It will be able to do what JD already does plus download multiple files from multiple filehosts simultaneously for free users by using proxies. Isn't open source great! 
|
|
#12
12.07.2010, 15:17
|
||||
|
||||
|
proxy rotation will come, parallel proxy usage maybe.
why maybe...its very easy...and you already gave the answer... the main reason why parallel proxy support would be bad is that free hosters are forced to limit download even more....and in the end use reCaptcha as well...and then your cool little parallel proxy feature resulted in what you did not wanted...even more limitations for free users. also note that by adding parallel proxies, everyone things..hey proxy..cool...i use it and sooner or later every public known proxy will be 1.) unusable 2.) not there anymore (died because of traffic costs) 3.) blocked by hoster 4.) those sites blocked and that are the reasons why we havent decided yet....yes it would be a nice feature..but give this feature to the masses of jd users and it will end in more pain than joy. many favorite hosters had to start using reCaptcha or other techniques (slower downloads, longer waittimes) once the ciritical mass of free users reached....imagine parallel proxy support and those numbers will double/tripple...in very short time... i can predict for sure that such a feature in a manager with high user numbers will make hoster change their minds. we already experimented with parallel proxy support and yes it works. but that was just an experiment, far from ready to publish state. as to your multiple tab view....yes i also want a filtered view and that is what most ppl want....but only having a filter is not enough..you always have to thing about 1.) noobs must be able to use the tool too 2.) how to manage dragdrop, list controlling in a filtered view 3.) multiple tabs...still dont see any advantage that a filtered view cannot offer too 4.) the same reason why we did not add subpackages, how should a normal user be able to control that thing...he/she only wants to add link and click download, not search the tab with the download and then wonderung why its not running because on tab 19 the download is taking the max download slot 5.) im sorry but you cannot compare file hosters with torrents. in torrents you dnt have to deal with different speeds, different waittimes, different features (resume yes no, chunks yes no), different limits (1 file, unlimited), free or premium usage and so on. in the end....you can easily avoid limitations with reconnect or a premium account and if you think jd is useless when having a premium account, then you dont know what jd is all about and never used its powerful features.greetings jiaz
__________________
JD-Dev & Server-Admin
|
|
#13
12.07.2010, 15:22
|
||||
|
||||
|
My 2 cents:
If you want to get good speeds for free buy premium for premiumhosters OR use fast free filehosts then you'll get the good speed without having to use proxies. This is the easiest solution for this "speedproblem"^^ GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
|
|
#14
12.07.2010, 17:11
|
|||
|
|||
|
I have nothing against JD's support for premium users. I simply don't intend to pay a cent to file hosts because I can instead pay for usenet or just pay for the crap I download. But I'm not going to pay for that either. I only spend money on tangible things necessary for survival (food, etc.) and tangible things that can improve quality of life. That's my personal opinion/policy and it's not going to change.
If all file hosts were like MU or MF then I would be content. But sites like RS and HF have long waiting times. 30mins between downloads and 30mins to download a link means 7 links in 7 hours. I can download at least 4x as much from MU or MF within the same time. The uploaders chose the file host, not the downloaders. HF pays them money. Reconnecting is not an option with simultaneous downloads from different file hosts. There was a time when RS had 45mins between downloads. People used proxies with USD to bypass the limit and it affected the download speed of premium users. RS realized and tried to combat it with weird captchas but there was a site that allowed users to manually enter captchas in exchange for points so others would automatically enter captchas for them. RS had no choice but to expand its network to maintain QOS for its premium users. All the while sites like MU, without ridiculous waiting times, were still doing fine and continued offering good service to both free and premium users. Moral of this story: don't screw with free users and they won't screw you! I won't stop trying to find a way to download with parallel proxies. As to the effect on proxy hosts, they will be collateral damage, a means to an end, that's unavoidable. I expect them to do what they have to to maintain QOS, but by then I would've cleared my backlog of links and switched over to torrents. The multilevel view allows for a level of granularity between the single flat view and the single-level tree of packages within it; sub-packages only extend the level of the tree, it's still 2D and it is still 'fixed' i.e. links can still only belong to (associated with) one sub-package . Tabs/groups add another dimension to link-management by grouping links based on whatever properties/settings they want to apply only to that group but as a flexible logical view. For example, links for music could be downloaded and extracted to a different folder than links for videos. JD doesn't need to look in the rars to see the file type, I will simply place those links in the Music tab or Video tab. Filtered view is logically independent of tab view - you can filter in the global tab as well as in individual tabs. The order of downloading links can be tricky, maybe top to bottom for each tab and left to right for 'user activated' tabs. It probably makes more sense if each tab is mostly used for a single filehost. This also gives me the idea of tagging links. Say, a user can tag links as music then setup rules/settings for all links tagged with music as well as having rules/settings for links of a particular filehost. So those rules would have to be prioritized by prioritizing tags. Download priority would then be based on tag priority. Tagging of links is only an idea, btw. There are various reasons why users may want to uses multiple instances of JD to have flexible control over links, so tabs/groups/multilevel view is for them, other users can consider it an advanced feature that is off by default. The torrent client comparison was only to indicate that each link must store a lot of information about itself in order to have link-level control, just as you currently have 'downloads' control using global JD settings. Link-level control allows higher-level views to manage link-level settings as it pertains to a group of links. I understand that more free connections will increase server load of filehosts and they will eventually switch to reCaptcha and put other measures in place. But free users will always find a way to bypass these limits, which will either make or break those filehosts. RS was able to expand its network and provided better QOS for both its free and premium users for a time. Why can't those filehost be more considerate to free users like MU and MF? If that was the case then we wouldn't be having this discussion. Also, maybe 1/50 free easy-to-find public proxies are currently fast enough to download large files fast. So I'm not too worried about the mass JD user-base taking all the good proxies. I expect, out of the users who decide to try proxies, 49/50 users will give up when they realize most public proxies are too slow. Which means filehost don't have much to fear from proxy users as much as they have to fear from their growing downloader user-base as they get more popular (unless someone can show me credible statistics that proves otherwise). Last edited by Bill; 12.07.2010 at 17:17.
|
|
#15
12.07.2010, 19:18
|
|||
|
|||
|
I think all the Hosters who know about JD tries to block it some way or the other (mostly because of its reconnect feature). By using proxies, JD will seriously violate the TOS of those hosters & work as an illegal leeching software giving undue advantages to free users.
I am not in its favor.
|
|
#16
13.07.2010, 00:45
|
|||
|
|||
|
Anti-captcha isn't illegal?
The point of captcha is to ensure every download is manually done by individual users, not automated software. JD has no problems in bypassing that limitation and giving it to the masses. IMO anti-captcha is at least an order of magnitude higher on the leeching scale because it requires absolutely no effort on the part of the user (as opposed to finding fresh proxy lists regularly) and works 98% of the time (as opposed to 2% of proxies being fast). Anti-captcha is killing filehosts and causing those hosts to impose greater restrictions on free users. So if you insist on being sympathetic to file hosts then turn off anti-captcha. 
|
|
#17
13.07.2010, 02:16
|
||||
|
||||
|
Quote:
If a filehost wants to prevent users from downloading 100% automated they can use Re Captcha and their problem is solved. Do you think JD is killing hosts like megaupload by "bypassing" their captcha ? NO, if MU wouldn't want the captcha recognition they would change their captchas @pauldmps Nope, actually we support about 200 hosters (i think) and i think less than 5 of all those try to block jd GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
|
|
#18
13.07.2010, 02:29
|
|||
|
|||
|
Why do you think RS tried those weird captchas before, then eventually doing away with captchas entirely when they expanded their network? Why do you think HF is using ReCaptcha now and they didn't before?
File hosts are definitely feeling the effect of automated anti-captcha. It's only a matter of time before MU and others switch over to ReCaptcha once they are sure they have a stable implementation with all the bugs worked out.
|
|
#19
13.07.2010, 02:44
|
||||
|
||||
|
@Bill
Well but Anti-Captcha isn't killing filehosts, that's what i said and i'm right. If they wanna have good captchas they know what to do. All hosters who use weak captchas are doing this on purpost. We also have captcha recognitions for filehosts who NEVER changed their captchas for years so not having the recognitions for those would be a waste GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
|
|
#20
13.07.2010, 03:15
|
||||
|
||||
|
The bottom line is that if you use multiple fast proxies, you are reducing the bandwidth available to other free users. It has nothing to do with sympathy for the companies.
--------------------------------- Your explanation about what you want to do with multiple tabs has changed. I suspect your ideas are changing. I rejected your proposal because it did not add enough to the program to encourage developers (volunteers) to want to implement it in the next couple of years. Right now, the database and download page need major improvement before any major new features can be added. Stability improvement, speed improvement, memory reduction, status management, and redesigning plugins all have priority. Give me an attractive proposal. I now could write one, but I need to read your ideas (which might be more imaginative).
|
|
#21
13.07.2010, 05:13
|
|||
|
|||
|
@pspzockerscene
There are a few reasons I can think of why file hosts aren't using reCaptcha: 1) Illegible catpcha - strike-through, circles, bad handwriting 2) English words - compounds problem in 1) for non-English users 3) It's a non-standardized closed source 3rd party plugin/tool/framework as seen from the point of view of file hosts and other clients. It's the same excuse Apple uses for justifying why it doesn't support Flash and other 3rd-party frameworks. That third point is probably the most important reason why file hosts still use their own proprietary captcha framework. I suspect only people who develop software in a large consulting company would understand the full benefit of maintaining a proprietary fully-internal development environment and workflow by using open standards and not using 3rd-party frameworks. 3rd-party frameworks require waiting on 3rd-party developers to fix problems, and in the meantime you are losing clients/money because you can't fix the problem yourself because the plugin is closed source and is not your code to begin with. Quote:
Without anti-captcha, users would have to constantly check and manually enter captchas between downloads. That 'user delay' could range from minutes to hours or several hours (while sleeping) between downloads. The manual way would probably not be sustainable by most average JD users for long, that's why they use JD in the first place- because of its anti-captcha! Also, it is highly unlikely an average JD user will find "multiple fast proxies" frequently and in large quantities. If they spend some time on it they may get 3 fast proxies on one day, but then they would have to search for new proxies almost on a daily basis to keep up the speed. If you want to belive that support for potential use (i.e. not fully functional by default, as opposed to simply checking a box to enable anti-captcha) of parallel proxies is going to be more detrimental to file hosts and its premium and free users than the impact of automated anti-captcha is currently having, then so be it - I have already made my point. Quote:
My use-case is to avoid a long 1000+ list of packages in one panel. I would separate them by filehost first because that differentiates the links the most and the type of filehost also affects the downloading of links the most (captcha, wait time, speed). The list per filehost would still be too long so I would create more than one filehost tab, say 3 tabs, with fhtab1 having links dated 2010, fhtab2 with links dates 2009, and fhtab3 with links dates before 2009. Another user may use tabs to separate types of downloads, eg. music, movies, apps. It is important to realize that what I am suggesting is not a simple search/filtering functionality, instead it is cataloging functionality! Cataloging of links! Hence the idea of using tags, in particular multi-level hierarchical tagging functionality (i.e. tag metadata), but that is a feature for another thread. As an example of the use of multi-level tags i.e. tag metadata, links can be automatically tagged by date, so 2010, 2009, 2008 are all tags, but those three tags will be tagged 'date'. Similarly, music, movies, apps, are tags but those tags will also be tagged as 'type'. Those tags should be easy to create/modify/delete by the user. JD can also have default tags and meta-tags as well. So in the case of tags, a tab is just a logical view of filtering the links by tags and keeping that query-view persistent as a tab. The second and main reason for using tabs is to make it easier to view and manage the use of parallel proxies. Similar to the analogy of parallel threads in a process, each filehost tab (i.e. top-level tab) can have several 'worker threads'/tabs (nested one-level within filehost tab) which will treat the links in the filehost tab as a queue and automatically download the links in parallel, (each worker-tab using a different proxy) in the same order as the links in the filehost tab. So each top-level tab can act as a separate instance of JD with different settings like extraction paths, etc. So tabs still offer some level of management functionality and are not only for views. The overall order of downloading links can be as simple as, always download links from top to bottom for each tab and start from left tab to right tab. Only worker-tabs will do actual downloading of links - one link per worker-tab, so they limit how many links are being downloaded simultaneously per top-level tab (e.g. filehost tab). There would then be global JD rules/limits that limit total number of worker-tabs active at any given time. The idea of 'worker-threads'/tabs can simply be implemented as a scroll wheel 'max. concurrent downloads' per tab, but then that won't allow reordering of the link queue per worker-thread. It should be possible to temporarily 'disable' tabs so those links won't be downloaded. It should also be possible to set bandwidth download limits per tab, etc., Again, those are settings that would normally only apply to a single JD instance, but now would apply per tab/group. Last edited by Bill; 13.07.2010 at 05:50.
|
|
#22
13.07.2010, 09:08
|
|||
|
|||
|
Quote:
I said that using proxies & other brute-force methods of bypassing the limits placed by hosters on free users is illegal. This system of downloading should not be compared to P2P networks (torrents). This is because there are no storage or bandwidth costs associated with torrents. The hosters do have to maintain their servers & pay for bandwidth. They need a source of income. Thta is why they try to push premium.
|
|
#23
13.07.2010, 09:57
|
||||
|
||||
|
i think we dont have to continue this discussion as its more a point of
1.) why someone does not pay premium but pay for internet 1.1.) with a premium , thats very cheap too, you can leech as much as you want (depending on hoster) 2.) why ppl always want everything for free 3.) why ppl always want MORE for free (in case of parallel proxies) as i said its not yet discussed in team if or when we will add this parallel proxy feature. also i STILL dont see any advantage of multiple tabs..only many disadvantages. and you still MIX controlling(what proxy to use) with viewing(what are my links). when we add such an feature then in a way the user has nothing to do exept to add the proxies..why should a user interest the proxy..the user wants to download and thats all...jd should manage all proxies, waittimes, host usage and so on...if you want control over such things then you can use self written wget scripts and then you can control every little detail....you dont ask your browser to show seperate config for each tab at this moment i still think there are much better ways to improve usability like: 1.) subpackages 2.) filtered views 3.) 3 tabs (history, downloadlist, current downloads) and so on.....im sure 2 and 3 will come cause i think they help most. greetings jiaz
__________________
JD-Dev & Server-Admin
|
|
#24
13.07.2010, 10:05
|
||||
|
||||
|
@ pauldmps, I think that we should stop that side discussion or move it.
@ Bill, This would be almost the same as the existing category proposal. You understand that none of the tag data you are referring to can be automated. Most of the time, the link source is not known. The Host is known, and in some cases a host may have categories, but that is not normal. The Host really does not do a lot of good except for controlling download, but you can already do that with sort. I agree with either a tree of categories or a collection of tags. We don't have the resources to mine data (such as automating category by searching for the title). We usually don't even have a useful file extension. I would propose that the download list be kept as a single list and that View tabs be added (filtered and organized by any column or by tags possibly add "AND" and "OR" operations to tags.). Keeping the download list itself unchanged would prevent this from requiring massive changes in the download engine (and you REALLY don't want that). What do you say? @ Jiaz, Sorry, we overlapped again. I think we should split the multiple proxy idea to a different thread. There isn't really much to discuss. The proxies have nothing to do with categories or multiple download tabs. If you do not disagree, I will split the topics tomorrow (some posts will be split and I will mark that). I agree with you that the discussion should separate the features that are needed and the engineering. For example, if we add Views, users can control order of downloading by using the existing Priority mechanism (allow all of the context commands in the View, but apply it to the Link, Assembly, sub-package (if that comes), or Package). Link, Assembly (what Link Grabber assembles), and Corrected-Assembly are natural. Packages are a convenience. Sub-packages might not be necessary with tags (especially if one of the tags can be the container name). One suggestion: Make status available as tags. That is, Downloading, Downloaded and Finished would be tags. Allow users to deal with old links in the same view as new ones. Allow users to search by tag for unusual circumstances. Again, what do you think? Last edited by drbits; 13.07.2010 at 10:23.
|
|
#25
13.07.2010, 10:07
|
||||
|
||||
|
thats what i mean
eg 3 tabs 1.) a history of what has been loaded 2.) the complete list (move, dragdrop and such thins possible) 3.) all current downloads (so you can see the real downloading ones without having to scroll a big downloadlist) 4.) (maybe) additional tabs with filtered views (eg only packages with xy)...no dragdrop and such things here
__________________
JD-Dev & Server-Admin
|
|
#26
13.07.2010, 23:48
|
||||
|
||||
|
Btw. easy-share.com now also changed to Re Captcha.
Good for us so we don't have to play the "we crack your captchas and you change them again" game anymore GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:
A users' JD crashes and the first thing to ask is: Quote:
|
|
#27
14.07.2010, 08:15
|
|||
|
|||
Multi-level Tabs, Link Cataloging, Parallel Proxies
Quote:
Visualize, if you will, the current JD downloads panel - it is a tab called 'Downloads'. I now create 3 tabs nested within the Downloads tab, called Apps, Music, and Movies. JD will automatically create a 4th tab called Remainder. Why a Remainder tab? The intent is to partition the Downloads links into mutually exclusive subsets to allow for a more organized downloads list. Links that aren't in any of its siblings but are in its parent will be in Remainder. Thus, Remainder = Parent - Parent's user-defined children. So all links in the parent tab are automatically copied initially to the Remainder tab. The parent tab (in this case Downloads) will always show all the links (as a union of the links) in its children tabs. Therefore, Downloads still remains the 'global' tab when clicked on. I then move links from Remainder (or copy links from Parent, JD should know and deal with the difference appropriately) to the other tabs. From there, I then create filehost tabs within each of Apps, Music, and Movies. (Reason for filehost tabs: JD currently allows parallel downloads from different file hosts, but I have to scroll 1000+ links to see what's downloading and slightly change the order of the links if I want. Under no circumstances do I want to reorder the links by clicking the column header! because that's the order I want the links to be downloaded.) So for example, Music will have RS, HF, and MU tabs as well as an automatically created Remainder tab that will contain a mix of other filehosts like MF. So I move the links from Remainder to the appropriate filehost tabs. JD can help by allowing the user to search/filter Remainder by filehost, thus making it easier to find the appropriate links to move across to the other tabs. The RS tab in Music still contains 500+ links, so I want to better organize them visually. I create 2 tabs within RS called 2010 and 2009. The Remainder is again automatically created below RS and filled with all Music->RS links initially. I manually move the links to the appropriate 2010 or 2009 tabs. Again, JD can help by allowing the user to filter by date added. (P.S. Live search/filtering functionality should be available for any tab in general.) _________________________________________________________________ The use-case given above describes what I REALLY want. It uses a certain kind of multi-level tabs, which I prefer to call 'Partition' tabs. Partition tabs partition the set of links in the Parent tab into mutually exclusive subsets of links, i.e. 1-1 relationship between an exclusive subset and a Partition tab. 'View' tabs are intended to be persistent (only closed by user) real-time-updated (updates itself when links change/added/removed) views of search/filter/query results. For example, when a new View tab is created (say, right-click on tab bar and select 'New View') it should be blank with a search box at the top right corner where a search term (e.g. a tag) is entered and the results are returned and filtered in real-time as the user types additional terms in the box. (Actually, it can be as complex as allowing users to do complex SQL-type queries on the links and the links' metadata. But that's up to the implementation.) It should be possible to drag'n'drop an existing (nested) tab onto the tab bar and a new View tab would be created that is a copy of the tab - queries on the View would only work on that dataset. The only difference between View and Partition tabs is the constraint that sibling Partition tabs should be mutually exclusive to each other in terms of their links, and that constraint should always be enforced by JD when new links are added to the Parent tab. The link-level functionality (e.g. tagging) offered to the user should be the same for both View and Partition tabs, except that Partition tabs nested in Downloads are the only tabs that affect the download order of links because that is the main downloads list. It should also be possible to automatically partition a View. For example, a View of all RS links can then be partitioned into 2010, 2009 and the automatically created Remainder tab will have RS links not in 2010 or 2009. N.B. By using nested tabs instead of nested subpackages I can use the entire width and height of the JD window to see the links in the tab currently selected. Subpackages is a tree view, where all the tree nodes and leaves share the same screen and I would have to keep scrolling up and down to explore the tree. Tabs provide a horizontal B-tree view of nodes (sibling tabs are side-by-side horizontally and on the row below the parent tab). P.S. Being able to backup a tab (with links and links' metadata) would allow for some flexibility. For example, I can then load the tab into another instance of JD with its own proxy list. P.P.S All these multi-row tabs would have to be thinner to avoid taking up too much screen space. _________________________________________________________________ An alternative to manually moving links from Remainder to sibling tabs, is to specify/associate a filter/query string with the user-defined tab. Thus, Music->RS would have a query string like "filehost=RS", which would automatically place all RS links within the Music dataset into the RS tab. The query string will be persistent, so new RS links added to Music will automatically be placed in the RS tab. Of course, that automatic method will only work with attributes JD already knows about, so it won't work for file types like Music. The Link Grabber should somehow present the tabs that are nested one level (or maybe more) below Downloads to the user for category selection. It should automatically place links in the downloads list according to tab query strings if applicable. For example, if there was a tab just below Downloads called RS with query string "filehost=RS", then new RS links would be automatically added to RS tab instead of Remainder, if nothing else was specified when adding the link. Implicitly, the full query could be like "RS tab = tab=downloads & filehost=RS". If tabs just below Downloads (Apps, Music, Movies) don't have associated query strings, then those tabs should be user-selectable as mutually exclusive options (like a scroll box ) when the links are added, otherwise they will go in Remainder. The exact implementation and view details may not be like that, but that's the idea. So for example, RS links can be added to Link Grabber and Music selected from the scroll box. The links will then be added to Music->RS (in my example it will be added to Remainder next to 2009, 2010 tabs). The internal query string might be "Music->RS tab = tab=Downloads->Music & filehost=RS". Tagging of links can happen in Link Grabber as well. _________________________________________________________________ I strongly recommend using meta-tags with tags, otherwise there will be a mess of tags with no structure or relationships. For example, if you have tags for: Queued, Downloading, Finished then those tags should be tagged Status, where Status is the meta-tag. A simple usage of Status metatag is to create a View tab called Status and search for all links tagged Status, which in-turn would return links tagged with Queued, Downloading, Finished. The meta-tag itself won't be used to tag a link directly (as it does not (or generally should not) provide enough specific useful information to describe a link on its own - it describes tags, not links). So JD must know how to make appropriate use of the connection/relationship between tags and their metatag. Implementing multilevel download tabs and tags will require working at the link-level. So with regard to downloading an individual link, the link (in a queue) can be assigned to one of many 'download-threads'. It should then be trivial to either use the user's IP address for downloading if they aren't using proxies, or simply select a proxy from a rotating proxy list shared by all download-threads. Having download-threads as tabs may make it easier to visualize and control by the user, but it is not necessary. Parallel proxies can simply be viewed by the user as a scroll wheel 'max. concurrent proxies' at the base of the JD window. _________________________________________________________________ In summary, I discussed the following requests: 1) Multi-level tabs; including 2 types of tabs: Partition, View 2) Cataloging of links; using multi-level tags: tags (leaves) and meta-tags (non-leaves) 3) Parallel proxies Those 3 features will significantly improve the usability and efficiency of JD in-keeping with my usage model of JD. 
Last edited by Bill; 14.07.2010 at 11:54. Reason: different example of Partition tabs in Views
|
|
#28
14.07.2010, 09:04
|
||||
|
||||
|
you cannot change order of links when not viewing all of them
imagine P1 F1 F2 F3 F4 now seperated in tab T1 T2 F1 F2 F3 F4 i want F2 to be loaded first, now what? also i still see no advantage for controlling as you loose overall informations and this also will break mirrormanagement... user is in T1 and wants to redownload a mirror link that got deactivated by jd. now he 1.) first have to check all tabs for the same package with the mirror 2.) enable it there without tabs, the mirror is direct reachable! i still think what you really need is a better downloadlist managing, eg subpackages. parallel proxy has NOTHING to do with that at all. user does not want to know why its loading and so on..he/she just wants to load. also you have a hughe ressource increase when you want live tabs and so on (multiple tabs, keeping controll across all of them) live status updates, live inserts, filter rules and so on.... im very sure that multi tab like you described it will not come..best i can hink of is history,download(with maybe subpacakges or something like that), download view and some kind of view filtering this thread is about managing complex download lists (and i must say that a normal user does not queue up alot of files, because its some kind of useless...having 10k links and in moment yu want to download the links are all gone..) parallel proxies, multi ip and so on..has nothing to do with that at all and does not belong here
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 14.07.2010 at 09:07.
|
|
#29
14.07.2010, 12:08
|
|||||||
|
|||||||
|
Quote:
Users should be able to (numerically) prioritize tabs and links within tabs. The priorities add up - prioritized link in prioritized tab has higher priority than prioritized link in non-prioritized tab unless the link priority is high enough. Also, it should be possible for users to 'disable' tabs so that those links aren't downloaded. Thus F2 should be set to a high priority, or move across T2 to 1st position. If the links in T1 are from different filehosts than F2 then F2 can be downloaded in parallel with links in T1. If, say, F2 is from the same filehost as F3 then there are two choices: either remain in T1 (i.e. DFT) or start with top link of T2 (i.e. BFT). You can use BFT as the default if you want and allow the user to change traversals as they see fit. That's the disadvantage of working in higher dimensions - more degrees of freedom - which is also the main advantage. Quote:
Quote:
Read again my comment about subpackages vs B-tree tabs. Quote:
Quote:
 Quote:
Quote:
|
|
#30
14.07.2010, 12:10
|
||||
|
||||
|
Given what you say, I still like the idea of being able to filter by tags (rather than categories), and some tags could be automatic (status).
That said, when looking at the download list, eventually one should be able to "drill down" from a package to an assembly (what is required to restore files from an archive), to mirrors for a file in the assembly. I am not sure what extra functionality arbitrary subpackages would add.
|
|
#31
14.07.2010, 12:51
|
|||
|
|||
|
@Bill
I've been following this interesting discussion ([unsolvable] tags attract me in particular) and I would like to see a graphical presentation (mockup, diagram) of what you propose. Can you give an example of an application with those cataloguing/viewing features? I currently use packages as categories (name and download folder) and that works fine. My categories correspond to the sources of my links. Mostly they correspond to hosts as well, because link publishers prefer some hosts over others. These categories would automatically correspond to different media, because my sources tend to stick with one type of media (music, movies, etc.) as well. There still are other tools like FreeRapid that support multiple, parallel proxies. Thanks to jD these tools stay alive. I don't think jD should protect file hosts. I discovered that file hosts are still operating in a free market and only the best will survive. (It'll give pspzockerscene a real life, because he won't have to babysit hundreds of hosts.) The same with proxy servers. Currently, more than 10% of people are using them. If demand grows, more proxy server services will be offered. My view on Captchas is that they're illegal because they discriminate between people who have good eyes and those who have bad eyes or are blind. Blind people definitely need anti-captcha tools to get access to the sites who're using them. Thanks to anti-captcha tools like jD blind people can now download whatever they want. ReCaptcha is still illegal, but that doesn't surprise me, because Gogol (remember the word "evil"?) invented them. @drbits : good luck with splitting this brain storm session into cleanly separated subjects. I would use tags.
|
|
#32
14.07.2010, 13:10
|
|||
|
|||
|
Quote:
I'll try FreeRapid.
|
|
#33
15.07.2010, 11:59
|
||||
|
||||
|
There is no problem separating anticaptcha, proxies, and things like that in this thread. Future posts will have that edited out or the post will be rejected. Those issues have their own threads and do not belong here.
____________________________________________________ Remi asked about searching with tags. I usually think of uTorrent when I think of Categories. I usually think of blogspot and other more generic tools for tags. Generally, a tab based system allows one to search by entering keywords and then matching those keywords with title words, ordering by relevance, and returning the results. Even Web Search Engines use tags (the <Meta keywords="..."> statements). There are three ways to deal with a multiple "keyword" (tag) request. 1) If any keyword matches, then the item is a match (vBulletin uses this and it drives me crazy). This is absolutely wrong, unless you have a powerful relevance engine. 2) If all keywords match then the item is a match (this is less common). This allows people who know what they want to specify what they want and the set of results is narrowly defined. This is usually too restrictive, so people add complicated features like stemming (which drives a lot of people crazy) 3) Expressions. (tag + keyword) * (search + sort + find) a) This allows one to search for tag OR keyword and for at least one of the final three words. b) One of the options is whether to assume these are prefixes, or a character at the end of the word is used to mean "truncatation" (*, $, and #), that this is a match if the first part matches (sometimes this is always done). Stemming requires the most of the code of a spelling checker, while truncation is just matching. c) Every modern programming language has a PERL like comparison. Although, usually it is a full *nix regular expression. d) + is often replaced by | or OR. - is often replaced by NOT, and * is often used to replace by & or AND. Relevance: There is a scientific definition of relevance in search results and there is a real-world definition. The real-world definition is whether it sorts the desired items to the top of the list. The scientific definition is a statistical way to make that happen most of the time. The value of a word match is log(#occurrences+1)/log(Total occurrences+1). To combine word match values, OR becomes addition, AND becomes multiplication, and NOT is usually a problem, but it is usually subtraction. To count the # of occurrences, one counts a word in text as 1, a word in an inside title as 2, a word in an object title as 4, and a tag match as 6 (the number can be adjusted). When only part of the word matches, this can be used to adjust the value of the occurrence. There is no SEO problem, because people are working to make desired things easy to find, not to compete for the top spot. The final addition that is interesting (but probably not necessary for JD) is to redefine the value of an occurrence of a word using an algorithm like a neural network.
|
|
#34
15.07.2010, 17:17
|
|||
|
|||
|
I view tag relationships as a graph structure. A tag can have multiple parents (generalized categories for which that tag can be a part/subset of), siblings (synonyms) and children (specific sub-categories of the tag).
A tag is just another link metadata attribute. So for example, Windows Search Advanced Query Syntax could be used since it is a complete/robust (i.e. not random guess work) query syntax. @Remi From what I can tell, Videophile II uses a (logical) graph-like structure to implement tagging/cataloging functionality. It does not have anything to do with tabs. The multi-level tabs I described is an application of a B-tree structure to efficiently organize data.
|
|
#35
16.07.2010, 07:50
|
||||
|
||||
|
Bill,
How useful is it to know that the tag SciFi is a child of (book, tv, and film)? One does not usually go to the parent tag. Cataloging tags is a way to help people find the right tags in huge collections, not in a few hundred (or even tens of thousands) of assemblies. To use a catalog, one always goes down or sideways (to a sibling), but none of that has to be automated for this kind of application. Let's not blow this out of proportion. We are not tagging Wikipedia. A total of 10-20 tags will be enough for most people (music, tv, clip, movie, software, book, and a few for special interests). Only the most general tags would be predefined, if that. Things like host are not tags, each is just an automated piece of information about the link (inherited by assemblies and packages). For those people who want to work hard, the web site they got the link from might be useful (for most people, that is 2-3 sites per category). According to Bell's theory, the number tags needed grows approximately with the log of the number of assemblies. This takes care of link sources, categories, file types, and interests. I estimate 3*log2(assemblies) at the maximum for people with medium sized collections (add a few for small collections). That is under 50 tags for a reasonable life-time of downloading (3*log2(60K) assemblies is 48).
|
|
#36
16.07.2010, 11:41
|
|||
|
|||
|
@Bill
B-tree structures are used as technical implementation solutions for databases an file systems. I still would like to see an example of an application that uses such a B-tree structure in a tabbed human machine interface. The tag relationships you're describing are typical for ontologies used in semantic web technologies. I think this would be overkill for jD as jD can't cope with millions of links. jD is not a media storage or cataloguing application but a download manager.
|
|
#37
16.07.2010, 11:54
|
|||
|
|||
|
Quote:
Instead of using all the rigid forum structures and rules that are not understood by (mostly) new members, we could tag the threads/posts in order to make them more easy to search. It would avoid all the nervous breaking, off-topic accusations, forcing some moderators to take pills because they have to waste their precious time merging, moving and cleaning threads. It doesn't work as I've numerous examples of threads that have slipped through the maze. BTW, this might be an off-topic post as well.
|
|
#38
16.07.2010, 17:16
|
||||
|
||||
|
Quote:
It is important to realize that all links point to files to be downloaded, so tags would usually describe those files. Therefore, this is a file cataloging problem. I'm not sure if tags should be used to describe the link itself, instead place that description in the link's comment. It is still important to tag tags, i.e. meta-tagging. That is what cataloging is about: structuring data by relating them to each other - tags are no different; tags are metadata, but still data! There should also be a way to do implicit group tagging. With flat-level tags, each file must be explicitly (manually or automated) assigned and associated with all the tags that describe it. That results in unnecessary redundancy because the tags themselves have no relationships. Hierarchical meta-tagging allows for defining relationships between tags. It also allows files to be implicitly tagged with a metatag if the file is already tagged with a subcategory/tag of the metatag. In this case a metatag can also be used to tag a file directly if the user does not want to specify a more specific tag to describe the file. In short, the tagging concepts used by Videophile II is probably how I would implement the use of tags. Quote:
Quote:
Link-source should be a separate link metadata attribute, just like comment. A forum thread can be a link source. If they want to search for links from a particular forum then the appropriate query should be used on the link-source string. E.g., Search for link-source:*jdownloader.org* You can even have special tags that aren't manually tagged to a link but instead return links that correspond to a user-specified search query that is associated with that tag name. So the tag is really a View, but the resulting links are treated as thought they were tagged with that tag. Quote:
Each clickable tab/node in the muti-level hierarchical tabs represents a dataset of links; nested tabs are subsets of that dataset. The panel/area below the tab-rows displays the links in the corresponding dataset of the tab currently selected.
|
|
#39
17.07.2010, 01:59
|
|||
|
|||
|
Hey Bill,
You show a great understanding of my program, Vidophile II. If you ever want to help out with testing and so on, join my forum to get involved... Cheers, VPII
|
|
#40
17.07.2010, 03:20
|
|||
|
|||
|
Quote:
However, Videophile II could have great potential as a general file cataloger. I read about its tagging concepts on the forum; the concepts are exactly what I am looking for to tag media files (photo, audio, video). I don't know of any application that uses tag categories (meta-tags) and tag links the way Videophile II does to setup non-trivial relationships between tags and then apply it to general files. If a generalized version of Videophile II could be tightly integrated into Windows Explorer (via context menu) then I could browse files and use the usual functionality (and usage model) of Explorer with Videophile II acting as an intelligent cataloging database/library in the background helping me to easily catalog and relate files to each other without having to switch between applications/interfaces.
|
 |
|
|
Linear Mode
