JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 05.08.2017, 17:31
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Lightbulb Connection Manager Improvements

Hi, due to the annoying limitations imposed by MEGA, since some weeks ago I'd been experiencing with proxies' use and when it comes to a plenty of them, I found out it is kind of cumbersome to deal with them. So, I thought of some little improvements that you might find useful to implement, and thus making JD2's proxies management more practical and user friendly.

1- Letting JD2 grabs the proxies directly from the clipboard (via a "paste from clipboard" button), and from a text file, through drag and drop use.

2- As it currently does it with download links detection/parsing, make it detects what kind of proxies it is importing, among their ports as well as their login data, if present. Usually, proxies lists have a "IP:PORT" format, but maybe JD2 could even grab them by doing a lines level analysis.

3- Maybe it is already possible, but I didn't found a quick way of toggling on/off the check mark of several or even all (except the "No Proxy" default one) the available proxies; so currently I have to do it in a one by one basis, if needed.

4- Make the "Filter" feature works simultaneously for any amount of selected proxies, and not in a one by one basis.

5- Add some "Delete All" or "Reset" proxies button, which blanks out all the currently listed proxies, excluding the default "No Proxy" one.

6- If for some reason proxies' detection couldn't be automatically performed, let the "Proxy type" function works for any amount of selected proxies simultaneously, so the pertinent ones can be set manually, and thus avoid having to give them a specific text format so they get exactly recognized.

7- Make JD2 gets a ping/response time value out of proxies.

8- Let the proxies' columns actually arrange them.

9- This one specifically about MEGA, but I think it still could be useful for any other hosting services which impose downloading quotas/limits: avoid waiting times acceptance. Whenever a kind of "Bandwidth Limit Reached" message shows up for a used proxy, don't obey the given waiting time but keep trying to start or resume the downloads with another available proxies instead.

10- Considering there could be some working but slow proxies in a list of many ones, it would be great if JD2 manages to start doing a short speed monitoring right after a download have been initiated, so it can just discard them and try another ones instead. So, if a proxy doesn't meet a previously set speed rate during a certain amount of time, it should be rejected.

Hopefully, at least some of the above requests make sense how to be implemented somehow.
Reply With Quote
  #2  
Old 06.08.2017, 04:38
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

1 and 2 allows for mass imports and parsing, though they have to be in the correct format
settings > connection manager > import
so protocol://user:pass@hostORip:port where user:pass@ is optional.
or the up arrow has 'load proxy setup', this is either saved jd proxy list or I believe it does support pac though not mentioned.

3, should be highlight objects by keyboard or mouse and spacebar though that doesn't work.... I've mentioned this long ago mainly for account manager or account usage rules

4, grouping. You could effectively use packages and the packages contain the rules. I've also mentioned that in the past.

5, realise if you're going to delete all, you just highlight and click delete? takes like one second

6, I have started to make more in-depth proxy module, which does test proxies to see if they work and which region they are in. But I still think you should provide the type(protocol) when adding them to reduce the time to add and test.

7) sending ICMP packets I don't think is required, the test module I have in mind will keep track of latency via http request response times.

8) arrange? sort views and backend order like download/linkgrabber tables? Keep in mind that entry order is how the proxy selection works at this time

9) that's already done/possible within current frame work, its specific to the exception thrown in plugin. we throw IP_BLOCKED exception that triggers IP reconnection event or marks current connection as blocked. JD will try another connection if available. Keep in mind that some of the blocks for MEGA are based on the API key and not the IP session, so 'Reconnecting/Using another connection' wont bypass this restriction within application.

10) test module was going to do some sort of test, just not sure what and how. But I was also going to write a statastical module also, to keep track of download speeds of actual use. Think that would be more useful than a static test as real life speed would be a indicator of how good it is. As test happens once, and stats update each time you use it.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 09.08.2017, 01:35
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default

1&2- Yes, I know one can import a bunch of proxies into JD2's connection manager that way, and that's the way I'm currently doing it; but I wish it could be way easier and practical than it is now, kind of in the fashions I mentioned before. As the great links' detection/parsing feature JD2 already has, which saves a lot of time and nuisances when it comes to get desired download links, it would be really useful if JD2 could be the one who does the annoying job about proxies' importation and related tasks, without previous special formatting requirements.

For example, right now to import "socks 5" type proxies which originally are listed as simple "IP:PORT" format in a line by line basis, I first have to create a text file out of them; then using a "search & replace" text editor's function, to adapt them to the JD2's required format, and finally copy and paste them into JD2's importation window. When it would be quite easier and quicker if JD2 could simply allow to paste them directly (via a paste button or a text file drag/drop), and then being JD2 itself who do the pertinent tasks, as what kind of proxies they are and the like. But even in the case JD2 wouldn't be able to detect the exact proxies' type, if it still allows importing the proxies in the way I told before, the only one would have to do afterwards is selecting the exact type of proxies they are manually.

3- Yes I tried to use the "space bar" keyboard button, but as you said it doesn't work. I bet adding that specific missing function shouldn't be that difficult, and selecting/unselecting the check boxes to mark active or inactive proxies would be really useful (mostly when dealing with a large list of them).

4- I know the "packagizer" feature, but you say it will let me, let's say, select a bunch of proxies from the already imported ones and make them work by a whitelist/blacklist basis? As far as I can see, the proxies' filtering works in a one by one basis.

5- I could even do a "Ctrl+A" and "Delete" combination, but having a simple button who does it at once, would be better. Moreover, if that button's action omits to delete the default "No Proxy" one.

6- If you think it is better to do it that way, what about importing it like I mentioned in point 1&2, and then manually applying the right proxy type after selecting them? That way, using a text editor wouldn't be necessary anymore.

7- That's OK.

8- Yes, I wanted to say "sorting" them. "Priority" column couldn't be sorted for obvious reasons, not at least without applying a numbering on them, but what about could have the proxies sorted by at least some of the other columns, like "Proxy Type" or "Host/IP/Address" ones?

9- As to MEGA, having the "Direct" connection in first place, and a list of proxies below, I see JD2 tries to use one of the available proxies whenever my own IP reached the downloading quota limit. But that behavior is kind of erratic sometimes, in a way JD2 will still obey the imposed waiting time if some of the proxies it tries has reached said limit as well, trying to start another download in the list, instead retrying with the same one.

The waiting time for the proxies facing a "Bandwidth Limit Exceeded" message is actually 50'; and about 5 hours for my own IP. Once I use the "Start ALL downloads" button, JD2 will still try to use the "No Proxy" connection before trying each one of the proxies, but it would be better if it just omit since it makes non sense to keep trying with the direct connection, not at least the time limit has ended.

10- Once downloads using MEGA server starts, they use the "Direct" connection first; then, once my IP reached the limit, I usually have a list of proxies which might work or not, and with different speeds between them. But let's say I have a list of 5 proxies, even if all of them work, there could be at least one of them which is real slow, but since it had a higher rank in the priority list, that will be the one that keeps being used, when there could be faster ones between the remaining four. So, letting JD2 knowing how to deal with said "slow" proxies, it should discard them and keep trying with another ones instead, if needed.
Reply With Quote
  #4  
Old 09.08.2017, 07:09
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

1&2)
its simple to add the prefixes for example in notepad++ with regex, or macro, or type it with mass edit (alt + left click select first and last locations which will be beginning of the line and type socks5://). It will take you all of second.

3) hopefully the default behaviour will change.

4) not the packagizer a package. big difference

5) I really don't see the need, just lead to errors or more dialogs are you sure! as its dangerous action without fail safe. at least selecting first is a little prevention.

6) its not about best, its about knowing where proxies are located. I need this information for functions I've been trying to develop, for example automatic bypass geoblocking/region locking content (should be illegal as an anti-completive practice). it wont have a part 1 or 2 component at this stage.

8) order within list is the preference, so changing backend order (like download/linkgrabber tabs, ctrl + click) will perminately change the order. It would also have issue with 'default proxy/no proxy'.

9) proxy use is based on order of entry of list, then are there IP block exception?, then availability (are there slots left) maxsimldl plugin setting?
since mega is unlimited max sim dl for free, it will try the first proxy until ipblock. if you want to load balance say 1 connection on each of the proxies in blanced mode you need to change an advanced setting. GeneralSettings.freeproxybalancemode. Keep in mind you will probably suffer more exceptions with this, as proxies are generally shared resource. You don't know what others are doing, or have been doing(historically). so you could spend lots of time trying to connect to other proxies when you know one of the others that you're is working and could work again... if that makes sense? this is a trade off.

10) speed stats are all and good, just need to find a way to make them useful as a preference.. it wont work within the current frame work, as you would need to alter position in connection manager.


raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #5  
Old 11.08.2017, 01:16
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default

1- I know it isn't a really big deal and it will took me just a few seconds to do it manually, but the question here is why? When JD2 could actually do it by me in a blink of an eye.

I still think having the "pasting" as well as "drag & drop" features would be great, if possible.

And further features like proxies' parsing and analysis would be great to have also, but I bet those would mean a considerable amount of work, so it is understandable if they couldn't be implemented.

4- Would be great to see JD2 letting do that kind of filtering tasks for a nth amount of proxies, instead one by one.

5- I don't see why it has to be problematic, since one should know what he/she is doing before using that feature; adding it like an optional/advanced feature should be enough to avoid scatterbrain people use it.

I consider it would be "useful" since it will mean less steps to accomplish a simple task. Something that clears all but the "No Proxy", with just a click an no selections at all.

6- I think being able to import the proxies using just the IP:PORT (even if initially JD2 assign them an incorrect type) format and then manually select the right type for them (actually possible but in a single proxy basis) will be quicker, but if that interferes with what you said before, it's OK.

9- I'm already using the balanced proxy mode, and as you said, since they are mostly public proxies sometimes they work, sometimes they don't; but I was wondering why JD2 wouldn't just skip them right away, once it gets that bandwidth relate message. And that applies for my own IP waiting time as well, in the sense of not keep retrying/waiting time in vain.

:thumbup:
Reply With Quote
  #6  
Old 11.08.2017, 02:46
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

1&2
is no different than any other url into jd you must have protocol prefix!

4. agree.

6 my idea for 6 has nothing todo with yours' yours is more so re:1&2
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #7  
Old 13.08.2017, 20:09
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default

OK, thanks for your replies.

Hopefully some of the improvements be useful enough how to be implemented in the short/medium term.
Reply With Quote
  #8  
Old 14.08.2017, 06:51
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

yes it will be taken aboard for sure

thankyou
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 20.12.2017, 03:47
ElCho ElCho is offline
Tornado
 
Join Date: May 2014
Posts: 245
Default

It would be great if you could please implement the missing feature from my #3 request, since I'm dealing with a few hundreds (manually added) proxies which work intermittently, and each time I want to to disable/enable all of them, I have to delete them instead, thus avoid doing it a a one-by-one basis; and later add them once again, when I need them.

Thanks.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 06:33.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.