JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 03.06.2009, 21:02
shawn8888
Guest
 
Posts: n/a
Default Proxy rotate support

Instead of reconnect/redial, maybe JD can use a proxy pool to walk around the hosters' same IP restriction.

The idea is, you can put multiple working proxies in JD. Then JD can switch proxy for each download job, so each job can get a new IP.

Some download tools has this feature and I think it is a very good alternative when reconnect is not possible.

Last edited by shawn8888; 03.06.2009 at 21:06.
Reply With Quote
  #2  
Old 03.06.2009, 21:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

As far as i know this was already declined but let's see what the devs say, just wait for them to post!
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 03.06.2009, 21:52
shawn8888
Guest
 
Posts: n/a
Default

It was declined before? why?
JD has already supported proxy. All you guys need to do is to add the rotate feature, right?
Reply With Quote
  #4  
Old 03.06.2009, 22:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

we are already working on that , but at the moment we have no plans for unlimited length of list....for example we think that 5 proxies in rotation are enough

because large list to rotate would destroy the hosts AND the free proxies
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 04.06.2009, 02:22
shawn8888
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by Jiaz View Post
we are already working on that , but at the moment we have no plans for unlimited length of list....for example we think that 5 proxies in rotation are enough

because large list to rotate would destroy the hosts AND the free proxies
For me, 2 or 3 are good enough not to use the reconnect function.
Thanks a lot for putting it on your todo list!
Reply With Quote
  #6  
Old 04.06.2009, 15:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Changed Prefix to in progress but that doesn't mean that this will be added the next days/weeks, i just marked it because Jiaz wrote he'll implement it!
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 05.06.2009, 20:11
wolframio
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by shawn8888 View Post
For me, 2 or 3 are good enough not to use the reconnect function.
Thanks a lot for putting it on your todo list!
agree, my router/cable modem combination its not supported by jdownloader, also that can be speed up the download, even if you can do 2 proxie conection for no overload the free servers!

Quote:
Originally Posted by pspzockerscene View Post
Changed Prefix to in progress but that doesn't mean that this will be added the next days/weeks, i just marked it because Jiaz wrote he'll implement it!
take your time (but any work on this feature its very nice notice for all)
Reply With Quote
  #8  
Old 06.06.2009, 07:34
shawn8888
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by Jiaz View Post
we are already working on that , but at the moment we have no plans for unlimited length of list....for example we think that 5 proxies in rotation are enough

because large list to rotate would destroy the hosts AND the free proxies
I think there are two ways to use multiple proxies:

1. Using them simultaneously. For example, if you have 3 working proxies, you can download 3 RS links at the same time and it certainly can speed up downloading.
If JD is even more smart, you can put 9 proxies in the pool and ask JD to open 3 downloads at the same time, and rotate the 9 proxies to avoid the waiting time.

2. Only use proxy to replace the reconnect function. When one download is finished, instead of reconnect, use a different proxy to continue download.

What I wanted is the the latter option, which I think is easier and has less impact to the hosters.

What do you think?
Reply With Quote
  #9  
Old 21.07.2009, 23:44
shawn8888
Guest
 
Posts: n/a
Default

Any updates on this?
Reply With Quote
  #10  
Old 22.07.2009, 14:20
remi
Guest
 
Posts: n/a
Cool

If you can't wait for this rotating proxy list, you can use the new FreeRapid downloader.

Be aware that besides the proxy treatment and a higher usability level it has less features than jD.
Reply With Quote
  #11  
Old 06.08.2009, 19:35
shawn8888
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by remi View Post
If you can't wait for this rotating proxy list, you can use the new FreeRapid downloader.

Be aware that besides the proxy treatment and a higher usability level it has less features than jD.
I tried FreeRapid and it does support a proxy list and it works fine.
But, it doesn't support socks5 proxy, which is a pity.

I wish to see this new feature in JD soon and until then I have to use FreeRapid instead.
Reply With Quote
  #12  
Old 07.08.2009, 00:50
wolframio
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by shawn8888 View Post
It was declined before? why?
JD has already supported proxy. All you guys need to do is to add the rotate feature, right?
maybe that its best and short way to do this, a feature to select if you want to dowload a package with your real ip or use the proxy that you put on the options (in the package list and link grabber)... and the automatic way, if you are downloading 2 or more files from the same server, dowload one using the real ip and the same time download another with proxy, just 1 proxy must used but using 2 ips (the real one and the proxy one)
Reply With Quote
  #13  
Old 07.08.2009, 11:13
remi
Guest
 
Posts: n/a
Default The earlier you commit a design fault, the costlier it gets to correct it afterwards

I get the impression that this feature is difficult to implement in jD, because of architectural reasons.

This important feature should have been taken into account at the start of the design but it was overlooked. You can deduce that from the fact that you need to restart jD when you change the proxy address.
Reply With Quote
  #14  
Old 07.08.2009, 13:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

im sorry but you are wrong
this feature is possible and on our list for long time.....the reaseon why you have to reastart is, because you set a global proxy by using other functions for that....(developer thingy)......
when we have time we will add this feature....
__________________
JD-Dev & Server-Admin
Reply With Quote
  #15  
Old 07.08.2009, 15:00
remi
Guest
 
Posts: n/a
Cool

In the beginning of March this year two threads were started independently from each other.

started by gokaycagri on 02.03.2009
http://board.jdownloader.org/archive....php/t-56.html

started by proximity on 04.03.2009
http://board.jdownloader.org/showthread.php?t=129

I think there is sufficient evidence that it is difficult or needs a lot of time, otherwise it had been implemented in the previous 5 months.
Reply With Quote
  #16  
Old 07.08.2009, 15:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

it does not need alot of time and is not difficult...just no time to do it, because more important stuff on todo list
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 07.08.2009, 16:13
remi
Guest
 
Posts: n/a
Cool

I agree concerning all the bugs caused by the redesign, but I don't agree on the Human Machine Interface changes.

jD version 0.4. had an excellent GUI, but according to standard usability criteria, it got a mediocre GUI in v.0.5.. A lot of time has been lost there.

Please, don't interprete my words as accusations. I wouldn't spend so much time arguing if I wouldn't care about your brilliant child. Everybody makes errors - we are all humans.
Reply With Quote
  #18  
Old 10.08.2009, 19:24
shawn8888
Guest
 
Posts: n/a
Default

I have to admit that I begin to fall in love with FreeRapid just because of this proxylist feature.
I agree that JD is still better than FR, but overall they're all downloading tools.

I have a bunch of small files on RS to download and using FR saved me 2 or 3 times of time comparing JD.

So, again, this is a very important feature. please make it high priority.
Reply With Quote
  #19  
Old 17.08.2009, 00:24
banished
Guest
 
Posts: n/a
Default

I hope you implement support for an arbitrary amount of proxies - I for example currently use a couple of machines from my university as SOCKS proxy using ssh - they each have an individual public IPv4 ip, so parallel downloading of rapidshare links works just fine using a little script I wrote and FoxyProxy - it's however pretty time consuming to deal with every discrete tab and download link, still it causes problems when encountering encrypted link files.
So don't limit the maximum amount of proxies, there are perfectly legal ways to get hands on several dozens of proxy computers ;-)

Code:
#!/bin/bash

function createProxies {
	for proxy in ${plist[@]};
	do
# check weather the host is alive (otherwise we would have to wait ages for ssh timeout) and also check weather ssh is working at all for this host before, to not create gaps - uname is likely never to fail, so if this fails, we know it's ssh
  		if ping $proxy.$domain -c 1 -w 1 > /dev/null && ssh $USR@$proxy.$domain uname > /dev/null; then
                	echo Creating SOCKS proxy using $proxy on port $port
	                ssh -D $port -N $USR@$proxy.$domain &
        	        let port=$port+1
	        else
        	    	echo "Can't connect to $proxy"
	        fi
	done
}

port=8000 	# Port to begin with, counts up

USR=ash
plist=(Bulbasaur Ivysaur Venusaur Charmander Charmeleon Charizard Squirtle Wartortle Blastoise Caterpie Metapod Butterfree)
domain=park.kanto.org

createProxies

USR=foobar
plist=(moskau berlin xian peking london wuhan shanghai paris chengdu roma leningrad jekatarinburg prague)
domain=somewhere.else.edu
createProxies
You should use Priv/Public key authentication (otherwise this would be a major pain in the ass).
Reply With Quote
  #20  
Old 17.08.2009, 11:15
remi
Guest
 
Posts: n/a
Cool

Quote:
Originally Posted by Jiaz View Post
it does not need alot of time and is not difficult...just no time to do it, because more important stuff on todo list
The rotating proxy list is not a high priority for the devs. This feature is probably one of the highest priorities for the customers for a long time.

The obvious question is why devs' and customers' priotities are not aligned...

@banished : Thanks for the script but I don't know how to use it. Would you be able to provide an ad interim solution with your script?
Reply With Quote
  #21  
Old 17.08.2009, 11:29
wolframio
Guest
 
Posts: n/a
Default

ok, i think like a lot users of jdowloader... but in this momment they are working to redo the redo it interface of jdownloader, testing, retesting adding and removing things, fixing some hoster plugins and a lot of things, i am using nightly 7220 and have to admit it... i want proxy support, but... the changes that the jd-team are doing are good for all kind of users...
Reply With Quote
  #22  
Old 18.08.2009, 00:24
banished
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by remi View Post
@banished : Thanks for the script but I don't know how to use it. Would you be able to provide an ad interim solution with your script?
Well, not everybody can use this method, you of cause have to have a shell account on the machines you want to use as a proxy.
USR is your account on those machines
plist is a list of machines you have access to with said user name in
domain. This is split, cause the domain is usually the same for all the computers in the pool. Say you have the account remi that has acces to all the machines of pool.youuniversity.edu which are foo.pool.youuniversity.edu, bar.pool.youuniversity.edu and baz.pool.youruniversity.edu, so you get USR=remi domain=pool.youuniversity.edu plist=(foo bar baz). (so the script will ssh remi@ foo.pool.youuniversity.edu, etc..)
for public/private key authentication you have to add your ~./ssh/id_rsa.pub to ~/.ssh/authorized_keys on the machines you want to connect to.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 01:16.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.