JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #941  
Old 29.03.2015, 16:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

@Llamatron
1. The behavior you posted with these settings unchecked is exactly correct:
JD tries via account --> Limit --> Next account --> Limit --> next account

With the settings enabled, JD compares the timestamps on the account (or also on the plugin itself) before asking for captchas.

From what you write it looks as if it is all working as desired
EDIT
By the way, the additional one minute waittime was intended...if you know I can change it to 10 seconds or so but I don't know how exact the uploaded system counts

GreeZ psp
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 29.03.2015 at 16:57.
Reply With Quote
  #942  
Old 29.03.2015, 19:18
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
@Llamatron
1. The behavior you posted with these settings unchecked is exactly correct:
JD tries via account --> Limit --> Next account --> Limit --> next account

With the settings enabled, JD compares the timestamps on the account (or also on the plugin itself) before asking for captchas.

From what you write it looks as if it is all working as desired
Yes, I think so. With the settings unchecked, the timeout on one of the accounts (the first one I added a while ago, not from this recent testing) usually is set to 5 minutes after a failed attempt. This is what makes JD2 horrible to use (captcha spam) with ul.to free accounts out of the box. (the waittime options default to off. Changing them to on-by-default would help.)

One thing that isn't working as desired is trying to use multiple accounts in parallel, like the datafile.com plugin does. The limits for this need to be explored more, but the worst that happens if you do too much in parallel is that downloads from your IP get blocked for something like 3 hours (possibly from the end of one of the downloads, rather than the usual start.) I changed my log settings to keep 60 days of old logs, but I need to do the parallel DLs from inside JD2 for it to all get logged in the same place. I'd rather just have logs than have to manually make notes of when every DL started.

Also, with multiple connections enabled, ul.to won't start a download on my proxied-to-another-IP connection unless I uncheck "direct" temporarily, so it sees the proxy as the only available connection, and then realizes it's not downloading on it. (even though there's still a "direct" download running.)

datafile and rapidgator were leaving the extra connection underutilized, too. k2s seemed to be starting new DLs on it as soon as possible, though.

Quote:
EDIT
By the way, the additional one minute waittime was intended...if you know I can change it to 10 seconds or so but I don't know how exact the uploaded system counts

GreeZ psp
I think 0 seconds is enough. Even negative 30 seconds would work, because I think the 180min limit is checked at the end of the 30-sec wait after bringing up the captcha.

I'd recommend 0 seconds, though, because being a few seconds early is WAY worse than a few seconds late. The captcha wait-time alone is enough to make sure the next download starts after the 180min limit, with 30 seconds of safety margin.
Reply With Quote
  #943  
Old 29.03.2015, 20:08
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

1. As said, the uploaded plugin is not designed to start multiple free downloads at all but with the new JD2 controlling every account has its own download slots (jd intern) so that makes it possible - which is fine!
For the uploaded unregistered free mode, it is limited to max 1 download at the moment!

2. Okay, I removed the 60 extra seconds...waiting for your feedback then
And the idea that they probably check the waittime after the captcha is good!

GreeZ psp
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #944  
Old 29.03.2015, 22:55
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
1. As said, the uploaded plugin is not designed to start multiple free downloads at all but with the new JD2 controlling every account has its own download slots (jd intern) so that makes it possible - which is fine!
Ah, I see, so JD2 kind of runs multiple instances of the plugin, one for each account.

That's not what JD2 is doing for me, in practice, though. I have waittime options #1 and #2 checked. I've never seen JD2 even try a second download on the same IP when one is already running. (with #1 and #2 unchecked, too.) And to get it to start one on the proxy connection, I have to disable my "direct" connection, so the proxy i the only connection left.

I haven't restarted JD2 for over 24 hours, so maybe an on-the-fly plugin update is behaving badly, and will work right after a restart. (Or maybe I confused something when I added a connection to a simple proxy that didn't support HTTPS, without first disabling HTTPS in the ul.to and k2s.cc plugins. I also tried adding a "direct gateway" and a "socks" connection, but neither of those worked. I'll post a new thread about that. But after messing around with a bunch of stuff, I went with tinyproxy on the non-default gateway on my network.)

Hmm, I guess I could run multiple proxies to trick JD2 into doing more parallel DLs, but much better to just get it working the right way.

Quote:
2. Okay, I removed the 60 extra seconds...waiting for your feedback then
And the idea that they probably check the waittime after the captcha is good!
I'm pretty sure they don't check the waittime until after the 30-sec countdown, so even if you solve the captcha really fast, you still have that margin. I think that's what i remember figuring out once, after setting a timer for myself before I was using JD2.
Reply With Quote
  #945  
Old 30.03.2015, 01:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

As said, inside the uploaded.net plugin free downloads are limited to 1.
Same way it was with datafile but I did change that.
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #946  
Old 30.03.2015, 02:25
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
As said, inside the uploaded.net plugin free downloads are limited to 1.
Same way it was with datafile but I did change that.
Did you see my post earlier about ul.to allowing parallel downloads from the same IP, on different accounts?

I tried to start a new thread for this topic, but the forum auto-moved my post into this thread: https://board.jdownloader.org/showth...745#post328745

From a single IP, it's possible to actually have 3 or 4 downloads running at once, not just skip wait times between downloads, by using multiple free accounts. There's still some kind of limit you can run into, which I guess is per-IP, but there are still huge throughput gains easily available.


Also, I just restarted JD2. Even with a clean restart, it won't start a download from my proxy while one is running from direct connection. So is it just that plugin limit being set to 1 that's preventing separate downloads on separate connections, too? Because even without this newly-discovered ability to download from ul.to in parallel on a single IP, that was limiting JD2's multi-connection capability.

Last edited by Llamatron; 30.03.2015 at 02:31.
Reply With Quote
  #947  
Old 30.03.2015, 03:01
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

Hi
1. As said, unregistered mode is hardcoded limited to 1 - so are free accounts.
2. Because I never use proxies I can't really tell you anything about JDs behavior with proxies but I guess every connection has its own limits at least that would make sense and no, plugins don't interfere with that!

Then again, how did you start simultan downloads?
Is it also possible without using multiple free accounts (as unregistered user)?
How long is the minimum waittime between downloadstarts?

GreeZ psp
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #948  
Old 30.03.2015, 03:54
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
Hi
1. As said, unregistered mode is hardcoded limited to 1 - so are free accounts.
This is why I'm telling you it shouldn't be... The current wait-time behaviour is perfect for each account separately, though.

Quote:
2. Because I never use proxies I can't really tell you anything about JDs behavior with proxies but I guess every connection has its own limits at least that would make sense and no, plugins don't interfere with that!
Ok, well something is putting an unnecessary limit on starting dowloads through different connections, because JD2 doesn't even try for uploaded.net. I think k2s does work, and I have gotten rapidgator to start parallel DLs after some prodding (enable / disable one connection or the other.)

If you can connect multiple computers to your cable modem or DSL and your ISP gives them each a different global IP address, then you can run tinyproxy on one, and point JD2 to it. (Only works if you get two real IPs, not _http://en.wikipedia.org/wiki/Private_network#Private_IPv4_address_spaces 10.*, 192.168.*, or 172.16.0.0/12 behind a NAT firewall like some ISPs hand out. Luckily mine hands out real IPs for up to a few connected machines that aren't behind a NAT firewall, aka router.)

Quote:
Then again, how did you start simultan downloads?
Is it also possible without using multiple free accounts (as unregistered user)?
Unregistered no. Free account yes. (Actually, you can do unregistered at the same time as a free account, but you need at least one free account to get any parallelism. And unregistered has a gimped bandwidth, so it's best to just let users add enough free accounts. With a free email provider that lets you add aliases that all come to the same inbox, like gmx.com, it's pretty easy to make several accounts.)

Quote:
How long is the minimum waittime between downloadstarts?
zero seconds, AFAIK. Like I said in https://board.jdownloader.org/showthread.php?p=328745, every account appears to have its own 3h timer.

I tested with firefox's "new private window", so I could log in on one ul.to account while making another. relogging to change accounts after starting a download works, too.

Or just browser-login with a different account than the one JD2 is using.
Reply With Quote
  #949  
Old 30.03.2015, 03:58
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

Hm some time ago this was different - I'm sure!
...but anyways - if you are right the current handling should be fine for multiple accounts and there should be no issues!

About multiple connections:
Jiaz is the man for that - he can answer all your questions about this

GreeZ psp
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #950  
Old 30.03.2015, 04:56
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
Hm some time ago this was different - I'm sure!
...but anyways - if you are right the current handling should be fine for multiple accounts and there should be no issues!
You mean after you lift the hardcoded one download at a time across all free accounts limit it currently has, right?

But then yes, I think the current wait-time handling should work well. If it already tracks wait-time per account, then it should be able to avoid wasted captchas by not trying accounts that are still on cooldown.
Reply With Quote
  #951  
Old 30.03.2015, 05:02
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

No why lift it?
...or were you able to start multiple free downloads with ONE account?
At the moment:
Unregistered: 1
Free account: 1 (per account)
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #952  
Old 30.03.2015, 05:28
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
...or were you able to start multiple free downloads with ONE account?
no

Quote:
At the moment:
Unregistered: 1
Free account: 1 (per account)
Well JD2 is only ever doing one download globally for me. Not per-account or per-IP, just one, ever. It can start another one right away after that finishes, but won't start a new one while one is still running.

So maybe I guessed wrong about the cause being the plugin's hardcoded limit applying across accounts, instead of per account. But there's a problem somewhere. Logs for this will be in with the datafile.com logs I PM you, if I can figure out how to add an attachment to a PM.

I have
max chunks = 1
max simultaneous DLs = 20
max per hoster = 5

Last edited by Llamatron; 30.03.2015 at 05:32.
Reply With Quote
  #953  
Old 30.03.2015, 16:48
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

Hm didnt it work fine for other plugins with 1 per account?
I m not sure about that either!
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #954  
Old 30.03.2015, 17:09
Llamatron
Guest
 
Posts: n/a
Default

Let's get the ul.to plugin working for parallel downloads using multiple accounts on one IP. Then worry about using multiple connections / triggering reconnects when hitting a per-IP instead of per-account limit.
Reply With Quote
  #955  
Old 30.03.2015, 20:10
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

Enabled simultan usage of uploaded accounts - try again after the next update!
For me it worked fine.
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #956  
Old 31.03.2015, 17:44
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
Enabled simultan usage of uploaded accounts - try again after the next update!
For me it worked fine.
Working perfectly so far. All 5 accounts queued up a download. They all finished before 3 hours, and all 5 accounts queued up a second download at 3 hours.

They all used my direct connection, not via the proxy, which is the desired behaviour. I'll have to see what happens if I run into a per-IP cap. Hopefully attempts that run into unexpected waittime on the primary connection will fall back to trying other connections, in case it's an IP block rather than the account being used outside of JD2.

Setting the "account usage rules" to bump unregistered up into the same group as my free accounts, JD2 popped up a download start on each of my 2 connections (direct and proxy). The "direct" one disabled itself right away, without even a captcha. (I'm not sure if that's correct. I thought ul.to gave unregistered it's own 3hour timer, like it was an account.)

Thanks for taking the time to get this plugin working really nicely, based on my testing of hoster behaviour.

Oh, another thing I could do for testing: use JD2's "pause" button to make downloads run so slowly that the 3h timer would be up while a download is still running. There aren't per-hoster throttles, though. I don't have QoS throttling / priority set up on my router, either, so it'd be an effort to set that up. :/

Per-hoster rate limits is probably only useful for testing, so not maybe worth implementing in JD2. Although if you were confident your connection was stable, you could use it to get a crapton of datafile.com downloads running, starting one per hour, by preventing the earlier starts from finishing and triggering the block.

Last edited by Llamatron; 31.03.2015 at 17:51.
Reply With Quote
  #957  
Old 31.03.2015, 18:00
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

Per Hoster or individual link speedlimit will be available in the future - but not exactly the near future!
Thanks for your feedback!
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #958  
Old 31.03.2015, 21:24
Llamatron
Guest
 
Posts: n/a
Default

Ok, so after DLing 5 files in parallel, then another 5 after 3 hours, the attempt to start another set failed.

Browser testing with one of the 5 free accounts confirms that it's blocked on the IP JD2 used. However, the same account is fine to download on another IP, but JD2 didn't try.

31.03.15 12.06.59to31.03.15 17.45.39 jdlog://5652027486241/

Look at around 3:08 to 3:10pm in the log for the attempt to connect.

The 5 accounts each put link a link into captcha state, IIRC, but after solving them, the DLs didn't start. (since that IP was blocked.) The account manager showed all 5 free accounts temporarily disabled for 5 minutes, with an unkown error.

Then after 5 minutes, without another ul.to connection attempt, the accounts went to "ok", and reconnect-waittime appeared in the ETA column for 3 of the 5 links. Gateway = direct for all 5, and Download Mode = different account for each of the 5 next ul.to links in my list. The reconnect-waittime is counting down to 3 hours from when the captchas came up and they got temp-disabled. However, the waittime in the ETA column is EXACTLY the same for all 3 accounts (and for unregistered via the proxy, which makes no sense for that to be linked). Each account should have a separate waittime. I assume this is just the GUI failing to display different waittimes from the same hoster well.

Another "unregistered" download was able to start, but it hasn't shown up in the log file. I think the logging code must do some buffering so lines from different threads don't get mixed together, but the filename and the url from the currently running ul.to download don't appear in uploaded.to.log.0 at all.

IDK if any relevant stuff about errors with the 5 accounts is missing from the logs, too.


Oh, important point: browser testing showed that while the account I tested was blocked from the IP JD2 used it on, the other IP I have access to was able to download a file with that account.

So when the ul.to plugin hits an error trying to start an account it expects should be able to start, it should do whatever is needed to get JD2 to try the same account on another connection, before giving up on the account for 3 hours.
Reply With Quote
  #959  
Old 31.03.2015, 22:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,703
Default

Reconnect waittimes sit on the hoster and also each account has only one timestamp of the last download which is not depending on the connection.
What you re trying to do here is simply crossing the boundaries of what is possible with JD.
JDownloader is made to improve free- and premium mode of one click hosters and direct links but the cases you re doing here are quite abnormal.

Also, whenever a reconnect wait is shown, it sits on the host and all links of a host (excluding the ones which are still being downloaded at that time) get the wait.

The "Unknown error" in the account is just a bug instead of displaying the real message but as you correctly discovered, after 5 minutes the accounts will get an ok state again and new download attempts can be performed - if the waittime still exists the account will got into error state again.
If you want I can also set the timeout to the total waittime so in such a case accounts would be temporarily disabled (yelloe color) for 3 hours instead of 5 minutes.

I agree that JD should try the other connections but this is a thing you have to ask jiaz.
Its also no uploaded issue but more (if there is an issue...) a controlling design issue/bug.

GreeZ psp
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #960  
Old 31.03.2015, 23:09
Llamatron
Guest
 
Posts: n/a
Default

Quote:
Originally Posted by pspzockerscene View Post
I agree that JD should try the other connections but this is a thing you have to ask jiaz.
Its also no uploaded issue but more (if there is an issue...) a controlling design issue/bug.
Ok, thanks for the quick explanation on JD2's design for tracking wait-times.

So it's sort of a design limitation, preventing trying other connections, I guess.

It's a limitation that only affects ul.to and any other rare hosters with a limit model that mostly tracks timers per account, not per client IP. So it affects ul.to, but getting the ideal behaviour would require core changes, not just this plugin, right?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 22:48.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.