#81
|
||||
|
||||
Can't test this way as I cannot reset my connection. So I have LONG wait times. But when I have about 10 tries in JDownloader with 15KB/s and all in Firefox with 75KB/s I'm pretty sure that's not coincidence ;-) And that now I got the first time 75 in JDownloader also not. But I know from other things I programmed, that some services use the header lines (most importantly user agent, but also others) to decide how to react to a request. E.g. Amazon presents you Captchas much much faster when you access it with an older Firefox version.
|
#82
|
||||
|
||||
Use a VPN though this can also falsify your results as filehosts like upstore often try to detect VPN IPs.
Quote:
Still ~75KiB/s in JD. Here is what you could try additionally: 1. Start a download in JD and get the direct URL from your log/IDE console. 2. Now check if you can get better speeds when downloading the direct-URL generated vy JD in browser. --> If that is possible, you got unlimited tries with that direct-URL and your current IP. Of course you can also do this experiment the other way around and generate a directURL in browser and check what happens when using that in JD. Yap that's common practice.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#83
|
||||
|
||||
@sdi: I would not spend much more time in this but of course, you may spend as much time as you like there are many other services out there that do provide much better speeds. I don't think it has to do with headers or anything like that but something very simple like different download servers
__________________
JD-Dev & Server-Admin |
#84
|
||||
|
||||
It's more about the "Why". Because that again may help me with another project in the future ;-)
P.S. In case somebody knows: I'm still searching a working Javascript interpreter solution for either perl direct or the Linux shell command, so I can call specific functions of websites. Automating website access gets more and more complicated by JavaScript Mumbojumbo. |
#85
|
||||
|
||||
Following seems to do the trick for me. Not 100% tested, but I'm at least 90% sure.
I tested that when I get a 15Kb Link this also is 15KB in Firefox, so the User-Agent does a server choice, not a different transfer handling. I assume the additional "Ubuntu; " somehow disturbs the Firefox version detection. For my openSUSE Firefox there is nothing there in the agent string, but I see on my servers that the Ubuntu string is unchanged also with version 102 so I did not checkin this change yet as it may disturb others. HTML Code:
Index: src/jd/http/Request.java =================================================================== --- src/jd/http/Request.java (Revision 46473) +++ src/jd/http/Request.java (Arbeitskopie) @@ -481,11 +481,12 @@ } // [url]...[/url] // 08.05.2020 - current is 76.0 - // 30.11.2020 - current is 83 - final String firefoxRevision = "76.0"; + // 30.11.2020 - current is 83.0 + // 16.07.2022 - current is 102.0 + final String firefoxRevision = "102.0"; if (archString != null && osString != null && !new Regex(archString, "(arm|aarch)").matches()) { // do not return ARM based strings as it will revert to mobile websites. We do not want that behaviour by default -raztoki - return "Mozilla/5.0 (X11; Ubuntu; " + osString.trim() + " " + archString.trim() + "; rv:" + firefoxRevision + ") Gecko/20100101 Firefox/" + firefoxRevision; + return "Mozilla/5.0 (X11; " + osString.trim() + " " + archString.trim() + "; rv:" + firefoxRevision + ") Gecko/20100101 Firefox/" + firefoxRevision; } else { switch (CrossSystem.getOSFamily()) { case WINDOWS: @@ -507,9 +508,9 @@ } default: if (CrossSystem.is64BitArch()) { - return "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:" + firefoxRevision + ") Gecko/20100101 Firefox/" + firefoxRevision; + return "Mozilla/5.0 (X11; Linux x86_64; rv:" + firefoxRevision + ") Gecko/20100101 Firefox/" + firefoxRevision; } else { - return "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:" + firefoxRevision + ") Gecko/20100101 Firefox/" + firefoxRevision; + return "Mozilla/5.0 (X11; Linux i686; rv:" + firefoxRevision + ") Gecko/20100101 Firefox/" + firefoxRevision; } } } |
#86
|
||||
|
||||
@sdi: you may change the used UserAgent within the plugin but general version bump is currently not advised because of too many possible negative side effects.
__________________
JD-Dev & Server-Admin |
#87
|
||||
|
||||
Quote:
Quote:
__________________
JD-Dev & Server-Admin |
#88
|
||||
|
||||
I thought so, that a global change is probably not wise. Yeah I know that "Ubuntu; " is still there for Ubuntu systems. Do you get 75KB/s or 15KB/s when downloading in the firefox browser direct?
|
#89
|
||||
|
||||
Quote:
75 in JD and Chrome. (Current version of Chrome + U Block Origin)
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#90
|
||||
|
||||
Well, Chrome has another User-Agent, so that doesn't count ;-)
|
#91
|
||||
|
||||
Why not?
The main point is still the same: Speeds are the same in JD and browser, at least for me.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#92
|
||||
|
||||
Update:
I've tested Firefox anyways now: Firefox Portable (no adblocker) --> Also 75KB/s This time I've used a VPN because I got a static IP and reached the limit on that atm.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 21.07.2022 at 19:21. Reason: Fixed typo |
#93
|
||||
|
||||
Well, is there the additional "Ubuntu; " string in your JDownloader agent, i.e. do you download with a Linux system? The change actually moves the rv: string one field to front like for Windows, MacOS. If there is a more or less simple regexp on the serverside that can be enough to allow the version check to succeed.
|
#94
|
||||
|
||||
Firefox portable sounds much like Windows? Totally different User-Agents used there.
|
#95
|
||||
|
||||
Quote:
So your conclusion is which now? Which User-Agent is supposed to get you better speeds? The User-Agent of the Firefox I've tested with was: Code:
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#96
|
||||
|
||||
@jiaz: Hmm, untested yet (I have a working download I don't want to cut :-), but that should do the job without side effects?
HTML Code:
Index: src/jd/plugins/hoster/UpstoRe.java =================================================================== --- src/jd/plugins/hoster/UpstoRe.java (Revision 46489) +++ src/jd/plugins/hoster/UpstoRe.java (Arbeitskopie) @@ -141,6 +141,8 @@ if (!(browserPrepped.containsKey(prepBr) && browserPrepped.get(prepBr) == Boolean.TRUE)) { super.prepBrowser(prepBr, host); prepBr.setCookie("**External links are only visible to Support Staff**, "lang", "en"); + /* Must be relatively recent, shouldn't have the "Ubuntu; " part of Firefox or downloads onyl 15KB/s instead of 75Kb/s */ + prepBr.getHeaders().put("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:102.0) Gecko/20100101 Firefox/102.0"); } return prepBr; } |
#97
|
||||
|
||||
@sdi
I don't mean to be rude but I still don't get the results of your tests. From my understanding, I always get 75KB/s no matter which User-Agent I've used. Also independent from browser addons/adblocker usage and in JD and browser. Now according to your experiments there is one specific User-Agent that gets you better speeds. Just in case that's true: This would mean that upstorewould determine the download speed not only based on the User-Agent but also on other factors like IP(Country)/current time/other headers (?) EDIT Don't get me wrong of course we can try your User-Agent. If you want, I can add a plugin setting for upstore where you can define your desired User-Agent string this way you can update it in the future. I just still don't get the point because...it doesn't seem to have an influence on the speed here^^
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#98
|
||||
|
||||
Do you get 75KB/s with the JDownloader Linux agents of the form
"Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:86.0) Gecko/20100101 Firefox/86.0" or "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0" The tests under Windows don't really apply to this problem as agent on Windows in JDownloader is different. |
#99
|
||||
|
||||
Haven't tried yet.
Quote:
I'm getting the same speed in JD than in browser and vice-versa...
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#100
|
||||
|
||||
Well, that's reversed. There is a specific set of User-Agents, namely the ones with "Ubuntu; " inside which seem to cause worse results (i.e. they don't match upstore.net's expectations of agent strings.
|
#101
|
||||
|
||||
Regarding the last patch - I wanted to take the default string and only modify it, but it seems that's much more complicated than simply setting one string which works.
|
#102
|
||||
|
||||
@sdi: just set a custom tested/working ua in the plugin. no need to mess with the default UA on global level if not required
looks good, https://board.jdownloader.org/showpo...0&postcount=96
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 21.07.2022 at 20:07. |
#103
|
||||
|
||||
But that does not explain what the testfile has good speed in JDownloader for me with same UA as the slow speed file I still think the cause is different but if you have good speeds with different UA, it's okay to change it in plugin
__________________
JD-Dev & Server-Admin |
#104
|
||||
|
||||
I did one more test for you:
Used User-Agent: Code:
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0 addons.mozilla.org/de/firefox/addon/uaswitcher/ VPN used: Yes, Belgium Testlink: Same as for my other tests Cookies: Cleared before Adblocker: Yes, U Block Origin Result: Unable to download (Wait-time bug and "File not found" issue) I'm done with this. Takes too much time + captcha attempts. Just in case you come to the conclusion that changing the UA helps you, as said, I can add a plugin setting for you.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#105
|
||||
|
||||
Quote:
We'll see the effect when new core is released. For me it works now on multiple machines and different providers. Using the Windows string also on Linux shouldn't have any negative side effects so it does no harm. |
#106
|
||||
|
||||
Quote:
The "File not found" seems like an CloudFlare mechanism to me. E.G. I'm unable to access any files from Hetzner servers. They simply block IP ranges and return 404 code. So probably the VPN endpoint you used is blocked. Let's wait for feedback after next release... |
#107
|
||||
|
||||
P.S. Interesting side note - Essentially I found the timing issue because of the too-fast counter in Firefox. Because I needed to use an own clock I always did wait until next full 10 seconds before start as this is easier to remember. That's why it worked for me often and not for you :-) But I only know this now after I found the reason.
|
#108
|
||||
|
||||
Quote:
...though I would still consider it as a bug because most of all users who know the website will need less than 6 seconds to hit the two checkboxes and execute the click, leading to an irritating errormessage ~60 seconds and one captcha later -> You may call it bad usability, I'd call it a bug. Quote:
Quote:
...and I'd definitely call this a bug although they must have intentially made it this way Now after another 3 attempts with Firefox and this time without VPN, I always ran into the wait time issue again so I've changed the User-Agent in JD and did my experiment this way. Result: You seem to be right! Used User-Agent: Code:
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0 addons.mozilla.org/de/firefox/addon/uaswitcher/ VPN used: No Testlink: Same as for my other tests Adblocker: Yes, U Block Origin Result: 15KiB/s Also, using that pre-generated direct-URL in another browser (other User-Agent) did not help: The limit is caused by the User-Agent and bound to it when it gets generated! I've also counter-tested this using a VPN. I got 75KiB/s, same file, same dataserver in generated direct-URL! Great investigation and nice that it wasn't a waste of time in the end! I've added a plugin setting so users can set their own preferred User-Agent value for this host. I've set your proposed User-Agent as default.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 22.07.2022 at 13:55. |
#109
|
|||
|
|||
Hello, I'm trying to download from upstore too.
I tried with many browser combinations and with JD2 but I've never been able to download a single file. When I download from browser I get one of these errors: -"No slots for free users in your area at the moment, try again later" -"It seems you did not wait 60 seconds of the Captcha loading, please start from the beginning" -No error but download doesn't start, instead the countdown starts again "Please wait 60 seconds..." When I download from JD2 once I solve the captcha I get the second error. Browsers used are firefox-esr, firefox and brave. My JD2 is the flathub app and my system is debian 11. I even used a different user agent with the extension cited above and a different IP with a VPN but no luck so far. Any ideas? What am I doing wrong? |
#110
|
||||
|
||||
@kak7
Short summary: - The website is kind of broken - We've found a workaround so after the next update, it should work again using JD - If you want the website to be working again, contact the upstore support and ask them to fix free download via website
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#111
|
||||
|
||||
After using the fixed variant for some time I noticed something regarding the 15KB/s issue.
Sometimes it still happens. When it happens starting and stopping, even after waiting a day does not help. It's only for a single file! Others aren't affected. Downloading with another user agent helps (or maybe also IP change?)! Maybe we should auto-switch the user agent for the file and retry procedure for that file in case a 15kb/s download is detected ? :-) |
#112
|
||||
|
||||
I'm not planning to add handling for that.
Just in case an IP change helps: Use an EventScripter script that redirects if speed is below X for at least X time. Check the following thread and/or ask there: https://board.jdownloader.org/showthread.php?t=70525
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#113
|
||||
|
||||
@sdi: You could also easily add support for auto random UA, eg different Version number (from x to y) for each JDownloader session or download.
__________________
JD-Dev & Server-Admin |
#114
|
|||
|
|||
It's currently not working. Could you please check why? Maybe the workaround you mentioned needs a little modification.
|
#115
|
||||
|
||||
Works fine here.
Please post your log-ID here If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having. Bitte poste deine Log-ID hier. Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|