#21
|
||||
|
||||
Wait for next update, then request limit will be *limited* to api only and mirror handling also be be working correct (you have to re-add the links for this to work)
__________________
JD-Dev & Server-Admin |
#22
|
||||
|
||||
Limiting the request interval to api only will be available with next core update next week. I'm not finished yet
__________________
JD-Dev & Server-Admin |
#23
|
|||
|
|||
Quote:
Quote:
Considering the JSON already contains the filesizes and JD2 doesn't need to hit the image-server to get that info, it shouldn't be a problem at all. ----- Another question: Is JD2 somehow limiting the concurrent downloads from cloudflare-protected links or something? When I start to download a package with ~30 images (~11MiB total) from 4chan with 5 concurrent downloads, it takes about 3 minutes for it to finish, with most of them being stuck at "Starting..." for 10-15 seconds. Or are you seeing the same behavior and it's being limited server-side? I think it changed to this about a week ago. EDIT: Another try yesterday: 140 files, 75 MiB, 3 concurrent downloads, took 8 minutes. Last edited by Derp; 23.11.2020 at 09:29. |
#24
|
||||
|
||||
@Derp: Has nothing to do with cloudflare. At the moment the request limit also affects the downloads. Once the next core update is live, the request limit only limits api requests, current limiter doesn't support different subdomains.
I will release core update later this day, see https://board.jdownloader.org/showpo...1&postcount=22
__________________
JD-Dev & Server-Admin |
#25
|
||||
|
||||
* this update has not been released yet.
Please be patient. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#26
|
|||
|
|||
Quote:
|
#27
|
||||
|
||||
@Derp: Yes, change is live and working fine here
__________________
JD-Dev & Server-Admin |
#28
|
|||
|
|||
Too bad, download is just as slow as it was before.
But now the downloads aren't stuck at "Starting..." for a while, but at "Download". Many files have to be retried due to timeout, like this: **External links are only visible to Support Staff****External links are only visible to Support Staff** Did another test with a set of 50 URLs and 1 simultaneous download per host: JD2 took 160 seconds, the wget-batchfile 47. Raising the number of simultaneous downloads helps a bit: 30s when setting to 5 downloads. I guess that's as good as it can get? ------ Found another bug with regard to filenames: **External links are only visible to Support Staff****External links are only visible to Support Staff** Post #104266878 has a file with the uploaded filename ".jpg", just the extension, no name-part. JD2 doesn't add this file to the Linkgrabber-list when copying the 4chan-thread-url, regardless of the state of the filename-setting. |
#29
|
||||
|
||||
your log shows
1.) very slow server response times Request-Time: 1758ms 2.) read timeouts Caused by: java.net.SocketTimeoutException: Read timed out I can reproduce both and looks like server side throttling. Same happens for me in browser too Fast start and then takes ages for the last percent or end up in read timeout as well.
__________________
JD-Dev & Server-Admin |
#30
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin |
#31
|
|||
|
|||
Quote:
----- Another minor issue: Since the change, JD2 is adding the MD5-hash to every image file, except WEBM-files. You can use the URL from post #28 for an example. Last edited by Derp; 01.12.2020 at 15:43. |
#32
|
||||
|
||||
@Derp:
MD5 Hash is available to every link for me. Plugin doesn't process/add items without md5. Example link from #28 worked fine for me. Is it missing on linkcrawler or after download?
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 01.12.2020 at 16:10. |
#33
|
|||
|
|||
Quote:
In a new JD2-install, it works for me, too. But in my existing install, the WEBMs don't get the hash added, see pic attached. Both updated to Build Thu Dec 03 19:21:05 CET 2020 Any ideas? |
#34
|
||||
|
||||
Any Eventscripter Scripts active? I cannot reproduce the issue.
Can you try to provide a log? Is this screenshot from download or linkcrawler?
__________________
JD-Dev & Server-Admin |
#35
|
|||
|
|||
No.
Quote:
Found the culprit: A "learned file extension" Linkcrawler-rule from over five years ago. Code:
{ "enabled" : true, "cookies" : null, "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "id" : 1434728570583, "name" : "Learned file extension:webm", "pattern" : "(?i).*\\.webm($|\\?.*$)", "rule" : "DIRECTHTTP", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : null, "rewriteReplaceWith" : null } No idea why the rule is even in there, because webm was added to the default supported extensions nine years ago. Shouldn't be happening even with that rule in place, right? Last edited by Derp; 04.12.2020 at 12:18. |
#36
|
||||
|
||||
Thanks for taking deeper look.
Normally this is expected behaviour because the rule has higher *order* but for this type of rule I've updated JD to forward the meta information correctly. Wait for next core update
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 04.12.2020 at 13:24. |
Thread Tools | |
Display Modes | |
|
|