[rabidman]

I have a 300/30 Mbps connection from my ISP, and a paid MEGA account.
If I use the official Mega client, with 8 concurrent connections, it maximizes my bandwidth usage very well. This works fine, as long as I'm only downloading one MEGA folder. If I put several in a queue, the MEGA app trips all over itself and there is corruption or missing files in the downloads. Thus, I can only really download one folder at a time with that app. I uninstalled it due to the corruption issue.

If I use JDownloader2, I can put any number of MEGA URLs, and they all download sucessfully without corruption. However, they do so very slowly. Sometimes the bandwidth utilization is very close to zero. Other times, there are lots of small files, and there seems to be a delay before moving on to the next file, which does not occur with the native Mega app. I have played with Settings -> General -> max simultaneous downloads / max downloads per host / max chunks per download. No combination I have found causes the bandwidth utilization to go up.

What settings should I use to maximize the bandwidth usage ?

[Jiaz]

@rabidman: can you define *very slow*? what speed do you get? is your account added and does it show correct state/expire date? maybe try different location , see Settings->Plugins->mega.nz (there are two entries, one for download plugin, one for folder plugin) and there you can change cdn

[rabidman]

Originally Posted by Jiaz

@rabidman: can you define *very slow*? what speed do you get? is your account added and does it show correct state/expire date? maybe try different location , see Settings->Plugins->mega.nz (there are two entries, one for download plugin, one for folder plugin) and there you can change cdn

As low as 0 - completely stalling.

The download plug-in shows the correct account status, traffic & expiration date.

I didn't touch the other plug-in, which is designated as "Crawler" in the UI. I don't see anything there that I would expect to affect the bandwidth.

[Jiaz]

@rabidman: There are no known issues with mega and this sounds more like some error on your side/environment. Does this happen with any mega download? Also happoens with single (max downloads = 1 ) download? Can you please create a log in such situation, see https://support.jdownloader.org/de/k...d-session-logs
wait for this to happen, then create log and post logID here

[pspzockerscene]

Please also go all the way through the following troubleshooting:
https://support.jdownloader.org/de/k...-downloadspeed

[rabidman]

I went through the checklist, and I don't see anything that applies.

I just uploaded some logs :

09.12.24 18.55.12 <--> 09.12.24 18.46.05 jdlog://3927411370661/

At many points during this session, the bandwidth dropped all the way to 0 bytes per second.

There are several odd things I'm seeing :

1) frequently, JD seems to assign all the connections to other hosts, and none to Mega . Those other hosts can be a lot slower, since I don't have premium accounts. Is there a way to fix this ?

2) After disabling all the non-Mega URLs, the throughput picks up, but it is still far short of my ISP limit. right now I'm seeing 3-12 MB/s which is only 24-96 Mbps, well short of the ISP capacity. The Mega app usually achieves close to 300 Mbps. I don't have it installed right now because of the bugs it has, but I can certainly try it again with some of the same Mega links.

3) the "Finalizing" status takes an inordinate amount of time in some cases, even for very small files. This holds the startup of subsequent connections. A checksum of a 12MB file shouldn't take several seconds, for example. I have seen it happen with regular links, but not with Mega links.
Edit: never mind, I just saw it with some Mega links, too. 3 seconds for a 6MB JPG file.

FYI, the hardware I'm running on is pretty beefy. AMD 5700G APU, 16GB of RAM, 20 TB of SSD (5 x 4TB in stripe, each Team Group MP34 4TB NVMe). There is plenty of CPU and I/O available. The boot drive runs on a slower Patriot Ignite 256GB SATA SSD. OS is Win 11 Pro 24H2.

[rabidman]

Here is another log.

19.12.24 08.22.00 <--> 19.12.24 08.30.45 jdlog://5057411370661/

I have got 670 packages from various hosts, many of them MEGA, totaling over 3TB, and JDownloader is just sitting at 0 bytes/s for long periods of time, such as >1 minute.

It does eventually download some files, for a very brief period, but then stops again, for reasons that are unclear.

If I temporarily disable the packages from non-Mega hosts, it helps a lot. The bandwidth still sometimes drops to a small fraction of what the ISP can do, though, as low as 8 Mbps, when 300 Mbps is typical, and Megasync can get close and sustain it.

[Jiaz]

Thanks for the logs. sorry, oversaw the first one, will check asap

Sorry for the missunderstanding but I need the log to be created in the exact moment when the "0 byte/s for long period" is happening. The log must be created while this issue is present/happening, not after. Both logs were not created while downloads were running.

But log does show similarities to underlying cause of this https://board.jdownloader.org/showthread.php?t=96477
But need a log that is made when this issue actual is happening.

Originally Posted by rabidman

1) frequently, JD seems to assign all the connections to other hosts, and none to Mega . Those other hosts can be a lot slower, since I don't have premium accounts. Is there a way to fix this ?

That's not how JDownloader works. Other downloads do not *steal* or get *assigned* connections over other downloads. Every downloads has it's own connection pool, non shared.

Originally Posted by rabidman

The Mega app usually achieves close to 300 Mbps.

Don't forget to check/compare number of connections. Mega app might use more connections/chunks as JDownloader do and hence better overall speed per downloaded file.

Originally Posted by rabidman

3) the "Finalizing" status takes an inordinate amount of time in some cases, even for very small files. This holds the startup of subsequent connections. A checksum of a 12MB file shouldn't take several seconds, for example. I have seen it happen with regular links, but not with Mega links.
Edit: never mind, I just saw it with some Mega links, too. 3 seconds for a 6MB JPG file.

THAT sounds like being caused by mentioned https://board.jdownloader.org/showthread.php?t=96477 Would be really helpful if you can create a log when finalization takes unusual longer.

Originally Posted by rabidman

I have got 670 packages from various hosts

according to log

670/111601

with 111601 links spread in 670 packages
and that's also a huge sign for being the same mentioned issue


Short explanation: the underlying issue I'm currently working on is that the processing (searching for the next download) might stall/block pending/ongoing downloads as such as finalization has to wait for this search and with large packages (of same hoster) this can take very long time and for example because of this speed drops to 0 because download cannot start/stop. due to missing optimization.


Would be really great if you can provide new logs when spoken issues are actual happening

Thanks for your help with this

[shinji2009]

when jd can't start new download opening and closing settings panel on the downloads tab (where chunks per download, max sim downloads, etc) works for me. if i have a package with a lot of files in downloads list, i leave it for night with autoclicker opening and closing panel every few seconds and to the morning everything is downloaded.

[rabidman]

Originally Posted by Jiaz

Thanks for the logs. sorry, oversaw the first one, will check asap

Sorry for the missunderstanding but I need the log to be created in the exact moment when the "0 byte/s for long period" is happening. The log must be created while this issue is present/happening, not after. Both logs were not created while downloads were running.

First, sorry about the very late reply. I had not figured out how to get email notifications on the forum until now, and thus had no idea you had responded. I was looking for the option under my profile in the top right. I finally found the user settings under Extended. But that took me 6 weeks to figure out, sadly. I would humbly suggest that "user settings" and most of "misc" should be in their own, better-named separate menu. Or these options should be under the profile. But I realize you are probably using some third party forum software.

But log does show similarities to underlying cause of this **External links are only visible to Support Staff**...
But need a log that is made when this issue actual is happening.

How am I supposed to do that ? Create the log on startup ? Then send it when I get the problem ? And is there a way to automate the creation ?

That's not how JDownloader works. Other downloads do not *steal* or get *assigned* connections over other downloads. Every downloads has it's own connection pool, non shared.

I think I would need more details to understand. There is a global connection limit of 20 as far as I understand. I saw that it's possible to get an extra 40 for the "force download" advanced setting. Correct me if I'm wrong, but doesn't this mean that there at most 20 (or 60) active socket connections at any given time in JD2 ? And if so, how does JD2 decide the order of active file downloads ?

Don't forget to check/compare number of connections. Mega app might use more connections/chunks as JDownloader do and hence better overall speed per downloaded file.

The Mega app has a hard maximum of 8 concurrent connections, even if chunking is used with parallel connections for a single file. What I have been seeing is that there is no delay at all from one download to the next in the Mega app, whereas there are fairly long delays between file downloads in JD2. Some of them may be related to the finalization problem mentioned below. But I don't think that's all of it. I have also seen the Mega downloads in JD2 slow down massively over time, from 30MB/s (near my ISP limit) down to sometimes as little as 5 KB/s. This is the case even with a premium Mega account. I have tried on VPN and 3rd party HTTP proxy as well - it's all the same.

I haven't the Mega app installed in a couple months, but I need to reinstall it to check the throughput behavior again.

I determined that the problems I was having with the Mega app are related to the browser integration when using Firefox. A lot of the Mega download info is stored on the client side. It gets corrupt when using a long queue, leading to failures. I may try with Chrome again.

The Mega app also has major problems with file and directory names containing certain Unicode characters, which are very hard to resolve - requiring some crazy Windows command running as administrator to move or delete these files/dirs. This is a huge PITA. FYI, the command to erase was "rmdir /S /Q "\\?\C:\Users\<username>\Desktop\Mega\<undeletable_dir>" .
There is also another whole set of problems when moving files to the ZFS on my NAS. Some characters allowed on NTFS are not allowed on ZFS. JDownloader is doing some transformation/normalization that causes this problem not t exist. I would much rather use JDownloader than the Mega app for all these reasons, and many more, but JD download performance with Mega still lags.

THAT sounds like being caused by mentioned **External links are only visible to Support Staff**... Would be really helpful if you can create a log when finalization takes unusual longer.


according to log with 111601 links spread in 670 packages
and that's also a huge sign for being the same mentioned issue


Short explanation: the underlying issue I'm currently working on is that the processing (searching for the next download) might stall/block pending/ongoing downloads as such as finalization has to wait for this search and with large packages (of same hoster) this can take very long time and for example because of this speed drops to 0 because download cannot start/stop. due to missing optimization.


Would be really great if you can provide new logs when spoken issues are actual happening

Thanks for your help with this

Thanks. Yes, that sounds like the same problem. Have you made any progress with solving it ?
I don't understand why this wait is necessary. I haven't taken a look at the code, and I'm mainly a C programmer, but can't you dispatch these file finalization jobs on a queue on a separate thread(s) ? Other file downloads, even on the same hoster, should not need to wait for that. Only when the package is complete should there be a wait for the finalization jobs before marking it finished.

I haven't seen the issue in a while, but if I do, I will create logs.

[Jiaz]

@rabidman: by default in JDownloader only one download can decrypt file after download -> Disable this via Settings->Plugins->mega.nz and multiple downloads can decrypt in parallel and thus don't have to wait for each other. depending on file size, this can lead to less concurrent downloads but also less pressure on cpu/disk

Originally Posted by rabidman

but can't you dispatch these file finalization jobs on a queue on a separate thread(s) ?

That's not the issue. DownloadWatchdog(Controller) has queue system and processes all events (start download, stop download..) in it. Problem is that large package with many links in it, currently don't check for pending queue items. When JDownloader looks for next possible download to start and large packages, this can lead to many checks.
A package with 100 links in it, already results in about 10.000 checks, for that package alone. that is blocking all other (start, stop of downloads...). a package with 500 links in it, already 250.000 checks, each time.
Stopping downloads currently *starve* while waiting for those checks to finish -> new one don't start before finished ones are processed.

Originally Posted by rabidman

Have you made any progress with solving it ?

yes, but not finished yet, maybe this week