[zdzichu476]

I was using Process Monitor to analyze something else and I noticed by chance that JDownloader uses weird buffer size for file writes. It writes files in very weird chunk sizes, for example 413696 bytes, 425984 bytes, 409601 bytes. Two of those are divisible by 4096, so that's at least something, but you need to know that not everyone uses default 4KB cluster size (allocation unit size) for their filesystem. That's why chunk sizes should be "powers of 2".
You should fix it by using 524288 bytes or even 1048576 bytes.
Check attached image for what I'm talking about.

[Jiaz]

@zdzichu476: See Settings->Advanced Settings->
GeneralSettings.maxbuffersize (default 500 kb , 500*1024)
GeneralSettings.flushbuffertimeout (default 2 mins)
GeneralSettings.flushbufferlevel (default 80%)
set flushbufferlevel to 100 and JDownloader will wait until buffer is full or flushbuffertimeout is reached.
with flushbuffertimeout (default 2 minutes) is used to specify the maximum data (that was downloaded in that time) that may be lost (eg crashes/killing the process..)
set flushbufferlevel to 100 and maxbuffersize to wished buffer size and writes (what we tell Java to write/flush to disk) will be always full buffer size.
in case you downloads are slow and not able to full buffer to 100% in flushbuffertimeout , then you should increase that as well. if you don't want time based flushes, then set this value to something very high, like 1 day or 1 year


personally I think this is over optimization as I expect os/filesystem to know better when to write/flush to disk.

also please know that process monitor only shows what is send to the api but not the actual write IO to the storage device, meaning that the OS is allowed to cache/reorder/optimize those writes.

[zdzichu476]

Thanks, setting maxbuffersize to 1024 and flushbufferlevel to 100 made it just the way I wanted. And I know about system's file cache and merging together multiple write operations, etc . But it only works on internal drives. It doesn't work on USB drives (unless you enable write caching on those in device manager) and SMB shares (NAS, etc). So JDownloader could by default use optimized values to benefit every use case.

[Jiaz]

@zdzichu476: I would not recommend setting to 1024 as there will be huge write overhead then. better leave multiple of 4096 and set flushbufferlevel to 100

[zdzichu476]

1024 * 1kb = 1mb

Offtopic: I can't use uppercase KB and MB above, it changes to lowercase

[Jiaz]

Originally Posted by zdzichu476

1024 * 1kb = 1mb

Ah okay, misinterpreted it as 1024 b = 1 as value, my fault

[Jiaz]

@zdzichu476: with next core update JDownloader will query OS for allocation unit size and then allign writes to it. Implemented/tested for Windows/Linux/BSD, MacOS not enabled yet as not tested