#1
|
||||
|
||||
Size and Size on Disk discrepancy
I've noticed a lot of folders that contain content downloaded from Instagram lately have been largely different between Size and Size on Disk.
I also am using the new setting of original filenames in instagram plugin. My filesystem is NTFS, Windows 8.1. |
#2
|
||||
|
||||
I guess high filesystem fragmentation or not good blocksize chosen for filesystem or some ntfs features enabled that cause this.
JDownloader has no influence on that.
__________________
JD-Dev & Server-Admin |
#3
|
||||
|
||||
Interesting. My drive is not fragmented for other files, my blocksize is fine (again, for other normal picture and video files), and I have not tweaked my NTFS to be anything but the default.
Are you sure it's not JD2 having writing issues? Edit: The reason I posted this is that I have never encountered these issues with JD2 before - only from what I'm seeing now with .jpg files saved by JD2 from instagram. Before, every other filetype has been fine as far as I've seen in their Size/Size on Disk ratio. Last edited by xped123; 02.09.2016 at 20:38. |
#4
|
|||
|
|||
(890306560-748671920)/4347 = ... Cluster size 32KB? Google "file slack".
|
#5
|
||||
|
||||
@SourTon: yes thanks, I found same google results
It must have to do with: -cluster size -fragmentation -data streams (maybe some tools append additional infos) @xped123: no changes in JDownloader
__________________
JD-Dev & Server-Admin |
#6
|
||||
|
||||
and if you use raid you can compound the issue as its raid block size * cluster size.
raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#7
|
|||
|
|||
The average slack space (difference between physical and logical size) per file is roughly half of the cluster size. For e.g. the total file slack for 4347 files depending on the cluster size would be:
4K = 8.49 MB 8K = 16.98 MB 16K = 33.96 MB 32K = 67.92 MB 64K = 135.84 MB While the difference between size and size on disk is comparatively small with 4K cluster, the same will be considerably high with a 64K cluster. |
#8
|
||||
|
||||
@mgpai: thanks for the numbers
__________________
JD-Dev & Server-Admin |
#9
|
||||
|
||||
Thanks for all the support ! I will inspect what's going on with all the resources here and find what's my own problem
|
#10
|
||||
|
||||
I'd like to revive this thread.
I would like to think it's something relating to BOTH JDownloader and my system. Here's an example: Image downloaded by JDOWNLOADER from INSTAGRAM using "Original Filename" Image COPIED by the system to a different folder, same drive: What gives? Why is there a descepancy between my Size and Size on Disk? I checked out my CLUSTER, it's 4,096 bytes (4K) allocation. There's no reason why some of these images should be almost double in their size when downloaded by JDOWNLOADER. EDIT: Checking another folder of mine relating to "imagefap.com", I can find in 2015, between MAY and AUGUST 2015, the folders then start having issues of these size discrepancies. This is also on a DIFFERENT drive, also have 4K clusters. i.e. Folders BEFORE May 2015: i.e. Folders AFTER August 2015: Windows 8.1, OS install predates both of the drives in question. Both drives are 2TB in question. Both subfolders are NOT within a Junctioned Directory. Both drives are 4K Cluster size. Windows has not been updated in 2 years, so no Microsoft wankery. The only thing that has changed between these variables is literally JD2 and updating it and resuming the same methods of download (Change directory for when JD2 parses the page=> to new directory of downloading than the Default Download Folder). Last edited by xped123; 30.09.2016 at 03:11. |
#11
|
||||
|
||||
A possible cause of this may be the use of sparse files.
I will add advanced option to disable it soon How many connections do you use to download?
__________________
JD-Dev & Server-Admin |
#12
|
||||
|
||||
Quote:
4-5 max simultaneous downloads I hope that helps, let me know whatever else I can provide for my current JD2 settings + my OS settings. I swear I'm not crazy. Nothing else on my system has yielded this much of a difference between Size and Size on Disk. Thank you for your hard work! |
#13
|
||||
|
||||
I will inform you once I've added support to disable sparse, then you can check. I'm sure it is because of many concurrent downloads in combination with multiple connections. Without sparse flag it should no longer hapen
__________________
JD-Dev & Server-Admin |
#14
|
||||
|
||||
Thanks for figuring out and catching that issue with instagram. I appreciate it.
I managed to get around it by a simple copy paste, delete of original, but obviously unless I use xcopy through cmd in Windows, I won't preserve proper file timestamps for created/modified. Cheers |
#15
|
||||
|
||||
Any progress on this sparse setting feature?
Thanks :D |
#16
|
||||
|
||||
I#m sorry, not yet but I will work on this next week.
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|