I did some simple file system file allocation comparison. How files are allocated when allocation happens differently. I did this just for fun. Results are quite interesting. Especially if you don't have time or will to repeat tests. Initialization: I1: I formatted multiple 100 gigabyte volumes systems, one using ext4, one using ntfs and one using ntfs. I2: I copied about 300k randomly sized files to file system, totalling about 6.9 gigabytes. Tests: T1: Copied 3.1 gigabyte file to volume. Final size was known and therefore pre-allocation could be used. T2: Quickly created same file, final size unknown to system (pre-allocation unavailable), file allocated when it grows quickly. (about 80Mbytes/s) T3: Slowly created same file, final size unknown to system (pre-allocation unavailable), file allocated when it grows slowly. (about 2Mbytes/s) Results for BtrFS: B1: Excellent result, 7 extents. B2: Strange result, 46 extents, which some are very small. B3: Very strange result, 531 extens, which almost 1/3rd contain only 1 block. Results for Ext4: E1: Excellent result, 29 extents. Please note that Ext4 got maximum extent size limit, only three of extents (excluding last one) are smaller than maximum. E2: Very good result, 37 extents. E3: Excellent result, 29 extents. Results for Ntfs: N1: Perfect results, 1 contiguous file. N2: Skipped. N3: Bad result, 5063 fragments. 614 kilobytes/fragment and for comparison it makes about 157 blocks(clusters)/fragment. Conclusions: Ext4 is very good what comes to avoiding fragmentation in general. BtrFS still got clear delayed allocation flaws, slow allocation is actually quite bad. NTFS, it behaves just it has behaved in all of my previous tests. When using pre-allocation results are perfect and with growing files, data is scattered all over disk. Notes: NTFS allocation was done using 64bit Windows 7, not Linux NTFS allocator, so can't claim that it's just Linux NTFS allocator that would cause fragmentation. There was plenty of free disk space during these tests. It really outlines how bad BtrFS is now with slow allocation. Same file content was used for all tests. BtrFS supports zlib and lzo. Also file content was already compressed using LZMA2. If you want to use exactly same files for similar tests, I do have deliverable as one large bittorrent file. Here's more detailed results if you're really interested. BtrFS copy allocation: File size of copy is 3342903522 (816139 blocks, blocksize 4096) ext logical physical expected length flags 0 0 3900207 40720 1 40720 4086804 3940926 34784 2 75504 4070400 4121587 2640 3 78144 4157441 4073039 147936 4 226080 4332544 4305376 248784 5 474864 4594688 4581327 232448 6 707312 4856832 4827135 108827 eof copy: 7 extents found BtrFS fast allocation: File size of fast is 3342903522 (816139 blocks, blocksize 4096) ext logical physical expected length flags 0 0 4957187 1 1 1 4965660 4957187 77887 2 77888 4856832 5043546 70688 3 148576 4927521 4927519 480 4 149056 4957186 4928000 1 5 149057 4928002 4957186 29183 6 178240 5043548 4957184 9696 7 187936 5053245 5053243 40512 8 228448 5093758 5093756 2048 9 230496 3900207 5095805 44449 10 274945 3958348 3944655 18047 11 292992 4007289 3976394 29440 12 322432 4037177 4036728 18880 13 341312 4003250 4056056 3232 14 344544 3977514 4006481 6176 ... 41 659809 5283659 5310731 17855 42 677664 5301515 5301513 9216 43 686880 5322247 5310730 58432 44 745312 5381120 5380678 48768 45 794080 5429889 5429887 22059 eof fast: 46 extents found BtrFS slow allocation: File size of slow is 3342903522 (816139 blocks, blocksize 4096) ext logical physical expected length flags 0 0 2294997 1 1 1 2255636 2294997 463 2 464 2256100 2256098 128 3 592 2257205 2256227 1 4 593 2256229 2257205 367 5 960 2257206 2256595 1 6 961 2256597 2257206 319 7 1280 2257207 2256915 1 8 1281 2256917 2257207 287 ... 95 28576 2273440 2284336 1 196 28577 2284338 2273440 191 197 28768 2273441 2284528 1 198 28769 2284530 2273441 207 199 28976 2273442 2284736 1 200 28977 2284738 2273442 175 201 29152 2273443 2284912 16 202 29168 2284914 2273458 32 203 29200 2284947 2284945 48 204 29248 2273460 2284994 1 ... 524 813744 3160980 3159665 1 525 813745 3159667 3160980 223 526 813968 3160981 3159889 1 527 813969 3159891 3160981 1084 528 815053 3160982 3160974 1072 529 816125 3162068 3162053 1 530 816126 3162055 3162068 13 eof slow: 531 extents found Ext4 copy allocation: File size of copy.7z is 3342903522 (816139 blocks, blocksize 4096) ext logical physical expected length flags 0 0 374784 18432 1 18432 395264 393215 30720 2 49152 428032 425983 30720 3 79872 460800 458751 30720 4 110592 493568 491519 30720 5 141312 565248 524287 24576 6 165888 591872 589823 30720 7 196608 624640 622591 30720 8 227328 657408 655359 30720 9 258048 690176 688127 30720 10 288768 722944 720895 30720 11 319488 755712 753663 30720 12 350208 788480 786431 30720 13 380928 821248 819199 30720 14 411648 854016 851967 30720 15 442368 886784 884735 30720 16 473088 919552 917503 30720 17 503808 952320 950271 30720 18 534528 985088 983039 30720 19 565248 1017856 1015807 30720 20 595968 1083392 1048575 30720 21 626688 1116160 1114111 30720 22 657408 1148928 1146879 30720 23 688128 1181696 1179647 30720 24 718848 1214464 1212415 30720 25 749568 1247232 1245183 30720 26 780288 1280000 1277951 30720 27 811008 1312768 1310719 4096 28 815104 1311233 1316863 1035 eof copy.7z: 29 extents found Ext4 fast allocation: File size of fast is 3342903522 (816139 blocks, blocksize 4096) ext logical physical expected length flags 0 0 2230272 30720 1 30720 2263040 2260991 30720 2 61440 2295808 2293759 30720 3 92160 2328576 2326527 30720 4 122880 2361344 2359295 30720 5 153600 2394112 2392063 30720 6 184320 2426880 2424831 30720 7 215040 2459648 2457599 30720 8 245760 2492416 2490367 30720 9 276480 2525184 2523135 30720 10 307200 2557952 2555903 30720 11 337920 2590720 2588671 28672 12 366592 253952 2619391 2048 13 368640 258048 255999 2048 14 370688 280576 260095 2048 15 372736 286720 282623 2048 16 374784 270336 288767 4096 17 378880 290816 274431 4096 18 382976 301056 294911 2048 19 385024 313344 303103 2048 20 387072 321536 315391 4096 21 391168 374784 325631 18432 22 409600 395264 393215 30720 23 440320 428032 425983 30720 24 471040 460800 458751 30720 25 501760 493568 491519 30720 26 532480 565248 524287 24576 27 557056 591872 589823 30720 28 587776 624640 622591 30720 29 618496 657408 655359 30720 30 649216 690176 688127 30720 31 679936 722944 720895 30720 32 710656 755712 753663 30720 33 741376 788480 786431 30720 34 772096 821248 819199 30720 35 802816 854016 851967 24576 36 827392 852481 878591 45 eof fast: 37 extents found Ext4 slow allocation: File size of slow is 3342903522 (816139 blocks, blocksize 4096) ext logical physical expected length flags 0 0 374784 18432 1 18432 395264 393215 30720 2 49152 428032 425983 30720 3 79872 460800 458751 30720 4 110592 493568 491519 30720 5 141312 565248 524287 24576 6 165888 591872 589823 30720 7 196608 624640 622591 30720 8 227328 657408 655359 30720 9 258048 690176 688127 30720 10 288768 722944 720895 30720 11 319488 755712 753663 30720 12 350208 788480 786431 30720 13 380928 821248 819199 30720 14 411648 854016 851967 30720 15 442368 886784 884735 30720 16 473088 919552 917503 30720 17 503808 952320 950271 30720 18 534528 985088 983039 30720 19 565248 1017856 1015807 30720 20 595968 1083392 1048575 30720 21 626688 1116160 1114111 30720 22 657408 1148928 1146879 30720 23 688128 1181696 1179647 30720 24 718848 1214464 1212415 30720 25 749568 1247232 1245183 30720 26 780288 1280000 1277951 30720 27 811008 1312768 1310719 16384 28 827392 1311233 1329151 116 eof slow: 29 extents found NTFS, incremental allocation: File is 816139 physical clusters in length. File is in 5063 fragment(s). NTFS, copy with pre-allocation: File is 816139 physical clusters in length. File is in 1 fragment(s). Using pre-allocation with NTFS file system is very important. It should be used when ever it is possible (when final file size is known). Unfortunately many applications aren't using this important feature. Linux kernel used: 2.6.38-8-server |