Benchmarks
Here you can see speeds and comparisons of releases of MiniINI in some of the benchmarks included in the package. Comparisons are shown with absolute and relative values. These comparisons were made using optimized builds of MiniINI.
Run time of a benchmark is compared by instruction reference count measured by Callgrind (part of the awesome Valgrind debugging and profiling suite). This is good for relative comparisons, not so much for measuring absolute time a run takes. However, I don't have time to ru n thousands of runs of every benchmark in order to minimize error, so I prefer Callgrind data, which doesn't change between runs and is accurate enough.
To give you a vague idea, on my 1.73 GHz dual core Pentium machine (of which MiniINI only uses one core), 1 billion (10^9) instruction references translates to about 1300 milliseconds. Note that this is a very inaccurate idea, but the real performance will at worst be within an order of magnitude from that.
Total time taken by reading files, parsing them and extracting data from them as well as program shutdown is measured.
In memory usage comparisons, only peak memory usage is shown, i.e. the maximum memory used by the program. Average memory usage is lower. Memory use is measured by Massif, another Valgrind tool, and is a sum of heap and stack size.
Tags benchmark
This benchmark's purpose is to measure dependence of MiniINI performance on number of tags in sections. This includes 5 sets of runs with files containing varying number of tags in identical number of sections. Time for every set is a sum for 4 test runs, one with a file full of string values, one with ints, one with floats and one with bools. These tests and total filesizes of INI files processed are:
004 232.97 kiB 012 634.82 kiB 032 1640.19 kiB 096 4859.39 kiB 256 12896.61 kiB
Test names correspond with number of tags per section in files used. All files have 512 sections. The non-STL methods were used in this benchmark. With STL methods, you can expect run time to double.
Time: (millions of instruction refs)
ver 0.4 0.5 0.6 0.7 0.8 0.9 004 17.63 14.74 15.12 15.23 13.89 13.33 012 39.30 32.71 33.67 34.14 30.07 29.60 032 96.64 80.77 83.30 84.55 73.67 73.52 096 298.54 252.88 260.40 264.20 231.52 232.37 256 878.54 758.07 778.03 788.12 700.93 704.08
Time: (relative)
ver 0.4 0.5 0.6 0.7 0.8 0.9 004 100% 83.568% 85.74% 86.37% 78.75% 75.57% 012 100% 83.212% 85.67% 86.86% 76.51% 75.31% 032 100% 83.579% 86.19% 87.49% 76.23% 76.07% 096 100% 84.707% 87.22% 88.50% 77.55% 77.84% 256 100% 86.288% 88.56% 89.71% 79.78% 80.14%
Peak memory usage (kB)
ver 0.4 0.5 0.6 0.7 0.8 0.9 004 183.70 183.81 192.41 192.41 200.70 200.70 012 455.78 455.88 465.82 465.82 474.11 474.01 032 1138.75 1139.80 1150.29 1150.29 1158.68 1158.68 096 3320.84 3325.03 3326.08 3326.08 3348.10 3348.10 256 8762.95 8771.34 8772.39 8772.39 8817.48 8817.48
Peak memory usage (relative)
ver 0.4 0.5 0.6 0.7 0.8 0.9 004 100% 100.06% 104.74% 104.74% 109.25% 109.25% 012 100% 100.02% 102.20% 102.20% 104.02% 104.00% 032 100% 100.09% 101.01% 101.01% 101.75% 101.75% 096 100% 100.13% 100.16% 100.16% 100.82% 100.82% 256 100% 100.10% 100.11% 100.11% 100.62% 100.62%
Sections benchmark
This benchmark measures performance with varying numbers of sections in a file. This includes 5 runs with files containing varying number of sections with identical number of tags. These tests, sizes of files processed and numbers of sections are:
01 26.92 kiB 16 sections 02 119.31 kiB 64 sections 03 488.73 kiB 256 sections 04 1944.84 kiB 1024 sections 05 7656.29 kiB 4096 sections
All files have 512 sections. The non-STL methods were used in this benchmark. With STL methods, you can expect run time to double.
Time: (millions of instruction refs)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 2.64 2.39 2.43 2.45 2.46 2.46 02 6.97 5.88 6.06 6.16 6.19 6.20 03 24.31 19.82 20.57 20.98 21.09 21.15 04 93.34 75.48 78.43 80.07 80.53 80.79 05 372.85 302.52 314.14 320.59 322.40 323.41
Time: (relative)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 100% 90.58% 92.18% 93.02% 93.24% 93.18% 02 100% 84.26% 86.88% 88.32% 88.72% 88.88% 03 100% 81.55% 84.61% 86.31% 86.78% 87.03% 04 100% 80.87% 84.03% 85.78% 86.27% 86.55% 05 100% 81.14% 84.25% 85.98% 86.47% 86.74%
Peak memory usage (kB)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 68.00 68.07 68.24 68.24 68.56 68.56 02 287.85 288.56 288.77 288.77 289.79 289.79 03 1168.11 1166.02 1172.31 1172.31 1176.50 1176.50 04 4642.05 4635.75 4644.14 4644.14 4676.65 4676.65 05 18276.68 18266.19 18339.59 18339.59 18402.51 18402.51
Peak memory usage (relative)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 100% 100.09% 100.35% 100.35% 100.81% 100.77% 02 100% 100.25% 100.32% 100.32% 100.75% 100.68% 03 100% 99.82% 100.36% 100.36% 100.72% 100.72% 04 100% 99.86% 100.05% 100.05% 100.75% 100.75% 05 100% 99.94% 100.34% 100.34% 100.75% 100.69%
Multi value tags benchmark
This benchmark measures performance at reading of arrays of various sizes from multi value tags. This includes 5 sets of runs with files containing varying number of same size multi value tags per section, and one set with few very long multi value tags per section. All files contain the same number of sections. Multi value tags were added in MiniINI 0.7, so there is no data from older versions.
Run time of every set is a sum of 4 test runs, one with a file full of string values, one with ints, one with floats and one with bools. These tests and total sizes of files processed are:
01 35.80 kiB 02 82.91 kiB 04 197.31 kiB 08 557.58 kiB 16 1426.62 kiB long 1301.79 kiB
Test names correspond with number of multi value tags per section in files used. In the long test, there are 2 tags per section. The first 5 sets have 8 to 24 values per tag and the long set has 64 to 192. All files have 128 sections. The non-STL methods were used in this benchmark. With STL methods, you can expect run time to double.
Time: (millions of instruction refs)
ver 0.7 0.8 0.9 01 4.46 4.12 3.81 02 7.47 6.45 6.14 04 14.63 11.90 11.60 08 36.28 27.85 27.57 16 88.02 65.80 65.54 long 75.14 53.45 53.10
Time: (relative)
ver 0.7 0.8 0.9 01 100% 92.32% 85.43% 02 100% 86.27% 82.20% 04 100% 81.35% 79.29% 08 100% 76.78% 75.99% 16 100% 74.76% 74.46% long 100% 71.13% 70.67%
Peak memory usage (kB)
ver 0.7 0.8 0.9 01 36.79 38.87 38.84 02 69.30 71.38 71.35 04 155.14 157.29 157.29 08 423.73 425.78 425.78 16 1034.94 1033.86 1033.90 long 976.69 978.74 978.74
Peak memory usage (relative)
ver 0.7 0.8 0.9 01 100% 105.65% 105.57% 02 100% 103.00% 102.96% 04 100% 101.39% 101.39% 08 100% 100.48% 100.48% 16 100% 99.90% 99.90% long 100% 100.21% 100.21%
Arrays benchmark
This benchmark measures performance at reading of arrays of various sizes from sequences of numbered tags. This includes 5 sets of runs with files containing varying number of same size arrays per section, and one set with few very long arrays per section. All files contain the same number of sections.
Run time for every set is a sum for 4 test runs, one with a file full of string values, one with ints, one with floats and one with bools. These tests and total sizes of files processed are:
01 41.70 kiB 02 105.81 kiB 04 263.30 kiB 08 776.37 kiB 16 2052.51 kiB long 2138.36 kiB
Test names correspond with number of arrays per section in files used. In the long test, there are 2 arrays per section. The first 5 sets have 8 to 24 tags per array and the long set has 64 to 192. All files have 128 sections. The non-STL methods were used in this benchmark. With STL methods, you can expect run time to double.
Time: (millions of instruction refs)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 6.78 4.94 5.01 5.05 4.42 4.25 02 13.18 9.56 9.68 9.81 7.98 7.81 04 30.28 21.65 21.93 22.27 17.55 1.74 08 83.10 65.29 66.17 67.16 53.21 53.06 16 218.95 197.99 200.26 203.18 166.50 165.95 long 224.03 165.70 168.23 170.86 134.51 134.20
Time: (relative)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 100% 72.94% 73.91% 74.59% 65.19% 62.68% 02 100% 72.53% 73.46% 74.46% 60.54% 59.29% 04 100% 71.49% 72.42% 73.54% 57.95% 57.40% 08 100% 78.57% 79.62% 80.81% 64.03% 63.85% 16 100% 90.43% 91.47% 92.80% 76.04% 75.79% long 100% 73.96% 75.09% 76.27% 60.04% 59.91%
Peak memory usage (kB)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 41.15 41.22 43.11 43.11 45.21 45.18 02 92.50 92.35 94.37 94.37 96.49 96.46 04 216.27 216.68 218.42 218.42 220.57 220.47 08 622.08 620.44 624.13 624.13 625.46 625.46 16 1620.05 1622.15 1622.15 1622.15 1624.24 1624.24 long 1653.60 1653.60 1661.99 1661.99 1664.09 1664.09
Peak memory usage (relative)
ver 0.4 0.5 0.6 0.7 0.8 0.9 01 100% 100.15% 104.75% 104.75% 109.85% 109.78% 02 100% 99.85% 102.03% 102.03% 104.38% 104.28% 04 100% 100.19% 100.99% 100.99% 101.99% 101.94% 08 100% 99.74% 100.33% 100.33% 100.54% 100.54% 16 100% 100.13% 100.13% 100.13% 100.26% 100.26% long 100% 100.0% 100.51% 100.51% 100.63% 100.63%