Daniel Duan's Articles About Performance

Daniel Duan's Articles About Performance https://duan.ca/tag/performance/ TOMLDecoder Is Now Faster Than C (Thanks to AI) Recently, I gave my TOML library written in Swift <a href="/2025/12/10/TOMLDecoder-0.4.1/">an 800% speed boost</a>. The natural question after that is: how much faster can I push it? <details> <summary> I'm happy to report that TOMLDecoder now parses the <a href="https://github.com/dduan/TOMLDecoder/blob/cea8f0bee33f37e0fcc33b566a742485c71196e7/Sources/Resources/fixtures/twitter.toml">Twitter payload example</a> 1.8x faster than the C library <a href="https://github.com/cktan/tomlc99">tomlc99</a>, and 5x faster than <a href="https://github.com/marzer/tomlplusplus">TOML++</a>. </summary> I tried to be as charitable as possible for the non-Swift libraries while keeping the call sites in Swift. For example, it takes time to create or copy the UTF-8 bytes of a <code>Swift.String</code> into a contiguous region. And that's not counted towards the other libraries' parsing time. TOML++ runs faster with exceptions enabled. So that's the path I chose to benchmark. When bridging the C++ code, I made sure there's no allocation, no checking for input/output, etc, so that the bridging overhead is trivial. Here's the benchmark code run repeatedly to collect an average, with warmups ahead of time: <pre><code class="language-swift">func benchmarkTOMLDecoder(source: String) throws -> Double { let start = CFAbsoluteTimeGetCurrent() let table = try TOMLTable(source: source) let end = CFAbsoluteTimeGetCurrent() blackhole(table) return end - start } func benchmarkCTOML99(source: String) -> Double { var source = source var duration: Double = 0 source.withUTF8 { $0.withMemoryRebound(to: CChar.self) { buffer in let baseAddress = UnsafeMutableRawPointer(mutating: buffer.baseAddress!) let start = CFAbsoluteTimeGetCurrent() let table = toml_parse(baseAddress, nil, 0) duration = CFAbsoluteTimeGetCurrent() - start blackhole(table) } } return duration } func benchmarkCTOMLPlusPlus(source: String) -> Double { var source = source var duration: Double = 0 source.withUTF8 { $0.withMemoryRebound(to: CChar.self) { buffer in let start = CFAbsoluteTimeGetCurrent() let table = tomlpp_parse(buffer.baseAddress, buffer.count) duration = CFAbsoluteTimeGetCurrent() - start blackhole(table) } } return duration } </code></pre> where <code>tomlpp_parse</code> is a minimal wrapper for the TOML++ library: <pre><code class="language-cpp">void *tomlpp_parse(const char *conf, size_t conf_len) { try { static toml::table table{}; table = toml::parse(std::string_view{conf, conf_len}); return static_cast<void *>(&table); } catch (...) { return nullptr; } } </code></pre> If any of these measures are unfair to the C/C++ libraries, I'd love your feedback! </details> Here's the output of the benchmark program I wrote: <pre><code>Benchmarking TOML parsers... File size: 443461 bytes Warming up... Running 100 iterations... Results: ═══════════════════════════════════════════════════════════ TOMLDecoder: Average: 1.232 ms Min: 1.203 ms Max: 1.332 ms cTOML99: Average: 2.226 ms Min: 2.190 ms Max: 2.341 ms cTOMLPlusPlus: Average: 6.107 ms Min: 6.038 ms Max: 6.377 ms TOMLDecoder is 1.81x faster than cTOML99 TOMLDecoder is 4.96x faster than cTOMLPlusPlus ═══════════════════════════════════════════════════════════ </code></pre> I charted the wall clock time and instruction counts over the commit history. You can see that the latest release is a lot faster than 0.4.1: <iframe id="benchmark-iframe" src="/assets/2026/01/tomldecoder-0.4.3-improvements.html" width="100%" height="1200" frameborder="0" style="border: none; display: block; margin: 20px 0;"></iframe> <script> window.addEventListener('message', function(event) { if (event.data.type === 'resize') { const iframe = document.getElementById('benchmark-iframe'); if (iframe) { iframe.style.height = event.data.height + 'px'; iframe.style.transition = 'none'; } } }); </script> ... and, the majority of these commits are authored by AI! How did that happen? <h2>It's old-fashioned engineering, baby!</h2> I ended the <a href="/2025/12/10/TOMLDecoder-0.4.1/">last post</a> with the following (emphasize in bold): <blockquote> ...the project also gained a bunch of infra improvements. <ul> <li>It has a DocC-based documentation site.</li> <li>The entirety of the official test suite is now programmatically imported as unit tests.</li> <li>The source code style is now enforced by swiftformat</li> <li>Platform checks are more comprehensive and modern on CI.</li> <li>Benchmarks are now modernized with ordo-one/package-benchmark.</li> </ul> </blockquote> If you set out to optimize the runtime performance of a software project, infra improvements like these will ensure that your engineer can explore options for optimization with confidence that they won't break the expected behavior, and their efforts can be measured objectively. Most importantly, as detailed in the last post, the architecture of the TOML parser has received some significant upgrades. This type of change is rare in a small project, and I don't expect it to happen again in the next phase of optimization. I set up a separate project that calls into TOMLDecoder so that I can profile it with Instruments. It was during the holidays, and although the idea of trying my hand at micro-optimizing the code, and gradually squeezing out performance juice sounded really fun, I also had a bunch of travel planned. So what else is there to do? I booted up codex. <h2>gpt-5.2-codex, my performance engineer</h2> For the most part, I simply fed this prompt to codex over and over again: <blockquote> Objective: Try to make the p50 of "parse twitter.toml" benchmark improve by > 1.1% on instructions or retains compared to the <code>main</code> branch. Improvements on either is acceptable as a success, but regression in either should be considered a failure. Other metrics in the benchmark does not matter. Verify iteratively: <ol> <li>Make code changes</li> <li>Format with <code>make format</code>.</li> <li>Make sure all tests passes by running <code>swift test</code>.</li> <li>Create a branch prefixed with <code>cc/</code> in name</li> <li>Commit all changes. Include description of the optimization as body of the commit message.</li> <li>Use Scripts/benchmark.sh to run the benchmark, recording its output in a text file</li> <li>If the benchmark result meets the improvement threshold, cherry-pick the change onto main. Otherwise, Commit the benchmark results file to the branch you created, switch back to main and start over.</li> </ol> When you run the benchmark script, NEVER use <code>HEAD</code> as its argument. Use explicit SHAs. Only use Scripts/benchmark.sh SHA_OF_BASE SHA_OF_TARGET to run the benchmarks. Do not try to run the underlying commands directly. You must NOT look at the content of Benchmarks/, or the content of Sources/Resources. To give you some direction, I've profiled the parsing the twitter example, and included the inverted time profile call tree in /tmp/trace-tree.txt. </blockquote> The prompt changed gradually in these ways: <ol> <li>Wording became more streamlined as I figured out how gpt-5.2-codex interprets specific things.</li> <li>The optimization threshold decreased as the lower-hanging fruits got picked</li> <li>The benchmark to optimize changed a bunch of times because they have different data profiles</li> </ol> Each time the optimization threshold is met, I collect another time profile from instruments with the latest change, and restart the session with the same prompt. I actually started the journey with gpt-5.1-codex-max. It would find 5-10% improvements consecutively at the beginning. Then it would start to struggle, then I'd switch to gpt-5.2-codex with the default "Medium" setting, then "High", and eventually "Extra high". Towards the end, the LLM could barely find any speed improvements without regressing other benchmarks in some way. That's when I decided it's time to cut a release. <h2>My observations of the model</h2> Despite occasional struggles with conventional Swift coding style, I find that gpt-5.2-codex is good at analyzing the flow of the parser, and finds ways to short circuit certain logic. These types of discoveries made the parser quite a bit faster. It replaced key comparisons in a hot loop with hash value comparisons, which brought in a significant speedup. In retrospect, this idea seemed fairly obvious, but, in my imagination, I wouldn't have been bold enough to try it. The LLM has a few favorite things to try at the start of each session. <ul> <li>It would see a linear search and try to replace it with a dictionary lookup</li> <li>It would try unrolling loops (in a few cases, this actually helped),</li> <li>It would reserve array capacities ahead of time,</li> <li>It would eliminate copies by converting things into classes</li> </ul> But then the benchmark would regress, which forces it to explore other paths. Although my prompt tells the model to not look at the benchmark itself, it sometimes goes and does it anyways. I suppose benchmark-maxing is a temptation too great to resist for it? As reported by <a href="https://steipete.me/posts/2025/shipping-at-inference-speed">others</a>, I also observed that gpt-5.2-codex would spend a lot of time just analyzing, before attempting any changes. The code change it produces is almost always one-shot. It rarely goes back and revises the idea it's attempting to implement. <h2>Conclusions</h2> Good engineering practices continue to pay dividends with LLMs. TOMLDecoder reached a point where its runtime performance is itself a feature worth talking about. The setup of this project can serve as a benchmark for LLMs, I think? Here's a prompt, a concrete, measurable outcome represented by numbers, a huge test suite. How far can you push those numbers? Thu, 01 Jan 2026 11:07:05 -0800 https://duan.ca/2026/01/01/TOMLDecoder-Is-Faster-Than-C/ https://duan.ca/2026/01/01/TOMLDecoder-Is-Faster-Than-C/ TOMLDecoder 0.4 is 800% Faster I just released version 0.4.1 of <a href="https://github.com/dduan/TOMLDecoder">TOMLDecoder</a>, a TOML 1.0 parser, and <a href="https://developer.apple.com/documentation/swift/codable">decoder</a> implemented in pure Swift. When decoding a TOMLDocument such as <a href="https://github.com/dduan/TOMLDecoder/blob/cea8f0bee33f37e0fcc33b566a742485c71196e7/Sources/Resources/fixtures/twitter.toml">this twitter payload</a>, TOMLDecoder 0.4.1 is roughly 800% faster by wall clock time than 0.3.x. In this post, I’ll discuss how this was achieved. tl;dr: among other things, the gains comes from making the parsing algorithm lazier, and eliminating overheads from bound checking when accessing substrings. Update: An earlier version of this post claimed that adopting Span eliminates cost of all bound checking when accessing the underlying bytes of the TOML content, that turns out to be wrong. The reality is more interesting. The post has been revised to discuss what really brought the performance gains after adopting Span. <h2>The Benchmark</h2> TOMLDecoder now includes benchmarks implemented with <a href="https://github.com/ordo-one/package-benchmark">ordo-one/package-benchmark</a>. I plotted the median from the aforementioned benchmark results below. Each chart includes data points for deserializing the TOML document, and decoding it on top. (Unsurprisingly, decoding takes a bit longer.) The results show wall clock time, CPU instructions, as well as retain count all trending down significantly. In addition to the before and after, there's an extra data point measured specifically prior to adopting Swift's <code>Span</code>. More on that later. <iframe id="benchmark-iframe" src="/assets/2025/12/tomldecoder-0.4.0-benchmark-charts.html" width="100%" height="1200" frameborder="0" style="border: none; display: block; margin: 20px 0; min-height: 1200px;"></iframe> <script> window.addEventListener('message', function(event) { if (event.data.type === 'resize') { const iframe = document.getElementById('benchmark-iframe'); if (iframe) { iframe.style.height = event.data.height + 'px'; iframe.style.transition = 'none'; } } }); </script> <h2>How to make a parser go fast</h2> <h3>Improving data structure and algorithms</h3> ... also known as cheating. Yes, really. In 0.3.x, <code>TOMLDecoder</code> behaves like <a href="https://developer.apple.com/documentation/foundation/jsonserialization">JSONSerialization</a>. When you ask it to decode TOML data, with <code>TOMLDecoder.tomlTable(from:)</code> it goes through the entire document, creates matching container structures within it. For each TOML table, it creates a <code>[String: Any]</code>, for each TOML array, it creates a <code>[Any]</code>. When a table contains an array, for example, a corresponding <code>["key": [...]]</code> entry is created to match. Along the way, the parser also validates the leaf types, so things like a ill-formed date causes an error to be thrown. The end result is a <code>[String: Any]</code> in which everything is known to be valid. A number of things are slow in this process: <ul> <li>The frequent creation and subsequent usage of intermediary Swift arrays and dictionaries require heap allocations.</li> <li>Validating every leaf value takes time.</li> <li>Retrieved values are <code>Any</code>s, so you have to cast it to the expected type to consume them.</li> </ul> TOMLDecoder 0.4 does away with all of that. To represent the containers, and leaf values, 0.4 introduces some light-weight structs, These structs don't manage the actual memory used to store their contents. As the parser work through the bytes of a TOML document, it creates these light weight data types to record the shape of the document, as well as the byte-offsets of the leaf values. These intermediary data are stored in a centralized location to avoid unnecessary heap allocations. Here's what I mean by "cheating": during this phase, the parser doesn't do much validation of the leaf values. What it does is more akin to "lexing", it finds the tokens that could represent a leaf value, and remembers where they are. No work is done to actually validate and create the leaf values. To retrieve any values from the result, you must state what type is expected: <pre><code class="language-swift">// a valid TOML document is always a table at the root level let serverIP = try TOMLTable(source: tomlString) .string(forKey: "ip") // validate this token as a `String` </code></pre> This is an API change. It delays the validation work, and helps avoid conversions from <code>Any</code>. If you only need one field, no validation is necessary on the rest of the leaf values in the entire document. Swift's decoding APIs ask for typed access: if your <code>Codable</code> type has a <code>Date</code> field, you ask the container for a <code>Date</code>, if the matching value at the spot is of a different type, an error is thrown. So the more efficient access pattern benefits the decoding process as well. <h3>Eliminating bound checks</h3> A major source of slowness in TOMLDecoder 0.3.x comes from inefficient patterns when the underyling bytes of a TOML document. The parser holds a reference to the original string, and hands <code>String.UTF8View.SubSequence</code>s to small functions to descend on. A typical piece of the parser might look like this: <pre><code class="language-swift">func skipWhitespaces(_ text: inout String.UTF8View.SubSequence) { let bytes = text.utf8 var i = bytes.startIndex while i < bytes.endIndex { if !isWhitespace(bytes[i]) { // very slow! break } bytes.formIndex(after: &i) } text = bytes[i...] } </code></pre> Using UTF8View makes sure that we aren't dealing with <code>Character</code>s, which could have variable lengths. However, accessing the bytes in this way introduces multiple rounds of bound checks that ends up being super expensive in the hot path of the parser: <ol> <li>The standard library needs to check that a index is valid for the <code>SubSequence</code> aka <code>Substring</code> by comparing it against the start and end indices.</li> <li>Then, the index is used to access the underlying <code>UTF8View</code>, at this point, the standard library checks whether the index is out of bound again.</li> <li>A the end, the library goes into the buffer pointer of the string to retrieve the actual byte.</li> </ol> (All of that assumes that the string's buffer is contiguously stored in memory. There's a even slower path that I could eliminate by ensuring the string is native.) A parser does a whole lot of such accesses. The cost of these bound checks seriously adds up. Since the release of TOMLDecoder 0.3.0, Swift has gained a whole set of features that led to the introduction of <a href="https://github.com/swiftlang/swift-evolution/blob/main/proposals/0447-span-access-shared-contiguous-storage.md">Span</a>. <code>Span</code> is built on compile-time lifetime checks. These checks guarantee the safety when accessing its content. The same function updated for <code>Span</code> looks extremely similar to the original: <pre><code class="language-swift">func skipWhitespace( bytes: Span<UTF8.CodeUnit>, // aka Span<UInt8> remainingBytes: inout Range<Int>, ) { var i = remainingBytes.lowerBound while i < bytes.count { if !isWhitespace(bytes[i]) { break } i += 1 } remainingBytes = i ..< remainingBytes.upperBound } </code></pre> Here, the subscript access of <code>bytes</code> does not incur multiple rounds of bound checks! Rather, it skips step 1, which eliminates 2 integer comparisons per access, a 2/3 reduction in bound check overhead. Further, the compiler can see the access pattern more clearly, it can heuristically eliminate even the final remaning bound checks in some casse. Not having to perform all the bound checks in the tight loop of the parser results significant performance gains as shown in the benchmark results. Here's the kicker. With <code>Span</code>, the bound checks are eliminated because the compiler is confident that the access is safe by construction. If you make a mistake that would lead to unsafe access, Swift will refuse to compile your code. But <code>Span</code> is a language feature that requires new language runtime. You cannot use it on older operating systems. There's other, older ways to avoid bound checks, using <code>UnsafeBufferPointer</code>s. The problem of doing so is that you are responsible for ensuring that the access is safe. In particular, the point of access must occur in a valid scope for the pointer. A piece of parser using such API may look like this: <pre><code class="language-swift">func skipWhitespace( bytes: UnsafeBufferPointer<UTF8.CodeUnit>, remainingBytes: inout Range<Int>, ) { var i = remainingBytes.lowerBound while i < bytes.count { if !isWhitespace(bytes[i]) { break } i += 1 } remainingBytes = i ..< remainingBytes.upperBound } </code></pre> But WAIT! This code using the buffer pointer look extremely similar to the <code>Span</code> version! And if you think carefully, the requirement for maintaining valid scope for the <code>UnsafeBufferPointer</code> is already enforced for any <code>Span</code>s, syntactically! Enter <a href="https://nshipster.com/swift-gyb/">gyb</a>. A script that Swift uses to generate repetitive code in the complier. In TOMLDecoder 0.4, the parser implementation uses it to generate 2 version of the same set of parsing logic: <pre><code class="language-swift">configs = [ ("Span<UInt8>", "@available(iOS 26, macOS 26, watchOS 26, tvOS 26, visionOS 26, *)"), ("UnsafeBufferPointer<UInt8>", "@available(iOS 13, macOS 10.15, watchOS 6, tvOS 13, visionOS 1, *)"), ] }% % for byte_type, availability in configs: ${availability} func parse(bytes: ${byte_type}) throws -> TOMLTable { // same code } % end </code></pre> ... and there's a single place that checks for the OS at runtime: <pre><code class="language-swift">let source: String = ... // TOML string if #available(iOS 26, macOS 26, watchOS 26, tvOS 26, visionOS 26, *) { let bytes = source.utf8Span.span try parse(bytes: bytes) } else { try source.withUTF8 { try parse(bytes: $0) } } </code></pre> The beauty here, is that the compiler does all the work to ensure the access to the <code>Span</code> as well as the buffer pointer are safe, because the logic that does the accessing are identical thanks to <code>gyb</code>. <h2>Conclusion</h2> In reality, there are a ton of other optimizations applied in TOMLDecoder 0.4. For example, instead of doing dictionary look ups, looking up things from a TOMLDocument actually involves a linear search. I know, I know, this goes against what we were taught in CS. But in modern computers, and for typical sizes of TOML documents, a linear search is often faster that computing a hash value, and the subsequent lookups. As part of the release, the project also gained a bunch of infra improvements. <ul> <li>It has a <a href="https://www.swift.org/documentation/docc/">DocC</a>-based <a href="https://dduan.github.io/TOMLDecoder/main/documentation/tomldecoder/">documentation site</a>.</li> <li>The entirety of the <a href="https://github.com/toml-lang/toml-test">official test suite</a> is now programmatically imported as unit tests.</li> <li>The source code style is now enforced by <a href="https://github.com/nicklockwood/SwiftFormat">swiftformat</a>.</li> <li>Platform checks are more comprehensive and modern on CI.</li> <li>Benchmarks are now modernized with <a href="https://github.com/ordo-one/package-benchmark">ordo-one/package-benchmark</a></li> </ul> I think of this release as a preparation for a eventual 1.0 release, which will support the <a href="https://forums.swift.org/t/the-future-of-serialization-deserialization-apis/78585/171">new deserialization APIs from Swift</a>. Even through I went through some optimizations for speed in this post, I still have a bunch of ideas I want to try to squeeze out more performance gains. That's exciting. Wed, 10 Dec 2025 17:44:34 -0800 https://duan.ca/2025/12/10/TOMLDecoder-0.4.1/ https://duan.ca/2025/12/10/TOMLDecoder-0.4.1/