This Shit is Hard: Nobody has ever been pwned by Helvetica…or have they?
View all articlesDustin Kirkland
SVP, Engineering
Chainguard
Dustin Kirkland SVP, Engineering
The biggest pile of unverified binaries in a from-source Linux distribution is not the compiler, the kernel, the JIT, or the crypto library. It is the fonts.
Debian, Fedora, Arch, Alpine, openSUSE, Gentoo, NixOS, and, until this month, Chainguard OS all ship at least some fonts the same way: download the upstream TTF or OTF files, verify a hash, and copy them into a package. Nobody decided fonts were exempt from the from-source rule. They just sit in a blind spot. They are not code, so the toolchain people do not own them; they are not documents, so the content people do not either. They never crash, half of them have not had a release since the Obama administration, and nobody looks.
Chainguard OS is built from source: more than 10,000 package definitions, and in the OS tier, roughly 1% reach upstream as compiled artifacts. In September 2026, we audited that one percent: every package, every installed file, and its source. Strip out the proprietary-by-nature (drivers, partner binaries) and the deliberate (Gradle distributions kept bit-identical to upstream so Chainguard Libraries rebuilds can be verified against them), and the largest remaining class was fonts: 19 source packages, 129 APKs, 2,379 distinct font files, 3.2 million glyphs, 354 families. font-noto alone accounted for 2,171 files.
The instinctive reaction, from some of our own folks included, was "who cares?!?" Nobody has ever been pwned by Helvetica. This post is about why that instinct is wrong, what it took to fix, and, because the honest answer matters more than the flattering one, how much it actually buys you.
A font is a program. Yes, really.
A TrueType or OpenType file is a container of tables, and several of them execute. Hinting (fpgm, prep, and per-glyph instructions) is bytecode for a stack machine that the rasterizer runs for every glyph at every size. CFF outlines are Type 2 charstrings, PostScript-derived programs for another virtual machine. GSUB and GPOS are state machines with chained lookups and recursion limits. Color fonts embed PNGs, SVG documents, or COLR v1 paint graphs, and variable fonts apply interpolation deltas via pointer arithmetic over glyph data.
All of this is parsed by decades-old C and C++: FreeType, HarfBuzz, Skia, CoreText, DirectWrite, the Windows kernel's font drivers, Java, Ghostscript, the X server. After finding eight bug classes in the charstring interpreters of Windows and Adobe Reader in 2015, Google's Project Zero called fonts "one of the best imaginable attack vectors," running "turing-complete virtual machine environments for running untrusted" programs.
Turing-complete. In your fonts. What every distribution actually ships is programs, in a binary format nobody reads, executed by some of the most historically fragile parsers on the system, that nobody built from source.
Fonts have been pwning people since Windows XP
You remember the headlines. You probably do not remember that a font was the door.
2011, Duqu. The Stuxnet-adjacent toolkit arrived in a Word document with an embedded TrueType font that exploited the win32k.sys parser for ring-0 code execution (CVE-2011-3402).
2011, JailbreakMe 3.0. Visiting a web page jailbroke an iPhone, via a PDF whose Type 1 font hit a FreeType signedness bug (CVE-2011-0226).
2015, BLEND and Hacking Team. Project Zero's audit of Adobe Type Manager in the Windows kernel produced a run of charstring memory-corruption bugs (CVE-2015-0093, CVE-2015-3052). The Hacking Team leak contained another, in atmfd.dll (CVE-2015-2426), which Microsoft patched out of band.
2020, ADV200006. Microsoft warned, without a patch, of two exploited Type 1 parsing bugs (CVE-2020-1020, CVE-2020-0938). The advice: turn off the Explorer preview pane, because previewing a folder with a malicious font was enough.
2020, CVE-2020-15999. A FreeType heap overflow in embedded PNG bitmaps, used in the wild against Chrome as part of a sandbox-escape chain.
2025, CVE-2025-27363. An out-of-bounds write in FreeType's variable-font parsing, exploited on Android. The fix had been in FreeType for two years; the victims were running old binaries.
The build tools are not bystanders either. In late 2025, the Zero Day Initiative disclosed a dozen FontForge CVEs in its file parsers, including the .sfd source parser: the program you build fonts with. None of this is exotic. Complex format, old code, untrusted input, and memory unsafety. The only unusual thing about fonts is how completely everyone has agreed not to think about them.
Everyone agrees. Almost nobody does it.
Distribution policy documents are unanimous that fonts should be built from source. The spec files tell a different story. Two representative cases: DejaVu, where upstream ships .sfd sources and a Makefile, and Noto, where upstream ships .glyphs/UFO sources that need fontmake.
Distribution | DejaVu | Noto | Policy |
Debian / Ubuntu | Source | Prebuilt | DFSG requires buildable source; still being argued in 2026 |
Fedora / RHEL | Source | Prebuilt | Fonts "SHOULD be built from source" |
Arch | Source | Prebuilt | None |
NixOS | Source | Prebuilt | None |
Gentoo | Optional, off by default | Prebuilt | None |
openSUSE | Prebuilt | Prebuilt | None |
Alpine | Prebuilt | Prebuilt | None |
Guix | Prebuilt | Prebuilt | None |
When upstream provides a make target, most distributions use it. The moment the toolchain is anything more than FontForge, everyone ships the binaries. Noto is 200 families built with fontmake and Google's gftools, the fallback font for half the world's scripts, and all ten distributions we checked, including Guix, the one most likely to be stubborn about source, ship Google's binaries.
As far as we can tell, Chainguard OS is now the first general-purpose distribution to build Noto from source and the first to do so across the entire font set. That is not a brag. It is strange: the tools are open source, Google publishes the recipe, and nobody has done it.
What it took
Three weeks after the September 12 audit, every font in Chainguard OS with a buildable source form is built from it: 2,347 font files, 2.4 million glyphs, 323 families, 59,754 Unicode code points. Seventeen packages with no source anywhere remain as 205 prebuilt files. They are mostly CJK and carry another 35,687 code points, almost entirely Han. That is the one honest hole in the claim.
Noto: 202 per-family packages, built with a new gftools pipeline using Google's own recipe, pinned to per-family upstream tags. We dropped 110 stale files that the release site never pruned, after confirming that nothing used them. One was a Sinhala family whose Regular and Bold were different versions.
FontForge .sfd sources: DejaVu, FreeFont, Linux Libertine, Culmus, LKLUG, Samyak, and the Lohit extras.
Glyphs/UFO via fontmake: Caladea, Amiri, and Open Sans.
Everything else: GNU Unifont (which, despite a comment saying otherwise, was not actually building its fonts), Twemoji's 3,689 color glyphs traced from SVG, Nerd Fonts Symbols via upstream's font-patcher, and STIX Two on aarch64.
Along the way, we packaged Perl and Python dependencies that did not yet exist as source builds, and found that skia-pathops's build script downloads a prebuilt gn and a ninja wheel to compile itself. Turtles.
Making the exceptions visible
Converting fonts is a project. Keeping them converted is a policy, and a policy nobody enforces is a blog post. So our linter now fails any recipe that copies a compiled font from an upstream download without a font compiler in the build, unless the step carries a comment giving one of six reasons (no sources, license forbids rebuilding, missing toolchain, broken sources, untagged upstream, conversion in progress) and a tracking issue.
The linter also delivered the most embarrassing finding. The audit counted 19 prebuilt font packages. The linter found 26, including 17 that the audit never saw: Bitstream Vera, Ubuntu, Carlito, the CJK collections, and more. The audit looked at which recipes were downloaded; it missed recipes that clone a git repository and copy the committed binaries. Most of the new finds that had sources were converted within ten days, including Ubuntu, which we had initially written off because of a stale note in the recipe. Read the sources, not the note about the sources.
Noto Color Emoji followed: 4,004 bitmaps that, on x86_64, decode to the same pixels as Google's. On arm64, about ninety did not. It was not floating point. pngquant builds its palette by walking a Rust hash table, which probes sixteen slots at a time with SSE2 and eight with NEON, so the walk order differs, and a hundred k-means iterations later, so do the palettes. One line that sorts the histogram fixes it, and that line is going upstream.
Rebuilding the small packages also revealed bugs unrelated to building from source. Nobody had ever rebuilt them and compared them. The four Lohit extras packages were shipping each other's fonts, and the Bengali extras package had, for years, installed its fonts as directories named JamrulNormal.ttf with the real font inside.
Verification is the hard part
Building a font is easy. Knowing you built the same font is not. Font binaries are never byte-identical across toolchain versions, so we compared meaning instead of bytes: fontTools for glyph sets, cmaps, metrics, flags, and features; HarfBuzz and FreeType to render every code point in both fonts unhinted and diff the pixels; and then we read whatever was left.
Most of it matched. DejaVu's 160,757 glyphs were 99.99 percent identical, and Amiri, STIX, and Nerd Fonts had bit-identical outlines. The residue was more interesting:
A flag that used to mean "no hinting" now means "no glyphs." FreeFont's generator passes no-hints to FontForge, which, on today's release, drops the glyf table entirely. The build "succeeded."
Debian has shipped a stale binary for 19 years. LKLUG's repository has a 2005 TTF that is newer than its source, so make does nothing and packagers ship the 2005 file, with wrong side bearings on 411 of 511 glyphs.
Noto Serif Myanmar's 2022 release draws a vowel sign in the wrong place: 1,141 units below the consonant instead of above, in all 36 faces. We spent a week treating our correct rebuild as the regression before tracing it to a 2022 glyphsLib anchor bug.
The compiler was wrong, and we nearly shipped it. The October 2025 FontForge release truncates the metrics table of any font it considers monospace, giving every combining mark a full-cell advance. Our comparison flagged it in Culmus; we misread it as rounding noise, because a mark rendered alone looks identical either way. A reviewer caught it. We patched FontForge and rechecked. GNU FreeMono's 199 marks were broken too, as they are in GNU's own 2012 binaries, so nobody using those binaries has ever seen them positioned correctly. Liberation Mono, which our recheck list had missed, turned up only when we swept all 2,500 or so font files through the harness as a build gate.
That last one taught us two things. HarfBuzz zeroes mark advances after positioning regardless of the metrics table, so a shaping test passes on the broken font and the fixed one alike; the only honest assertion reads the table directly. And a table difference is never noise until you can name the field and say why.
None of these are security issues. All of them are things you cannot know about a binary you did not build, and cannot know about one you did, unless you compare it against something and take every difference seriously.
The bugs that bit us
When we split Noto into per-family packages, apk upgrade installed the new packages first, refused to overwrite files still owned by the old one, and then deleted them when the old package upgraded. 1,831 font files silently vanished from in-place upgrades for four days. Fresh builds never saw it, and neither did CI. Every conversion now gets an upgrade test.
Later, a merge that added a package and a dependent on it at the same time left apk add font-noto failing on fresh systems for a few hours, because the build farm published the dependent first. The rule now is one package per merge, dependency first, wait for the index. The follow-up fix then sat unbuilt for a day and a half behind an unrelated OpenSSL sweep that broke Rust, and with it uv, gftools, and Noto Sans Bengali. A spill of the same shape had been fixed by hand in April, with a note that a lint check should prevent it. The lint now exists. A rule in a commit message is one that the next automation will not read.
Seven packages to build two fonts
The last holdouts were SIL's Padauk (Burmese) and Abyssinica (Ethiopic), which use SIL's own build system, smith. Getting there took seven new packages and six pull requests.
Then the comparison told us something we had not gone looking for. font-padauk declared version 6.000 and had been shipping 5.100. font-abyssinica declared 2.300 and shipped 2.201. Both installed convenience binaries from upstream's references/ directory that were two releases stale, and had been for years. That is the whole argument for building from source, made by accident: the version on a binary you did not build is a claim, not a fact.
Our builds match SIL's releases exactly across about 673,000 HarfBuzz shaping cases. Padauk's maintainer pointed out that Myanmar shaping often needs several features active at once, which our one-feature-at-a-time cases never exercise, so every build now also runs SIL's own shaping tests against SIL's release. And the Graphite tables we assumed we would have to reproduce turned out not to exist in any current source or release, only in those stale binaries. The old file had simply looked more capable than the current source.
OK, so what could a poisoned font actually do?
Here is the threat model without spin. No gas station pills.
Exploit delivery. A font in a container image is parsed by everything that renders text: headless Chrome, LibreOffice, ImageMagick, Java, PDF generators, matplotlib. A tampered font targeting a renderer bug fires the first time a report is rendered, on every host running the image. The vector is the upstream release artifact, which makes this a supply chain problem, not an input validation one.
Content substitution. Swap the outline behind a code point, and a rendered document says something different without its text changing: digits on an invoice, a hostname in a screenshot. The attacker does not need to control the text, only how it renders.
Storage. Parsers ignore unknown tables, and no scanner opens fonts. That makes them a convenient place to park a payload.
Provenance. The least dramatic and most concrete. We can say a compiled font was built from a given source at a given commit. We cannot say that about a downloaded one, and reproducible builds make tampering detectable.
Have any of the first three happened with a distribution's shipped fonts? Not that we can find. Every attack above delivered the font to the victim; none poisoned a distribution's copy.
Is this worth it? Honestly.
Yes. It is also not the most important thing we did this year. If you came to be told fonts are the next xz, we are not going to tell you that.
The direct exploit risk is real but low-probability: it needs a compromised upstream font release and an unpatched renderer bug, and we patch renderer CVEs within hours regardless. If that were the only argument, the effort would be hard to justify. What carries it:
Completeness. "Built from source" is a property, not a percentage. Fonts were the biggest asterisk in every distribution's core, including ours.
Verifiability. Every problem above was invisible in the binary and obvious in the build.
Fixability. As Fedora's policy puts it, automating the build "ensures we'll be able to fix the fonts when problems are reported and upstream is not responsive." Half of these fonts have not had a release in a decade.
Licensing. The GPL has no fonts exception, and "the TTF is probably the preferred form of modification" is not a settled argument.
We would rank it below the toolchain, runtime, and library work the rest of this series covers, and above most of what people ask us about at conferences. Nobody has ever been pwned by Helvetica. We would like to keep it that way and be able to prove it.
What is still hard (it is in the series title)
Upstreams that do not ship their build. Linux Libertine's source says "run ./make" and does not include it, so we reverse-engineered the release from the shipped TTFs.
Toolchains that drift. FontForge changes defaults and drops flags. fontmake and glyphsLib change output between minor versions, and 2023 requirement pins do not install on Python 3.13. Even zopfli, the PNG recompressor in Google's emoji build, segfaults under GCC 16 at -O3 due to an unaligned read from an upstream bug open since 2015. And FontForge's TrueType output differs by a few units between x86_64 and aarch64, so "reproducible" has an asterisk until someone pins down the drift.
Sources that do not build. Noto Sans Bengali broke in every current glyphsLib release. We bisected it and sent a fix, and the maintainer merged a tightened version within two days. Twelve more Noto fonts silently produced nothing because of one wrong number in their sources; two upstreams had already fixed it without cutting a release, and the third merged our patch the same morning. That is the good version of this story: open toolchains, responsive maintainers, and fixes that protect everyone who builds Noto after us. Inconsolata is the other version. We found and helped fix two glyphsLib bugs, shipped in 6.16.0, and it still does not build, because 58 of its 74 faces are generated by a script that only runs inside the Glyphs desktop app. You fix the thing you can name, and the next one is still there.
Tools whose parsers are the attack surface. FontForge upstream has fixed four of the twelve late-2025 CVEs and says it does not prioritize reports about untrusted input files, which is defensible for a desktop editor and awkward for anything shipped in a container. One of those four fixes bound only half of the vulnerable function, so the fifth patch is ours. The fix is four lines. Knowing they are the right four lines took a day of building variants and feeding each one a bitmap crafted to sit exactly on the boundary.
Tests that pass for the wrong reason. Those CVE regression tests were green on the first run because every image import was failing in the test environment, including good files. Two missing runtime dependencies meant image import had been broken in our shipped container all along. Every assertion that bad input is rejected now needs a good-input control in the same environment. Otherwise, it is a mood, not a test.
Fonts that are not drawn at all. Noto Sans Duployan is generated by 20,000 lines of Python code that drive FontForge from 2019. Running it on the current release took five generator patches (every dot in the font initially came out empty) and, after the arm64 build crashed where x86_64 did not, a generator that retries each stroke in a forked child. Upstream's own build has been red since August 2026, and nobody noticed, because the release binaries were already on disk, and that is all anyone ever installs. That sentence is the whole problem in miniature.
Fonts with no sources. Droid, IPA, Nanum, WenQuanYi, Bitstream Vera, Carlito, the Arphic collections: the TTF is the only form that exists. These stay as data, classified as such.
Verification that needs a rasterizer. There is no diff for fonts. We built a harness that renders every glyph twice, and we expect to keep it.
Share this article
Related articles
engineeringThis Shit is Hard: Building a vulnerability database from private reports
engineeringThis Shit is Hard: Building forkstore
engineeringThis Shit is Hard: Making FIPS boring, fast, and post-quantum
engineeringThis Shit is Hard: Getting commercial software into regulated environments using AI
engineeringThis Shit is Hard: Factory-scale toolchain management
engineeringWhat it took to reach 1 billion build manifests