Measured, Not Assumed: Five Times the Obvious Answer Was Wrong

The bugs that ate the most of my time on this site weren't hard. They were hiding behind a perfectly reasonable assumption: a report I trusted, a fix that looked right, or a line in my own notes saying something was done. Every time, the cure was the same boring one. Stop reading the summary and go measure the actual thing.

Here are five of them. None needed fancy tools, just a mildly suspicious streak and a willingness to be wrong in public.

1. The build report said the chat was free

When I added the chat assistant to the site, I checked what it cost the usual way: run a production build and read the First Load JS column. The homepage came in at 211 kB with the chat and 211 kB without it. A whole AI chat for zero bytes. Incredible. Suspiciously incredible.

So I added up the real size of every chunk the homepage loads up front, straight from the build manifest:

Imported the normal way, the chat added 246 kB to every page on the site, for a feature most visitors never touch. Loading it only when someone clicks the launcher brought the extra down to 25 kB, which is just the button and its label.

What I took away: a summary is somebody else's decision about what to count. When it tells you something's free, go see what it left out.

2. Making the shadow darker made the text harder to read

The headers on the contact, work, blog and résumé pages have animated light waves next to them. In my first version the waves ran behind the text, with a dark gradient (a scrim) on top to keep the words readable.

Then I measured it. The muted text over the waves came out at 4.45:1, just shy of the 4.5:1 that WCAG AA asks for. Easy fix, I thought, just make the scrim stronger. I measured again and got 2.33:1. I'd made it worse by making it "better," which is a genuinely impressive way to lose an afternoon.

The waves move, and that's the whole problem. A scrim that looks safe in one frame isn't safe in the next, because a bright crest wanders through whenever it feels like it. Tuning the gradient only changed which frame failed. The real fix was layout: the waves now sit completely beside the text column, never behind it, and a test checks both the layout and the brightest pixel that actually renders under the words.

What I took away: when something's animated, one good-looking screenshot proves nothing. Measure the worst frame, or design so there isn't a bad one to find.

3. My best lead source was me

I'd set up contact-form submissions as leads in Google Analytics, and the numbers looked encouraging. Then I broke them down by hostname, and most of the recorded leads came from localhost. My most enthusiastic prospect, it turns out, was my own test suite.

It came from three sensible decisions stacked on top of each other. The tests fake a successful form submission so they never send real email. The success screen fires a lead event. And the analytics tag only stayed off in development builds, not during test runs or on preview deploys. So every test run counted as a real lead, and as a real ad conversion too. Very flattering. Not very useful.

The fix was small: the tags now load only on the production site and nowhere else. Spotting it took one filter that nobody checks by default.

What I took away: when a number looks great, find out where it came from before you celebrate. Test traffic can look a lot like success.

4. The flaky test wasn't flaky

Three different chat tests failed across three full runs in a row, and every one of them passed when I ran it alone. The easy answer is to shrug, call it flaky, and hit retry until it goes green.

What was really going on: the chat button is rendered on the server, so the test tool could see it and click it before the page's JavaScript had hooked up the click handler. Under light load that gap is tiny. Under a full parallel test run, it was wide enough to swallow a click whole. The fix was a small helper that opens the panel, checks it actually opened, and safely tries again if it didn't. The next two full runs were clean.

A different test had a cousin of the same problem. The WebGL waves from story 2 were heavy enough under software rendering to keep other animations on the page from ever settling down. Capping the frame rate and rendering at half resolution (you can't tell on a soft glow) took it from 4 failures in 24 stressed runs to zero.

What I took away: before you blame flakiness, check whether the same test fails twice. If different tests are failing at random, the real culprit is usually load, and load is something you can measure and fix.

5. "It redirects" meant the homepage only

In September I retired my old WordPress site at damienkomala.com and pointed it here. My notes said the old domain "301s here," and technically it did, for the homepage. Registrar forwarding doesn't keep the rest of the URL, so all 26 old blog links returned 404 for five days while their new versions sat here working fine. Nobody checked, because my notes made it sound finished. I'd basically written myself a confident little lie.

Fixing it took 26 explicit redirects, attaching the domain to the hosting project, and then actually testing all 26 old URLs on live DNS. There was one more surprise along the way: the www certificate didn't cover www, even though the dashboard swore the domain was configured correctly.

The blog images had an even closer call. Most of the old posts loaded their featured images straight from the WordPress server, so when it went away, they all broke. I got them back from a site export I happened to take before the shutdown. That was luck, not a plan, and I'm not counting on getting lucky twice.

What I took away: "done" in your notes is a claim, not a fact. Test the deep links, not just the homepage. And anything you load from a domain you don't control is living on borrowed time.

The habit behind all five

None of these fixes were clever. Each one came from not accepting a comforting answer without peeking underneath it. Measure the real thing instead of the report about it, whether that's the bytes instead of the build summary or the live URLs instead of the note. Go looking for the worst case: the brightest frame, the busiest test run, the deepest link. Pick apart good numbers, because those are the ones nobody questions.

And write down what you measured, right next to the code it applies to. That one matters more than it sounds. Every story here lives in a comment or a project note, numbers included, so the next time something "costs nothing," the receipt is already sitting there.

It's how I like to work on client projects too, on the design side and the build side. If that's what your project needs, say hello.