Debugging with AI: Two Strategies for Finding the Truth

I run two websites from the same codebase: assumptology.com and erotetics.com. Both are static sites deployed to S3 via GitHub Actions, with Cloudflare in front for DNS, HTTPS, and caching. The setup is identical in architecture.

Except for one thing: assumptology.com (without the www) correctly redirected to www.assumptology.com. But erotetics.com did not redirect to www.erotetics.com. Visitors who typed the bare domain got an error page.

I knew the fix would be simple once I understood the cause. The question was: where is the redirect configured, and why is it missing for one domain but not the other?

I asked two AI models to help me debug this. What happened next was a study in two fundamentally different strategies for dealing with uncertainty.

Strategy 1: Reason from assumptions

The first model I tried — ChatGPT 5.2 Thinking — immediately began reasoning about what could be causing the problem. It constructed a mental model of the infrastructure and started generating hypotheses from first principles:

• “It’s probably a CloudFront distribution configuration issue”
• “You may need to set up an S3 redirect bucket for the bare domain”
• “Check if your ACM certificate covers both the apex and www subdomain”
• “The issue might be in your Route 53 hosted zone configuration”

Each suggestion was plausible. Each was technically correct in the sense that any of these could cause the symptom I described. But the model was reasoning from a generic understanding of AWS architecture, not from the specifics of my setup. It assumed CloudFront was involved (it wasn’t — I use Cloudflare). It assumed Route 53 (I don’t use it). It assumed the problem was in AWS (it was in Cloudflare).

The reasoning was sophisticated. The assumptions were wrong.

Strategy 2: Test and eliminate

The second model — Claude — took a different approach. Before theorising, it ran diagnostic commands:

# Check what DNS records exist for both domains
dig +short assumptology.com A
dig +short erotetics.com A

# Check what actually happens when you hit each bare domain
curl -sI https://assumptology.com
curl -sI https://erotetics.com

The results were immediately revealing:

• assumptology.com returned a 301 redirect to www.assumptology.com with Cloudflare headers (server: cloudflare, cf-ray: ...)
• erotetics.com returned a 522 error (Cloudflare could not reach origin)

Two commands, and the hypothesis space collapsed. We now knew:

1. The redirect was handled by Cloudflare, not S3 or CloudFront
2. Both domains used Cloudflare (both returned cf-ray headers)
3. assumptology.com had a redirect rule; erotetics.com did not

The fix was to find the redirect rule in Cloudflare for assumptology (it turned out to be under Page Rules) and create the same one for erotetics. Five minutes, problem solved.

What this tells us about assumptions

The difference between the two approaches was not intelligence — both models are highly capable. The difference was in how each handled what it didn’t know.

Reasoning from assumptions is powerful when your assumptions are correct. If I had been using CloudFront and Route 53, the first model’s suggestions would have been spot on. But the model didn’t check whether its assumptions held. It treated its generic mental model of “how AWS static sites usually work” as ground truth and reasoned forward from there.

Testing assumptions is slower per step but faster to the answer. By running two curl commands, we didn’t need to reason about whether CloudFront was involved, or whether the certificate was misconfigured, or whether S3 was set up correctly. We simply observed what was happening and let the evidence constrain the hypothesis space.

This is a pattern that shows up everywhere in debugging, and in thinking more generally:

• Assumption-first reasoning expands the space of possibilities. Each plausible explanation generates sub-hypotheses, and you can spend a long time exploring branches that don’t apply to your situation.
• Evidence-first reasoning contracts the space of possibilities. Each observation eliminates entire categories of explanation, converging quickly on the actual cause.

The meta-assumption

There’s an irony here. The model that tried to reason its way to the answer was making an assumption about the best strategy for solving the problem: that the fastest path is to think harder. The model that ran commands was making a different assumption: that the fastest path is to look.

In debugging, looking almost always wins. The real world is full of configurations, edge cases, and historical decisions that no amount of reasoning can recover. The gatepost was already there in the wall — we just needed to walk over and see it.

As Yogi Berra may or may not have said: “You can observe a lot by watching.”