Large language models struggle with generating clean code

Explore how large language models struggle with clean code generation, revealing high API misuse and the need for better reliability assessments.

The article discusses a study on the reliability and robustness of code generated by large language models (LLMs) for Java coding questions. The study evaluated four code-capable LLMs, including GPT-3.5 and GPT-4 from OpenAI, and found that they exhibited high rates of API misuse. The study also highlighted the importance of assessing code reliability beyond semantic correctness and emphasized the need for static analysis to ensure full coverage. Llama 2, an open model, performed the best with a failure rate of less than one percent.

Original article: Perhaps AI is going to take away coding jobs of those who trust this tech too much

Featured writing

Why customer tools are organized wrong

This article reveals a fundamental flaw in how customer support tools are designed—organizing by interaction type instead of by customer—and explains why this fragmentation wastes time and obscures the full picture you need to help users effectively.

Infrastructure shapes thought

The tools you build determine what kinds of thinking become possible. On infrastructure, friction, and building deliberately for thought rather than just throughput.

Server-side dashboard architecture: Why moving data fetching off the browser changes everything

How choosing server-side rendering solved security, CORS, and credential management problems I didn't know I had.

Books

The work of being available now

A book on AI, judgment, and staying human at work.

The practice of work in progress

Practical essays on how work actually gets done.

Recent writing

Dev reflection - February 20, 2026

I want to talk about the difference between execution and verification. Because something happened this week that made the distinction painfully clear, and I think it matters far beyond software.

Dev reflection - February 18, 2026

There's a moment in any system—a team, a company, a workflow—where the thing you've been optimizing for stops being the constraint. And you don't notice right away. You keep pushing on the old bott...

Dev reflection - February 17, 2026

I want to talk about staging areas. Not the technical kind—the human kind. The places where work goes to sit. The inbox you check before forwarding. The draft folder. The approval queue. The meetin...

View all writing →

Related thinking