Large language models struggle with generating clean code

Explore how large language models struggle with clean code generation, revealing high API misuse and the need for better reliability assessments.

The article discusses a study on the reliability and robustness of code generated by large language models (LLMs) for Java coding questions. The study evaluated four code-capable LLMs, including GPT-3.5 and GPT-4 from OpenAI, and found that they exhibited high rates of API misuse. The study also highlighted the importance of assessing code reliability beyond semantic correctness and emphasized the need for static analysis to ensure full coverage. Llama 2, an open model, performed the best with a failure rate of less than one percent.

Original article: Perhaps AI is going to take away coding jobs of those who trust this tech too much

Featured writing

When your brilliant idea meets organizational reality: a survival guide

Transform your brilliant tech ideas into reality by navigating organizational challenges and overcoming hidden resistance with this essential survival guide.

Server-Side Dashboard Architecture: Why Moving Data Fetching Off the Browser Changes Everything

How choosing server-side rendering solved security, CORS, and credential management problems I didn't know I had.

AI as Coach: Transforming Professional and Continuing Education

Transform professional and continuing education with AI-driven coaching, offering personalized support, accountability, and skill mastery at scale.

Books

The Work of Being (in progress)

A book on AI, judgment, and staying human at work.

The Practice of Work (in progress)

Practical essays on how work actually gets done.

Recent writing

The bully pulpit: why AI slop only matters to people who write about AI slop

This article exposes how the 'AI moral crisis' narrative is amplified by the very people who control media—and why the 90% of workers actually using AI don't share the panic.

Why your job matters more than mine: the selective morality of job loss

This article reveals the uncomfortable pattern behind which jobs get moral protection and which get called 'market forces'—and what that means for everyone outside the creative class.

AI in writing: the end of a professional monopoly

This article reframes the AI writing debate: the panic isn't about creativity—it's about a professional class losing control of the systems they've gatekept for a century.

View all writing →