Why Does Software Keep Breaking?

Software keeps breaking because modern systems depend on a long chain of external assumptions—APIs, frameworks, services, and layers of abstraction—that can change independently. Even if each dependency is “likely” to keep working for a year, the combined probability that everything still works drops fast as the number of calls grows. The result is a practical reality: maintaining software becomes less about building features and more about constantly repairing breakages.

A central point is quantified with a simple probability model. If a single external dependency has a 90% chance of remaining compatible after a year, then code that relies on multiple such dependencies becomes fragile quickly; the chance the whole system still works after time declines roughly exponentially with the number of dependencies. Even when assuming an unusually high per-API reliability—like 99%—the overall outlook still looks bad once software uses multiple services. The takeaway is not that one API is unreliable, but that typical software stacks are too interconnected for “mostly stable” to be stable in practice.

That fragility creates knock-on effects beyond annoyance. Developers feel like they’re drowning in constant churn: libraries evolve, services get deprecated or replaced, and teams spend time shuffling components to prevent collapse rather than doing satisfying engineering work. This also helps explain why AI coding tools are attractive—when the environment is punishing and repetitive, automation feels like relief. But the criticism is that AI is often being used to patch over a broken ecosystem rather than fixing the underlying causes of instability.

Security is framed as the most serious downstream risk. When systems change constantly, vulnerabilities spread through unfamiliar combinations of frameworks and services, and defenders struggle to know what to secure or even where the attack surface lives. A concrete example comes from an AI-assisted login flow: when asked about session security, the assistant suggested using JWTs, and the discussion highlights how easy it is to miss the right security questions unless someone already knows what to look for. In a world where AI-generated code is widely copied, the “where did this exploit come from?” problem becomes harder—unlike past framework vulnerabilities where the community can track and patch known issues, AI-produced patterns could embed weaknesses across many codebases without clear provenance.

The conversation also raises a strategic fear: cheaper software creation can increase the volume of malware and attacks. If AI reduces the cost of building code, it may also reduce the cost of building insecure or malicious code, intensifying the cat-and-mouse dynamic between exploit developers and defenders. Even if AI improves defensive coding, training and deployment cycles may lag behind attackers who can iterate faster.

Finally, there’s a more optimistic thread about education. AI can help learners by providing explanations and guidance, but the best learning outcomes may require AI tutors designed to avoid handing over complete answers—encouraging students to struggle productively, debug, and build understanding. The overall message is a balancing act: AI can reduce friction for learning and some engineering tasks, but the broader software ecosystem’s instability—and its security consequences—still demands serious attention.