Before AI Gets Regulated, Software Engineering Already Has a Rulebook

Imagine two pieces of software fail at the exact same moment. One is a hospital's dosage calculator. The other is a game app that tracks your high score. Both crash. Only one of those failures could hurt someone. That distinction, obvious as it sounds, is the entire premise behind a decades-old engineering standard that deserves far more attention as governments scramble to write rules for artificial intelligence.

The standard is IEEE 1012, and it has quietly guided software quality assurance for years. At its core is a simple but powerful idea: not all software carries the same risk, so not all software should face the same scrutiny. The standard sorts systems into integrity levels, a kind of risk rating that combines two factors, how bad the consequences would be if something went wrong, and how likely that failure actually is.

Think of it like building codes. A garden shed and a hospital wing are both "buildings," but nobody expects them to meet the same structural requirements. The shed might just need four sturdy walls and a roof that doesn't leak. The hospital wing needs seismic bracing, fire suppression, backup power, and inspections at every stage of construction. IEEE 1012 applies that same logic to software. A integrity level assessment asks two questions: if this fails, how severe is the harm, and how often might that failure actually occur? The answers to those two questions, plotted against each other, tell engineers how rigorously a piece of software needs to be tested, documented, and verified before it ships.

That framework matters right now because artificial intelligence systems are being deployed across sectors with wildly different consequence profiles, often under a single, blunt regulatory approach. An AI model that recommends movies and an AI model that helps triage emergency room patients are not the same kind of risk. Treating them identically, whether through overly loose oversight or overly heavy-handed rules, fails both the public and the industry trying to serve it.

A Framework Already Tested in High-Stakes Systems

What makes IEEE 1012 particularly relevant is where it has already proven itself. This is not a theoretical academic exercise. The standard has been applied in software verification and validation for systems where failure has real consequences, the kind of engineering environments where a bug isn't just an inconvenience but a potential catastrophe.

The genius of the consequence-and-likelihood matrix is that it forces engineers to be honest about risk instead of guessing. A system with catastrophic potential consequences, think loss of life or major environmental harm, gets assigned the highest integrity level regardless of how rare the failure might be. Meanwhile, a system where failures are both minor and unlikely can be verified with a lighter touch. This isn't about being lenient with dangerous systems. It's about not wasting scarce engineering and regulatory resources on systems that don't need the same level of scrutiny.

For AI specifically, this kind of tiered thinking solves a problem that has plagued early regulatory proposals. Blanket rules that apply the same testing and documentation burden to every AI system, regardless of application, tend to either overwhelm low-risk innovation or, worse, underprotect the public in high-risk domains because everyone is drowning in the same paperwork. A consequence-and-likelihood approach lets regulators and engineers calibrate. Higher stakes demand higher integrity levels. Lower stakes get proportionate oversight.

That said, mapping AI onto this framework isn't a plug-and-play exercise. Traditional software, the kind IEEE 1012 was designed for, behaves predictably. You can trace a bug to a specific line of code. Machine learning systems, especially large, opaque neural networks, don't offer that same traceability. Their failure modes can be subtler and harder to predict, which complicates the "likelihood" half of the equation. Engineers and regulators adapting this standard for AI will need to grapple with that uncertainty rather than paper over it.

There's also the question of who decides. Assigning an integrity level requires judgment calls about consequence severity and probability, and those judgment calls can be shaped by industry pressure, limited data, or simple underestimation of edge cases. A standard is only as good as the rigor and independence of the people applying it. Without proper oversight, a consequence-and-likelihood matrix could become a checkbox exercise rather than a genuine safeguard.

Still, the alternative, regulating AI with no risk-tiering at all, is worse. Right now, much of the public conversation around AI regulation swings between two poles: either AI is treated as an existential threat requiring blanket restriction, or it's treated as harmless software deserving no more scrutiny than a spreadsheet app. Neither extreme reflects reality. Some AI applications genuinely are low-stakes. Others, in healthcare, transportation, criminal justice, and infrastructure, carry consequences that demand the kind of rigorous verification IEEE 1012 was built to enforce.

Why It Matters

The people who will feel the effects of how AI gets regulated are not abstract stakeholders. They're patients relying on diagnostic algorithms, drivers trusting autonomous vehicle systems, and job applicants screened by hiring software. When oversight is too loose, these are the people who absorb the harm when systems fail. When oversight is too rigid and undifferentiated, innovation that could genuinely help these same people gets strangled in unnecessary red tape.

IEEE 1012 offers a way out of that false choice. It's not a perfect fit for AI as written, and adapting it will require real technical work to account for the unpredictability of machine learning systems. But the underlying philosophy, that risk should be measured honestly and oversight should scale with consequence and likelihood, is exactly the kind of grounded, engineering-first thinking that AI regulation badly needs. Lawmakers reaching for a framework don't have to start from scratch. Software engineers have been solving versions of this problem for decades. The tools are already sitting on the shelf.