ISO 9001 — a QMS built, then audited through the API

Anyone can show a quality management system to someone who already knows where everything is. The interesting test is the opposite one: can a system be audited — by someone working from queries, without a guided tour, who has to establish what the population is before sampling it?

So a complete QMS was built for a fictional manufacturer, and then audited against ISO 9001:2015. Deliberate gaps were seeded during the build to see whether the audit would find them.

What was built

A library of 65 requirements, R01–R65, mapping the clauses of ISO 9001:2015 — the criteria, held as items so they can be linked to, inherited from, and audited as documented information in their own right.

Then Aureon Smart Systems, a fictional company making smart radiator valves and a connected hub: its scope document, its processes, its products down to component level, its records — management review, internal audit programme, nonconformity and corrective action, design and development, monitoring and measurement.

The company is typed against the requirement library, which turns the checklist into a compliance register: 69 requirements in scope, 98 rows once roll-ups are counted, reconciling exactly.

The audit

65 requirements verdicted:

The seeded gaps were found. But the result worth reporting is the one nobody planted.

The finding that was not seeded

Of the 36 minor nonconformities, 16 were "content present, link absent".

The QMS held a record that would satisfy the requirement. Nothing was missing. But no edge connected the record to the requirement it answered, so conformity could not be established by query — only by someone happening to know the document existed and remembering to go and look.

That is a failure mode a register in a spreadsheet structurally cannot see, because in a spreadsheet the connection between a requirement and its evidence is somebody's memory. Here it showed up as a countable, addressable gap.

A real certification audit would aggregate many of those into a handful of system-level findings — documented-information control, context and interested parties. They are listed one per requirement here because the exercise scored per requirement.

Working the way an auditor works

The audit surface is built around the questions an auditor actually asks:

• What is the population? Listing, always with a total, because a list of nine with no denominator is indistinguishable from the first nine of three hundred.
• Did the record move while I was reading it? Reads pinned to the opening meeting, and a drift report since any date.
• What is this database, exactly? A provenance block naming the file, the counts, the last write and the configuration revision.
• Can I show what I examined? An append-only journal of coverage — written beside the database, not inside it, and recording what was read rather than what came back.

The auditor's guide ships inside the product, at /info/audit, along with an engagement checklist to settle before the opening meeting. See auditing.

What it did not do

The claim under test was "the audit can be conducted from KF alone". The honest answer is partly.

Every requirement was reachable and every verdict was reached from KF data. But three things had to come from outside it, and one of them was load-bearing: judging whether evidence is sufficient is not a query. KF can show you that a management review record exists, is linked to the requirement, and was written on a date inside the audit period. Whether its content actually demonstrates that top management reviewed the quality system is a human judgment, and no amount of structure removes it.

That is a limit of auditing, not of KF — but a product page that implied otherwise would be lying to you.