In short
- Annex IV of the AI Act lists 9 numbered points of technical documentation. Point 2 alone breaks into 7 sub-points covering development method, architecture, data and validation.
- Article 18 requires providers to keep that documentation for 10 years after the system is placed on the market or put into service.
- ISO/IEC 42001 adds 38 reference controls in 9 areas, each of which an auditor will ask to see evidence for, not a policy describing it.
- The recurring finding is not a missing control. It is a control that cannot produce an artefact with a date on it.
- Point 4 of Annex IV asks why your performance metrics are appropriate, which is a question most model cards do not answer.
What do AI assistants say about your organisation?
Model answers are becoming a channel your communications team does not control. EntityRise audits how assistants describe your company and whose name they give instead. From $19.
An audit is not a test of whether you have thought about a risk. It is a test of whether the thinking produced something with a date on it.
That is why the same finding recurs across organisations with good intentions and thorough policies.
01What does Annex IV actually demand?
Nine numbered points, and the second one carries most of the weight.
| Annex IV point | What it asks for | Artefact that closes it |
|---|---|---|
| 1 | General description, versions, interfaces | System description with version history |
| 2, sub-points a to g | Development method, architecture, data, validation | Design record plus dated test reports |
| 3 | Capabilities, limits, accuracy by group | Evaluation results disaggregated |
| 4 | Why these performance metrics are appropriate | Written justification, signed |
| 5 | Risk management system per Article 9 | Risk register with review dates |
| 9 | Post-market monitoring plan per Article 72 | Plan plus evidence it ran |
Point 4 is the one that catches mature teams. Reporting an accuracy figure is straightforward. Explaining why accuracy is the right measure for this system, and why the alternatives were rejected, is a written argument that nobody produces unless asked.
02Why does the same finding keep recurring?
Because a policy describes intent and an auditor is testing operation.
A procedure stating that every model is validated before release is a control description. The evidence is a validation record for the model currently in production, dated before the release, naming who ran it. Organisations that fail here usually do the validation. They do it in a notebook that was not kept.
The rule of thumb worth adopting is that a control which cannot produce an artefact does not exist for audit purposes. It may still be a good control. It is not evidence.
03What does the retention period change?
It changes where the documents live.
| Record type | Minimum retention | Provision |
|---|---|---|
| Technical documentation and QMS records | 10 years | Article 18 |
| Automatically generated logs, provider | 6 months | Article 19 |
| Automatically generated logs, deployer | 6 months | Article 26(6) |
| Declaration of conformity | 10 years | Article 47 |
Ten years outlasts the tooling, and it dwarfs the six month floor that applies to the logs themselves. It outlasts the ticket system the design decision was recorded in, the wiki that was migrated twice, and usually the team. Documentation that lives in the tool the engineers happen to be using is documentation that will be unavailable at the moment it is needed.
The practical answer is unglamorous: an exported, versioned, human readable pack per system, refreshed at each material change, stored where records are stored rather than where work happens.
04How does ISO 42001 evidence differ?
It is broader and shallower, and it is about the system that produces the systems.
The 38 reference controls in Annex A of ISO/IEC 42001 span AI policy, roles, resources, impact assessment, lifecycle, data, information for interested parties, use of AI systems and third-party relationships. An auditor will sample them and ask for evidence of operation over a period, not for a single artefact.
That is a different question from Annex IV, which is about one product, and the distinction is the practical reason the two frameworks are not alternatives. An organisation can hold the certificate and still fail to produce the technical file for a specific high-risk system, because nothing in the management system audit required it to exist.
05What should be assembled first?
The two artefacts that unblock everything else.
An inventory of AI systems with an owner and a classification for each, because every later question is scoped by it. And, for each system classified as high risk, the dated evaluation record, because it is the input to Annex IV points 3 and 4 and the one that takes longest to reconstruct after the fact.
Everything else in the pack can be written in a week. Those two are the ones that cannot be written retrospectively without inventing something, which is the line an assurance function should not cross.
06Frequently asked questions
How long must AI technical documentation be kept?
Ten years. Article 18 requires providers of high-risk AI systems to keep the technical documentation, quality management system documentation and conformity related records at the disposal of national authorities for ten years after the system is placed on the market or put into service.
What is in Annex IV?
Nine numbered points: a general description, a detailed account of elements and development process, monitoring and control information, justification of performance metrics, the risk management system, lifecycle changes, standards applied, the declaration of conformity, and the post-market monitoring plan.
What is the most common audit finding?
Evidence that exists only as a policy. A documented procedure describing how models are validated is not evidence that a specific model was validated, and the artefact that closes the finding is the dated record of the run, not the procedure.
Does a model card satisfy Annex IV?
Partly at best. A model card typically covers the general description and some performance information, and rarely covers the development process sub-points, the risk management system, lifecycle changes or the post-market monitoring plan.
07References and method
- Regulation (EU) 2024/1689, Annex IV, for the nine numbered points of technical documentation and the seven sub-points under point 2.
- Regulation (EU) 2024/1689, Article 18, for the ten year documentation retention period, and Article 47 and Article 72 as referenced by Annex IV points 8 and 9.
- ISO/IEC 42001:2023, Annex A, for the 38 reference controls across nine control areas.
- Observations about common findings are editorial, prepared by The Guardrail from publicly documented certification requirements. Practice varies between certification bodies and we mark that where it matters.