What to document so you can answer “What will this change affect?”
https://iceshore.ai/en/blog/a-p2-1/
On this page
- Reading existing code tells you the structure, but not what the system does for the business
- The unit that can answer impact questions is the use case
- Do not over-document
- A good fit for the age of AI
- If your development is outsourced
- When AI drafts and people correct the draft, the AI's understanding changes too
“Will this change stop tomorrow's appointment booking?” When someone asks you that, the code alone often cannot answer it.
Scanning the code tells you the system's structure, but what that structure does for the business is buried in documents outside the code and in the developers' memory.
This article shows what you should write down, as documentation, to fix that. If your development is outsourced, also see the section “If your development is outsourced” later in this article.
The short answer: what you should write down is use cases. Describe, in business terms as far as you can, who triggers each one and why, which resources the request passes through in order, and what information ends up where. If you keep records at this level, finding the use cases that include the resource you are changing gives you a list of what the change might affect.
Reading existing code tells you the structure, but not what the system does for the business
What you can extract mechanically by scanning code is which function calls which, and which tables they reference. What you cannot extract is what those calls accomplish for the business.
Take an appointment system for a clinic. Say there is a process that updates one row in the appointments table. Whether it is part of the front desk staff's appointment management or part of patients' online booking is not written anywhere in the code. In practice, it may be called from both.
You are asked, “Will this change stop tomorrow's appointment booking?” but a call graph can only tell you, “This function is called from three places.” That is why you need a record, written in business terms, outside the code.
The unit that can answer impact questions is the use case
What you increase here is not the amount of records but the unit of record. A use case is one continuous storyline: from the moment someone does something, through the resources the request passes in order, to where some information ends up.
A format for writing records at this level is already published as open source. It is ADL (Architecture Description Language), a specification for writing architectures as JSON text. Reindeer Technology Pte. Ltd. publishes it under the Apache License 2.0, and it does not require a contract for any particular product (<https://github.com/reindeer-project/ArchitectureDescriptionLanguage>). Because the specification is public, files written in this form stay readable even after your contract with any system that uses them ends.
The ADL root object has six required fields, and one of them is useCases. A single use case has four required fields: a name, a description, trigger (the event that starts it), and traffics (its path).
In trigger, you write the moment it starts: who does what. There are two kinds: a user's web access and a timed action. traffics lists the path from there one segment at a time, and the start of each segment points to the trigger before it or to the previous segment. If a request goes from CloudFront through API Gateway to Lambda, you create three segments and chain them in series. This example appears word for word in the schema's own description.
With records in this form, you can trace an impact question in two directions. Find the segments that end at the resource you are changing, and you get the use cases that depend on it. Follow the chain back to its start, and you get the people and the business affected if it stops. References connect the single line you changed all the way to the business.
Each segment must also reference an information type, and an information type has a confidential flag and a personal-data flag. Following these flags is enough to see how far the segments that carry personal data extend. When someone asks about impact outside the company, this is the first place to look.
Do not over-document
ADL's resources holds only references to external files. It has two fields, a kind and a reference, and you can choose from seven kinds, such as CloudFormation and Terraform. The reference must be a .json, .yaml, .yml, or .tf file. Instance sizes, environment variables, and IAM settings do not go here.
Because it is a structure of references, you do not need to copy what is already written in your CloudFormation or Terraform definition files. That keeps down the cost of maintaining what you wrote and of resolving discrepancies.
The ADL schema's character limits also serve a purpose. A use case name can be up to 70 characters, and its description up to 500. Five hundred characters are enough to say why that piece of business exists and who is affected if it stops. The limits act as a mechanical check against writing too much.
On the other hand, ADL does have quite a few required fields. A web access trigger has 17, and each segment of a path has 12. Most of them are numbers for estimating performance and cost, such as the daily active rate and page size; the fields a person has to decide come to about five. If all you need is to answer impact questions, you can put rough estimates in on your first pass and move on.
A good fit for the age of AI
As agent-driven development becomes the norm, let AI write the ADL draft and have people fix only the parts that look doubtful. Each ADL node has an optional field called _meta that holds three things: the author, the time of the last update, and a confidence score. The author is either ai (written by an AI agent) or h: followed by a user ID (a human note written by a person). The confidence is a number from 0 to 1 and only means something for values the AI generated. A node without _meta is treated as created by AI with a confidence of 1.0.
What makes this design work is that you can tell afterward where a person stepped in. When a person fixes a segment, its author changes to the h: form. The next AI that reads this architecture can also tell which parts are estimates and which are human decisions. The same marks help when you read it again yourself six months later.
Where to draw the line between AI and people follows directly from the facts at the start of this article. AI fills in what can be inferred from the code: how resources connect, port numbers, and whether traffic goes over the internet, because the definition files give grounds for those. What AI cannot fully fill in are the descriptions of the use case, the trigger, and the segments, plus the two flags on the information type. AI cannot say with confidence, from the code, whether personal data travels along this path. That part has to be a person's call.
Two things tripped us up when we let AI write it. The validator caught both.
info.versionis an integer. It is a revision number that starts at 1, and it is not yours to choose. The tool increments it by 1 every time you save.info.authorsis an object, not an array. Its keys are author identifiers, and each author requiresrolesandjoinedAt.
If your development is outsourced
The steps so far assume you can reach the definition files. If your development is outsourced, that assumption does not hold. In that case there are three things to do, and the contractor does the hands-on work for all of them.
- Add ADL to the list of deliverables. The language specification is public and does not require a contract for any particular product. The files you receive stay readable after the contract ends.
- Have the contractor's AI write the draft. Most of the structure can be taken from the definition files.
- When you receive it, read only five fields yourself: the descriptions of the use case, the trigger, and the segments, plus the confidential and personal-data flags. These are not fields the contractor should decide.
The third point is the key one. Only the side that owns the business can decide these five. Put the other way, of everything the contractor delivers, the client only has to review five fields, and that is fixed from the start. You do not have to read the whole thing and relearn it.
One question is enough when you read: does the description say who is affected if it stops? If it does not, it is still just a copy of the structure.
When AI drafts and people correct the draft, the AI's understanding changes too
Finally, a word on who these records are for.
IceShore is a configuration management platform that prompts your AI to organize use cases in this form and hands the result back to your AI. Because the format is centered on use cases, the AI's draft starts from the business side. People fix it from the same point of view. Every fix leaves the author's mark. Over time, the business-side information your AI can refer to grows.
It is your AI that answers impact questions. What IceShore provides is the material.
The steps up to having your AI draft its first use case are in “From zero to your AI's first answer, without signing up: connect, check, and ask.” No signup is needed. (Some ways of using IceShore, such as adding the connector on the claude.ai site, require signing up.) Once you are connected, start by asking, “Does this use case description say who is affected if it stops?”