Intro

The “invention” of Architecture Decision Records (ADR) is attributed to Michael Nygard.

Particularly he addresses the following:

One of the hardest things to track during the life of a project is the motivation behind certain decisions. A new person coming on to a project may be perplexed, baffled, delighted, or infuriated by some past decision. Without understanding the rationale or consequences, this person has only two choices: Blindly accept the decision…

with all the negative consequences… or..

Blindly change it. … changing the decision without understanding its motivation or consequences could mean damaging the project’s overall value without realizing it.

ADRs initials focus is also coined around better understanding of the project history and continuity of the project vision. Michael Nygard recognizes a value in “additional work” put in extra documentation, if that leads to better avoidance of either blind acceptance or blind reversal of architectural decisions. So he describes lean doc and lean process.

I’m personally started to deepen in ADRs when i was charged with the organization of “architecture process” in and organization that tried to be agile. In that context the Decision-making via the “Advice Process” by Andrew Harmel-Law helped me to connect the dots and his description become the ground for our version of the ADR process back then.

In this article i’ll share my observations, learnings, real world ADR examples and the details of an existed ADR process.

What are Architecture Decision Records (ADR)?

Before we deepen, here is the small refresher.

ADRs are lightweight documents that capture important architectural decisions along with their context and consequences. They typically include:

  • Context: Why was the decision needed?
  • Decision: What was chosen and why?
  • Consequences: What are the trade-offs and impacts?

This structure makes it easy to understand the rationale behind a decision, even months or years later. ADRs prevent knowledge loss and promote continuity when team members leave or new ones join.

ADR example

Let me share a real world (anonymized) ADR, that was first of a kind, in a agile team that has not really had any experience with ADRs before.


## 001 - SOPS with KMS

Status: ACCEPTED
Last Status Date: 3 Jan 2022
Driver @Jan
Contributors: @Jan, @Alexander, @Chris

## Context and/or Problem Statement
We need a concept of working with secrets in our new K8s clusters. We would like to be able to provide well understood and simple to used and to maintain a solution that still provides an industry level of security.
## Decision
We go for SOPS. KMS should be managed by AWS. SOPS should bve used for Kube installed secrets.

### Consequences
SOPS will be responsible for encryption and decryption of secrets we need to have in our K8s cluster. 
 (-) We understand that SOPs cli has to be installed on (local) systems where it has to be used for encryption
 (+) However, it's natively supported by Flux, b that there isn't no much to do, to get started.
 (+) Encrypted Secrets can be freely checked in to git repos. No extra storage to consider. 
 (++) No secret keys are exchanged. The keys and the fine-grained control of them is managed by AWS KMS, by that we adhere to best security patterns. 

## Options Considered
 
### 1 SOPS with KMS
KMS is managed by AWS, keys management is completely solved. No runtime services or other moving parts needed.
Compatible with Flux (GitOps Operator). But SOPS client is needed, to provision keys.

### 2 Self hosted Hashicorp Vault
Vault is very popular and feature rich.
However it would require versatile additional running components and their management. It would require process definition around of Vault (+ disaster recovery, + hardening, + auditing etc.)

### 3 Pure AWS secrets
 
* (+) Convinient 
* (-) not cost efficient for us. 
* (?) too much legacy in the used components.

## Advice
 
* 9 Jan 2022: @Jan: We have gathered some experience with SOPS on their projects already. I favor its simplicity and lack of need for any infrastructure. 
 as well, Flux docu suggest SOPS.

* 10 Jan 2022: @Chis: The simplicity of SOPS is interesting, but i doubt some of our devs would like to install cli. Also documentation efforts are nto mentioned. I do expect some confusion among devs. But im fine to try 1 or 2. I Also see the cost of 3 not justified.

* 20 Jan 2022: @Alexander: The feature set of Vault is superior, however i cannot detect the match and need of of those in our context. Our Problem is very simple atm, we would like to eliminates any plain text secrets shared openly or committed into the codebase. The secret infrastructure needs to be robust. Option 1 seems to answer that best.

In this example we learn about our decision for SOPS. I recall that this decision went relatively easy and super fast. So the ADR doc is more for capturing the motivation and context for the decision (Basic ADR motivation)

How it evolved

As our organization has grown in team count and in the number and quality of ADRs some experienced and motivated engineers started to use new challenges arose. My company has hired many engineers wit very different cultural and engineering backgrounds. I’ve started to face a lot of questions and comments.

This radically liberal and inclusive process, which not only allowed but in some cases even demanded broad participation in architectural decisions, divided the engineers.

Engineers that have experienced agile and empowered team before, who also had ideas on their mind, quickly realized to use ADRs as a leverage to pursue their ideas. Other felt unsure or stayed in unbelief that the ADRs written decisions will be applied. According to their socialization, there still need to be CTO, Architect, or someone with a title to approve the ADR and make sure this is implemented. This topic is very interesting and i’m not going to deep into Engineering Culture aspect in this article.

At this point, i just want to document the more Formal internal process description that should help the engineers with answering the details, but also be written and more official by that.

Example ADR process

# ADR process in a nutshell


The AD Process is the core element of a decentralized approach to architecture. Let’s define it like this:
    1.  Anyone (or any team) can make architectural decisions.
    2. Before making the decision, the decision-maker must consult two groups:
        The first is everyone who will be meaningfully affected by the decision.
    The second is people with expertise in the area the decision is being taken.
    That’s it.
    While decision-takers are in no way obliged to agree with the advice the folks in these two consulted groups give them, they must seek it out, and they must listen to and record it. We are not looking for consensus here, but we are looking for a broad range of inputs and voices.

That’s it.
While decision-takers are in no way obliged to agree with the advice the folks in these two consulted groups give them, they must seek it out, and they must listen to and record it. We are not looking for consensus here, but we are looking for a broad range of inputs and voices.

## Process in Steps
1. Create a new (empty) ADR with ADR Template in your team ADR track.
2. Give your ADR a number and Title and set the status to  PROPOSED.
3. Describe Problem Statements / Context.
4. Describe solution options you see that can mitigate the problem.
5. Provide Pro and Contra arguments for your options.
6. Actively seek advice and record advice you have received or ask people to write it directly in your ADR. You don’t have to agree to it, the arguments don’t have to convince you at this point. If you like, call for feedback for your ADR in #adr Slack channel (up to you how to gather advice  ).
7. Depending on feedback arrange a meeting to clarify disagreements.
8. After you’ve received the needed advice take your decision and edit the status of your ADR to  ACCEPTED.
9. Let other people know about the accepted ADR in #adr Slack channel (could be interesting to other teams and people that were not part of the process).

## ”Where to get advice” checklist
* Don’t be shy to request more advices! There is no veto rule with them.
* Get advice from ambassadors (not necessarily team leads) of the teams who are your consumers.
* Get advice from people who elaborate on their answers, you learn form them.
* Get advice from people who would disagree with you - you will discover new consequences potentially.
* Is API affected? --> Check with Staff Engineering league
* New Microservice considered? –> check with the Staff Engineering league
* Security Considerations?  -> Speak to Slack#it-security
* Product/UX change needed? -> Speak to PMs.
* Infrastructure affected? -> speak to Cloud Infra Team
* Data Pipelines Affected -> speak with #data-platform-public
* Dev Tools affected? Speak to Dev Team…
* Unsure? -> Speak to Staff Engineers league

## How to Give Advice
Read the Context Problem Statement. Do you understand it?
If the problem statement is not clear, please approach the ADR author or try to improve on your own.
Write what you think in the Advice section
Be guided by
 * _Architecture Principles_ 
 * it’s more than fine to add new Options 
 * Yes, you should challenge Pros and Contra if you disagree.
 
## How is the decision actually taken?
The driver of the ADR is the one who drives the decision. As said there is no veto in the process, you’ll will see if there is a strong argument in the advice and it for sure will influence you. However, you take the decision and describe all consequences you see…
 It’s fine to ask for help with that. Try to ask

As you have eventually realized by the very first passage our ADR Process version goes hand in hand with Andrew Harmel-Law.

Since this process was driven by me, id like to share my believes. On general level i think that empowered, well equipped teams of professionals can do a lot of architecture inside of their respective teams. Generally an empowered and self-organized team should be able to know best when in needs external support and where it’s just external dependency that just slows down or offloads responsibility to someone else. Architecture is not different. Architecture happens on many abstraction layers and big portion of architecture can for sure can and should be driven inside Engineering teams. This is not only grounded in my personal experience, but meanwhile we have good data out of research work done e.g by Google.

However, Engineering team do not exist in vacuum and team overarching concerns do exist. More generalized how to make sure the org (teams in the org) working on “the right stuff” is topic in itself. Staying with architectural concerns and this article, let me just add the following. ADRs have been working well on the overarching context between teams. Of course this ties even more to your company culture, you need to have working communication between teems.

Often we’re not in charge of changing the organizations or cultures, let’s keep it aside and focus on Engineers.

Why Engineers profit form Architecture Decision Records

If you still reading, at this point i should have been convinces you, that ADRs are more than just documentation. Combines with process/rituals it becomes powerful communication tool that has all capabilities to foster a culture of transparency and team alignment. It helps engineers leverage their knowledge while ensuring that decisions are well-informed and collaborative.
I do remember a feedback from an engineer, that sounded like:

“Despite years of experience, I’ve found that the best decisions come from informed collaboration, not so much from heroic selfish work.”

Experienced engineers often have the technical intuition to make decisions quickly. However, they also understand that:

  • No decision is made in isolation. It impacts multiple systems and teams.
  • Documenting decisions reduces misunderstandings and assumptions.
  • Informed collaboration leads to better outcomes than solitary decision-making.
  • Better acceptance of the decision outcomes. The participation of engineers in the decision-making process, who have to deal with the consequences of the decisions, leads to better acceptance.

By using ADRs engineers transform decision-making into a team sport, leveraging collective knowledge while maintaining architectural integrity.

ADR Tool

Lastly I was refreshing my Golang skills and have build a Simple console ADR tool. I would be happy if you give a try with your next project.

Have you used ADRs in your team? Share your ADRs and experiences or questions in the comments below!