How Does The Role of Architect Change in Agile?

Premise

The role of “Architect” in an agile transformation sometimes causes confusion. Traditionally there has been a need for someone to not just worry about the current features to be delivered but to also look ahead and understand how they can position the Teams so they can work coming requirements. There has also been a need for consistency in an overall enterprise offering so that customers of that offering feel like the solution could have come from one person, to reduce the amount of re-invention we have to do, and to ensure we have clean implementation regarding non functional issues such as security, performance, and so on. That need does not go away just because we now have empowered Teams. In fact, it could be argued these issues are even more critical in the new world.

Note: This page does not try to define the role of “architect” in its various forms. This is more a discussion on the changes in thinking we need to have to be a successful agile architect.

Intentional and Emergent Architecture

Early implementations of agile often seem to hide the need for good design. Most of the time the assumption was that “if you have good people then good design will follow”. The Agile Manifesto argued that “The best architectures, requirements, and designs emerge from self-organizing teams.” This has proven to be true - there is a huge benefit in having the Teams do design as they do the work assuming Teams with strong technical skills. This worked for many situations, but as you had more and more Teams contributing to the solution, or as people started to apply agile into more and more existing IT situations, it became increasingly clear that something more was required. The question then became “how do we leverage this emergent property of design, while also keeping overall guidelines in place and doing some level of look ahead planning?”

SAFe in particular recognized the problem and developed the notion of intentional architecture in addition to emergent architecture. One of the key ideas was that for a Train program that the System Architect works with the Teams to build an “architectural runway” of new “enablers” that enables the Train to develop new capabilities better, faster, and cheaper.

The problem with this distinction is that, for many, the idea that we need “intentional” architecture is an excuse to say “the role of the architect does not really change” but this misses the point. The discussion must be more nuanced than this. A couple of examples:

  • Lets say we are about to embark on a significant re-design / re-factor of an existing legacy, “hairball” application that is worked by multiple Teams. In this situation if we just have each Team go off and refactor at will, the application will probably not work for a long period. Some level of alignment across Teams, with driving principles and approaches needs to happen to reduce the risk to the application. One the other hand, an architect deciding that “here are all the interfaces” to develop to might miss key issues and functionality that only the people working the code can see. What approach might we take. A first step might be simply to have all Teams adopt a common function / variable naming convention across the application and use the IDE to enforce this so the real structure of the application become more clear. So the first principle might be to establish this approach, and to work with the Teams on making it happen.
  • Lets say we are about to select a foundation technology to build our new capability on. In this situation, if we don't have some degree of commonality, then we could up with an application that does not talk to itself. The traditional approach is to have the architects determine the best foundation right at the beginning of the project, and ensure that the Teams comply with this approach. There is no doubt that a single solution is required, but agile approaches use the collective knowledge of people to generate data to come up with the “best” solution. The architectural approach in this case might be to work with each of the Teams on a different foundation technology (so multiple experiments) and then use the data generated from those experiments to make a decision as late as possible (“the last responsible moment”)

From these example you can see that architects need to adopt a different approach. Part of the problem is caused by the traditional way architects try to address these issues. The mindset of command and control applies in this space as it does in many others. So an architect will issue guidelines and demand reviews of designs, for example. From the perspective of the Teams, Architects are often perceived as providing guidelines that do not work in reality and as the place where innovation stops as architectural approval is required, and that is often slow in coming. It’s not that Architects want to be in this position; they are senior people working hard. But that is how it works out.

What we are trying to get to a more collaborative approach for this role, just like we are with any leadership role. Like all agile, there is a mindset shift required from Architects as they a work with agile Teams:

  1. Collaboration: According to James Coplien, the mindset required is the same as all agile and lean - “Everybody, all together, from early on.“ The idea is that instead of treating the job as if it can be independent of the work, Architects need to be involved from the beginning all the way to end. This is interesting in that again, per James Coplien “Maybe half of software development is about nerd stuff happening at the whiteboard and about typing at the keyboard. But the other half is about people and relationships.” This is a significant change for many Architects.
  2. Architecture as a Product or Service: We need to position Architects so that they are seen to provide a valuable service to trains and their “product” is the service of helping with good design etc. We have to encourage architects to ensure that their “products” are attractive to customers (Teams on Trains) so they want to use it, not required to use it “because we said so”.

And we are leverage key agile ideas to help us improve the result. From the examples above:

  • De-centralized decision making: Understand when decisions need to be centralized versus de-centralized.
  • Keeping options open: To maximize the data we have to make a decision and so reduce the risk of that decision.

So how does this work practically?

How Do We Leverage Collaboration?

One of the frustrating aspects of being an architect is knowing that there is a particular approach or technology that will help the Teams but also facing resistance from the teams in actually implementing that idea. Through a collaborative approach you will find there is help in getting some of these ideas to move. This is, in fact, one of those places where there is a significant benefit to having a large scale organization to work with.

In my early career as an agilist for a product development shop, we knew there was a benefit to doing test driven development but for a number of reasons (feature pressure, concern of the unknown, disbelief in the approach, …) we were unable to get teams to really bring the practice into their world. No amount of presentation, cajoling, threats made any kind of impact. Then I found a team that had actually done it, and were very pleased with the results. We had that team talk about their experiences and suddenly all kinds of teams were now adopting the practice.

It turns out that, no surprise, people listen more to the peers than anyone else. And in a large organization there is probably someone who has tried whatever practice or technology you are interested in working. Architects who are open to these experiences and work directly with teams are best positioned to leverage these peer effects.

How Do We Work on the “Big” Architectural Initiative?

The base recommendation is that, where possible, you split large chunks of work into smaller ones but focus on still providing end user value even when you do the split. In other words as you divide epics / feature / stories up ensure there is some valuable functionality with each as you deliver. This contrasts with the approach a typical technical organization would take when given a large amount of work to do where the instinct is to split the effort along architectural boundaries (data access layer, middle tier, client tier) or components and then build all the real functionality over once the base layers are in place. The problem with the traditonal architectural approach is that:

  • Architectural Development needs to continue for a long time before anything is provided to an end user.
  • Architectural Development will often proceed in isolation, so that integrating all layers happen late, adding risk.
  • There is no proof that the architecture will work in the context of the end user requirement until the end when end user functionality is exposed.
  • There is a tendency to do “technology for technology sake” resulting in wasted effort that does not provide value to end users (“but we might need it”) which means we add to the future maintenance and support burden.

In other words we need to resist dividing the epics / features / stories up along “technical” boundaries and components as this will result in iterations of development with nothing complete (and valuable) to the end user, waste, and added risk to the schedule.

Rather we need to deliver new architecture in a useable context. So instead of building all the architectural components and then combining them together to get something done for the end user, you build a thin sliver of functionality that passes through all the layers and components and determine how to pull together all the systems required to make this happen as you go. Even if the end user cannot actually do any real work at this stage, you can get feedback from the customers. The value here is “learning” but that learning is not “this is how the widget works” but rather “this is how all these widgets work together to deliver value and how do can use / deploy it”.

When building in a DevOps world, this approach has been extended and labeled as the Walking Skelton approach - see Kickstart Your Next Project with a Walking Skeleton. And then there are various “riffs” on this walking skeleton approach.

As these implementations are put in place, the people doing the work should keep certain principles in mind. Principles include things like “single responsibility”, common naming / set up for interfaces, rule for interfaces (eg you cannot delete a public interface, only add to), etc, etc. This is where architecture role really adds value and this is why they will need to collaborate through the whole implementation process.

How Do Architects Ensure Consistency?

Let’s take the example. One of the roles of architects is to establish the guardrails by which the organization (enterprise, train, team) operates. However, we need to be careful that we say role is to just “establish guardrails”. This is a problem in that:

  1. It is a negative job from the perspective of a person on a Train (ie architects say “no”).
  2. It does not allow for creativity from the teams (ie “we want to try this”)
  3. It means architects are a potential bottleneck (eg needs approval)
  4. Its not too far from what people think of as architecture today (ie an ivory tower where commandments come from)

Instead of this approach, part of the new job of the Architect is to work directly with Teams to help them understand why the guardrails are in place and, as Teams work solutions, to work those new ideas into the guardrails and distribute those ideas to others.

How Does the Architect Interact With the Product Management?

Traditionally architects develop an independent understanding of what the customer needs. This is best developed by direct conversation with customers, but in many instances is developed independent of that discussion, and also independent of the overall product strategy.

This also needs to be a more collaborative effort and so need to set the expectation that Architects are involved directly with customers through PM / PO discussions (not separate). If there is a customer meeting, then the architect should be involved. After all, one way to consider the establishment of the “architectural runway” is that the architect, having heard what customers are thinking of longer term asks themselves the question “what do we need to put in place so that our teams can deliver these upcoming requirements better, faster, cheaper?”

What Are the Benefits of Good Architecture?

From the above, and to paraphrase Scott Ambler, the benefits of agile architecture can be summed up using these three words:

  • Better. An agile architecture enables agile teams to produce better quality solutions for their stakeholders by providing a more reliable ecosystem with which to work.
  • Faster. An agile architecture enables agile teams to deliver solutions to market faster due to improved reuse and infrastructure quality.
  • Cheaper. An agile architecture enables agile teams to deliver solutions to their stakeholders at a lower cost due to improved reuse, greater quality, and greater platform consistency.

Perhaps a more succinct way of saying this is that the architect’s job is to focus on “design for deployability” based on doing the work aimed at increasing the flow of value.

These benefits need both intentional and emergent to fully leverage the combine intellect of the people doing the work.

Want To Know More?

You could leave a comment if you were logged in.
  • /home/hpsamios/hanssamios.com/dokuwiki/data/pages/how_does_the_role_of_architect_change_in_agile.txt
  • Last modified: 2019/01/08 15:53
  • by hpsamios