Lean-Agile
Coaching and Training
Other
-
- Also some Useful Tools
Note that we have some Incomplete Pages That Require Work on this site:-)
Note that we have some Incomplete Pages That Require Work on this site:-)
History is that Scaled Agile has purchased licensing rights to Icon (Mark Rix) material and with be releasing a course based on this shortly. There are a couple of differences in approach but basics are the same. Differences include:
From introduction:
“In this two-day course, you’ll gain an understanding of the role of DevOps in a Scaled Agile organization. Unlike other training that focuses only on the mechanics of DevOps, Icon’s DevOps for the Agile Enterprise course explores the role of DevOps in the context of the entire IT value stream, and prepares you to successfully plan and implement DevOps capabilities that significantly extend the benefits of Agile transformations. This includes understanding the full- spectrum software delivery ecosystem, where Agile transformations typically are and are not effective; how Agile and DevOps are “better together;” and what people, processes, and technology must be aligned throughout the organization, to achieve true enterprise agility.
Objectives
Teacher: Dan James
Support: Adam Beck
Base thinking is “agile is good at developing code but this does not reduce time-to-market”. Time to market is the business need. “Agile gets developed; DevOps gets deployed”.
Goal: New code has opportunity to go all the way to release.
Difference between development and operations:
Agile @ Scale (Concept to Code) | DevOps (Code to Cash) |
---|---|
Upstream | Downstream |
Focus: Design & Build | Focus: Package & Deliver |
Analog: New Product Development | Analog: Manufacturing |
High uncertainty | Low uncertainty |
Change is welcome | Change is NOT welcome |
Output is non-deterministic / unpredictable | Output is deterministic / predictable |
Lean underpinnings | Lean underpinnings |
Development and operations mindset start at odds to eachother as development is all about change, and operations is all about stability.
Definition from Wikipedia: https://en.wikipedia.org/wiki/DevOps Note: this has changed from definition given in course:
“DevOps is practice that emphasizes the collaboration and communication of both software developers and other information technology (IT) professionals while automating the process of software delivery and infrastructure changes. It aims at establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably.”
No one way of doing it.
Software is eating the world
Example: Amazon
Example: Facebook
2016 State of DevOps report details benefits
Principles:
But actually implemented more as SMLAC (as culture comes last)
Start with Sharing
Break-down silos, visualize how the work is really done today
Form dedicated cross-functional team(s) to address
Know all your customers
Include third parties
Inspect and adapt based on what you are seeing
When you have visualized the work you will get benefits:
Then measure so that:
Agree of common (SMART?) objectives / targets
Record baseline
Inspect and adapt
When you measure the work you will get benefits:
Now lean out (streamline) the process
Focus on
Batch reduction: “WIP is a surrogate for productivity in the minds of managers” - watch “One Piece Flow Versus Batch Production”: https://www.youtube.com/watch?v=JoLHKSE8sfU
“We want coders to test, and testers to code”
Mature teams must do a few of the XP practices
When you do this you will get:
Now automate the process to maximize speed and quality
Don't automate anything until you understand the process and the process is sound
Often lowest hanging fruit is to automate the acceptance testing
When you do this you will
Which should lead to competitive advantage
Now use the outcomes to drive widespread cultural change
Over share the vision and the wins
Focus on
Aim is to get products to market faster by building a deployment pipeline
Metric: Deployment Lead Time
This is why we focus on making work visible, reducing batch sizes and WIP
Build quality in - do not allow defects to flow downstream
Cost of defect:
This is hard - want it set up so that we have steps 1 to step n always flow in one direction (and never loop backward)
This means you need to measure % complete and accurate at each step to understand where to focus
Theory of constraints quote “In any value stream, there is always a direction of flow, and there is always one and only one constraint; any improvement not made at that constraint is an illusion”
Note: from class you really should not move to the second way until you have the first way done.
Goal is to increase value delivered by carrying lessons learned into the next round
Make sure everyone is informed of issues
Business metrics drive this:
This is where we use A/B testing (ie experiment with 2 outcomes and you set the system up, say, so that one group of users see one screen, and another group of users see the other and you use data to understand which meets the business need more effectively)
Inspect and adapt
Goal here is to create competitive advantage and a self-healing organization
We want experimentation and risk taking (safe environment)
Scientific method - “hypothesis driven development”
Self-sabotage as a tool (chaos monkey and its brethren)
Taking many small risks, not one big risk
Quotes from “The DevOps Handbook”
Business objective: maximize development productivity through continuous integration.
Agile context: working software every team increment (team / sprint DoD)
Development responsibility. Aim is to deliver bug free tested code into next cycle. Called “continuous integration” in the slides.
Aim is to be able to do this at least once every 2 weeks. Note: this is lazy - should be able to do on every “story”
Sample / generic build workflow:
Workflow Step | Primary Patterns | Helper Patterns | Measures | Trigger | Example Tools |
---|---|---|---|---|---|
Build | Build automation; Version control | API-driven development; Code reviews | Frequency (how often to build); Cycle time (how long to build) | Code commit | ANT, Maven, MSBuild, Make, … |
Unit test | Test automation | Code reviews; TDD | Test coverage; Frequency; Cycle time | Successful build | JUnit, Unit, … |
Integrate | CI System; Gated (automated) commits | Trunk based development; static code analysis; modular architecture | Frequency; Cycle time | Successful unit tests | Jenkins, Bamboo, VSTS, CircleCI, TeamCity, … |
Acceptance test | Test automation | Environment configuration; Service virtualization | Failure rate; Cycle time; Frequency | Successful (CI) integration run | Selenium, FitNesse, VSTS |
Business objective: maximize delivery efficiency through continuous delivery.
Agile (well SAFe) context: supports effective system demos and, when done right, means you have potentially shippable features every system increment.
General target discussion. You have a certain amount of time it takes you to push a hot fix through, say 48 hours. If this is the case then you should set things up so that all code can run through this at “hot fix” speed, at least initially. Ask yourself “how do you make this happen?”
Sample / generic workflow steps:
Can (should) be all done in parallel.
Workflow Step | Primary Patterns | Helper Patterns | Measures | Trigger |
---|---|---|---|---|
Integration testing | Test automation | Deployment automation; Test data management; Environment configuration automation | Failure rate; Cycle time; Test coverage | Successful acceptance test |
Regession testing | Test automation | Deployment automation; Test data management; Environment configuration automation | Failure rate; Cycle time; Test coverage | Successful acceptance test |
Exploratory testing | Session based testing? | Deployment automation; Test data management; Environment configuration automation | Failure rate; Cycle time; Test coverage | Successful acceptance test |
Performance testing | Test automation | Deployment automation; Test data management; Environment configuration automation; NFR's | Failure rate; Test coverage | Successful acceptance test |
Business objective: maximize speed to market though continuous deployment.
Agile (SAFe) context: release level definition of done. Goal is to be able to release on demand.
Note: you can have a released system that the customer never sees.
This is all about production, so there are no “mocks” in this part of the cycle
Workflow step:
Workflow Step | Primary Patterns | Helper Patterns | Measures | Trigger | Example Tools |
---|---|---|---|---|---|
Configure | Automated provisioning (infrastructure as code); Teams self-service | Version control; Cloud computing; Technical standards | Cycle time (one day or less) | Successful test cycle | Chef, Puppet, Ansible, AWS, Azure, … |
Stage | Automated environment configuration | Deployment automation; Blue / green deployment; Demo | Cycle time; Frequency (at least bi-weekly) | Successful test cycle | |
Deploy | Deployment automation | Automated environment configuration; Self-service | Cycle time; Frequency; Lead time | Customer readiness | Octopus Deploy, MS Release Manager, CA Release Manager, UrbanCode, XLDeploy, etc … |
Verify | Test automation; Automated rollback | Selective deployment; Version control | Cycle time (desire 30mins or less including rollback); Failure rate; Rollback cycle time | Production deployment |
Business objective: maximize business value
Agile context: operationalize critical feedback loops. Deployed solutions are viable and have sufficient business value (and we are measuring it)
Workflow steps:
Workflow Step | Primary Patterns | Helper Patterns | Measures | Example Tools |
---|---|---|---|---|
Monitor | Telemetry | Data collection; visual displays | % of stack monitored; Custom SLAs and KPIs | Dynatrace, App Dynamics, New Relic, Splunk, … |
Respond | Proactive detection (work to prevent) | Alerts and notifications; Server reboot avoidance; Cross-team collaboration (original team fixes problems) | Mean time to restore (MTTR) | |
Stabilize | Business continuity | Failover / disaster recovery; Cyber security; Design for operations | Outage frequency; Emergency change % | “Simian Army” |
Deliver | Business alignment | Product adoption; Transparency; Continuous improvement | (Business) customer satisfaction; (Business) return on investment |
Immutable architecture: From https://thenewstack.io/a-brief-look-at-immutable-infrastructure-and-why-it-is-such-a-quest/
“Immutable simply means something that is created and left unchanged. Immutable things are understood as is, without need to mutate them, and by that have no need to be mutable. Infrastructure of course is word taken from the domain of architecture and physical design of space, for our virtual realms of servers and software applications. Combined we have a created and unchanged set or piece of architecture for hosting applications.
Unchanging infrastructure might sound like the opposite of what you’d want in an agile environment. But what it means is that once the image is working, only a working image is deployed. When it’s time to make changes a new is created, but the previous image is still available for rollback of the environment itself. The fact that we can now create exact, versioned, timestamped versions of an entire environment removes troubleshooting broken instances almost entirely. And thanks to OS-level virtualization, which is spearheading the movement for immutable infrastructure, these images are extremely fast to deploy.”
Netflix's Simian Army (see https://medium.com/netflix-techblog/the-netflix-simian-army-16e57fbab116)
Need to align your objectives and deployment pipeline to the value streams, and you might need to find the these first
Then position DevOps to streamline the Code-to-Cash process
Then analyze the current deployment pipeline and measure:
See exercises
DevOps is an enabler at the portfolio level
Like most organizational transformations need to establish a transformation team:
Value is a function of timely adoption, not raw capabilities. Focus on value. Go all in. Rack up wins. Over-communicate.
Strategies:
The automation paradox “Automation is essential to DevOps but the wrong tools can bury you in a hurry. Don't be hypnotized by shiny objects.”
Develop the DevOps backlog. For example:
PDCA.
Never be satisfied.
What kinds of things should you put in version control? Everything:
Assuming you have an idea of your value stream and now want to understand the deployment pipeline assoicated with that value stream:
Cycle | KPI | Baseline Value | Target Value | Source of Truth |
---|---|---|---|---|
Build | Build frequency | |||
Build | Code review % | |||
Build | Merge to trunk frequency | |||
Test | % Acceptance tests automated | |||
Test | % Integration tests automated | |||
Test | % Regression tests automated | |||
Release | Environment setup cycle time | |||
Release | % operational assets in version control | |||
Release | Deployment lead time | |||
Release | Mean time to resolve |
Build | Test | Release | Transition | |
---|---|---|---|---|
Goal | Continuous integration | Continuous delivery | Continuous deployment | Continuous value |
Roles | Developer, Tester, Build engineer, Database developer, Architect | Tester (end to end), Tester (NFR) | Configuration manager, System administrator, Release engineer | Production support, System engineer, Network engineer, Security engineer, Operations engineer |
Skills | Trunk-based development, API-driven development, Code review, Environment configuration, Build automation, Continuous integration, Test driven development, Test automation, Service virtualization, Mocking / stubbing, Feature toggles, Application telemetry, Cyber security, Version control | Test automation, End to end testing, Environement configuration, Test data management, Version control | Automated provisioning, Environment configuration, Automated deployment, Selective deployment, Blue / Green deployment, Automated rollback, Version control | Monitoring (full stack), Incident management, Problem management, Disaster recovery |
Like the structure of this presentation - practices to “business why”. Some of the mapping might be a little artificial but nice way to hang it all together.