The fallacy of self-organising teams

"The best architectures, requirements, and designs emerge from self-organising teams” - Agile Manifesto

I respect the intent of this line from the Agile Manifesto but in my many years of experience in Agile teams I've noticed a common theme.  Yes teams need to self-organise but they always need a leader and teams rarely self-organise without this leadership.

For example...

• Its natural for techies to scope creep through sheer enthusiasm (or perhaps just taking their eye off the prize) and it's not always obvious to a non-technical product owner when this is happening.
  
• Don't expect "the team" to pick up individuals who aren't performing or perhaps are forgetting all of the agreements in the definitions of done.

I've always felt that someone in the team (I'm talking technically here) need to understand the direction they are taking and on occasion be a bit grumpy when they deviate from this.   Yes architecture can evolve but doesn't it help to have someone have some idea of where they are trying to go? 

I've just not seen teams do this naturally without prompt and that's why in a lot of teams there is some level of seniority - perhaps an architect or lead developer.  As much as the manifesto would like everyone to be equal in an Agile team, back in the real world someone needs to stand up and show some leadership qualities.

Perhaps this is symptomatic of having always worked for large enterprises.  The culture of Agile is evolving in this types of places and that takes time.

Its the same with DevOps.  Sticking a load of devs together with a load of infrastructure people together in a room and expecting them to instantly bond and start hugging each other is just not going to happen.  Sometime you need someone to lead from the front and (to begin with at least) TELL them what to do.

Of course, the easiest way NOT to influence someone's thinking is to tell them how to do their job - you have to make them think its their idea.

I catch myself from time to time telling and slap myself on the wrist.  Sometimes, even if I know they are doing the wrong thing you just need to let them get on with it - some people need to learn through failure.  Saying "I told you so" afterwards isn't going to win you any friends either.

Not to say I don't make mistakes myself.  Of course I do but then I'm happy to put my hands up, admit it and learn from it (which is what Agile is about after all).

Certainly our DevOps movement has been top-down rather than down-up as most purists would be prefer and would be interested to hear if this is the same experience in your organisation?

Infrastructure as Code - what and why?

One of the core concepts behind our Cloud adoption strategy is that of Infrastructure as Code (IaC).

IaC is commonly associated with DevOps - in fact the two are sometimes (I believe incorrectly) used interchangeably, but I can see how this arises.

Often the greater the need for the increase in pace of change, the greater the need for collaboration, empathy and automation.  IaC can certainly help with automation but what about the other two?

In order to achieve a high level of pace of change all the cogs of the IT delivery engine need to be aligned – there is no point in having high level of application change throughput if you have to wait an age for an Ops guys to make the necessary infrastructure changes to support it.  If you put a Skoda clutch into a Ferrari then you are going to severely dampen the effectiveness of the car...

Let's take a step back and have a look at what IaC actually is first...

What is IaC?
Infrastructure as Code means writing code to manage configurations and automate provisioning of infrastructure in addition to deployments.

This is not simply writing scripts, but involves using tested and proven software development practices that are already being used in application development.  For example: version control, testing, small deployments, use of design patterns etc.

IaC differs from infrastructure automation (scripting), which just involves replicating steps multiple times and reproducing them on several servers.

In other words, IaC uses software engineering principles to produce repeatable infrastructure.


Why use the IaC approach?

• Its easier to associate, develop, version control and test the application along with the infrastructure that it resides on
• This aids collaboration between Dev and Ops teams
• By using similar techniques and approaches to the delivery of change there should be an increase in empathy between both camps
• Infrastructure delivery is reliable, repeatable and fast
• Its possible to spin up entire environments and the applications that sit on them for a version control repository.  That's cool on a number of levels - could this help your DR capabilities for example?

Software engineering has been around a long time and the principles are relevant to infrastructure as they are with applications.

• Why shouldn’t infrastructure be version controlled?
• Why shouldn’t infrastructure be built and deployed automatically?
• Why should infrastructure be tested automatically with every change that is made?

Principles and features of our IaC approach

• Immutable - a change to the application or infrastructure stack triggers a full rebuild (where possible - databases are harder) of the whole environment.  That way we know what is in version control is exactly what is going to be deployed.
• Full Stack Engineering - we should version control the application alongside
• The network
• Security roles, firewalls, authentication mechanisms etc
• Databases
• Compute
• etc
• Everything is in version control (we use Git and BitBucket)
• Human readable - all the configuration should be in easy to read configuration files
• Tested - we test everything automatically using BDD.  For example, run a test to ensure that only port 80 is open on a webserver.  We run these every time someone makes a change through Continuous Integration (we use Bamboo).
• No changes in the portal - we are using IaC on our Cloud project.  We don't allow any config changes via the Azure portal.  Everything has to be done in code.
• Hands-free deployment - deployment of infrastructure is completely automated apart from someone pressing the "Go" button

Back from the future

I'm just back from Dublin where we went to review the Microsoft Azure data centre.

I can't say too much because of NDA but I think it's ok to say 'wow'.

Cloud isn't anything magical and ethereal it's essentially a load of big data centres and some clever software over the top.  However that's really doing it an injustice.

The scale and attention to detail is just astonishing.

If you think you can run an IT operation or data centre better than that you need to visit a padded cell.

A big barrier to Cloud adoption is that of trust.  Do I trust their procedures?  Do I trust their infrastructure? Etc

It's safe to say one of the roles most untrusting or Cloud is that of the Security Manager.  We are lucky enough to have an open minded one that is up for change but even then it's job to distrust...

Think it's fair to say he's placated! Well at least from a physical security and operations point of view.

The place is so process driven to the most minimal of detail.  For example cutting up old disks in millimetre pieces and having multiple pairs of eyes ensure that they are disposed of appropriately.  

Azure data centres are annally retentative.  

In a good way....

This has to be the future - everything else seems amateurish in comparison.

Interesting (ish) blog stats

You are probably a man
You are probably 30-40
You are probably in the USA
You are probably using Firefox
You are probably using Microsoft Windows (would have thought iOS)
You probably found the site via LinkedIn
You probably read an average of 4 blog posts
You probably spend 5-10 minutes on the site before moving off


The blog receives anywhere between 200-500 visits a day which is surprising (to me anyway!).  Especially as I don't post daily and secondly because it's largely a stream of consciousness rather than a well structured piece of journalism.


Not exactly Google scale but each one of those visitors comes to the blogs on purpose (via a Twitter, LinkedIn or devops.com referral).  No one comes via Google.  Excellent SEO :)


That's kind of interesting to me because that's 500 DevOps people (so nicely targeted) rather than 500 member of the general public that find the site by accident and much more than I would have thought.


Thankyou!

DevOps is ALL about culture

Criticisms of defining DevOps with the word "culture"


I've heard a number of lazy criticisms of DevOps recently particularly where individuals talk about culture.

 

"It's fluffy"
"It's in-actionable"
"DevOps is just automation"
"Cultural change isn't relevant in a bank - its a term used by the Facebook's of this world"

 

I think part of this is the continuing confusion over what the term means, along with DevOps being used as an umbrella term for a dozen different other things (Continuous Delivery being the most prevalent).

 

I've talked at length about what DevOps is to me so I'm not going to cover that again.  However, I am going to talk about culture.

 

Let me start with a great definition of what culture is (it's not my quote but I wish it was) - apologies I forget the source.

 

Culture is what you do when there is no one around to tell you what to do

 

I think that's brilliantly simple and bang on the money.

 

DevOps is concerned with bringing two disparate groups together (unsurprisingly Developers and Operations) and encouraging empathy and collaboration between them.

 

Nothing in that talks about automation.

 

The companies with the right culture between Dev and Ops might land as automation as a solution to curing some of the challenges of unshared incentives and goals  However, it doesn't predicate it.

 

A strong DevOps culture might turn to Continuous Delivery as a process solution to meeting their demands for an increased pace of change.  But again it doesn't predicate it.

 

Its really important that people understand the differences and goals between DevOps, Agile, Continuous Delivery etc.

 

They all very closely relate and link but they are different things (take a look here for a glossary).

 

Back to the criticisms

 

Now lets have a look at some of those criticisms of using the word DevOps to represent culture...

 

"It's fluffy" - Culture is closely linked to human behavioural patterns and humans are very different.  So its a fairly loose term but does it make less important?  Not for me.  How your employees and colleagues behave and interact is THE most important thing in a company.  There are a squillion clichés out there about how your employees are the most valuable asset but its absolutely true.  Great people and accomplish great things.  Great technology is nothing without great people.

 

"It's in-actionable" - again this is a very lazy criticism.  Culture is actionable its just hard to change.  Changing the way you recruit, sitting people together in cross-functional teams, not micro-managing, encouraging lunch & learn presentations etc are all actionable ways of affecting culture.

 

Culture is hard to change though as we are dealing with people.  People have different reactions to change and are sometimes difficult to second guess.  It takes time and effort but that doesn't make it in-actionable.

 

"DevOps is just automation" - think I've done this to death but for clarity...the right culture could well lead towards automation as a solution.

 

"Cultural change isn't relevant in a bank - its a term used by the Facebook's of this world" - why is having high performing people in a high performing environment only relevant to the unicorns of this world (i.e. Netflix, Facebook etc)?  Its a daft comment in my eyes.

 

Besides who are the companies earning the big bucks these days?  Oh yeah its not the banks it's Facebook, Google etc...

 

 

Characteristics of a DevOps culture

 

Here are my top twelve characteristics of a DevOps culture.  How far are you off?

1. Measure everything, always
2. Individuals have empathy for the rest of the team (i.e. they don't pass the buck)
3. Shared goals and incentives
4. Don't reward the fire fighter, reward the fire preventer
5. Reward innovation and challenging the status quo
6. Don't punish people when they try something new but fail
7. There is no IT and "business".  IT as much "the business" as the sales people.
8. Seeking to break down silo's
9. "It's not my job" doesn't exist
10. "Its my server/code/network/database" doesn't exist
11. Individuals are empowered to make decisions
12. Top-up management rather than top-down

 

 

I gotta (DevOps) feeling

I recently returned from Atlanta, US and A (as Borat would say), where I was helping set up a new Agile development team.

Actually, they are a little bit more than a dev team - they are one of these product teams I'm so keen on.  The teams owns the full lifecycle of change; development, testing, support, deployment etc all in one team.  The idea is that they start to make decisions as a team rather than individual silos and that they have empathy for the impact of their changes on others.  For example, if a change is poorly tested that is going to affect the velocity of the sprint due to rework.  Or if they develop a change that is difficult to deploy then the team are going to feel that pain.

I think of it as gradle to grave responsibility.  Your job doesn't stop at checkin.  Its only just begun.

Only once software has hit production does it realise any business benefit.

It's not the first time I've set up a team like this but I get the feeling that its going to be a fantastic success.

We are still in sprint zero so how can I make such a bold statement?  Largely its a gut feel but because I've done this a number of times I know what "good" feels like.

That's a bit of cop out though and not very interesting for anyone kind enough to read this blog.

Let's drill into a bit further and have a look why the ingredients of success are there:

Budget/investment in tooling
We are putting together a kick-ass CD pipeline with Bamboo, Jira, HipChat and Bitbucket all very neatly integrated.   Add into the mix Puppet, Urbancode Deploy and a load of other tooling and they have everything they need for a very fast pipeline.

What's more, there is budget for some consultancy if we get stuck or need some training.


Right people in the right team structure
We've been able to cherry pick some of the best people that already work for us and go to market for the rest. This means we can really closely get in the skillets we need "off the shelf" rather than having to train & influence a set of existing individuals.

When you employ people that already live Continuous Delivery, BDD, Git etc then your job is considerably easier.

We've really concentrated on the soft skills of the guys & gals that make up the team too.  Attitude for me is 99% of the job and technical skills 1%.  People with the right attitude will learn the tech.  The culture of the team is absolutely important.

Some banking type told me this week that culture is just fluffy and that its just about tooling and automation.  I bet you £10m that this team will be more successful than his.

Its all about culture.  Tooling and automation comes out of that culture.


Agile training
Teaching a development team the Scrum process is actually pretty simple.  What is often neglected though (I'm guilty of this one) is what it means to the business stakeholders outside of the core dev team.  In my experience, business users often want to assert some requirements and then walk away.  They also want the full product and the idea of Minimal Viable Product is a hard pill for them to swallow.

In an Enterprise you also get a million and one different stakeholders all fighting for their piece of the pie.  Having to prioritise the backlog is difficult when you have different stakeholders from different parts of the business and with a different management hierarchy.  You end up with stakeholders trying to reserve story points in a sprint and other work arounds.  This is a bad place to be in.

However, we've (the royal we) have spent considerable effort and time spending time with the business stakeholders training them in Scrum.  How does prioritisation work?  What are the sprint ceremonies?  Why should the Agile team be self managing etc etc

The fact that we spending the first month or two on technical setup (CI processes etc)and not on business change gives me confidence that the stakeholders "get it".

Aligned goals (minimal number of stakeholders)
Agile is easier the fewer people that are involved trying to pull the team in different directions.  We are lucky that we have a very senior stakeholder that is able to arbitrate and protect the team for the prioritisation noise and that has a clear direction of what they want to achieve.


Self organising
Agile teams need to be protected from outside interference so that they can concentrate on the task at hand.

Too often have I seen other interfere with the running of an agile team and decisions made on their behalf.

Go away!  Leave them alone!  The Agile team best knows how to organise themselves for optimum performance.

I see really good signs that the team aren't being micro-managed and are protected from the scary, outside world with a good Product Owner.

As long as the results are there, the stakeholder don't seem to care too much about how those results are achieved.  That seems simple but too often people hang on too tightly thinking that it will give them control and will help ensure success.  Its actually the opposite; don't strangle the team, let them breathe and hey will get you where you want to go.

Again this is aided but not having too many cooks (stakeholders).

Don't believe what I tell you

I presented at the Computing DevOps summit in London yesterday which was great and I met some really nice people.


There were some really interesting presentations from individuals and the usual bombardment of tooling choices from vendors.


Most of the people I spoke to at the event came from the "where do we start" camp with adopting DevOps.  It surprises me how immature the idea still is across the Enterprise - there are far more that haven't even started vs. the ones that are "doing" it.


I listened to a few panellists speak and a couple of presentations but it struck me that after a few hours no one could arrive at a common definition of "What is DevOps"? 


If we can't define it then how on earth can we put a process together for adopting it?


Before my session I asked the audience to stand up and put their hands on their heads which they obligingly did without question.

The pic is a bit dark but it amused me to get people to do this!  Power Hungry PowerPoint Fiend.


The point of this though was not to "follow the leader".  DevOps is a cultural idea - what it means to you and how to implement it will change between different companies.


For many, the pursuit of DevOps is in response to increasing the pace of change.


For some its about increasing quality.


Or it could be about reducing costs.


The point is that you need to work out what you are trying to achieve and you need to land on your own definition of what DevOps is.


Listen to me and others.  But don't blindly follow us. 


DevOps is an ideology so its open to interpretation and that's a good thing.