Saturday 5 March 2016

The fallacy of self-organising teams

"The best architectures, requirements, and designs emerge from self-organising teams” - Agile Manifesto

I respect the intent of this line from the Agile Manifesto but in my many years of experience in Agile teams I've noticed a common theme.  Yes teams need to self-organise but they always need a leader and teams rarely self-organise without this leadership.

For example...


  • Its natural for techies to scope creep through sheer enthusiasm (or perhaps just taking their eye off the prize) and it's not always obvious to a non-technical product owner when this is happening.
  • Don't expect "the team" to pick up individuals who aren't performing or perhaps are forgetting all of the agreements in the definitions of done.


I've always felt that someone in the team (I'm talking technically here) need to understand the direction they are taking and on occasion be a bit grumpy when they deviate from this.   Yes architecture can evolve but doesn't it help to have someone have some idea of where they are trying to go? 

I've just not seen teams do this naturally without prompt and that's why in a lot of teams there is some level of seniority - perhaps an architect or lead developer.  As much as the manifesto would like everyone to be equal in an Agile team, back in the real world someone needs to stand up and show some leadership qualities.

Perhaps this is symptomatic of having always worked for large enterprises.  The culture of Agile is evolving in this types of places and that takes time.

Its the same with DevOps.  Sticking a load of devs together with a load of infrastructure people together in a room and expecting them to instantly bond and start hugging each other is just not going to happen.  Sometime you need someone to lead from the front and (to begin with at least) TELL them what to do.

Of course, the easiest way NOT to influence someone's thinking is to tell them how to do their job - you have to make them think its their idea.

I catch myself from time to time telling and slap myself on the wrist.  Sometimes, even if I know they are doing the wrong thing you just need to let them get on with it - some people need to learn through failure.  Saying "I told you so" afterwards isn't going to win you any friends either.

Not to say I don't make mistakes myself.  Of course I do but then I'm happy to put my hands up, admit it and learn from it (which is what Agile is about after all).

Certainly our DevOps movement has been top-down rather than down-up as most purists would be prefer and would be interested to hear if this is the same experience in your organisation?

Friday 12 February 2016

Infrastructure as Code - what and why?

One of the core concepts behind our Cloud adoption strategy is that of Infrastructure as Code (IaC).

IaC is commonly associated with DevOps - in fact the two are sometimes (I believe incorrectly) used interchangeably, but I can see how this arises.

Often the greater the need for the increase in pace of change, the greater the need for collaboration, empathy and automation.  IaC can certainly help with automation but what about the other two?

In order to achieve a high level of pace of change all the cogs of the IT delivery engine need to be aligned – there is no point in having high level of application change throughput if you have to wait an age for an Ops guys to make the necessary infrastructure changes to support it.  If you put a Skoda clutch into a Ferrari then you are going to severely dampen the effectiveness of the car...

Let's take a step back and have a look at what IaC actually is first...

What is IaC?
Infrastructure as Code means writing code to manage configurations and automate provisioning of infrastructure in addition to deployments.

This is not simply writing scripts, but involves using tested and proven software development practices that are already being used in application development.  For example: version control, testing, small deployments, use of design patterns etc.

IaC differs from infrastructure automation (scripting), which just involves replicating steps multiple times and reproducing them on several servers.

In other words, IaC uses software engineering principles to produce repeatable infrastructure.


Why use the IaC approach?
  • Its easier to associate, develop, version control and test the application along with the infrastructure that it resides on
  • This aids collaboration between Dev and Ops teams
  • By using similar techniques and approaches to the delivery of change there should be an increase in empathy between both camps
  • Infrastructure delivery is reliable, repeatable and fast
  • Its possible to spin up entire environments and the applications that sit on them for a version control repository.  That's cool on a number of levels - could this help your DR capabilities for example?
Software engineering has been around a long time and the principles are relevant to infrastructure as they are with applications.
    • Why shouldn’t infrastructure be version controlled?
    • Why shouldn’t infrastructure be built and deployed automatically?
    • Why should infrastructure be tested automatically with every change that is made?
Principles and features of our IaC approach

  • Immutable - a change to the application or infrastructure stack triggers a full rebuild (where possible - databases are harder) of the whole environment.  That way we know what is in version control is exactly what is going to be deployed.
  • Full Stack Engineering - we should version control the application alongside
    • The network
    • Security roles, firewalls, authentication mechanisms etc
    • Databases
    • Compute
    • etc
  • Everything is in version control (we use Git and BitBucket)
  • Human readable - all the configuration should be in easy to read configuration files
  • Tested - we test everything automatically using BDD.  For example, run a test to ensure that only port 80 is open on a webserver.  We run these every time someone makes a change through Continuous Integration (we use Bamboo).
  • No changes in the portal - we are using IaC on our Cloud project.  We don't allow any config changes via the Azure portal.  Everything has to be done in code.
  • Hands-free deployment - deployment of infrastructure is completely automated apart from someone pressing the "Go" button