Spiderman and Azure

I've been really impressed by Azure at the TechEd conference in Barcelona this week.  We are starting our Cloud journey in 2015 so it's been great to see how far things have moved on in the last year.

Because I'm a geek I signed up to the free Azure trial (http://azure.microsoft.com/en-us/pricing/free-trial/) and had a play around for an hour.

In 1 hour I created a load balanced pair of web servers and a geo-located SQL cluster (i.e. one node in Europe and the other in the US).

1 hour.

That's ridiculous.

I'm not sure we even have the technical skills to do something like that little lone the time it would take (literally months).

1 hour from a cold start with no real experience of doing it before.

So that's all very good.

However, earlier in the week the excelled David Chappell (http://www.davidchappell.com/) talked about the Cloud and how with great power comes great responsibility (like Mr Spiderman).  Couldn't agree more.

If its that easy to do and if you aren't careful, your company will be spinning up machines all over the shape and things will quickly get out of control.

Just because you can spin infrastructure quickly doesn't necessarily mean you should do...

One of way of getting around this might to be to put some kind of dashboard over the top of the Azure capabilities (I think System Centre can do something like this) which automates builds and adds in things like approvals.  I really don't like having to nanny individuals but the reality is that if you let people build servers and environments manually (as well as install applications) things will get out of sync pretty quickly.

And your credit card bill will be substantial (ooops I forgot to turn off those 100 test servers I built last year).


Automating Azure


I'm not speaking from a point of great experience so my thoughts on Microsoft's automation approach might be entirely misguided.  Apologies Microsoft is this is not the case but  I'm not convinced their automation approach for Azure is right.

I'm a big fan of idempotence, infrastructure as code and desired state.

I can talk about the above 3 for quite some time so I won't bore you here (if you aren't already!).

Microsoft's approach is heavily reliant on PowerShell for Azure automation jobs.  These jobs are basically script and therefore contain logic and without a very skilled developer they are not idempotant.  Moreover, its difficult to version control these scripts as it seems you enter the PowerShell code directly on the Azure portal (probably wrong here!).

Being a procedural script its difficult to see what the impact on script v1.1 has over script v1.0.

I would turn to Puppet but it doesn't have the maturity to hook into the vast Azure API.

So I'm going to be a little stuck with PowerShell for the time being.

PowerShell is very powerful but really I just want to define the state of an environment/server and let the software worry about how to get there (a la Puppet).

In order to make the migration from on-premise to Cloud (or use Cloud for DR) we also aren't just talking about VM's.  What about VLAN's, F5 load balancers and all the other stuff that makes up an application stack?

Puppet can define these and if I could potentially use the same manifests for on-premise and the cloud this would be very exciting.  The ability to abstract away implementation into a XML manifest is the dream.

I'm not sure if that's a deficiency in Puppet or Azure (or both).  Either way there is another opportunity for Microsoft here.

So please Mr (or Mrs) Microsoft extend Desired State Configuration so it can cover more than just servers and allow those same manifests to cover private-cloud and public-cloud.  It needs to be agnostic of whether you are using VMware, Hyper-V, F5, Alteon etc.

Write once, run everywhere.
   

GatesOps

Love-hate
It might be super-unfashionable to say but I have to admit to being somewhat of a Microsoft-phile over the years. 

I've made a decent living from consulting around their technologies over the last 16 years for starters (thanks for paying the mortgage Mr Gates!).

In any dealings with them I've had they have been nice individuals and they have gone out of their way to make me feel welcome.

Not to say I don't have any criticisms.  Quite the opposite.

Anything to do with licensing and Microsoft then just forget it. 

I also hate how far they are behind the curve nowadays - always picking up what others have already started;  GUI operating systems, Cloud computing and Tablets are just a couple of examples.

DevOps?
This brings me on nicely to DevOps.

I'm currently in Barcelona at their annual TechEd event and took part today in their DevOps focus group.  Surprisingly for such a "focus group" that I was really the only one around the table that had any kind of matured DevOps culture at work (and really we are still only beginners tbh). 

Whether this is representative of their customers I don't know. What it is says about their customers I also don't know...

Anyway that is a bit disappointing, I'm always keen to hear from other adopters especially ones that are further down the path than we are.

I digress.

It strikes me that Microsoft are in a fantastic position to sow the market place right up.  They have tools and technologies that range from cradle to grave, from requirements analysis, through to development, testing, provisioning, release, support etc...

Add in that they also cover on premise, hybrid and cloud solutions then you have the vast majority of the application life-cycle and deployment scenarios.

I can't think of any other companies that are in the same privileged position.

However, they aren't embracing this and they are missing a big opportunity.  If they want to be the premier Cloud player then they need to make it easy for people to get from idea in someone's head to the production space in a fast, repeatable and easy fashion.  Anyone can provide a load of compute power.  The real differentiator is being able to facilitate the whole pipeline; laptop to production.

There are a lot of different tools along their chain - the vast majority of which are very Microsoft-centric meaning its difficult (or at least harder) to integrate your favourite technologies if they aren't from Microsoft.

Tools that don't integrate become silos of information and reduce visibility and transparency across teams;  ultimately leading to reduced collaboration.

The sheer number of different tools and solutions makes it difficult to know where to start. 

Do I use TFS or GIT?  Or both? Should I put System Centre in before and get the Ops and provisioning bit right first?  Oh hang on I want to use the Azure pack but I use VMware?

Prompt head explosion.

My boss said something today and he's quite right.  If we knew were to begin with all this stuff and it arrived out of the box then we'd probably buy everything Microsoft have.  It's just too darn hard and too darn complex.  Microsoft seem to expect that you are Microsoft only and are on the very latest everything.  Not realistic.

I hate this term but if there was a "single pain of glass" across your SDLC with the ability to be agnostic of technologies then Microsoft would be onto a winner.

With a Microsoft "something" orchestrating the life-cycle we could integrate what we have right now and eek out the non-MS stuff over time rather than having to blow our data-center up and start again.

Credibility

If Microsoft were really embracing DevOps then they wouldn't be as late to the game as they are and so uninsipring in some of the their technologies. 

For example the release tools in Visual Studio 2013 are so behind the likes of UrbanCode Deploy it's a bit embarrassing.  PowerShell DSC (Desired State Configuration) is so far behind Puppet its redic (as they say in Essex).  Why would I just want to configure my Windows servers when Puppet can control network devices and a whole range of other stuff too?

I think its the same as a personal trainer. 

I wouldn't ask a fat personal trainer to train me.

If you don't live and breathe it then you can't be credible for trying to sell it.

P.S. yes Microsoft I am available to help you put this right in exchange for a small island or something like that (I hear the Bahamas are nice)  :)

Service virtualisation

You may have heard of Service Virtualisation (SV) recently.  It's an idea that seems to getting some traction especially in the Continuous Delivery space.

Not to be confused with Operating System virtualisation technologies such as Hyper-V and VMware.

My personal definition of SV is the productionisation of stubbing (granted this is a bit simplistic).

Technically SV can mock too but I didn't really want to get into a discussion of what is a mock and what is a stub :)

Imagine you have an application that is a composite of multiple other systems you have in your company.  It retrieves data from a web service, reads a file from an FTP site and runs a SQL query against a database from a different application and imports the data.

That's potentially problematic as all these systems needs to be available.

What happens if the people running the web services want to take it down for maintenance?  Does that mean you can't develop and test your application?

What happens if you don't have all these interfaces all the way through your environment stack?

What happens if you are reliant on a change that isn't going to be ready for another 10 weeks in one of these interfaces?  Are you going to be blocked from development for 10 weeks?

These are all the kinds of questions that Service Virtualisation hopes to address.

It provides an abstraction of other systems (such as SOA platforms) or even resources such as flat files you might have not have access to.

It can record the interactions between your code and the other system so that it's easy and quick to create the stubbed interface.  Commonly SV applications also provide an interface so you can change the values in these recorded stubs easily through the GUI.

We are looking at implementing something over the next couple of months for one of our key systems.  In a complicated SOA environment with logic managed by different teams and in different geographic locations it's really killing their pace of change.  They are unable to test until a full stack is in place (system test) which can be days/weeks until after they have made the original change.

You really want very short feedback loops between dev and test.  Ideally none at all and the developer can do a lot of testing using automated regression tools on his/her local developer machine.

Service Virtualisation should help them accomplish this rather than the dev, deploy, test cycle that they are currently going through.  It should really increase the pace of change and expect some big cost and efficiency gains.

Daddy DevOps

I found out this week I'm going to become a Dad.

It's a strange mix of emotions from excited, to terrified, to panicked and happiness.

Not too dissimilar to undertaking a big new project, scoping out how complex it is, getting close to go-live and then delivering it :)

[if any one you say I should use Agile processes for childbirth go to the back of the class - having a baby is definitely Waterfall]

At the moment I have time on my hands, which is probably one of the reasons I blog.  Time also gives you the room to think and ponder.  And because I spend all day, every day talking about DevOps I tend to think about it even in my downtime (and sleep) which is fairly plentiful despite a busy day job.

With my partner away this weekend, bored of playing on the Xbox and all my friends busy/married/parents, I've done a fair bit of sitting around of late.

I got around to thinking about how DevOps might compare to parenthood and I think there are some good analogies there.

My goals of being a good Dad are:

1. Ensure that the family is provided for
2. Ensure the health of my child
3. Ensure that the baby is happy
4. Ensure my partner is happy and supported
5. Ensure that the family are safe

My partners goals are probably very similar so we shouldn't have to worry about pulling in different directions too much.

So that's cool, we have 2 different parties (Mum and Dad) who's goals and incentives are aligned and will work together to accomplish them.  The missus might not know it, but she's soon to be living DevOps!

I want to be a good Dad (and partner) and I've been thinking about what I can do as an individual to contribute towards our shared common goals?  And how do I make it easier for my partner to achieve her goals?

Commonly, DevOps teams turn to automation in order to break down the silos between them and meet their common goals.

If I could automate the changing of a nappy that would help!

I'm not sure I can automate nappy changes (I'd be very rich if I could).  Does anyone know if Puppet have a Pampers module?

    
    nappy { '/baby/bottom/':
      ensure => clean,
      owner  => 'parents',
      type => 'huggies',
      source => 'crib',
    }

Alright so I'm going a little off topic here.  My point being that we have two different resources who goals are the same (i.e. ensure the health and happiness of little Johnny).

The stereotyped Dad in the 1920's that went to work and had little to do with bringing up the child is a good example of a silo.

I've gone to work, it's your responsibility to look after the child - it's not my problem

Apart from the fact I want to be involved and I want to help my other half, I'm incentivised to help as much as possible because if I contribute then ultimately the happier we'll all be.

That's DevOps.  Changing your behaviour in order to meet shared goals.

Super excited.  Lets just hope the delivery isn't Continuous.

Motivations of the techy

I spent an hour the other day with the team discussing what their personal motivations were for working at the company.

The team is a mix of contractors and permanent staff.

The idea was two fold. Firstly, by understanding their motivations I would be a better manager and could help incentivise them in the right way. Secondly, by revealing their ambitions they would be motivated to produce metrics which would help them achieve their goals.

These metrics are key to proving the business value of the team and will help prove its success and extend its influence (which are my personal goals).

I assumed that everyone (especially the contractors) would say money is a key driver for them. I was wrong.

Everyone and I mean everyone, talked about company culture as one of the prime reasons they work where they do.

Interesting.

It makes sense I guess. If it was all about $$$ then they might be doing more lucrative and money focused roles. Maybe that's too simplistic? I guess people (including myself) want it all but are more prepared to sacrifice money in the search for happiness.

I've only ever worked in an IT role but I know bankers, brokers and finance types and for the majority of them it's about the bucks.

No real end point to what I'm saying except to say you need to understand what motivates your team in order to incentivise them. That sounds super obvious but it was a little light bulb for me.

Finding the right consultancy is hard

There are lots of different types of IT consultancies; big, small, modern, old fashioned, expensive, even more expensive...

I have to admit a bit of a mistrust when it comes to most large consultancies and in fact I'm generally not a huge of fan of outsourcing come to think of it.

I've seen consultancies (charging astronomical rates), send 6-7 consultants to an engagement when 1-2 were required.  Why?  Ramp up the revenue of course.  How can you be expected to hire a company who aren't incentivised to do the best thing for you?

The aforementioned then tell you need to buy  software X from them in order to solve problem Y.   Again, self interest and a fundamental misalignment of incentives between you and them.

Lots of consultancies come in and tell you what you knew already, along with a Gartner magic quadrant to let you know what you should be doing in the future (its not hard to find out yourself).  OK, so some validation is nice once in a while but the majority of consultancy engagements I've worked on I've have wanted to solve real world issues and not death by SmartArt and PowerPoint.

Just tell me what buttons to press!

Perhaps I should point the finger back of blame back at myself and I haven't led the engagement effectively enough so that I achieved the results I wanted. However, at a squillion pounds a day you'd think they would get to the root of what you are trying to achieve pretty quickly.

I'm tarnishing the whole consultancy trade here and to be honest of course there are some good ones; they are just hard to find.  I tend to work better with the smaller consultancies, although I've had some really good engagements with ThoughtWorks.

Another pattern with the big consultancies is that they are staffed largely by consultants who are good at theory but haven't really ever lived the jobs they are talking about.  How can a consultant that's worked his way up the ladder from university ever tell me about how to structure my unit tests?  Or the best way to structure my automated deployment?  You need battle scars sometimes.

What has this got to with DevOps?  Well I guess if you are an Enterprise who wants to move to DevOps your natural reaction might be to find a correspondingly large consultancy.

Personally, I'd start really small and hire a really good focused team.  Don't engage on some 7 figure consultancy engagement.  Build a small team and grow success organically.

Why Continuous Delivery doesn't go far enough

My team “owns” (ensures that they are supported, provides best practises) a number of different tools and technologies in the DevOps space including:

• Puppet Enterprise
• IBM Urbancode Deploy
• Jenkins
• Teamcity
• TFS
• SVN
• Artifactory
• Selenium
• Octopus Deploy

...and numerous others.

That’s quite a long list for a Continuous Delivery tool chain (made worse by natural Java and .net divides).

Tools that don’t integrate become silos of their own and encourage a divide between different disciplines (such as Dev and Ops).

It’s important to see the aggregate efficiency of your delivery pipeline and having a thousand different tools involved makes this difficult.

• How many regression tests do we do a day?
• What is the ratio between bugs captured in CI against those in SysTest?
• Is there a relationship between the speed of a build and the number of deployment we do in a sprint?
• Etc etc

Only by capturing these metrics can we improve our efficiency by highlighting and attacking bottlenecks.

Continuous Delivery is a process to improve the delivery of software change into production. 

For me that doesn’t go far enough.  Are we saying we don’t care about it once our changes have gone live?

We have this nice, shiny Continuous Delivery pipeline to deliver value to the business – usually in the form of increasing revenues somehow.

So, ultimately it’s not really about builds, testing and deployments.  It’s about what impact our change team makes to the bottom line.

For example...

• What is the relationship between regression tests carried out in Systest and the number of errors that appear in production?
• What is the relationship between production errors and revenue collected?
• What is the relationship between how many story points we put in a sprint and the sentiment of our customers?

For me, this is the ultimate expression of DevOps.  We are aligning our delivery capabilities and processes in order to meet shared goals (in this example increased revenue).

I’m currently designing a proof of concept with Splunk to collate all these different types of metrics from the many  environments and tools we have in the delivery pipeline.   We should then be able to understand the different relationships between the capabilities in the team and the decisions we make.

Isn’t it powerful to have data which says to the business “stop putting too much change in a sprint”?  

Or that prioritising new features over technical debt is counterproductive to the bottom line?

I don’t expect that this will be an easy thing to accomplish and we can only infer these types of relationships.  However, the potential benefits are huge.