Tuesday, May 8, 2012

Deployment Management

Like many IT shops, we have a complex operational environment consisting of dozens of servers deployed across many network layers. Our production system is young, went live in July last year, and has healthy ongoing development activity. More, we have another major production system going live July this year. Critical to our success is intelligently managing our environments, code bases, and deployments. Here's our current strategy.


Three System Environments
  • DEV - Contains latest code with a copy of production data. Shared sandbox for our developers.
  • TEST - Contains copy of production code and data. Used as staging area for monthly service release, user training, and for troubleshooting current production issues.
  • PROD - Production code and data.
Except for a shared component or two, these are full end-to-end environments. Usually, a team can manage with just two such environments. We were forced into three since we had to branch the code after go live to implement major new legislation that went into effect six months later. So we needed an environment to support current production (TEST), and one to support all the new long-running development work (DEV).


Two Code Branches
  • develop - Latest developer tested code deployed to DEV environment.
  • master - Current code deployed to PROD environment.
Early this year, we successfully transferred the code base / maintenance responsibility from the vendor to our team. We loaded the system's code base into Git and as per best practice, isolated the current deployed production code (master branch) from the ongoing development work (develop branch). We use the centralized workflow model on the develop branch and the dictator (Deployment Manager) model on the master branch. This is done as per the long-running branches workflow described in Pro Git.

Developer Procedures
The procedures are simple one page guides (KISS). They're designed to keep us (about nine of us developers) from stepping on one another and to minimize management overhead. Please note that these procedures continue to evolve to fit the unique and dynamic characteristics of our team / project.

So putting it all together, we have developers making incremental commits (1) to the develop branch for the upcoming monthly production release. Then monthly, the Deployment Manager updates the master branch (2) with that month's service release. Occasionally, the Deployment Manager updates the master branch (3) with an urgent hot fix from the develop branch code stream.