Encapsulation with Vamos Deploy

Vamos Deploy encapsulates an application with it's binaries, all it's dependant libraries and it's runtime configuration. The 'capsule' is a self-contained runtime instance of an application which can be deployed to multiple hosts with a single command. Capsules do not conflict with each other and can happy exist side-by-side. There are no hard walls though, we rely on users abiding by the rules and good neighbourly conduct. In Vamos jargon, we call an encapsulated application a 'grid'.

On each host there can be one or more local repository. Once registered, a repository can be added to any number of grids. When a grid is 'deployed' each local repository, associated with that grid, is updated with all the required software packages and configuration. The application is ready to go! Takes just 2 commands:

$ vamos grid JupiterDev1 addrepo myrepo-myhostname
$ vamos grid JupiterDev1 deploy

All software and configuration is stored within the local repository, which on Linux is a directory structure under /vamos. Software is not scattered around the filesystems it's neatly contained in a dependable location, irrespective of the operating system.

In a future post we will examine how an application accesses its dependancies and runtime configuration from within the local repository.

Get in contact to see a demo of how this works : http://bitlq.com/5WLqU

Using Vamos Deploy

No Git knowledge is needed, no Ruby or Python knowledge is needed, no YAML knowledge is needed. No writing of modules is required. Anyone can learn to use Vamos Deploy in 15 mins.

With Vamos Deploy you don't need to learn a new language to deploy your applications. We think a deployment tool should be simple to use which makes it easy for you to learn.

Vamos Deploy has a simple CLI which is not overbloated:

Nine bullet points on how Vamos Deploy automates the delivery of software

• Vamos Deploy encapsulates an application with it's binaries, all it's dependant libraries and it's runtime configuration.

• A capsule Vamos can be deployed to multiple hosts with a single command.

• Multiple capsules can exist and function on the same host with no conflicts.

• Each application has a promotional lifecycle. It passes through a series of testing tollgates on it's path to production.

• A strong segregation of duties prevents dev teams from deploying to production.

• Reduces handover effort between dev and ops. Dev teams configure what is going to be deployed and Ops take responsibility of the orchestration (or how it gets released).

• No scripting is required just use the simple CLI.

• Everything is audited, all data is visible, it can integrate with other systems and its easy to use.

• Read more details and see our website.

Please get in contact if you are thinking of having painless software deployments.

The down-to-Earth view of application deployment

One area that I have been concentrating on for a while now is the automated delivery of software that actually works. This has recently become a hot topic with Continuous Integration and Continuous Delivery being discussed. The DevOps banner is being used for wider discussions which include the breaking down of barriers between teams to increase co-operation. This is all good stuff and a step in the right direction in an area which has long been neglected. Companies are now starting to invest in how they deliver their software. But they may just be saving up problems for the future.

Configuration management and release orchestration

The new technologies (Ansible, Chef, Puppet) in this space have the ability to drastically reduce the amount of time it takes to deliver a 'working' host. Many a time I've witnessed a host ping-ponging between Sys Admins and Dev teams before it's fit for use. The Dev teams don't specify exactly what they need and the System Admins don't automate it for future use. With DevOps we break this cycle and bring the efficiencies of automation to OS builds.

But these technologies leave fundamental gaps in the delivery of application software. Let's think this through a bit :

I have an application called Saturn that:

• comprises of 10 different processes that each serve a different purpose;
• each process has a different binary package;
• uses Java 1.6 and has a list of dependant libraries;
• is not resource intensive so its processes can co-habitate with other applications on a host and;
• has configuration that needs to differ between environments.

We don't have the luxury of a dedicated host so Saturn shares hosts with other applications. In fact, in Dev and UAT we have multiple instances of Saturn running and even many different versions of Saturn running on the same host. If we are to share it's essential for the application to be a good neighbour so it doesn't have to be isolated. But without the right tools, sharing of resources can cause conflicts. Here is why:

There is another team that are building Pluto and they are using Java 1.8. This is one of the applications that share hosts with Saturn. Instantly we see a conflict in the version of Java used. We could hardcode the Java version we use but that reduces our application portability.

We then discover that Saturn and Pluto use different versions of the same Java libraries. How do we cope with that? We could embed these libraries in our application but that seems rather clumsy, encourages duplication and reduces visibility of usage. This is a wider problem than just Java. There are two other teams that own Jupiter and Mars which are written in Python. They have very similar problems with the use of conflicting versions of Python and Python libraries. These problems stem from the reliance on the underlying OS build to fulfill runtime dependancies.

When resource sharing we need a tool that prevents dependancy conflicts.

There are similar conflicts when we look at the application binaries. We are making our lives unneccessarily difficult if we don't have tooling that provides functionality to:

• easily manage multiple versions of the application software on a single host;
• select/update binary versions used in each environment;
• give transparancy and visibility of environment and configuration settings and;
• automate the removal of unused software for effective disk management.

Storing your application binaries and libraries in a repository makes perfect sense. Yum is frequently used to hold RPMs that are built containing the application software. But this is for RedHat-based Linux distros; what about Debian-based distros and of course Windows and OS X? This is highly restrictive! We need a repository that can store software for all flavours of OS.

We need our repository to have functionality to:

• be OS and packaging standard independant
• distinguish between production and non-production packages
• retain meta data about each package:
  • when was it created and by whom
  • what level of testing it has had
  • which environments its used in
  • which hosts it exists on
• be immutable so you know it's not been tampered with
• restrict who can add or remove packages

Configuration management tools can make a big difference in release orchestration. Releases often comprise of many steps that need to be performed in a certain order over a number of different hosts. Software deployment is usually an essential part but there are frequently many other steps involved (ie process stop/start, database updates, batch scheduler changes etc).

There is however the temptation to mix up what is being released with how it is being released. The release runbook (sometimes called recipe or playbook) should contain all the steps involved and their dependancies. If, in the runbook, we start to include what is actually changing (ie versions of packages to install or remove) then it would need modifying for each release performed and needs to be tested, just like code. To lessen the testing burden it seems logical to separate what is being released from how it gets released. But what tools do we need to do this?

Using a configuration manager to quickly set the standard OS build and to orchestrate a release saves a lot of time and money and gives welcome consistency. But if your application is part of a larger business where resources are shared and is going to be changing frequently, a configuration manager has fundamental gaps. We need something more.

Logical or pseudo containers

Vamos Deploy has the functionality to encapsulate the components of an application with all its dependancies, libraries, binaries and configuration files so they can be deployed as one to any number of local Vamos repositories irrespective of the underlying OS. We call this grouping a Vamos Grid.

Deploying your applications via Vamos Deploy:

• frees you from relying on software that is installed as part of the underlying OS. This saves you from the conflicts that causes;
• enables multiple instances of the same application can be running on the same kernel that can be independantly re-configured or updated;
• enables many instances of different applications to happily and easily co-exist on the same host with different configurations;
• enables applications to be promoted once they pass different stages of testing. This test-promote lifecycle keeps non-production quality code well away from the production domain;
• brings the benefit of a full application lifecycle. The detection and removal, from local repositories, of application packages that are no longer used is running from day 1;
• allows you to update your development environment with exact copies of software used in production in a single command. You can even look back at a fixed point in time and replicate how production looked;
• avoids duplication due to the Vamos Grids sharing the underlying application software and libraries in the local repositories.

Vamos Deploy's promise is to distribute all the software components and configuration to the local Vamos repository on each host or client that you have told it to. This can be initiated from a configuration management tool as part of release orchestration. Vamos Deploy is responsible for what is being released and the configuration manager is responsible for how it gets released.

With Vamos Deploy you don't need to learn YAML, Ruby or Python. There is a simple Vamos CLI, which can be called from any configuration manager or embedded in any self-service environment manager. There is no need for lengthy training sessions on cutting-edge technologies and radically changing your deployment methodology.

Vamos Deploy is a down-to-Earth tool that takes the headaches out of managing the lifecycle of an application and it's deployments.

If you think that this product could help you then let me know. Or, if you have questions, don't hestitate to ask.

Deploy client and server side components

Having a different deployment procedure for client and server side components seems crazy to us...

Vamos Deploy not only facilitates the delivery of server side software components to all necessary hosts but it can deliver your client side as well.

Delivery of the client side UI can often be a time consuming cumbersome task that requires the input from a dedicated team. It can also be painful to rollback.

Vamos Deploy removes this inefficiency from your software releases.

Creating a Vamos Grid

In a previous article we added an application to Vamos so lets move to the next step of creating a Vamos grid.

First, a Vamos grid is an association of applications, libraries, properties and a list of hosts they are to be deployed to. We frequently think of a Vamos grid as an environment.

Lets create a grid, add an application, a property and a repo:

 $ vamos grid Jupiter_dev create
 $ vamos grid Jupiter_dev addapp JupiterServer 1.0.1
 $ vamos grid Jupiter_dev addproperty jupiter_http_port 8091
 $ vamos grid Jupiter_dev addrepo arepo-vm-barcelona-alpha

Lets look at a grid I prepared earlier:

 $ vamos grid Jupiter_dev info
 Gridname       : Jupiter_dev - For Jupiter developer testing purposes
 Edit status    : Unfrozen
 Change status  : Unchanged
 Owner group    : developers
 Release group  : developers
 Applications   : JupiterServer             1.0.32     dev        Linux
                : JupiterUI                 1.0.17     prod       Windows
                : PythonDjango              1.7.1      prod       ALL
                : PythonGoogleFinance       0.7.0      prod       ALL
                : PythonLinux               3.4.lin32  prod       Linux64
                : PythonRequests            2.5.1      prod       ALL
                : PythonRestFramework       3.0.1      prod       ALL
                : PythonWindows             3.4.win32  prod       Windows64
 Properties     : jupiter_http_host = 192.168.56.101
                : jupiter_http_port = 8091
                : jupiter_window_title = JupiterUI-Version:1.0.17-Build:#73
 Repos          : arepo-nathan-pc           ALL        Vamos      Windows    Windows64
                : arepo-vm-barcelona-alpha  ALL        Linux      Linux64    Vamos

Here we have a collection of applications including python libraries, python itself and the 2 Jupiter applications. There are three environment specific properties being set and two repos one Linux host and one Windows client.

When we deploy this grid Vamos will deliver the applications and properties to both of the repositories. So providing the versions of application binaries to use and the runtime environment including environment specific properties.

This is explained in more detail here and in this YouTube demo http://bitlq.com/R0Joq

Auto increment application versioning - MAJOR.MINOR.PATCH

We noticed that many teams implement a mechanism to increment their application version. Frequently, a script is used to look at the previous version used and increment. With Vamos Deploy this is built-in.
Vamos recommends the use of semantic versioning for your application versions.To use, simply embed '++' in the version pattern to indicate if MAJOR, MINOR or PATCH is to be incremented.

To increment the patch level from 1.0.0 use '1.0.++' as in example below:

 $ vamos application JupiterServer 1.0.++ create --directory=$HOME/JupiterServer
Successfully registered application JupiterServer 1.0.1 and copied to repo at /vamos/myrepo/applications/JupiterServer/1.0.1

To increment the minor level from 1.0.0 use '1.++' as in example below:

 $ vamos application JupiterServer 1.++ create --directory=$HOME/JupiterServer
Successfully registered application JupiterServer 1.1.0 and copied to repo at /vamos/myrepo/applications/JupiterServer/1.1.0

To increment the major level from 1.0.0 use '++' as in example below:

 $ vamos application JupiterServer ++ create --directory=$HOME/JupiterServer
Successfully registered application JupiterServer 2.0.0 and copied to repo at /vamos/myrepo/applications/JupiterServer/2.0.0

Other styles of versioning can be used but semantic versioning is the style we have implemented with an auto increment mechanism. This can be changed, by developing a new auto increment plugin for Vamos Deploy.

Adding a new application to Vamos Deploy

This post will show you how to use the CLI to add a new application to Vamos Deploy.

Let's assume that you have your application code in an SCM (git, svn etc..) and you have the ability to build any parts, that you need, into the artifacts that will be deployed. 

What directory structure is best to use for our package? With Vamos you are free to decide on this yourself. The only rule is that you have a directory called 'bin'. Vamos will give all contents execute permissions.

 $ vamos application JupiterServer 1.0.0 create --directory=$HOME/JupiterServer

The above command will create the application in the Vamos database and copy the code to the local repository. It will instantly be useable on this host. File permissions will be set to be read-only with executable permissions being given to all contents of the 'bin' directory. The local repository will be at the following location:

 /vamos/<our department>/applications/JupiterServer/1.0.0/  

The status of this application will be 'local' and it only exists locally on the host we created it on. We can commit this application to the Vamos filestore so it can be used by other hosts. This will also change the status to 'dev':

 $ vamos application JupiterServer 1.0.0 commit  

We need to add a subscription to the application. More details on this in another post but they are used to specify what operating system applications were built on and hence what this application is compatible with. For now let's set to ALL to indicate it can run on any o/s:

 $ vamos application JupiterServer 1.0.0 addsub ALL  

Let's now see the details of what we have created:

 $ vamos application JupiterServer 1.0.0 info  
 Application   : JupiterServer 1.0.0  
 Status        : dev  
 Owner group   : developers  
 Subscription  : ALL  

The owner group will be set to the default group that your userid was registered with. In this case the user is in the developers group. Vamos uses a simple group ownership model for all objects it creates. If you created it then your group owns it and your group can do whatever it wants with it.

First Steps in Automation

If you have got this far you are probably already thinking that your application deployments needs improvement. Maybe you have one or more of the following deployment 'smells':

• home grown 'release' scripts that differ between applications
• manually scp/sftp/rsync software
• manually edit property files in production
• complex release notes

These can result in slow, unreliable, unrepeatable deployments which will not scale. But where do we start in transforming a release process? The key to an effective release process is consistency. Before we get to discussing Vamos Deploy there are areas that can be tackled in preparation which will reap rewards later on.

Packaging:

Vamos does not enforce a packaging methodology, its very open. The only rule is that there needs to be a ‘bin’ directory and all of its contents will be granted execute permissions. You have the flexibility to build, as you require, on top of this single rule. Though we do recommend setting a basic standard layout so most packages have the same look and feel.

Naming conventions, with a little thought, can make life much easier in the long run. A common mistake is to embed the management structure rather than the business into the name. The management structure can change frequently so can make naming obsolete.

A good naming standard would contain the business name, the application suite, application name and application instance. You should end up with names in the region of 10-15 chars in length. CamelCase is a good idea also.

Version numbers used for applications should follow the semantic versioning pattern of MAJOR.MINOR.PATCH. See http://semver.org/

If you have packages that are very large they will take a long time to deploy. If you change just 5% of a large package then 95% will be redeployed for no gain. So, try to split up large packages into smaller ones. We will use a Vamos Grid to link them together. As a rule of thumb, a package of more than 250Mb sounds large.

How many is the right number of packages? This varies hugely but a good rule of thumb is to have between 10 and 50. More packages, will mean more to manage. Its a balancing act between speed and manageability.

One idea is to split packages, instead of by function, by the frequency of change. Package independently, parts that change frequently. Then your deployment is fast and you have few moving parts.

Runtime:

Standardisation of runtime environments can bring huge benefits. Most of the next points sound obvious and simple, which they are, but it’s amazing how frequently it’s not done.

All software you need should be deployed. Don’t rely on any software being present on a host! Don’t rely on someone else to put it there for you! It's worth listing what you rely on so later you can create them all as Vamos applications.

All applications should be good citizens. Applications should be able to run alongside any other application with no conflict. It shouldn’t need a dedicated user to run.

The scripts used to start and stop your application should be shipped with your application. No ‘support’ scripts should be held locally. They should all be held with your source code and deployed in the same manner.

Make sure when your application is running on a host that it is easily identifiable from the process list. Makes support much easier.

Select a standard area for placing the logs of your application. This should be agreed so all applications supported by one team use a standard directory layout. Make the log filename unique so you can identify the instance running.

Use a dedicated scheduler to start and stop your application using the scripts you deploy. It should be the only tool that does this in a production environment. The schedule needs to be tested before it’s used in production!

Keep all your environments clean. There no excuse for untidiness. Have an automated job that zips old logfiles and delete them after a configurable number of days. Remove all old core files. Only keep files on your system that you need - put the rest in the trash.

Application configuration:

A simple approach that works well in most cases is to categorise your properties into:

• Properties that need to change for each environment: Add these properties to a grid. Then use Vamos templatisation to inject them into property files for your application to read.

• Properties that change across all runtime environments: These can be added to property files and deployed with your application. A code change will be needed to change them.

• Properties that need to be secret: A common way is to use UNIX permissions to secure a password file owned by the runtime user. We prefer to use RSA to secure passwords. Anyone with the public key can encrypt but only those with the private can decrypt.

Do not have property files that are stored in an environment and changed locally. This is a recipe for disaster!
 

Finally...

Now that we have made you aware of some preparation steps we can move onto how to use Vamos Deploy. Starting with the next entry.

Application Deployment with Vamos Deploy

To demonstrate the ease of deploying applications using Vamos Deploy we have a demo application suite called Jupiter which has a client-server architecture.
Below shows the Jupiter_dev grid that we have setup: 

 $ vamos grid Jupiter_dev info
 Description    : Jupiter development
 Edit status    : Unfrozen
 Change status  : Changed
 Owner group    : developers
 Release group  : developers
 Applications   : JupiterServer             1.0.17      dev
                : JupiterUI                 1.0.37      dev
                : PythonDjango              1.7.1       dev
                : PythonGoogleFinance       0.7         dev
                : PythonLinux               3.4.lin32   dev
                : PythonRequests            2.5.0.1     dev
                : PythonRestFramework       3.0.0       dev
                : PythonWindows             3.4.win32   dev
 Properties     : jupiter_http_host = 192.168.56.101
                : jupiter_http_port = 8091
                : jupiter_window_title = JupiterUI-Version:1.0.37-Build:#52
 Repos          : arepo-nathan-pc             
                : arepo-vm-barcelona-alpha

A Vamos grid contains a list of applications, a list of properties and a list of repositories. When we deploy this grid the applications get deployed to the compatible repos with properties providing environment specific settings.
This grid is in a ‘Changed’ status because I updated JupiterServer to 1.0.17 earlier. To deploy this changed grid:

 $ vamos grid Jupiter_dev deploy
 Deploy of grid Jupiter_dev initiated. Release id 55
 Hint: Monitor deploy progress with:
       vamos deploy info --releaseid=55
 Monitor tasks with:      
       vamos task info --releaseid=55

Lets monitor the progress from the command line and from the Vamos Dashboard: 

 $ vamos deploy info --releaseid=55
 ReleaseId : 55
 Gridname  : Jupiter_dev
 Status    : Complete
 Start     : 28 Jul 2015 15:19:10 BST
 End       : 28 Jul 2015 15:19:10 BST
 User      : usera

Now thats completed we can start our new JupiterServer. This command line could be executed from a job scheduler. It doesnt need to change when a version of the application is deployed.

 arepo-vm-barcelona-alpha$ /vamos/ARepo/grids/Jupiter_dev/JupiterServer/bin/start_JupiterServer.sh

We can use VamosJump on my desktop (arepo-nathan-pc) to start JupiterUI:

In summary Vamos Deploy has delivered the JupiterServer application and its dependants to our Linux server and delivered the Jupiter UI to my laptop. We have environment specific properties set that enable the two to communicate.

Application Deployment Best Practises

Its 11:45pm on a Saturday evening and I’m waiting for the release of my application to the production environment. It was scheduled for 2:30pm but there was some delay with some other application. The production support team are almost ready to do my release but they have been saying that for 3 hours. It doesn’t have to be like this…

Having spent many years streamlining, standardising and simplifying complex deployment processes we have compiled a top 10 of best practises you need from a deployment process:

1. Automation. Get rid of all those manual tasks.
2. Consistency. Standardise as much as possible.
3. Audit trail. You need to be able to look back at who did what and when.
4. Visibility. You need to see what is happening now - to know the current status.
5. Promotional lifecycle. Only allow applications through the gate to the next level if they pass the tests.
6. Segregation of duties (SoD). Developers should not be gatekeepers to production. An access control mechanism should divide duties between different people or teams.
7. Reduce release notes to a minimum. The more steps in the release notes the more scope for human error. 
8. Deploy with consistency across all environments. The first time you fully deploy your  application should not in production.
9. Rollback strategy. Incase it all goes wrong you need to rollback to the previous working version as fast as possible. In reality, what is usually employed is a partial rollback or an emergency release if the fault can be fixed quickly.
10. Keep it simple and keep it clean!  Complexity is the enemy.

The good news is that Vamos Deploy ticks the box for all the above.

If you are lucky then you will have the time and personnel to redesign your entire release process end to end. In most cases the release process only gets attention when it gets really bad. Maybe the wrong application is deployed or it doesn’t work as expected in production and the delays cross into the unacceptable. So the little resources that does get allocated to improvements needs to be well spent. A deployment tool that can be integrated in phases is essential.