Continuous build and one click deployment

Wouldn’t it be great if we deployed every time we checked in our code to reduce the stress associated with deployment. With the goal of making deployment as easy as possible, we at MVBA follow a process that seem to work really well for us. We tend to deploy every two two weeks to production and numerous times on test servers. Even though, we had one click deployment process, it was becoming tedious to manage multiple rake scripts for build and deployments. So we decided to clean up the process. After couple iterations we had a script in our hands that did exactly what we were looking for at the moment. We use albacore for our Rake tasks. If you are not using albacore for Rake tasks, you should definitely check it out.  It has a number of predefined tasks and it also makes adding generic or custom tasks easier. We use TeamCity as our continuous integration server. settings.yaml is used to maintain settings for different environments (developer, build, test and prod). Before compiling the solution, a task replaces all the config files with the current environment settings. For example, rake compile[‘developer’] will update the web.config and App.config files with developer.yaml file. You can also add settings related to deployment like dist_directory (directory where distributable is available), website path (like C:\WebSite) etc to the yaml file that the rake file can pick up when deploying. We divided our Build configuration into three categories –

• Compile, run unit tests, run integration tests, run environment tests, create a temp distributable with version number that includes Web build, Windows Service build (we also deploy windows services as part of our deployment), TestBuild, DatabaseRebuild zip (scripts to rebuild the database), DatabaseUpdate zip (scripts to update the database) and Deployment Scripts zip (rake scripts that are needed to deploy the app).

• Set up deployment files: Unzips the latest deployment zip file into a folder from where you can deploy

• Deploy and run UI tests: Deploy the latest files onto web server, update your windows services, rebuild database (only on build and developer machines) and run the update scripts. Below is a rough draft of our deploy task. It can be re-factored to make it much cleaner.

class Deploy
include Albacore::Task

attr_accessor :website_base, :website_dir, :processor_dir, :tests_dir, :dist_directory, :web_dist, :service_dists, :tests_dist, :messages_dist, :messages_dir, :env, :database_build_type, :url, :rebuild_dir, :update_dir, :rebuild_dist, :update_dist

def initialize
@service_dists = []
@iis = "w3svc"
end

def execute
deploy
end

def deploy
puts "Deploying #{@web_dist}"

# stop iis
manage(@iis, 'stop', 1)

puts "Waiting for all the messages to be processed before stopping the service"
while(Dir["#{@messages_dir}/*.request"].count > 0)
sleep(2)
puts "."
end

# stop services
manage_windows_services(@service_dists, 'stop', 1)

# deploy website
deploy_app "#{@website_base}/#{@website_dir}", "#{@dist_directory}/#{@web_dist}"

# deploy services
@service_dists.each do |svc_name, svc_dist|
puts "Deploying #{svc_dist[:dist]}"
deploy_app "#{@website_base}/#{svc_dist[:dir]}", "#{@dist_directory}/#{svc_dist[:dist]}"
end

#deploy tests, messages and rebuild database
if (@database_build_type == "rebuild")
# deploy tests
puts "Deploying #{@tests_dist}"
deploy_app "#{@website_base}/#{@tests_dir}", "#{@dist_directory}/#{@tests_dist}"

# deploy rebuild
puts "Deploying #{@rebuild_dist}"
deploy_app "#{@website_base}/#{@rebuild_dir}", "#{@dist_directory}/#{@rebuild_dist}"

if File.exists? "#{@website_base}/#{@rebuild_dir}/rebuild_database.json"
puts "Rebuilding Database"
run_tests "#{@website_base}/#{@rebuild_dir}/#{@rebuild_dir}.dll"

# deploy messages
puts "Deploying #{@messages_dist}"
Dir.mkdir(@messages_dir) unless Dir.exists?(@messages_dir)
unzip("#{@website_base}/#{@rebuild_dir}/#{@messages_dist}", @messages_dir)
else
puts "There is no need to rebuild the database as there are no changes in the mapping files"
end
end
# deploy update
puts "Deploying #{@update_dist}"
deploy_app "#{@website_base}/#{@update_dir}", "#{@dist_directory}/#{@update_dist}"
puts "Updating Database"
run_tests "#{@website_base}/#{@update_dir}/#{@update_dir}.dll"

# start services
manage_windows_services(@service_dists, 'start', 1)

# start iis
manage(@iis, 'start', 1)
end

def deploy_app(working_dir, zip_file)
Dir.mkdir(working_dir) unless Dir.exists?(working_dir)
# unzip the files to working directory
unzip(zip_file, working_dir)
t = Dir.entries(working_dir).grep(/\.config\.#{@env}$/)
config_with_env = Dir.entries(working_dir).grep(/\.config\.#{@env}$/).first
return if (config_with_env.nil?)
config = File.basename(config_with_env, ".#{@env}")
FileUtils.mv "#{working_dir}/#{config_with_env}", "#{working_dir}/#{config}"
end

def manage(service, action, total_tries = 1, tries=0)
begin
sh "net #{action} #{service}"
rescue
tries +=1
puts "Caught error while performing #{action} on #{service}"
return if (tries == total_tries)
puts "Next try to #{action} the #{service} in 30 seconds"
sleep(30)
manage(action, tries) unless (tries >= total_tries)
end
end

def manage_windows_service(service_name, action, exe, tries)
if (action == 'start' && (`sc query #{service_name}`=~ /.*does not exist.*/) != nil)
sh "#{@website_base}/#{exe} install" unless (exe == nil)
end
manage(service_name, action, tries)
end

def manage_windows_services(services, action, tries)
services.each do |svc_name, svc_dist|
manage_windows_service(svc_name, action, "#{svc_dist[:dir]}/#{svc_dist[:exe]}", tries)
end
end

def unzip(file, dest)
unzip = Unzip.new
unzip.destination = dest
unzip.file = file
unzip.force = true
unzip.execute
end

def run_tests(assemblies)
nunit = NUnitTestRunner.new
nunit.command = get_nunit_x86
nunit.assemblies assemblies
nunit.execute
end
end

Before we deploy, we stop IIS, wait till all the services have finished processing before stopping all the windows services. We then deploy web and services. Tests are deployed only if the database_build_type is “rebuild” which is decided by :database_build_type in the settings file. :database_build_type is set to “rebuild” in developer and build, but it is set to “update” in test and prod (You don’t want to rebuild your production). We rebuild the database only if any of the fluent nHibernate mappings are changed in the latest check in. We then run the update scripts irrespective of what environment you are running on (We have an internal check that makes sure that a script is not run if it has already been run before). We then restart the windows services and finally IIS is restarted to complete the deployment process. Once the website is deployed, we run the UI tests and if all the UI tests are successful web dist, services dist, database update dist and deployment scripts dist are moved to its final destination folder from where you can deploy to test or prod. In our test and production servers we have a single batch script that calls 3 tasks

call rake set_up_deploy_files['test']
call rake copy_dist['test']
rake deploy['test']

The first line gets the latest deployment scripts or the deployment version you are looking for (you can pass in the version number as part of arguments to deploy a specific version like rake set_up_deploy_files[‘test’,’1.0.25.9217′]). Second line gets Web dist, services dist and database update dist. Third line deploys web, services and updates the database structure if needed. This has reduced considerable overhead in maintaining scripts for different needs ensuring that we are always using the right deployment script. Moreover, we now have just one RakeFile that is used to build and deploy making it easier to maintain. As a result, we are now more confident of our deployment process.