## Deployment at TrackMaven ![Alt text](lib/img/tm.png)
## About Me Matt Camilli @mlcamilli matt@trackmaven.com http://mattcamilli.com
## The Product
## Where we were ![heroku](lib/img/heroku.png)
## Where we ended up ![aws](lib/img/aws.png)
## Fabric ![fabric](lib/img/fabric.jpg)
Fabric is a python command-line tool for streamlining the use of SSH for application deployment or systems administration tasks. We use it to provide an easy interface for our developers to interact with Ansible and Boto.
## fab prod shell **fab** - Invokes a fabric command from the fabfile **prod** - Sets fabric's environment dictionary to production **shell** - Custom fabric method that will cd into the project directory on the host specified by the environment dictionary, and run the project shell ```python def shell(): ''' Calls the shell_plus of the TrackMaven project ''' with env.cd(env.project_path): env.run('foreman run python trackmaven/manage.py shell_plus') ```
fab prod shell ![shell](lib/img/shell.png)
## Boto ![corgicomp](lib/img/corgi_computer.jpg)
## Tagging ![tags](lib/img/tags.png) * Env - Prod, Puppystream, Puppycrate * Type - Web, Celery, Manager * Name - EnvType# (prodcelery1)
fab prod scale:web,3
## AWSManager ```python from boto import ec2 from boto.ec2 import elb class AWSManager(): REGION = 'us-east-1' def __init__(self, env): if not env: raise Exception('No environment set') self.env = env self.ec2 = ec2.connect_to_region(self.REGION, profile_name="trackmaven") self.elb = elb.connect_to_region(self.REGION, profile_name="trackmaven") def scale_web(self, number): ''' Scales the web workers to the number inputted ''' web_instances = self._get_instances('web') if number > len(web_instances): self._scale_up('web', number - len(web_instances)) elif number < len(web_instances) and number >= 2: self._scale_down('web', len(web_instances) - number) else: raise Exception('Cannot scale below 2 web instances or scale the same number') def _scale_up(self, instance_type, number): ''' Scales the specified instances up ''' # Fetch the main instance to base the new instances off of instances = self._get_instances(instance_type) main = next(instance for instance in instances if '1' in instance.tags.get('Name')) image = self.ec2.get_all_images(owners='self', filters={ 'tag:Name': main.tags.get('Name')}).pop() for i in range(0, number): reservation = self.ec2.run_instances( image.id, key_name=main.key_name, security_groups=[group.name for group in main.groups], instance_type=main.instance_type, placement=main.placement ) instance = reservation.instances[0] # Add the proper tags instance.add_tag('env', self.env) instance.add_tag('type', instance_type) instance.add_tag('Name', '{}{}{}'.format( self.env, instance_type, len(instances) + 1 + (i * 1))) # If it is a web worker, register it with the elb if instance_type == 'web': self.elb.register_instances(self.env, [instance.id]) ```
## Ansible ![ansible](lib/img/ansible.png)
Ansible is an open source orchestration engine that manages nodes over SSH. (Built on Python)
## Modules ![module](lib/img/module.png)
## Playbook Example ``` --- - hosts: webservers vars: http_port: 80 max_clients: 200 remote_user: root tasks: - name: ensure apache is at the latest version yum: pkg=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache - name: ensure apache is running service: name=httpd state=started handlers: - name: restart apache service: name=httpd state=restarted ```
## Inventory A file that describes Hosts and Groups in Ansible (INI or JSON) ``` mail.example.com [webservers] foo.example.com bar.example.com [dbservers] one.example.com two.example.com three.example.com [webservers:vars] some_var=somevalue ```
## Deployment ![maven](lib/img/maven.png)
fab prod deploy ENVRIONMENT=prod ansible-playbook deployment/playbooks/deploy.yml -i deployment/hosts -e "branch=master"
-i deployment/hosts ```python from boto import ec2 import os, json def main(): instances = {} instances['_meta'] = { 'hostvars': {}} env = os.environ.get('ENVIRONMENT') conn = ec2.connect_to_region('us-east-1', profile_name='trackmaven') prod_reservations = ec2.get_all_instances(filters={'tag:env': env}) for reservation in prod_reservations: for instance in reservation.instances: instance_type = instance.tags.get('type') # Group instances by type instances.setdefault(instance_type, []).append(instance.public_dns_name) # Group instances by name instances.setdefault(instance.tags.get('Name'), []).append(instance.public_dns_name) # Set instance specific variables instances['_meta']['hostvars'][instance.public_dns_name] = { 'type': instance_type, 'env': env, 'id': instance.id, 'region': 'us-east-1' } print json.dumps(instances) main() ```
output ``` { "prodweb1":[ "made-up-web1.compute-1.amazonaws.com" ], "web":[ "made-up-web1.compute-1.amazonaws.com", "made-up-web2.compute-1.amazonaws.com" ], "all":[ "made-up-web1.compute-1.amazonaws.com", "made-up-web2.compute-1.amazonaws.com", "made-up-celery1.compute-1.amazonaws.com", "made-up-manager1.compute-1.amazonaws.com" ], "prodweb2":[ "made-up-web2.compute-1.amazonaws.com" ], "prodcelery1":[ "made-up-celery1.compute-1.amazonaws.com" ], "prodmanager1"[ "made-up-manager1.compute-1.amazonaws.com" ], "celery":[ "made-up-celery1.compute-1.amazonaws.com" ], "manager":[ "made-up-manager1.compute-1.amazonaws.com" ], "_meta":{ "hostvars":{ "made-up-web1.compute-1.amazonaws.com":{ "type":"web", "env":"prod", "id":"someid", "region":"us-east-1" } } } } ```
playbooks/deploy.yml ``` --- - hosts : web vars: branch : "{{branch}}" roles: - deploy serial: 1 - hosts : celery:manager vars: branch : "{{branch}}" roles: - deploy ```
roles/deploy/tasks/main.yml ``` --- - name: Instance De-register local_action: ec2_elb args: instance_id: "{{ hostvars[inventory_hostname].id}}" state: 'absent' profile: 'trackmaven' ec2_elbs: "{{ hostvars[inventory_hostname].env}}" wait: 'yes' wait_timeout: 10 region: "{{ hostvars[inventory_hostname].region }}" when: '"{{ hostvars[inventory_hostname].type }}" == "web"' - include: deploy.yml - name: Instance Register local_action: ec2_elb args: instance_id: "{{ hostvars[inventory_hostname].id}}" state: 'present' profile: 'trackmaven' ec2_elbs: "{{ hostvars[inventory_hostname].env}}" wait: 'yes' wait_timeout: 20 region: "{{ hostvars[inventory_hostname].region }}" when: '"{{ hostvars[inventory_hostname].type }}" == "web"' ```
roles/deploy/tasks/deploy.yml ``` --- - name: turn on maintenance mode sudo: yes command: chdir=/home/web/www/ touch maintenance when: '"{{ hostvars[inventory_hostname].type }}" == "web"' - name: shut down supervisor sudo: yes service: name=supervisor state=stopped - name: pull the latest code git: repo=git@github.com:TrackMaven/TrackMaven.git dest=/home/web/www/TrackMaven force=yes update=yes version={{branch}} - name: install lastest requirements pip: requirements=/home/web/www/TrackMaven/requirements.txt virtualenv=/home/web/www/.virtualenvs/trackmaven - name: recreate supervisor conf files sudo: yes command: chdir=/home/web/www/TrackMaven/ /home/web/www/.virtualenvs/trackmaven/bin/python deployment/export.py -p {{ hostvars[inventory_hostname].procfile }} - name: remove pycs command: chdir=/home/web/www/TrackMaven/ find . -name '*.pyc' -delete - name: start supervisor sudo: yes service: name=supervisor state=started - name: Django collectstatic command: chdir=/home/web/www/TrackMaven/ foreman run python trackmaven/manage.py collectstatic --noinput - name: turn off maintenance mode sudo: yes command: chdir=/home/web/www/ rm maintenance when: '"{{ hostvars[inventory_hostname].type }}" == "web"' ```
## Questions? ![sunglasses](lib/img/sunglasses.jpg)