Create Ansible Playbook on GitHub to Build Mesos Clusters

Optional – Setup Git

If you haven’t already setup Git on your local system follow these steps:

1. Download and install the latest version of Git for your platform.
2. Launch the appropriate terminal or command line application.
3. Set the name used for Git commits.
   

   git config --global user.name "YOUR NAME"

4. Set the email address used for Git commits. This should match what you used when signing up for GitHub.
   

   git config --global user.email "YOUR EMAIL ADDRESS"

5. Configure Git to use the new “simple” style for pushing changes to a repository.
   

   git config --global push.default simple

Create a new repository on GitHub

1. In the upper-right corner of any page, click +, and then click New repository.
2. Enter a name for your new repository. I am calling it “ansible-mesos-cluster” for this project.
3. Add a description of your repository. For example, “Ansible playbook to build multi-node Apache Mesos clusters.”
4. Choose between creating a public or private repository. I’ll use public.
5. As long as you haven’t run

   git init

   locally you can select “Initialize this repository with a README.” This allows you to

   git clone

   the repository immediately.
6. Optionally, click “Add a license” and select one. I chose “MIT License” because it let’s people do anything they want with your code as long as they don’t hold you liable and provide attribution.
7. Click Create repository.

Checkout the new GitHub repository

1. Clone the HTTPS URL of your new repository. For example:
   

   git clone https://github.com/frankhinek/ansible-mesos-cluster.git

2. Move into the directory and create a gitignore file to prevent committing files that don’t need to be in the GitHub repository. For example,
   

   cd ansible-mesos-cluster/
   touch .gitignore

3. Enter the following contents into the .gitignore:
   

   .DS_Store
   ansible.cfg
   hosts
   *.retry

Basic Ansible Playbook setup

1. In the root of your project create a hosts file and build out the host groups. We use the

   zoo_id

   variable to set the ID of the ZooKeeper so each host knows its place in the cluster.

   touch hosts

   The content of the hosts file should look something like this, substituting the IP addressing for your environment:

   [mesos_masters]
   10.1.200.11 zoo_id=1
   10.1.200.12 zoo_id=2
   10.1.200.13 zoo_id=3
   
   [mesos_slaves]
   10.1.200.40
   10.1.200.41
   
   [marathon]
   10.1.200.30

2. In the root of your project create a

   site.yml

   file. This is the master playbook that defines the entire cluster infrastructure.

   touch site.yml

   The content of the hosts file should look something like this:

   ---
   # This playbook deploys the entire Mesos cluster infrastructure.
   # RUN: ansible-playbook --ask-sudo-pass -i hosts site.yml
   
   - name: deploy and configure the mesos masters
     hosts: mesos_masters
     sudo: True
   
     roles:
       - { role: mesos, mesos_install_mode: "master", tags: ["mesos-master"] }
   
   - name: deploy and configure the mesos slaves
     hosts: mesos_slaves
     sudo: True
   
     roles:
       - { role: mesos, mesos_install_mode: "slave", tags: ["mesos-slave"] }

Build Out Group Variables

Create a directory to store the variables used by the host groups in this playbook. Also add a blank file to store the variable definitions.

mkdir group_vars
touch group_vars/all

The content of the all file should look something like this:

---
# Variables here are applicable to all host groups

mesos_version: 0.20.0-1.0.ubuntu1404
mesos_local_address: "{{ansible_eth0.ipv4.address}}"
mesos_cluster_name: "Cluster01"
mesos_quorum_count: "2"
zookeeper_client_port: "2181"
zookeeper_leader_port: "2888"
zookeeper_election_port: "3888"
zookeeper_url: "zk://{{ groups.mesos_masters | join(':' + zookeeper_client_port + ',') }}:{{ zookeeper_client_port }}/mesos"

Building Out Role Scaffolding

Create a directory to store the roles used by this playbook. Use the

ansible-galaxy

mkdir roles
cd roles/
ansible-galaxy init mesos

Push Changes to GitHub Repository

1. Add all the files to Git locally:
   

   git add .

2. Commit you changes:
   

   git commit -m "Initial commit."

3. Push the changes to the GitHub repository. You will be prompted to enter your GitHub username and password.
   

   git push

Add the Mesos Role Handlers

Modify the

roles/mesos/handlers/main.yml

---
# handlers file for mesos
- name: Start mesos-master
  shell: start mesos-master
  sudo: yes

- name: Stop mesos-master
  shell: stop mesos-master
  sudo: yes

- name: Start mesos-slave
  shell: start mesos-slave
  sudo: yes

- name: Stop mesos-slave
  shell: stop mesos-slave
  sudo: yes

- name: Restart zookeeper
  shell: restart zookeeper
  sudo: yes

- name: Stop zookeeper
  shell: stop zookeeper
  sudo: yes

Add the Mesos Role Tasks

Modify the

roles/mesos/tasks/main.yml

---
# tasks file for mesos

# Common tasks for all Mesos nodes
- name: Add key for Mesosphere repository
  apt_key: url=http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF state=present
  sudo: yes

- name: Determine Linux distribution distributor
  shell: lsb_release -is | tr '[:upper:]' '[:lower:]'
  register: release_distributor

- name: Determine Linux distribution codename
  command: lsb_release -cs
  register: release_codename

- name: Add Mesosphere repository to sources list
  copy:
    content: "deb http://repos.mesosphere.io/{{release_distributor.stdout}} {{release_codename.stdout}} main"
    dest: /etc/apt/sources.list.d/mesosphere.list
    mode: 0644
  sudo: yes

# Tasks for Master, Slave, and ZooKeeper nodes
- name: Install mesos package
  apt: pkg={{item}} state=present update_cache=yes
  with_items:
    - mesos={{ mesos_pkg_version }}
  sudo: yes
  when: mesos_install_mode == "master" or mesos_install_mode == "slave"

- name: Set ZooKeeper URL # used for leader election amongst masters
  copy:
    content: "{{zookeeper_url}}"
    dest: /etc/mesos/zk
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master" or mesos_install_mode == "slave"

# Tasks for Master nodes
- name: Disable the Mesos Slave service
  copy:
    content: "manual"
    dest: /etc/init/mesos-slave.override
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master hostname
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-master/hostname
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master ip
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-master/ip
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master Cluster name
  copy:
    content: "{{mesos_cluster_name}}"
    dest: /etc/mesos-master/cluster
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

- name: Set Mesos Master quorum count
  copy:
    content: "{{mesos_quorum_count}}"
    dest: /etc/mesos-master/quorum
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "master"

# Tasks for Slave nodes
- name: Disable the Mesos Master service
  copy:
    content: "manual"
    dest: /etc/init/mesos-master.override
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Disable the ZooKeeper service
  copy:
    content: "manual"
    dest: /etc/init/zookeeper.override
    mode: 0644
  sudo: yes
  notify:
    - Stop zookeeper
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave hostname
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-slave/hostname
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave ip
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-slave/ip
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave ip
  copy:
    content: "{{mesos_local_address}}"
    dest: /etc/mesos-slave/ip
    mode: 0644
  sudo: yes
  when: mesos_install_mode == "slave"

- name: Set Mesos Slave isolation
  copy:
    content: "cgroups/cpu,cgroups/mem"
    dest: /etc/mesos-slave/isolation
    mode: 0644
  sudo: yes
  notify:
    - Start mesos-slave
  when: mesos_install_mode == "slave"

# Tasks for ZooKeeper nodes only
- name: Create zookeeper config file
  template: src=zoo.cfg.j2 dest=/etc/zookeeper/conf/zoo.cfg
  sudo: yes
  when: mesos_install_mode == "master"

- name: Create zookeeper myid file
  template: src=zoo_id.j2 dest=/etc/zookeeper/conf/myid
  sudo: yes
  notify:
    - Restart zookeeper
    - Start mesos-master
  when: mesos_install_mode == "master"

Create ZooKeeper Configuration Templates

1. Add blank files to store the ZooKeeper configuration file templates.
   

   touch roles/mesos/templates/zoo.cfg.j2
   touch roles/mesos/templates/zoo_id.j2

2. The content of the

   zoo.cfg.j2

   file should look like:
   

   tickTime=2000
   dataDir=/var/lib/zookeeper/
   clientPort={{ zookeeper_client_port }}
   initLimit=5
   syncLimit=2
   {% for host in groups['mesos_masters'] %}
   server.{{ hostvars[host].zoo_id }}={{ host }}:{{ zookeeper_leader_port }}:{{ zookeeper_election_port }}
   {% endfor %}

3. The content of the

   zoo_id.j2

   file should look like:
   

   {{ zoo_id }}

Push Changes to GitHub Repository

1. Add all the files to Git locally:
   

   git add .

2. Commit your changes:
   

   git commit -m "Initial commit."

3. Push the changes to the GitHub repository. You will be prompted to enter your GitHub username and password.
   

   git push

Pull Changes from GitHub

Here is an example of pulling down the repository to another host that you may use to build a Mesos cluster. It also indicates how to pull changes to the repository down to the host.

1. On the new system change to the working directory you want to clone the GitHub repository to.
   

   git clone https://github.com/frankhinek/ansible-mesos-cluster.git
   cd ansible-mesos-cluster/

You can now modify the hosts file to match the desired cluster node count and IP addressing scheme. Then run the Ansible playbook to build out your cluster. If you or someone else modifies the GitHub repository you can use the steps below pull the modifications down to your repository.

1. Make the entire working directory match the latest master branch commit and fetch a copy of the current branch and immediately merge it into the local copy.
   

   git checkout
   git pull

Summary and Next Steps

In this post we setup a repository on GitHub to store an Ansible playbook that builds Mesos clusters, cloned it to a local working directory, added the Ansible code and templates, pushed the changes up to the repository, and talked about how to pull them down on any system to execute the playbook. A role to deploy and configure Marathon is still needed, but rather than drag this post out I’ll leave that exercise to you. If you need some hints you can check out my repository.

The full Ansible Playbook with additional roles and capabilities is stored on my ansible-mesos-cluster GitHub repository. Feel free to explore the additional code I’ve added, clone the repository to test building a Mesos cluster in your lab, or fork to make your own changes.