An easy way to install a docker SWARM cluster with ANSIBLE ?

In a previous, we installed a prometheus stack with ansible : node exporter, prometheus, grafana… And I propose to install a new stack : a docker swarm cluster and some tools.

In this post, we’ll just install a simple swarm cluster. And we’ll use vagrant to have our infrastructure to develop it. And after, with few lines of ansible code, we’ll have our docker cluster. And this is very very easy.

Our vagrant infrastructure

Vagrant allows us to create virtual machines and orchestrate its. In our case, we just want to have some servers. Exactly we want 3 masters and 2 workers. I give you my standard vagrant file to do it. You can reuse it and maybe you need to change it for you use case.

The vagrantfile

Vagrant.configure(2) do |config|

	# Set some variables
  etcHosts=""
  common = <<-SHELL
  sudo apt update -qq 2>&1 >/dev/null
  sudo apt install -y -qq git vim tree net-tools telnet git python3-pip sshpass nfs-common 2>&1 >/dev/null
  sudo echo "autocmd filetype yaml setlocal ai ts=2 sw=2 et" > /home/vagrant/.vimrc
  sed -i 's/ChallengeResponseAuthentication no/ChallengeResponseAuthentication yes/g' /etc/ssh/sshd_config
  sudo systemctl restart sshd
  SHELL

	# set vagrant image
	config.vm.box = "ubuntu/focal64"
	config.vm.box_url = "ubuntu/focal64"

	# set servers list and their parameters
	NODES = [
  	{ :hostname => "devans1", :ip => "192.168.140.10", :cpus => 4, :mem => 1024 },
  	{ :hostname => "devans2", :ip => "192.168.140.11", :cpus => 4, :mem => 1024 },
  	{ :hostname => "devans3", :ip => "192.168.140.12", :cpus => 4, :mem => 1024 },
  	{ :hostname => "devans4", :ip => "192.168.140.13", :cpus => 4, :mem => 1024 },
  	{ :hostname => "devans5", :ip => "192.168.140.14", :cpus => 4, :mem => 1024 },
  	{ :hostname => "devans6", :ip => "192.168.140.15", :cpus => 2, :mem => 1024 },
  	{ :hostname => "devans7", :ip => "192.168.140.16", :cpus => 2, :mem => 1024 }
	]

	# define /etc/hosts for all servers
  NODES.each do |node|
   	etcHosts += "echo '" + node[:ip] + "   " + node[:hostname] + "' >> /etc/hosts" + "\n"
  end #end NODES

	# run installation
  NODES.each do |node|
    config.vm.define node[:hostname] do |cfg|
			cfg.vm.hostname = node[:hostname]
      cfg.vm.network "private_network", ip: node[:ip]
      cfg.vm.provider "virtualbox" do |v|
				v.customize [ "modifyvm", :id, "--cpus", node[:cpus] ]
        v.customize [ "modifyvm", :id, "--memory", node[:mem] ]
        v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
        v.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
        v.customize ["modifyvm", :id, "--name", node[:hostname] ]
      end #end provider
			
			#for all
      cfg.vm.provision :shell, :inline => etcHosts
			cfg.vm.provision :shell, :inline => common

    end # end config
  end # end nodes
end

I like to use the NODES list where I can describe the type of servers I want to use.

And now, ansible and the docker swarm

And first of all, we need to describe our infrastructure in the inventory file. We create two group, one for our managers and one for workers

This is our 00_inventory.yml file :

all:
  children:
    managers:
      hosts:
        192.168.140.10:
        192.168.140.11:
        192.168.140.12:
        #192.168.140.15:
        #192.168.140.16:
    workers:
      hosts:
        192.168.140.13:
        192.168.140.14:

I keep two nodes if we want to test to add a new manager.

And of course we need to install docker on all nodes. To do it I suggest you to create a docker role in roles directory.

mkdir -p roles/
ansible-galaxy init roles/docker

And we begin to edit tasks in the main file of tasks directory.

- name: add gpg key
  apt_key:
    url: "{{ docker_repo_key }} "
    state: present

- name: Add repository
  apt_repository:
    repo: "{{ docker_repo }}"

- name: install docker and dependencies
  apt:
    name: "{{ docker_packages }}"
    state: latest
    update_cache: yes
    cache_valid_time: 3600
  with_items: "{{ docker_packages}}"

- name: Add user to docker group
  user:
    name: vagrant
    group: docker

- name: start docker
  service:
    name: docker
    state: started
    enabled: yes

Great and we can set these jinja variables in the default directory.

docker_packages:
  - apt-transport-https
  - ca-certificates
  - curl
  - software-properties-common
  - docker-ce
  - docker-ce-cli
  - containerd.io
  - python3-docker
docker_repo: deb [arch=amd64] https://download.docker.com/linux/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable
docker_repo_key: https://download.docker.com/linux/ubuntu/gpg
docker_repo_key_id: 0EBFCD88

And now we need to create our playbook file with two roles : docker and swarm.

- name: install docker
  hosts: all
  become: yes
  roles:
   - docker
   - swarm

Good, of course we need to write our swarm role. And this is very very easy.

The docker swarm cluster installation

And we need to create our swarm role skeleton with ansible-galaxy CLI.

ansible-galaxy init roles/swarm

And of course, we edit our tasks file in the tasks directory.

Why it’s easy ? we just need to add 3 tasks to init our cluster, add managers and our workers.

- name: check/init swarm
  docker_swarm:
    state: present
    advertise_addr: enp0s8:2377
  register: __output_swarm
  when: inventory_hostname in groups['managers'][0]

- name: install manager
  docker_swarm:
    state: join
    timeout: 60
    advertise_addr: enp0s8:2377
    join_token: "{{ hostvars[groups['managers'][0]]['__output_swarm']['swarm_facts']['JoinTokens']['Manager']}}"
    remote_addrs: "{{ groups['managers'][0] }}"
  when: inventory_hostname in groups['managers'] and inventory_hostname not in groups['managers'][0]

- name: install worker
  docker_swarm:
    state: join
    timeout: 60
    advertise_addr: enp0s8:2377
    join_token: "{{ hostvars[groups['managers'][0]]['__output_swarm']['swarm_facts']['JoinTokens']['Worker'] }}"
    remote_addrs: "{{ groups['managers'][0] }}"
  when: inventory_hostname in groups['workers']

The subtlety is how we collect and use the join token (one for manager and one for worker).

In the first task we initialize our cluster on our first manager, the leader. To do it, we use the first index/node int the manager group. And we register the output of this initialize task in a register __output_swarm.

That allows us to add all other managers (not leader).

join_token: "{{ hostvars[groups['managers'][0]]['__output_swarm']['swarm_facts']['JoinTokens']['Manager']}}"

In this line we use on all other manager the join token that we have collected on the leader.

And we do the same task for workers.

That’s all we can run it now.

ansible-playbook -i 00_inventory.yml -u vagrant -k playbook.yml

And to check our swarm cluster, we can list all nodes in our docker cluster.

docker nodes ls

You can our leader, all manager and our worker.

Maybe you can add a node in your inventory and run your playbook again ??

All the code here : my gitlab

My blog : https://xavki.blog