An easy way to install a docker SWARM cluster with ANSIBLE ?

In a previous, we installed a prometheus stack with ansible : node exporter, prometheus, grafana… And I propose to install a new stack : a docker swarm cluster and some tools.

In this post, we’ll just install a simple swarm cluster. And we’ll use vagrant to have our infrastructure to develop it. And after, with few lines of ansible code, we’ll have our docker cluster. And this is very very easy.

Our vagrant infrastructure

The vagrantfile

Vagrant.configure(2) do |config|

# Set some variables
etcHosts=""
common = <<-SHELL
sudo apt update -qq 2>&1 >/dev/null
sudo apt install -y -qq git vim tree net-tools telnet git python3-pip sshpass nfs-common 2>&1 >/dev/null
sudo echo "autocmd filetype yaml setlocal ai ts=2 sw=2 et" > /home/vagrant/.vimrc
sed -i 's/ChallengeResponseAuthentication no/ChallengeResponseAuthentication yes/g' /etc/ssh/sshd_config
sudo systemctl restart sshd
SHELL

# set vagrant image
config.vm.box = "ubuntu/focal64"
config.vm.box_url = "ubuntu/focal64"

# set servers list and their parameters
NODES = [
{ :hostname => "devans1", :ip => "192.168.140.10", :cpus => 4, :mem => 1024 },
{ :hostname => "devans2", :ip => "192.168.140.11", :cpus => 4, :mem => 1024 },
{ :hostname => "devans3", :ip => "192.168.140.12", :cpus => 4, :mem => 1024 },
{ :hostname => "devans4", :ip => "192.168.140.13", :cpus => 4, :mem => 1024 },
{ :hostname => "devans5", :ip => "192.168.140.14", :cpus => 4, :mem => 1024 },
{ :hostname => "devans6", :ip => "192.168.140.15", :cpus => 2, :mem => 1024 },
{ :hostname => "devans7", :ip => "192.168.140.16", :cpus => 2, :mem => 1024 }
]

# define /etc/hosts for all servers
NODES.each do |node|
etcHosts += "echo '" + node[:ip] + " " + node[:hostname] + "' >> /etc/hosts" + "\n"
end #end NODES

# run installation
NODES.each do |node|
config.vm.define node[:hostname] do |cfg|
cfg.vm.hostname = node[:hostname]
cfg.vm.network "private_network", ip: node[:ip]
cfg.vm.provider "virtualbox" do |v|
v.customize [ "modifyvm", :id, "--cpus", node[:cpus] ]
v.customize [ "modifyvm", :id, "--memory", node[:mem] ]
v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
v.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
v.customize ["modifyvm", :id, "--name", node[:hostname] ]
end #end provider

#for all
cfg.vm.provision :shell, :inline => etcHosts
cfg.vm.provision :shell, :inline => common

end # end config
end # end nodes
end

I like to use the NODES list where I can describe the type of servers I want to use.

And now, ansible and the docker swarm

This is our 00_inventory.yml file :

all:
children:
managers:
hosts:
192.168.140.10:
192.168.140.11:
192.168.140.12:
#192.168.140.15:
#192.168.140.16:
workers:
hosts:
192.168.140.13:
192.168.140.14:

I keep two nodes if we want to test to add a new manager.

And of course we need to install docker on all nodes. To do it I suggest you to create a docker role in roles directory.

mkdir -p roles/
ansible-galaxy init roles/docker

And we begin to edit tasks in the main file of tasks directory.

- name: add gpg key
apt_key:
url: "{{ docker_repo_key }} "
state: present

- name: Add repository
apt_repository:
repo: "{{ docker_repo }}"

- name: install docker and dependencies
apt:
name: "{{ docker_packages }}"
state: latest
update_cache: yes
cache_valid_time: 3600
with_items: "{{ docker_packages}}"

- name: Add user to docker group
user:
name: vagrant
group: docker

- name: start docker
service:
name: docker
state: started
enabled: yes

Great and we can set these jinja variables in the default directory.

docker_packages:
- apt-transport-https
- ca-certificates
- curl
- software-properties-common
- docker-ce
- docker-ce-cli
- containerd.io
- python3-docker
docker_repo: deb [arch=amd64] https://download.docker.com/linux/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable
docker_repo_key: https://download.docker.com/linux/ubuntu/gpg
docker_repo_key_id: 0EBFCD88

And now we need to create our playbook file with two roles : docker and swarm.

- name: install docker
hosts: all
become: yes
roles:
- docker
- swarm

Good, of course we need to write our swarm role. And this is very very easy.

The docker swarm cluster installation

ansible-galaxy init roles/swarm

And of course, we edit our tasks file in the tasks directory.

Why it’s easy ? we just need to add 3 tasks to init our cluster, add managers and our workers.

- name: check/init swarm
docker_swarm:
state: present
advertise_addr: enp0s8:2377
register: __output_swarm
when: inventory_hostname in groups['managers'][0]

- name: install manager
docker_swarm:
state: join
timeout: 60
advertise_addr: enp0s8:2377
join_token: "{{ hostvars[groups['managers'][0]]['__output_swarm']['swarm_facts']['JoinTokens']['Manager']}}"
remote_addrs: "{{ groups['managers'][0] }}"
when: inventory_hostname in groups['managers'] and inventory_hostname not in groups['managers'][0]

- name: install worker
docker_swarm:
state: join
timeout: 60
advertise_addr: enp0s8:2377
join_token: "{{ hostvars[groups['managers'][0]]['__output_swarm']['swarm_facts']['JoinTokens']['Worker'] }}"
remote_addrs: "{{ groups['managers'][0] }}"
when: inventory_hostname in groups['workers']

The subtlety is how we collect and use the join token (one for manager and one for worker).

In the first task we initialize our cluster on our first manager, the leader. To do it, we use the first index/node int the manager group. And we register the output of this initialize task in a register __output_swarm.

That allows us to add all other managers (not leader).

join_token: "{{ hostvars[groups['managers'][0]]['__output_swarm']['swarm_facts']['JoinTokens']['Manager']}}"

In this line we use on all other manager the join token that we have collected on the leader.

And we do the same task for workers.

That’s all we can run it now.

ansible-playbook -i 00_inventory.yml -u vagrant -k playbook.yml

And to check our swarm cluster, we can list all nodes in our docker cluster.

docker nodes ls

You can our leader, all manager and our worker.

Maybe you can add a node in your inventory and run your playbook again ??

All the code here : my gitlab

My blog : https://xavki.blog

Microservices architecture and opensource. I’m maintainer of xavki https://youtube.com/c/xavki-linux about opensource. My blog : https://xavki.blog/