Infrastructure Management with SaltStack: Part 3 – Reactor and Events

By | March 19, 2014
Newton's Third Law

Newton’s Third Law

The Journey So Far:
Infrastructure Management with SaltStack: Part 1 – The Setup
Infrastructure Management with SaltStack: Part 2 – Grains, States, and Pillar

Introduction

By now I hope I have demonstrated the power of Salt and the simplicity underlying its configuration. Now we take another step down the salty path of Data Center management. The salt-master to salt-minion conversation to this point has been largely one way. Let’s discuss how Salt handles the other side, when minions talk back.

Minions talk back to the master by issuing Events. As of Salt version 0.11.0 an event handling system was introduced called Reactor.

Reactor, quite simply “reacts” to Events. To understand this we need to know two things: what is an Event, and what is a reaction.

Events and Reactions

The Reactor event system is technically a local ZeroMQ PUB interface. This means that the Events are sent and received via ZeroMQ. An Event is comprised of two parts: a “tag” and a dict type data structure. The tags in Events that match tags declared in the master config file cause SLS files to  be executed and actions performed. The data structure contains information about the event in a key:value format.

The data of the event can be used in conditional logic to decide what actions to take when a certain tag triggers reactor.

Reactions are contained in Reactor SLS files stored in /srv/reactor. Reactor SLS files are are alot like State and Pillar SLS files, however they also support the tag and data variables. The reactor config file that defines reactions based on tags is stored in /etc/salt/master.d/reactor.conf.

A Simple Example

Let’s walk through a simple example setup.

The reactor.conf does not exist when Salt is first installed, so it needs to be created. Change directory to /etc/salt/master.d and create the file.

cd /etc.salt.master.d
vi ./reactor.conf

salt301

Note: As always you need to type “i” to enter input mode in vi

The first line in the file defines that it is a reactor config.

reactor:

Next the tags are defined. The syntax is one indent + “-” + tag name + “:”
The tag name is always defined with single quotes.
This tag says “every time any minion service starts”.

  - 'salt/minion/*/start':

Now the event is mapped to a reaction SLS file. The syntax is two indents + “-” + path to the reactor SLS

    -/srv/reactor/start.sls

The file will look like this:

salt302
Save and quit vi by typing “escape”, “:”, and “wq”.

/srv/reactor/start.sls does not magically exist. So we need to create it and the /srv/reactor directory as well.

mkdir /srv/reactor
vi /srv/reactor/start.sls

Make sure to enter input mode with “i”.

salt303

The first line in this SLS file introduces a syntax beyond the YAML we have been using, Jinja, a template engine for Python.
The command is starting an if block matching the “id” key from the “data” dict to the string value “salt-minion”. The “id” key contains the hostname of the system running the minion.

{% if data['id'] == 'salt-minion' %}

The next line states that if the host is “salt-minion” use the reaction highstate_run. This reaction is defined in the lines that follow.

highstate_run:

The reaction is then defined as the Execution Module cmd.highstate.run. This executes the highstate.run function on the minion.

  cmd.state.highstate:

The next line is an argument for the previous line. It states that the target of the highstate.run function is the host “salt-minion”.

    - tgt: salt-minion

Finally the if block is closed out.
{% endif %}

The final file looks like this:

salt305

Save the file and exit vi.
Now we have a completely defined reaction.

Verification and Troubleshooting

To test the Reactor system we can fire off events from the minion from the CLI and listen for them on the master.

On The salt-master

To trap an event on the salt-master we need to run the eventlisten.py script. This is included in the Salt source. However if you installed from a package run the following:

wget https://raw.github.com/saltstack/salt/develop/tests/eventlisten.py

Now execute the script.

python eventlisten.py

On The salt-minion

Run the following command:

salt-call event.fire_master '{"id": "salt-minion"}' 'start'

If the event generation succeeds you will see this on the minion host:

salt306

On the salt-master you will see this:

salt307

This verifies that the Reactor system is working.

For further reading check out the SaltStack docs on Reactor.

In the next post I will discuss the use of Job Management in Salt. Stay tuned.

One thought on “Infrastructure Management with SaltStack: Part 3 – Reactor and Events

  1. adilroot

    Thank you very much Byron for the tutorial quality. It helps me a lot, now i can go confident configure my salt plateforme.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *