Install and Configure RabbitMQ Integration with vCloud Director

By | March 6, 2014

RabbitMQ.sh-600x600

Defining Automation With Messaging

In today’s world of distributed systems and cloud computing  independent compute nodes need to communicate with each other to enable end to end services. The process of enabling this communication is the core of automation. In the client-server model servers generally communicated with each other via messaging services like Microsoft’s Distributed Transaction Coordinator. As systems became more complex and optimized for speed the Advanced Message Queuing Protocol (AMQP) was developed. AMPQ has become the glue that binds the “cloud” together.

AMPQ leverages a broker to route binary objects (“messages”) through an exchange to a queue where it is retrieved by one or more consumers (recipient compute nodes). The letter-mailbox-postman metaphor is often used to explain this process, however it is a bit simplistic. The broker is the “post office” in this metaphor and current broker of choice for most folks is RabbitMQ.

vCloud Director has the ability to send messages over AMPQ natively. These messages can either be in the form of constant system notifications or generated by a Blocking Task. A Blocking Task is generated by a pre-selected action. These actions include the creation, modification, or deletion of most objects in vCD.

The most common end point for the messages generated from vCloud Director are vCenter Orchestrator Workflows leveraging the AMPQ plugin. This enables powerful possibilities for automation. My favorite example is Auto-Deploying hosts then adding them to a Provider vDC to  scale compute resources of an Organization vDC based on demand metrics. You can then leverage another Workflow to remove the hosts when demand subsides. All of this being done auto-magically because of message queues.

The first step in taking advantage of this power is deploying a RabbitMQ server and connecting it to vCloud Director. Here’s how it is done.

Deploying RabbitMQ 

1. Create a default Ubuntu 64-bit VM and install Ubuntu Server 64-bit 12.04 LTS (“Precise Pangolin”) or deploy an instance from a template.

2. SSH to the newly created server.

3. Login in as the user created during the install and then switch to root with this command

sudo su -

4. Append the following line to /etc/apt/sources.list

deb http://www.rabbitmq.com/debian/ testing main

5. Add the RabbitMQ public key to the trusted key list

wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

apt-key add rabbitmq-signing-key-public.asc

6. Update the apt repos

apt-get update

7. Install RabbitMQ

apt-get install rabbitmq-server

8. The RabbitMQ service will be started automatically once the install finishes.

9. Create a new user named “vcd”

rabbitmqctl add_user vcd password

10. Tag the user as administrator

rabbitmqctl set_user_tags vcd administrator

11. Give the user full permissions to the default vhost “/”

rabbitmqctl set_permissions -p / vcd ".*" ".*" ".*"

RabbitMQ is now setup.

Note: To setup web based management of the system do the following:

rabbitmq-plugins enable rabbitmq_management
invoke-rc.d rabbitmq-server stop
invoke-rc.d rabbitmq-server start

When it restarts verify the service by running:

rabbitmqctl status

The results should look like this:

RabbitRunning

 

Configuring vCloud Director

1. Connect to vCD as the System Administrator

2. Navigate to the Administration tab, Expand System Settings, and select Extensibility

vcdamqp

 

3. Click “Enable Notifications”.

4.Add the IP address of the newly created RabbitMQ server to the AMQP Host field.

5. Leave AMQP Port, Exchange, vHost, and Prefix as the default.

6. Leave SSL unchecked.

7. Change the user to “vcd” and input the password assigned when the user was created.

rabbituser

 

8. Click “Test AMQP Connection” to verify the connection

9. Click Apply.

Due to the way AMQP works an exchange and a queue will be created the first time that vCD connects.

10. Wait a few minutes and verify the  exchange exists by running the command:

rabbitmqctl list_exchanges

The results should look like this:

listexch

 

You are now feeding the message queue from vCloud Director.

Stay tuned for how to leverage this queue in vCenter Orchestrator in a new post very soon.