Categories
Python

Using RabbitMQ (pika) with Python

I have struggled finding what I would consider a nice way to connect to Rabbit MQ from a python program.

Some issues I have found that slack overflow answers are not great if you are reusing connections or have something that isn’t just a single sending program.

My use case, where I am programming around a narrative (in this new project I am using “The Watch” from Terry Pratchett’s Discworld.

In my last project I had what amounted to boiler plate code in each “Character” (python app). For this new project (the Quantum one), I wanted something a bit easier and cleaner.

So let me introduce TheLawsandOrdinancesoftheCitiesAnkhandMorpork.py

This is a “helper” module that I will import * from. This is how I wrote the messaging side. This produces less crap in the Rabbit MQ logs than calling a channel.close() also:

import pika
import json
import logging as l
exchange = ''

def sendMessage(person, patrol):                                                           
    try:                                                                                   
        patrol = json.dumps(patrol)                                                        
    except Exception as err:                                                               
        l.error(f'could not dump the patrol to a json')                                    
    try:                                                                                   
        with pika.BlockingConnection(pika.ConnectionParameters('rabbitmq')) as connection: 
            #connection = pika.BlockingConnection(pika.ConnectionParameters('rabbitmq'))   
            channelM = connection.channel()                                                
            channelM.queue_declare(queue=person)                                           
            channelM.basic_publish(exchange=exchange, routing_key=person, body=patrol)     
    except Exception as err:                                                               
        l.error(f'Problem sending rabbit message: {err}')                                  

On my screen that is a bit hard to read;

What I am doing is a function that takes the name of a “character” (ie. a queue) and a dict. I then encode this as a JSON, and using the “with” block, send it to the correct queue.

By avoiding the connection.close() call I no longer get and error in rabbitMQ saying connection closed unexpectedly.

It may not be the most efficient, not reusing an existing connection. But the connection bringup is tiny, and I would rather take cleanliness over efficiency in this case.

I will be running quantum simulations so the bottleneck will not be here!

2 replies on “Using RabbitMQ (pika) with Python”

Thanks for putting together a post about how you use RabbitMQ. There’s a couple of things I do not agree with.

“Connection closed unexpectedly” is a warning and not an error. It is described in https://rabbitmq.com/logging.html#logged-events.

Team RabbitMQ certainly recommends long-lived connections https://rabbitmq.com/connections.html#lifecycle. One-off connections lead to a very problematic scenario known as high connection churn https://rabbitmq.com/connections.html#high-connection-churn. Modern
RabbitMQ versions even have a metric that helps detect it.

Thanks for the feedback MK. In my case I’m looking at a few connections a second max, so I am hoping that won’t cause enough churn to matter.

In my other big app I kept a connection open, so if there is a an issue I can go back to doing it that way. Thank you for the links tho, I will have a read over them!

Leave a Reply

Your email address will not be published. Required fields are marked *