I have struggled finding what I would consider a nice way to connect to Rabbit MQ from a python program.
Some issues I have found that slack overflow answers are not great if you are reusing connections or have something that isn’t just a single sending program.
My use case, where I am programming around a narrative (in this new project I am using “The Watch” from Terry Pratchett’s Discworld.
In my last project I had what amounted to boiler plate code in each “Character” (python app). For this new project (the Quantum one), I wanted something a bit easier and cleaner.
So let me introduce TheLawsandOrdinancesoftheCitiesAnkhandMorpork.py
This is a “helper” module that I will import * from. This is how I wrote the messaging side. This produces less crap in the Rabbit MQ logs than calling a channel.close() also:
import pika
import json
import logging as l
exchange = ''
def sendMessage(person, patrol):
try:
patrol = json.dumps(patrol)
except Exception as err:
l.error(f'could not dump the patrol to a json')
try:
with pika.BlockingConnection(pika.ConnectionParameters('rabbitmq')) as connection:
#connection = pika.BlockingConnection(pika.ConnectionParameters('rabbitmq'))
channelM = connection.channel()
channelM.queue_declare(queue=person)
channelM.basic_publish(exchange=exchange, routing_key=person, body=patrol)
except Exception as err:
l.error(f'Problem sending rabbit message: {err}')
On my screen that is a bit hard to read;
What I am doing is a function that takes the name of a “character” (ie. a queue) and a dict. I then encode this as a JSON, and using the “with” block, send it to the correct queue.
By avoiding the connection.close() call I no longer get and error in rabbitMQ saying connection closed unexpectedly.
It may not be the most efficient, not reusing an existing connection. But the connection bringup is tiny, and I would rather take cleanliness over efficiency in this case.
I will be running quantum simulations so the bottleneck will not be here!
2 replies on “Using RabbitMQ (pika) with Python”
Thanks for putting together a post about how you use RabbitMQ. There’s a couple of things I do not agree with.
“Connection closed unexpectedly” is a warning and not an error. It is described in https://rabbitmq.com/logging.html#logged-events.
Team RabbitMQ certainly recommends long-lived connections https://rabbitmq.com/connections.html#lifecycle. One-off connections lead to a very problematic scenario known as high connection churn https://rabbitmq.com/connections.html#high-connection-churn. Modern
RabbitMQ versions even have a metric that helps detect it.
Thanks for the feedback MK. In my case I’m looking at a few connections a second max, so I am hoping that won’t cause enough churn to matter.
In my other big app I kept a connection open, so if there is a an issue I can go back to doing it that way. Thank you for the links tho, I will have a read over them!