redis – Blogosourous

My main source of data… jira service desk… cannot be trusted. Oh yes, 90% of the time it acts sane, but once in a while, someone misconfigures a big panda alert and we get 1000 new incidents in moments.

My colleague got flooded with pager duties and emails one Saturday, and I’m sure it made him cry. I need to rate limit these actions.

My plan is to use incr function to increment a redis key. The idea is that you create a new key every x seconds, with an expiry (equal to x) and then when you increment redis will respond with the current value, so you can just test the if loop.

First things first. I know how to do something every second int(time)) but every 10 minutes?

from time import time, sleep
while True:
     everyXSeconds = 10
     curTime = int(time())
     key = curTime - curTime % everyXSeconds
     print(key)
     sleep(1)

I’m only calling time() once because calling time() twice in the mod actually give different nanosecond values… and I just know I am going to hit some corner case where that crosses a second boundary and messes up EVERYTHING

BUT… the above is actually crap and not needed at all. That is a crap way to rate limit cause, for example, if I am trying to stop an email flood it will still send 10 emails every X seconds. As in, the count will be reset every X seconds.

To me the following makes a bit more sense:

while True:
    curTime = time()
    print(f"curTime is {curTime}")
    everyXSeconds = 10
    rateBeginTime = curTime - everyXSeconds
    count = red.zcount(queueName, rateBeginTime, curTime)
    print(f"current hit queue is :{count}")
    limitMap = {curTime: curTime}
    red.zadd(queueName, limitMap)
    sleep(1)
    remove = red.zremrangebyscore(queueName, 0, rateBeginTime)
    print(f"removed {remove} keys")

So thats just what I used to test…. but basically you are using a sorted list and adding and removing as needed

Use this within functions which are doing to use redis

     try:
         if reddisConnect():
             l.info('Redis connection is up')                                                                                                                                                                                             
             pass
     except Exception as err:
         l.error(f'Some problem with redis: {err}')

This is the connection function defined:

### redis config

def reddisConnect():
    redpass = "Hidden"
    global red
    try:
        try:
            if red.ping():
                l.info('Redis connecton set up and validated')
                return True
        except Exception:
            try:
                red = redis.Redis(password=redpass)
            except Exception as err:
                raise err
    except Exception as err:
        l.error(f'Problem setting up redis connection: {err}')

Hopefully that’s it. The postgres connection is a bit more brute force than that. I’ll fix it once I can see that global keyword is working as I hope it does.

I have been finding with the rabbitMQ call back functions that I need to double check that global variables are working like I hope they are!