Categories
Python

Bury the body

What should we do with the body?

Keep sending it back to Watson to check state until Watson burys it.

But… what happens if the body is floating around twice?

We have a problem here. This is option 1:

^^^ try follow that shit. Problem is you end up delaying processing an update best case scenario. Worst case is the body is lost somewhere due to an error and the whole thing Jams up.

Option 2:
Ahh, but what about the caseinfo? If I have sent on the body+case already I will loose data. Actually I am making sure I loose data… I will ALWAYS loose data doing this

I decided on option 3…

TBC

Categories
Python

Messaging and time

Remember, there is no guaranteed delivery time of a message; The destination module may be broken, or it may be busy.

On data changes verify the change prior to making it in a way that fixes the amount of time between validation and edit.

ie.

findings = body['caseinfo']['findings']
if 'Needs to be assigned to new team' in findings:                                    
    l.info(f'Moriorty to assign {body["ticket"]} to new team')                        
    teamAssignedField = 'customfield_1XXX'                                          
    #Need to validate current state as there are no time guarantees in the message   
    issue = getJiraTicketInfo(body['issueURL'])      # <--- this is a function within the module code, no messaging needed                                
    teamCurrent = ''                                                                 
    try:                                                                             
        teamCurrent = issue['fields']['customfield_10311']['name']   #who knows what crap will be returned                
    except:                                                                          
        teamCurrent = ''                                                             
    if teamCurrent == 'old-team':                                                      
        l.info(f'Validated {body["ticket"]} is assigned to old-team')                  
        putData = {}                                                                 
        putData['fields'] = {teamAssignedField : 'new-field'}          
        pushUpdate = postJiraTicketInfo(body, putData)  # <=== also a function in the module code                             

This way I don’t have to worry about material delays. Ideally the api should really accept a validation value but it doesn’t. The change will only be made if the data is in the same (relevant) state as when the original check took place .

Because of this the various modules tend to interact with one system, both reads AND writes….. Except the slack integration of course, I have Lestrad sending messages and Lestrad’s phone answering them 🙂

One issue I can see if the value in one system depends on the state of another system… but I’m not sure if that is an actual issue, as you could just send the body back through the processing chain to double check afterwards maybe… I’ll burn that bridge when I get to it.

Categories
Python

Tracking the body

I seem to be having a bit of an issue regarding the messaging paradigm.

Essentially, I am finding that as there is no time guarentees of events getting processed, we have to think about the flow of the “body”.

I am calling my message body when it is the incident + case info

I have to be careful about “duplicating” the body, that is, sending the message away in a queue, and continuing to take action here. as they will loose sync and state.

I would rather just try to ensure the statelessness nature of the message processing code.

In other words, Watson will mark a post as a “first post”, send it to Lestrad to post as a first post.

Initially I wanted to continue processing the body in Watson to further establish state, but there is no way to get the initial time stamp of the initial slack post back into the body that stayed in watson.

Although the timestamps are in postgres and pushed into redis, there is NO guarantee about how long that will take, and no clean way to check if the action has taken place yet or not….

because… I forgot to mention that only one module is allowed talk to redis and postgres. This may well be a terrible architectural decision….

For now, the only solution is to send the body back to watson from Lestrad with the new TS, and let Watson re-establish the state.

Left is Watson, middle is Lestrad, and right is the hound

Categories
Python

Checking redis livliness

Use this within functions which are doing to use redis

     try:
         if reddisConnect():
             l.info('Redis connection is up')                                                                                                                                                                                             
             pass
     except Exception as err:
         l.error(f'Some problem with redis: {err}')

This is the connection function defined:

### redis config

def reddisConnect():
    redpass = "Hidden"
    global red
    try:
        try:
            if red.ping():
                l.info('Redis connecton set up and validated')
                return True
        except Exception:
            try:
                red = redis.Redis(password=redpass)
            except Exception as err:
                raise err
    except Exception as err:
        l.error(f'Problem setting up redis connection: {err}')

Hopefully that’s it. The postgres connection is a bit more brute force than that. I’ll fix it once I can see that global keyword is working as I hope it does.

I have been finding with the rabbitMQ call back functions that I need to double check that global variables are working like I hope they are!

Categories
Python

Messaging paradigm in Python

I’m not a fan of async when programming in Python. Although I have programmed in C++, I am not a fan of classes. By not a fan I mean I have no idea how to use them.

What I have is a goal and a need. The idea is using the messaging paradigm as a structure of Python coding. This and a functional aspect.

Structure:

pipenv, docker-compose, rabbitMQ for messaging, postgres for some “data” data and redis for some state data (ie. persistent key values

Philosophy:

Idea is that the “app” can be stateless. Each .py program performs a task based on message traffic received, interacts with one system or does one thing, and then sends back a message.
At them moment I am sending the “initial message” with a trigger script called by cron.