Categories
Python

Nested Dict Gets in python

Update- the below doesn’t work at all actually.. you do need to do a try-except block cause if any of the values are, for example NONE. The whole thing craps out with an exception.

I’ll leave the below to…. Show my workings!

Trying to figure out the best way to parse out complex JSON.

Saying

Level = issue.get('fields').get('customfield_1', 'NA')

Doesn’t actually work if ‘fields’ is missing. Error is:

AttributeError: 'NoneType' object has no attribute 'get'

you actually need to say:

In [1]: d = {'parent':{'test':True}}

In [2]: d
Out[2]: {'parent': {'test': True}}

In [3]: d.get('parent')
Out[3]: {'test': True}

In [4]: d.get('parentt')

In [5]: d.get('parent').get('test')
Out[5]: True

In [6]: d.get('parenT').get('test')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-6-142a5af0ba34> in <module>
----> 1 d.get('parenT').get('test')

AttributeError: 'NoneType' object has no attribute 'get'

In [7]: d.get('parenT', {}).get('test')

In other words, in the case of nested gets, make sure the second attribute (the one that returns if your key is missing) is an empty dict… ie {}

Categories
Parenting

nappy rash, how to avoid and minimise

Use water wipes.
They are fine for wee and don’t cause a rash.
As your kids gets older and eats more food their poo gets way more horrible.
So you need to get a big tub of silcoks base from chemist.
Use with water wipes to clean poo off.
Don’t use soap, at this point I’m sure it’s when you clean them too much, and I’m guessing kill a lot of bacteria in the arse region, they get a fungus infection… Nappy rash.

Silcoks base avoids this but let’s you clean off the shit without using 200 wipes.

Categories
Python

Pop nested dict in python

I have struggled to find the following examples:

hello = {'a':{'b':2, 'c':3}}
#I want to pop c (which is nested in a)
hello.get('a').pop('c', None)

This took me too long to figure out

Categories
Python

Bury the body

What should we do with the body?

Keep sending it back to Watson to check state until Watson burys it.

But… what happens if the body is floating around twice?

We have a problem here. This is option 1:

^^^ try follow that shit. Problem is you end up delaying processing an update best case scenario. Worst case is the body is lost somewhere due to an error and the whole thing Jams up.

Option 2:
Ahh, but what about the caseinfo? If I have sent on the body+case already I will loose data. Actually I am making sure I loose data… I will ALWAYS loose data doing this

I decided on option 3…

TBC

Categories
Python

Messaging and time

Remember, there is no guaranteed delivery time of a message; The destination module may be broken, or it may be busy.

On data changes verify the change prior to making it in a way that fixes the amount of time between validation and edit.

ie.

findings = body['caseinfo']['findings']
if 'Needs to be assigned to new team' in findings:                                    
    l.info(f'Moriorty to assign {body["ticket"]} to new team')                        
    teamAssignedField = 'customfield_1XXX'                                          
    #Need to validate current state as there are no time guarantees in the message   
    issue = getJiraTicketInfo(body['issueURL'])      # <--- this is a function within the module code, no messaging needed                                
    teamCurrent = ''                                                                 
    try:                                                                             
        teamCurrent = issue['fields']['customfield_10311']['name']   #who knows what crap will be returned                
    except:                                                                          
        teamCurrent = ''                                                             
    if teamCurrent == 'old-team':                                                      
        l.info(f'Validated {body["ticket"]} is assigned to old-team')                  
        putData = {}                                                                 
        putData['fields'] = {teamAssignedField : 'new-field'}          
        pushUpdate = postJiraTicketInfo(body, putData)  # <=== also a function in the module code

This way I don’t have to worry about material delays. Ideally the api should really accept a validation value but it doesn’t. The change will only be made if the data is in the same (relevant) state as when the original check took place .

Because of this the various modules tend to interact with one system, both reads AND writes….. Except the slack integration of course, I have Lestrad sending messages and Lestrad’s phone answering them 🙂

One issue I can see if the value in one system depends on the state of another system… but I’m not sure if that is an actual issue, as you could just send the body back through the processing chain to double check afterwards maybe… I’ll burn that bridge when I get to it.

Categories
Python

Tracking the body

I seem to be having a bit of an issue regarding the messaging paradigm.

Essentially, I am finding that as there is no time guarentees of events getting processed, we have to think about the flow of the “body”.

I am calling my message body when it is the incident + case info

I have to be careful about “duplicating” the body, that is, sending the message away in a queue, and continuing to take action here. as they will loose sync and state.

I would rather just try to ensure the statelessness nature of the message processing code.

In other words, Watson will mark a post as a “first post”, send it to Lestrad to post as a first post.

Initially I wanted to continue processing the body in Watson to further establish state, but there is no way to get the initial time stamp of the initial slack post back into the body that stayed in watson.

Although the timestamps are in postgres and pushed into redis, there is NO guarantee about how long that will take, and no clean way to check if the action has taken place yet or not….

because… I forgot to mention that only one module is allowed talk to redis and postgres. This may well be a terrible architectural decision….

For now, the only solution is to send the body back to watson from Lestrad with the new TS, and let Watson re-establish the state.

Left is Watson, middle is Lestrad, and right is the hound

Categories
Python

Checking redis livliness

Use this within functions which are doing to use redis

     try:
         if reddisConnect():
             l.info('Redis connection is up')                                                                                                                                                                                             
             pass
     except Exception as err:
         l.error(f'Some problem with redis: {err}')

This is the connection function defined:

### redis config

def reddisConnect():
    redpass = "Hidden"
    global red
    try:
        try:
            if red.ping():
                l.info('Redis connecton set up and validated')
                return True
        except Exception:
            try:
                red = redis.Redis(password=redpass)
            except Exception as err:
                raise err
    except Exception as err:
        l.error(f'Problem setting up redis connection: {err}')

Hopefully that’s it. The postgres connection is a bit more brute force than that. I’ll fix it once I can see that global keyword is working as I hope it does.

I have been finding with the rabbitMQ call back functions that I need to double check that global variables are working like I hope they are!

Categories
Python

Messaging paradigm in Python

I’m not a fan of async when programming in Python. Although I have programmed in C++, I am not a fan of classes. By not a fan I mean I have no idea how to use them.

What I have is a goal and a need. The idea is using the messaging paradigm as a structure of Python coding. This and a functional aspect.

Structure:

pipenv, docker-compose, rabbitMQ for messaging, postgres for some “data” data and redis for some state data (ie. persistent key values

Philosophy:

Idea is that the “app” can be stateless. Each .py program performs a task based on message traffic received, interacts with one system or does one thing, and then sends back a message.
At them moment I am sending the “initial message” with a trigger script called by cron.