A reason to use state with Octoblu

I have been posting an 8 part series going though some advanced use of Octoblu.

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node
Part 6: Reformatting nested JSON with JavaScript
Part 7: Logical data nesting with your Octoblu state device
Part 8:

Back at the beginning I introduced the concept of a state device.

Now, if you aren't yet understanding why I might introduce a state device, consider this:

Have you ever found yourself using SetKey and GetKey within flows to persist data, even if only for a little while?

Have you ever run into complex timing issues where you would love to break something into multiple flows and instead end up with one huge complex one?

This is where the state device is an easy fit.  Persist your data, in a common object that you can reference between flows.

Then, instead of relying on some message chugging through the system, you act upon a change to your state device.  So you could dev null some message, perform no update and exit your logic stream.

In the example I have been laying out I have two primary scenarios: 

Scenario 1: there are multiple incoming data sources

I have multiple devices that are all similar and they are feeding in data that I need to evaluate in a common way.  Each flow can update my state device independently, and then I simply have one evaluation flow to determine if I am going to send out my alert.

Scenario 2: there are multiple data listener paths

Just the opposite.  I have one primary input data source, it is big and complex.

Then I have multiple flows, each of which evaluates a specific type of data or specific properties.

Either way, it allows me to compartmentalize my flow logic and reduce / remove redundancy across the system.

So I end up with something like this:

Combined with this:

To do what I was doing in the first screenshot.

The big upside for me is that I removed all of the hardcoded naming filtering that I started with in order to persist the data.

The flows are now able to be more dynamic and handle the same sets of data no matter if it was mine, or someone else's.

Referencing nested array values in JavaScript from my Octoblu state device

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node
Part 6: Reformatting nested JSON with JavaScript
Part 7: Logical data nesting with your Octoblu state device

Okay, here is the big post that I have spent an entire week working up to.

I have to admit, I don't write code every day and am self taught JavaScript (along with Python,  PowerShell, and batch) so this took me a while to work through.

From my last post, my incoming message looks this this:

{
  "msg": {
    "rooms": {
      "Redmond": {
        "lunch": {
          "motion": {
            "name": "Redmond_lunch_motion",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "motion",
          },
          "refrigerator": {
            "name": "Redmond_lunch_refrigerator",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "refrigerator",
          },
          "door": {
            "name": "Redmond_lunch_door",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "door",
          }
        }
      }
    },
    "fromUuid": "d5b77d9b-aaf3-f089a7096ee0"
  },
  "node": "b5149300-9cbd-1f1b56e5d7bb"
}
There can be a variable number of devices per room, and a variable number of rooms per map, and a variable number of maps.  My nesting pattern above is rooms.map.room.devices

Now for the hard part.
I want to evaluate the differences between values of different devices, per room.
This ends up being a lesson in how values are referenced in arrays in JavaScript.

Before I move forward, I have abbreviated the above JSON to spare you scrolling.  There are additional fields, and these additional fields contain Date objects that I am interested in.  And these Dates are formatted as odd numbers which are actually Epoch Time.

So, to give you the full treatment, here is a real message:

{
  "msg": {
    "rooms": {
      "Redmond": {
        "lunch": {
          "motion": {
            "name": "Redmond_lunch_motion",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "motion",
            "motion": false,
            "motion_updated_at": 1485539337.480442,
            "battery": 1,
            "battery_updated_at": 1485539337.480442,
            "tamper_detected": null,
            "tamper_detected_updated_at": null,
            "temperature": 21.666666666666668,
            "temperature_updated_at": 1485539337.480442,
            "motion_true": "N/A",
            "motion_true_updated_at": 1485539125.483463,
            "tamper_detected_true": null,
            "tamper_detected_true_updated_at": null,
            "connection": true,
            "connection_updated_at": 1485539337.480442,
            "agent_session_id": null,
            "agent_session_id_updated_at": null,
            "connection_changed_at": 1485175984.3230183,
            "motion_changed_at": 1485539337.480442,
            "motion_true_changed_at": 1485539125.483463,
            "temperature_changed_at": 1485529054.5705206
          },
          "refrigerator": {
            "name": "Redmond_lunch_refrigerator",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "refrigerator",
            "opened": false,
            "opened_updated_at": 1485539969.6240845,
            "tamper_detected": false,
            "tamper_detected_updated_at": 1476739884.682764,
            "battery": 1,
            "battery_updated_at": 1485539969.6240845,
            "tamper_detected_true": "N/A",
            "tamper_detected_true_updated_at": 1476739866.2962902,
            "connection": true,
            "connection_updated_at": 1485539969.6240845,
            "agent_session_id": null,
            "agent_session_id_updated_at": null,
            "opened_changed_at": 1485539969.6240845
          },
          "door": {
            "name": "Redmond_lunch_door",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "door",
            "opened": false,
            "opened_updated_at": 1485538007.9089093,
            "tamper_detected": null,
            "tamper_detected_updated_at": null,
            "battery": 1,
            "battery_updated_at": 1485538007.9089093,
            "tamper_detected_true": null,
            "tamper_detected_true_updated_at": null,
            "connection": true,
            "connection_updated_at": 1485538007.9089093,
            "agent_session_id": null,
            "agent_session_id_updated_at": null,
            "opened_changed_at": 1485538007.9089093
          }
        }
      }
    },
    "fromUuid": "d5b77d9b-aaf3-f089a7096ee0"
  },
  "node": "b5149300-9cbd-1f1b56e5d7bb"
}
Now, the output I am looking for is to take some of these sensor Date values and evaluate them between each of the three devices.  Such as: door-refrigerator, motion-door, motion-refrigerator and so on.

If these values were in the same part of the message, it would be really easy.  I could simply dot reverence the values and do the math.
But they are not.  Each sensor is in its own document, in an array.

Now, if you recall a few posts back, I have a naming convention and I am standardizing three of the names:  "door", "refrigerator", and "motion".  Those I am not allowing to change.  But the room can and the map can.

Recall, I began this exercise this with just an array of devices with values.  Processed them to group by a logical naming pattern, saved that to an Octoblu state device, and now I am further processing that into my actionable data which I can easily handle with Octoblu filters to handle alerting or whatever I want to do.

So, to get you to read to the end and not just steal my code here is the output that I am producing, per room.
This gives me a nice single document per room as output - I can pass that to a demultiplex node to break the rot array apart and evaluate each document. 

My output looks like this:

{
  "msg": [
    {
      "motion": "motion",
      "motionAt": 1485544607.3195794,
      "motionAtHuman": "2017-01-27T19:16:47.319Z",
      "mapTitle": "Redmond",
      "room": "lunch",
      "refrigerator": "refrigerator",
      "fridgeOpenedAt": 1485539969.6240845,
      "fridgeOpenedAtHuman": "2017-01-27T17:59:29.624Z",
      "door": "door",
      "doorOpenedAt": 1485538007.9089093,
      "doorOpenedAtHuman": "2017-01-27T17:26:47.908Z",
      "diffDoorsOpenedMinutes": 32,
      "diffDoorMotionMinutes": 109,
      "diffRefrigeratorMotionMinutes": 77,
      "sinceDoorOpenMinutes": 115,
      "sinceRefrigeratorOpenMinutes": 82,
      "sinceMotionMinutes": 5
    }
  ],
  "node": "98cb8680-a264-1b8483214e06"
}
Now, to end this long, long story the JavaScript is below.
What I tried to do was have an intuitive way to read the code and reference each level of the document arrays, so you could understand where you were in the hierarchy.

// array to output
var output = [];
for ( var map in (msg.rooms) ){
    for ( var room in msg.rooms[map] ){
        var doorOpenedAt;
        var fridgeOpenedAt;
        var motionAt;
        var roomOutput = {};
        for ( var sensor in msg.rooms[map][room]){
            switch ( msg.rooms[map][room][sensor].device ) {
                case "door":
                    doorOpenedAt = moment.unix(msg.rooms[map][room][sensor].opened_changed_at);
                    roomOutput.door = msg.rooms[map][room][sensor].device;
                    roomOutput.doorOpenedAt = msg.rooms[map][room][sensor].opened_changed_at;
                    roomOutput.doorOpenedAtHuman = doorOpenedAt;
                    break;
                case "refrigerator":
                    fridgeOpenedAt = moment.unix(msg.rooms[map][room][sensor].opened_changed_at);
                    roomOutput.refrigerator = msg.rooms[map][room][sensor].device;
                    roomOutput.fridgeOpenedAt = msg.rooms[map][room][sensor].opened_changed_at;
                    roomOutput.fridgeOpenedAtHuman = fridgeOpenedAt;
                    break;
                case "motion":
                    motionAt = moment.unix(msg.rooms[map][room][sensor].motion_true_changed_at);
                    roomOutput.motion = msg.rooms[map][room][sensor].device;
                    roomOutput.motionAt = msg.rooms[map][room][sensor].motion_true_changed_at;
                    roomOutput.motionAtHuman = motionAt;
                    break;
            } // close of switch
            roomOutput.mapTitle = msg.rooms[map][room][sensor].mapTitle;
            roomOutput.room = msg.rooms[map][room][sensor].room;
        }  // close of sensor
        roomOutput.diffDoorsOpenedMinutes = Math.abs(doorOpenedAt.diff(fridgeOpenedAt, 'minutes'));  //removing Math.abs will give a + - if the refrigerator opens and the door does not it will be negative
        roomOutput.diffDoorMotionMinutes = Math.abs(doorOpenedAt.diff(motionAt, 'minutes'));
        roomOutput.diffRefrigeratorMotionMinutes = Math.abs(fridgeOpenedAt.diff(motionAt, 'minutes'));
        roomOutput.sinceDoorOpenMinutes = moment().diff(doorOpenedAt, 'minutes');
        roomOutput.sinceRefrigeratorOpenMinutes = moment().diff(fridgeOpenedAt, 'minutes');
        roomOutput.sinceMotionMinutes = moment().diff(motionAt, 'minutes');       
        output.push(roomOutput);
    }  //close of room
} // close of map
return output;

Lots of leading up to this post.  But I like to expand folks' understanding along the way.
And I know we don't all tolerate long articles.

I can thank Tobias Kreidl for even getting me started on this series of posts.
He asked a simple question, and I had a final answer, but I wanted to tell the journey so that he understood how I got to where I did.
That leaves it up to you to take what you need.  That's just how I write and respond to questions.

Logical data nesting with your Octoblu state device

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node
Part 6: Reformatting nested JSON with JavaScript

In my last post I left you with some JavaScript to reformat a JSON message and come out with a nice new format.

I left you hanging with my key format though.
Why did I format my key names the way I did?

It is actually pretty simple in concept (but it took me a long time to get all the code right).

Previously I mentioned that after I $set the data on my Octoblu state device, I want to catch that data change in another workflow.
And I also mentioned that logically grouping that data would make it easier to visualize and work with farther down the chain.

So, back to the key name pattern in my output:

{
  "msg": {
   "rooms.Redmond.lunch.motion": {
     "name": "Redmond_lunch_motion",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "motion",
   },
   "rooms.Redmond.lunch.refrigerator": {
     "name": "Redmond_lunch_refrigerator",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "refrigerator",
   },
   "rooms.Redmond.lunch.door": {
     "name": "Redmond_lunch_door",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "door",
    }
  },
  "node": "e271a6c0-9f9b-8d7882b7836a"
}

This is all about the $set.
I want each set of devices grouped under their room which is under the map they correspond to.
Thus: rooms.map.roomName.deviceName

Which is passed to a JSON Template Node in Octoblu which very simply looks like this:

{
 "$set": {{msg}}
}

Here is where there are different patterns for referencing the message values in Octoblu.
If you are referencing a blob don't put quotes around the mustache notation like I did above.
If you are referencing a value, then put double quotes around the value like this:  "rooms.{{msg.name}}"

The hard thing to get right is the quotes.  Since you will get a false message that your JSON is improperly formatted from the editor, when the message that comes out is actually totally right.

Now, back to why I had the dot notation key name.

When I listen to my state device for a change I will get this nice hierarchy as the output.  And I persist my data nice and logically.

{
  "msg": {
    "rooms": {
      "Redmond": {
        "lunch": {
          "motion": {
            "name": "Redmond_lunch_motion",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "motion",
          },
          "refrigerator": {
            "name": "Redmond_lunch_refrigerator",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "refrigerator",
          },
          "door": {
            "name": "Redmond_lunch_door",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "door",
          }
        }
      }
    },
    "fromUuid": "d5b77d9b-aaf3-f089a7096ee0"
  },
  "node": "b5149300-9cbd-1f1b56e5d7bb"
}

Next post:  I am going to parse all that nesting back apart and make yet another message

Reformatting nested JSON with JavaScript

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node

In my last example for an Octoblu Function node I very simply took the string value from a key and using .split() broke it into new fields.

This had the dependency of a format convention for the string.

Now, what if I had an incoming array which contained that data. 
And I only wanted to select certain values within each array element, and I had additional data which was nested within another array that I want to bring up a level to make it easier to evaluate later on.

To describe this differently, lets look at (an abbreviated version of) my incoming message:

{
  "msg": {
    "data": [
      {
        "uuid": "3b8a7529-b0f1-ddba9dc4cc27",
        "desired_state": {
          "pairing_mode": null
        },
        "last_reading": {
          "connection": true,
          "connection_updated_at": 1480529975.6920671,
        },
        "hub_id": "509234",
        "name": "Redmond",
        "locale": "en_us",
        "units": {},
        "created_at": 1476738922,
        "triggers": []
      },
      {
        "last_event": {
          "vibration_occurred_at": null
        },
        "uuid": "4d60c8ad-b6d2-f17c5e4a1192",
        "desired_state": {},
        "last_reading": {
          "motion_changed_at": 1480490895.7546704,
          "motion_true_changed_at": 1480490698.2846074,
          "temperature_changed_at": 1480530247.413451,
          "connection_changed_at": 1480530247.413451
        },
        "name": "Redmond_lunch_door",
        "triggers": []
      },
  },
  "node": "5c0e3d40-bddd-6f97ce016844"
}

I have an incoming message (msg) it has an array (data) of documents.  The data within each document could be different as each is a different device with different capabilities and settings.

From this point I have a couple wants: I need the name information of the sensors (from my previous post), and I need to in-nest the values of last_reading to make it easier to handle down the line.

And, then I want to save this information to my Octoblu device (a few blog posts ago).

Lets just format on the array at this point, I don't want this to get too confusing.

//A document object to hold the sensors per room
var rooms = {};




for ( var i in (msg.data) ){




 var sensor = {}; //an empty document object to populate with new key:values




 sensor.name = msg.data[i].name; //the incoming name

 var dotName = (msg.data[i].name).replace(/_/g,"."); 

   //the name in dot notation instead of underscores (see the next post)


 // break the device name into its descriptors (from the last post)
 var descriptors = (msg.data[i].name).split('_');

 switch(descriptors.length){
  case 3:
   sensor.mapTitle = descriptors[0];
   sensor.room = descriptors[1];
   sensor.device = descriptors[2];
   break;
  case 2:
   sensor.mapTitle = descriptors[0];
   sensor.device = descriptors[1];
   break;
  case 1:
   sensor.device = descriptors[0];
   break;
 }

 // un-nest last_reading to make it easier to handle later on

 var last_reading = msg.data[i].last_reading;
 for ( var reading in last_reading ){
  sensor[reading] = last_reading[reading];
 }

 // only those devices with a room value
 if ( sensor.room ) {
  var room = {};
  dotName = "rooms." + dotName;
  rooms[dotName] = sensor;

  // in the end, I want the devices of a room under the key pattern for that room
 }
}
return rooms;

This is what I get back out:

{
  "msg": {
   "rooms.Redmond.lunch.motion": {
     "name": "Redmond_lunch_motion",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "motion",
     "motion": false,
     "motion_updated_at": 1485539337.480442,
     "connection_changed_at": 1485175984.3230183,
     "motion_changed_at": 1485539337.480442,
     "motion_true_changed_at": 1485539125.483463,
     "temperature_changed_at": 1485529054.5705206
   },
   "rooms.Redmond.lunch.refrigerator": {
     "name": "Redmond_lunch_refrigerator",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "refrigerator",
     "opened": false,
     "opened_updated_at": 1485539969.6240845,
     "connection_updated_at": 1485539969.6240845,
     "opened_changed_at": 1485539969.6240845
   },
   "rooms.Redmond.lunch.door": {
     "name": "Redmond_lunch_door",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "door",
     "opened": false,
     "opened_updated_at": 1485538007.9089093,
     "connection_updated_at": 1485538007.9089093,
     "opened_changed_at": 1485538007.9089093
    }
  },
  "node": "e271a6c0-9f9b-8d7882b7836a"
}

Next post:  How that dot notation key name pattern is useful to me.

Breaking a value into new keys with a function node

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu

Previously I had set custom keys and then listened for settings changes.

What if I wanted to set an array of key:values, or a larger document.  How might I handle that with Octoblu?

You have two primary options:

• collect nodes
• F(x) (function nodes)

Collect Nodes emit when the collection hits its minimum value until they reset when it reaches its maximum value.  And what you get is a collection of what came in to that node.

The key here is that you need to predict or hard code when the size of the collection resets back to zero and it begins again.  That is not always easy.

So I frequently end up leaning on F(x) or Function Nodes - this is arbitrary, linear, JavaScript.  It can't loop, it can't wait.  You must construct it so that it executes as a very quick function taking the incoming message, running, and then outputting whatever you tell it to return.

By default a Function node gives you the line return msg;
Which would output exactly the message (msg) that came in.

If you ant to reference the value of a specific key you simply use dot notation such as return msg.rooms;

Everything you do here, requires that you understand the data in your incoming message.  And for longevity sake, that format does not change.

Many of the methods that you have in JavaScript are at your disposal.  The not so easy part is the debugging.  Because you don't get very good debug detail.  But with a bit of effort many folks can work through that.

Now, for my first example:  I am going to create new key:values from an existing value.

I have established a naming pattern of:  map_room_device

This allows me to name my devices in a structured way and then deal with them in Octoblu without using a large number of filters and hard coded values.  In essence, devices came come and go as long as I stick with my naming convention.

Now, I need to make this naming convention more useful and easier to work with farther down the line as the messages become properties, so I want to make new keys from the name.

var descriptors = (msg.name).split('_');


switch(descriptors.length){
 case 3:
  msg.mapTitle = descriptors[0];
  msg.room = descriptors[1];
  msg.device = descriptors[2];
  break;
 case 2:
  msg.mapTitle = descriptors[0];
  msg.device = descriptors[1];
  break;
 case 1:
  msg.device = descriptors[0];
  break;
}


return msg;


I always have error handling.  And that is why I have three cases.  In case I have a device that does not have a map or a room in the naming pattern.

The output of this is the addition of 1, 2, or 3 key:values to the outgoing message.

Just a simple case of what can be done.

Next, straight to a really big message and reformatting arrays..