Home Assistant, CBus, MQTT and Philips Hue
This has been a coding adventure.
My aim? Simple, really: Get Home Assistant talking to CBus, replacing 'MisterHouse', which on my network dates from ten years ago (and is a non-maintained pig). And now dead to me.
My stetch goals? Add MQTT discovery for Home Assistant, to eliminate any manual config there. Plus return to previously set lighting levels like MisterHouse used to do when a 'turn on' command was issued. And work out how I can include Philips Hue hub devices in CBus scenes, like having 'All Off' from a button, instead of relying on barking "Hey Google, turn everying off", which would also turn off my electric blankets (plus more that I didn't want turned off...)
Sooo... My initial goal, and all stretch goals have been achieved!
The pieces of the puzzle include:
- Home Assistant 'HAOS', running as a virtual machine. Home Assistant 'Core' as a container is not enough, as add-ins are required to get a MQTT broker and more going (but you could use a separately installed broker elsewhere on your network). HA Cloud talks to Google Assistant/Alexa.
- Home Assistant plug-ins: The excellent 'Portainer', 'SSH & Web terminal' and 'Mosquitto broker'. File editor is also handy.
- A container created with Portainer to run hue2mqtt.js
- LUA code on a Clipsal SHAC/NAC or Wiser 5500AC2
If you don't care for integrating Philips Hue devices with CBus, then ignore Portainer and hue2mqtt.js (the LUA Hue code can stay there if you want, and will just be unused.) Plus SSH & Web terminal and File Editor is not required, just nice to have.
Article changes: (Like I said, it's been an adventure, which will probably continue as odd corner cases pop up, so I'll continue to evolve the content here.) Of note:
- 28/5/22 In MQTT send script: Add Hue dimmable devices
- 29/5/22 In MQTT send script: Switch away from loop_start() for Mosquitto client to stop crashes. This was a great stride forward in ensuring stability, as my previous code was evidently not thread safe for the MQTT callbacks despite best endevours (hence the watchdog code being there), which was exposed when trying to set many CBus groups at the same time. The code melted down every single time. This update eliminates threads, plus improves overall performance (up from running every tenth of a second to one hundredth). If you want to see all detailed logs though then lower the pace...
- 30/5/22 In MQTT send script: Add create/remove/update objects with HUE keyword change
Prepare Home Assistant
I don't cover installing Home Assistant here. You probably wouldn't be reading this if you weren't already an avid user, but if you are new then you want 'HAOS' installed somewhere (RPi, NAC, VM, old laptop, etc.), and the how-to guide you want will depend on that 'somewhere'.
Install the official Mosquitto broker. First up, create a HomeAssistant user 'mqtt', and give it a password of 'password' (used below), and probably hide it so it doesn't appear on dashboards (it doesn't need to be admin). Then go to Settings, Add-ons, and from 'official' add-ons install and start Mosquitto. Any HA user can be used to authenticate to this Mosquitto instance, explaining the creation of the user 'mqtt'.
Portainer might be new to you though, and this allows the creation and maintenance of containers other than those intended to be run alongside Home Assistant. In short, you can do pretty well whatever you want, and all from the comfort of a GUI (just don't ask the HA folks for help if you get stuck).
Go to Settings, Add-ons, Store, and using the little three-dot menu at top right, add the repository https://github.com/MikeJMcGuire/HASSAddons. Once you do, you'll be able to install Portainer 2 from Mike's repository. After it's installed, turn off the option 'Protection mode', enable the sidebar entry and start it. That will enable you to configure a new container for hue2mqtt.js.
The first login to Portainer requires setting an admin password, and from there click on 'Volumes' in the blue bar at left, and click 'Add volume'. I created a volume called 'hue2mqtt'. Store it wherever you like. I used local storage. The reason a volume is needed is so that Hue bridge configuration (a.k.a "press the Hue button") only needs to be done once, and will survive re-creation of the hue2mqtt.js container.
Then click 'Containers' in the blue bar, and 'Add container'.
- For the image, I used: jkohl/hue2mqtt:latest & always pull.
- Under 'Command and logging' tab, for the command, I used: '-b' '192.168.10.15' '-m' 'mqtt://mqtt:password@192.168.10.21' '-i' '1' '--insecure'. See https://github.com/hobbyquaker/hue2mqtt.js/blob/master/README.md for details.
- Under 'Volumes' tab, I added a new volume mounted in the container at '/root/.hue2mqtt' pointed at the volume hue2mqtt to allow persistent storage.
For me, 192.168.10.21 is the IP address of my Home Assistant server, which is now running a Mosquitto broker. 192.168.10.15 is my Philips Hue bridge (I set its IP address to fixed on my Unifi router by editing the client device, so DHCP always gives it the same address).
Once the container is running, go press the button on your Hue bridge, then drop to a HAOS terminal and execute 'docker logs hue2mqtt', and it should show 'bridge connected'. (Protection mode needs to be off in the SSH & Web Terminal info tab to be able to do this.)
If you want to, go grab MQTT Explorer by Thomas Nordquist at http://mqtt-explorer.com/, which is an excellent tool to gain visibility of what is going on behind the scenes. On second thought, definitely go grab it. If using Hue, then MQTT Explorer should show 'hue' topics after connection.
To the NAC / SHAC / AC
Honestly, this part of the puzzle was the hardest for me to get right. And I still don't know whether it's 100%, although it seems quite reliable.
On the reliability front, I've built in "dead man" switches that will re-start errant code, so transient issues will get sorted quickly and without intervention. To date, things are running most smoothly, without dead-man restarts, and with high performance and very low load on my SHAC.
Script names are important for the code below, given keep-alive and other re-starts, so adjust as necessary by examining thoroughly if you need to change the script names.
First up, 'MQTT send', as a resident, zero sleep, which handles pushing topics to MQTT:
The comments at the top of this script explain how to set up CBus objects with the right keywords to have them integrated. For Hue devices, create new CBus objects.
--[[
Push CBus and Philips Hue events to MQTT, and publish discovery topics. Used with Home Assistant.
Lighting, measurement and user parameter applications are implemented.
Add the keyword 'MQTT' to groups for CBus discovery, plus...
One of light, fan, cover, sensor or switch, plus...
sa= Suggested area
img= Image
pn= Preferred name (defaults to CBus tag)
dec= Decimal places
unit= Unit of measurement
scale= Multiplier / divider
Keyword examples:
MQTT, light, sa=Outside, pn=Outside Laundry Door Light, img=mdi:lightbulb,
MQTT, switch, sa=Outside, img=mdi:gate-open,
MQTT, fan, sa=Hutch, img=mdi:ceiling-fan,
MQTT, cover, sa=Bathroom 2, img=mdi:blinds,
MQTT, sensor, sa=Pool, pn=Pool Pool Temperature, unit= °C, dec=1,
MQTT, sensor, sa=Pool, pn=Pool Level, unit= mm, dec=0, scale=1000,
For Philips Hue devices, bi-directional sync with CBus occurs. Add the keyword 'HUE' to CBus objects, plus...
pn= Preferred name (used as the MQTT topic, which needs to match exactly the name of the Hue device.)
Keyword examples:
HUE, pn=Steve's bedside light
HUE, pn=Steve's electric blanket
A useful result is that Philips Hue devices can then be added to CBus scenes, like an 'All off' function.
A 'hue2mqtt.js' instance is required, and for Home Assistant this could be run as a container using
Portainer, or run as a separate container / process on another VM. hue2mqtt is used to sync a Hue bridge
with the MQTT broker.
The CBus groups for Hue devices are usually not used for any purpose other than controlling their Hue device.
Turning on/off one of these groups will result in the Philips Hue hub turning the loads on/off. It is possible
that these CBus Hue groups could be used to also control CBus loads, giving them dual purpose.
Note: This script only handles on/off, as well as levels for dimmable Hue devices, but not colours/colour
temperature, as that's not a CBus thing. Colour details will return to previously set values done in the Hue
app.
Queues are used for publishing discovery topics, which allows incremental publication of newly tagged objects
discovered with the keyword MQTT. Queues are also used for MQTT and CBus message processing.
Changes:
1.00 - 25/05/22 Initial version
1.01 - 26/05/22 Add trim() when examining keywords to remove leading and trailing spaces
1.02 - 28/05/22 Add Hue dimmable devices, prevent CBus/MQTT publishing loops for Hue, and clean up comments
1.03 - 29/05/22 Ignore older 'ignore' flags in MQTT/CBus publish functions, and fix an issue where Hue loads
that were initially first switched from CBus after script startup failed to set
1.04 - 29/05/22 Queuing of inbound CBus messages, and don't use loop_start() for Mosquitto client to stop crashes
1.05 - 30/05/22 Add create/remove/update objects with HUE keyword change, and fix a bug in Hue publish
--]]
logging = false
mqttBroker = '192.168.10.21'
mqttUsername = 'mqtt'
mqttPassword = 'password'
mqttClientId = 'shac-send'
mqttReadTopic = 'cbus/read/'
mqttDiscoveryTopic = 'cbus/'
hueTopic = 'hue/#'
hueSetTopic = 'hue/set/lights/'
mqttStatus = 2 -- Initially disconnected
mqttDisconnected = 0 -- Timestamp of MQTT disconnects, initially zero which will cause an immediate connection
notifyPublish = false
socketTimeout = 0.01 -- Essentially how frequently the main loop runs (lower = higher CPU) 0.01 = 1/100th of a second
setHeartbeat = 1 -- Send a heartbeat to 'MQTT lastlevel' every second
hue = {} -- Hue device details (name, state, dimmable, etc)
hueDevices = {} -- Quick lookup to determine whether an object is a Hue device
cbusMessages = {} -- Message queue
mqttMessages = {} -- Message queue
ignoreCbus = {} -- To prevent Hue message loops
ignoreMqtt = {} -- To prevent Hue message loops
ignoreTimeout = 2 -- Timeout for old ignore messages (two seconds is a long time...)
tagcaches = 0 -- Timestamp when create/update/delete was last executed
cover = {} -- Quick lookup to determine whether an object is a cover (blind)
userParameter = {} -- Quick lookup to determine whether an object is a user parameter
publishAdj = {} -- For user parameters, holds scale and decimals to apply
unpublished = {} -- The current set of objects to publish discovery topics for
published = 0
triggerPublish = false
getHue = true
heartbeat = os.time()
allgrps = grp.all()
function contains(prefix, text) local pos = text:find(prefix, 1, true); if pos then return pos >= 1 else return false end end
function trim(s) return s:match "^%s*(.-)%s*$" end -- Remove leading and trailing spaces
--[[
UDP listener - receive messages from the event scripts 'MQTT' and 'HUE'
--]]
server = require('socket').udp()
server:settimeout(socketTimeout)
server:setsockname('127.0.0.1', 5432) -- Listen on port 5432 for CBus level changes
--[[
Mosquitto client and call-backs
--]]
mqtt = require('mosquitto')
client = mqtt.new(mqttClientId)
if mqttUsername then client:login_set(mqttUsername, mqttPassword) end
client.ON_CONNECT = function(success)
-- Fast, minimal code only in this callback
if success then
log('MQTT send connected')
mqttStatus = 1
client:subscribe(hueTopic, 2) -- Subscribe to relevant topics
notifyPublish = true; triggerPublish = true -- Trigger a full publish
end
end
client.ON_DISCONNECT = function(...)
-- Fast, minimal code only in this callback
log('MQTT send disconnected')
mqttStatus = 2
end
client.ON_MESSAGE = function(mid, topic, payload)
-- Fast, minimal code only in this callback
n = #mqttMessages + 1
mqttMessages[n] = { topic=topic, payload=payload } -- Queue the MQTT message
end
--[[
Publish lighting group and user parameter objects to MQTT
--]]
function publish(net, app, group, level)
local alias = net..'/'..app..'/'..group
local state = ''
if cover[alias] then
state = 'stopped' -- For CBus blind controllers
else
state = (tonumber(level) ~= 0) and 'ON' or 'OFF'
end
if not userParameter[alias] then
client:publish(mqttReadTopic..net..'/'..app..'/'..group..'/state', state, 1, true)
client:publish(mqttReadTopic..net..'/'..app..'/'..group..'/level', level, 1, true)
if logging then log('Publishing state and level '..mqttReadTopic..net..'/'..app..'/'..group..' to '..state..'/'..level) end
else
local adjust = publishAdj[net..'/'..app..'/'..group]
if adjust then v = tonumber(string.format('%.'..adjust['dec']..'f', level * adjust['scale'])) else v = level end
client:publish(mqttReadTopic..net..'/'..app..'/'..group..'/state', v, 1, true)
if logging then log('Publishing value '..mqttReadTopic..net..'/'..app..'/'..group..' to '..v) end
end
end
--[[
Publish measurement application objects to MQTT
--]]
function publishMeasurement(net, app, group, channel, value)
local adjust = publishAdj[net..'/'..app..'/'..group..'/'..channel]
if adjust then v = tonumber(string.format('%.'..adjust['dec']..'f', value * adjust['scale'])) else v = value end
client:publish(mqttReadTopic..net..'/'..app..'/'..group..'_'..channel..'/state', v, 1, true)
-- if logging then log('Publishing measurement '..mqttReadTopic..net..'/'..app..'/'..group..'_'..channel..' to '..v) end
end
--[[
Publish Philips Hue objects to MQTT
--]]
function publishHue(net, app, group, level)
local state = (tonumber(level) ~= 0) and true or false
local alias = net..'/'..app..'/'..group
local hueState
payload = { ['on'] = state }
if hue[alias].dimmable then
hueState = tonumber(level)
if state then -- Only add 'bri' to the message when turning on
if hueState == 255 then hueState = 254 end -- CBus is 0..255, but Hue is 0..254
payload['bri'] = hueState
end
else
hueState = state
end
if hue[alias].state ~= hueState then
if ignoreMqtt[alias] and os.time() - ignoreMqtt[alias] > ignoreTimeout then -- Ignore older 'ignore' flags
ignoreMqtt[alias] = nil
if logging then log('Ignoring older MQTT ignore flag for '..alias) end
end
if not ignoreMqtt[alias] then
local j = json.encode(payload)
client:publish(hueSetTopic..hue[alias].name, j)
-- Publishing to MQTT here will result in outstandingMqttMessage() below setting the CBus state for the group.
-- This is undesired, so ignoreCbus[alias] is used to ensure that the MQTT change received does not set CBus.
if hueState ~= hue[alias].state then ignoreCbus[alias] = os.time() end
if logging then log('Published Hue state and level '..hueSetTopic..hue[alias].name..' to '..j) end
else
ignoreMqtt[alias] = nil
if logging then log('Ignoring MQTT publish for '..alias) end
end
hue[alias].state = hueState
end
end
--[[
Build and publish a MQTT discovery topic for light, switch, fan, cover or sensor
--]]
function addDiscover(net, app, group, channel)
local name = GetCBusGroupTag(0, app, group)
if not name then if not channel then do return end else name = 'measurement' end end -- Need a name from tag lookup for everything but measurement app
local pn = name
-- All other keywords except MQTT are optional. Defaults:
local sa = ''
local img = ''
local units = ''
local scale = 1
local decimals = 2
local dType = 'light'
local tagcache = ''
-- Build an alias to refer to each group
alias = '0'..'/'..app..'/'..group; if channel then alias = alias..'/'..channel end
-- Find the keywords (aka tagcache) for the group
for _, v in ipairs(allgrps) do
if v['address'] == alias then
tagcache = v['tagcache']
break
end
end
-- Extract MQTT topic settings
tags = string.split(tagcache, ',')
for _, t in ipairs(tags) do
tp = string.split(t, '=')
tp[1] = trim(tp[1])
if tp[2] then
tp[2] = trim(tp[2])
if tp[1] == 'sa' then sa = tp[2]
elseif tp[1] == 'pn' then pn = tp[2]
elseif tp[1] == 'img' then img = tp[2]
elseif tp[1] == 'unit' then units = tp[2]
elseif tp[1] == 'dec' then decimals = tonumber(tp[2])
elseif tp[1] == 'scale' then scale = tonumber(tp[2])
end
else
if tp[1] ~= 'MQTT' then dType = tp[1] end
end
end
if logging then
if sa == '' then dSa = 'no preferred area' else dSa = sa end
log('Publish discovery '..name..' as '..dType..':'..pn..' in area '..dSa)
end
-- Build an OID (measurement application gets a channel as well)
if not channel then
oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group
else
oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group..'_'..channel
end
-- Build the type-specific payload to publish
if dType == 'light' then
payload = {
['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/switch',
['bri_stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/level',
['bri_cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
['pl_off'] = 'OFF',
['on_cmd_type'] = 'brightness',
}
elseif dType == 'switch' then
payload = {
['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/switch',
['pl_on'] = 'ON',
['pl_off'] = 'OFF',
}
elseif dType == 'fan' then
payload = {
['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
['pl_on'] = 'ON',
['pl_off'] = 'OFF',
['pr_mode_cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
['pr_mode_cmd_tpl'] = '{% if value == "low" %} 86 {% elif value == "medium" %} 170 {% elif value == "high" %} 255 {% endif %}',
['pr_mode_stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/level',
['pr_mode_val_tpl'] = '{% if value == 0 %} OFF {% elif value == 86 %} low {% elif value == 170 %} medium {% elif value == 255 %} high {% endif %}',
['pr_modes'] = {'low', 'medium', 'high'}
}
elseif dType == 'cover' then
payload = {
['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
['pos_open'] = 255,
['pos_clsd'] = 0,
['pl_open'] = 'OPEN',
['pl_cls'] = 'CLOSE',
['pos_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/level',
['set_pos_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
}
elseif dType == 'sensor' then
if channel then
payload = {
['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'_'..channel..'/state',
}
else
payload = {
['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
}
end
end
-- Add payload common to all
payload['name'] = pn
payload['uniq_id'] = oid
payload['dev'] = { ['ids'] = oid, ['sa'] = sa, ['mf'] = 'Schneider Electric', ['mdl'] = 'CBus' }
if img ~= '' then payload['ic'] = img end
if units ~= '' then payload['unit_of_meas'] = units end
-- Publish to MQTT broker
local j = json.encode(payload)
local topic = mqttDiscoveryTopic..dType..'/'..oid..'/config'
client:publish(topic, j, 1, true)
-- Update the cover, user parameter and publish adjust lookups
alias = '254'..'/'..app..'/'..group -- Adjust alias for CBus local network
if channel then alias = alias..'/'..channel end -- Add channel if needed
if dType == 'cover' then cover[alias] = true end
if app == 250 then userParameter[alias] = true end
if decimals ~= 2 or scale ~= 1 then publishAdj[alias] = { ['dec'] = decimals, ['scale'] = scale } end
end
--[[
Queue up initial discovery and current state CBus objects
--]]
function publishCurrent()
local mqttP = GetCBusByKW('MQTT', 'or')
local n = 1
for k, v in pairs(mqttP) do
local app = tonumber(v['address'][2])
local group = tonumber(v['address'][3])
if v['address'][4] ~= nil then channel = tonumber(v['address'][4]) else channel = nil end
unpublished[n] = {app=app, group=group, channel=channel}
n = n + 1
end
n = n - 1
published = 0
log('Queued '..n..' objects with keyword MQTT for publication')
end
--[[
Create / update / delete Hue devices
--]]
function crudHue(initial)
local hueP = GetCBusByKW('HUE', 'or')
local found = {}
local addition = false
for k, v in pairs(hueP) do
local app = tonumber(v['address'][2])
local group = tonumber(v['address'][3])
local pn = GetCBusGroupTag(0, app, group)
local alias = '0'..'/'..app..'/'..group
table.insert(found, alias)
if not hue[alias] then hue[alias] = {}; addition = true end
for _, v in ipairs(allgrps) do if v['address'] == alias then tagcache = v['tagcache'] break end end
tags = string.split(tagcache, ',')
for _, t in ipairs(tags) do
tp = string.split(t, '=')
tp[1] = trim(tp[1])
if tp[2] then tp[2] = trim(tp[2]) if tp[1] == 'pn' then pn = tp[2] end end
end
hue[alias].name = pn; hueDevices[pn] = alias
end
-- Handle deletions
for k, _ in pairs(hue) do
local f = false; for _, v in ipairs(found) do if k == v then f = true; break end end
if not f then
kill = hue[k].name; hue[k] = nil; hueDevices[kill] = nil
end
end
if addition and not initial then -- Re-connect MQTT to get status of newly added devices
log('Hue object keyword added, re-subscribing to get retained status')
client:subscribe(hueTopic, 2) -- Re-subscribe
-- Ensure that newly tagged/removed groups with the MQTT/HUE keywords send updates
log('Restarting group change script: HUE')
script.disable('HUE'); script.enable('HUE')
end
end
--[[
Create / update / delete CBus MQTT discovery topics
--]]
function crudCBusTopics()
if currTagcache then
tagcaches = os.time()
allgrps = grp.all()
-- Check for differences in keywords set, and for new groups to publish
newTagcache = {}; for _, v in ipairs(allgrps) do if v['tagcache'] and contains('MQTT', v['tagcache']) then newTagcache[v['address']] = v['tagcache'] end end
unpublished = {}
n = 1
for k, v in pairs(newTagcache) do
modified = false
if not currTagcache[k] then modified = true
elseif currTagcache[k] ~= v then modified = true end
if modified then
p = string.split(k, '/')
app = tonumber(p[2]); group = tonumber(p[3])
if p[4] ~= nil then channel = tonumber(p[4]) else channel = nil end
unpublished[n] = {app=app, group=group, channel=channel}
n = n + 1
end
end
n = n - 1
published = 0
if n > 0 then log('Queued '..n..' objects with keyword MQTT for publication') end
-- Handle deletions
for k, v in pairs(currTagcache) do
if not newTagcache[k] then
tags = string.split(v, ', ')
for _, t in ipairs(tags) do
tp = string.split(t, '=')
if not tp[2] and tp[1] ~= 'MQTT' then dType = tp[1]; break end
end
parts = string.split(k, '/')
net = 254; app = parts[2]; group = parts[3];
if parts[4] then oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group..'_'..parts[4] else oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group end
local topic = mqttDiscoveryTopic..dType..'/'..oid..'/config'
log('Remove discovery topic '..topic)
client:publish(topic, '', 1, true)
local topic = mqttReadTopic..'254/'..app..'/'..group;
client:publish(topic..'/state', '', 1, true)
client:publish(topic..'/level', '', 1, true)
end
end
-- Update the current tagcache
currTagcache = newTagcache
else
-- Set the initial current tagcache
allgrps = grp.all()
currTagcache = {}; for _, v in ipairs(allgrps) do if v['tagcache'] and contains('MQTT', v['tagcache']) then currTagcache[v['address']] = v['tagcache'] end end
end
-- Update Hue devices
crudHue()
end
--[[
Publish the next queued discovery topic
--]]
function outstandingPublish()
if notifyPublish then
log('Publishing discovery and current level topics')
notifyPublish = false
end
addDiscover(254, unpublished[1].app, unpublished[1].group, unpublished[1].channel)
-- Measurement application
if unpublished[1].app == 228 then
publishMeasurement(254, unpublished[1].app, unpublished[1].group, unpublished[1].channel, GetCBusMeasurement(0, unpublished[1].group, unpublished[1].channel))
-- User parameters
elseif unpublished[1].app == 250 then
publish(254, unpublished[1].app, unpublished[1].group, GetUserParam(0, unpublished[1].group))
-- Lighting and other
else
publish(254, unpublished[1].app, unpublished[1].group, GetCBusLevel(0, unpublished[1].app, unpublished[1].group))
end
table.remove(unpublished, 1)
published = published + 1
if #unpublished == 0 then
log('Publishing completed for '..published..' discovery and current level topic'..(published ~= 1 and 's' or ''))
-- Ensure that newly tagged/removed groups with the MQTT/HUE keywords send updates
log('Restarting group change scripts: MQTT and HUE')
script.disable('MQTT'); script.enable('MQTT')
script.disable('HUE'); script.enable('HUE')
end
end
--[[
Publish the next queued messages for CBus
--]]
function outstandingMqttMessage()
while #mqttMessages > 0 do
topic = mqttMessages[1].topic
payload = mqttMessages[1].payload
local parts = string.split(topic, '/')
-- Messages from Philips Hue
if parts[1] == 'hue' then
if parts[2] == 'status' then
device = parts[4]
if hueDevices[device] then
parts = string.split(hueDevices[device], '/')
local net = parts[1]; local app = parts[2]; local group = parts[3]
local stat, err = pcall(function ()
j = json.decode(payload)
end)
if stat then
local alias = net..'/'..app..'/'..group
if ignoreCbus[alias] and os.time() - ignoreCbus[alias] > ignoreTimeout then -- Ignore older 'ignore' flags
ignoreCbus[alias] = nil
if logging then log('Ignoring older CBus ignore flag for '..alias) end
end
if not ignoreCbus[alias] then -- Only set the CBus status/level if this script did not initiate the change
if j['hue_state']['bri'] ~= nil then
-- Dimmable
if not hue[alias].dimmable then hue[alias].dimmable = true end
if j['val'] == 254 then j['val'] = 255 end -- Adjust for Hue having a dimming range of 0..254 / make CBus show 'on'.
SetCBusLevel(net, app, group, j['val'], 0)
else
-- Switchable
SetCBusState(net, app, group, j['val'])
end
oldState = hue[alias].state; if oldState == nil then oldState = -1 end
-- Setting CBus here will result in the HUE event script requesting publication of the state using publishHue() above.
-- This is undesired, so ignoreMqtt[alias] is used to ensure that the CBus change received does not publish to MQTT.
if oldState ~= -1 and oldState ~= j['val'] then ignoreMqtt[alias] = os.time() if logging then log('Setting ignoreMqtt, oldState='..tostring(oldState)..' j.val='..tostring(j['val'])) end end
else
ignoreCbus[alias] = nil
if logging then log('Ignoring CBus publish for '..alias) end
end
hue[alias].state = j['val']
else
log('MQTT unexpected message payload: '..payload)
end
end
end
end
table.remove(mqttMessages, 1)
end
end
--[[
Publish the next queued messages for MQTT
--]]
function outstandingCbusMessage()
while #cbusMessages > 0 do
cmd = cbusMessages[1]
parts = string.split(cmd, '/')
alias = parts[1]..'/'..parts[2]..'/'..parts[3]
if not hue[alias] then -- CBus message to MQTT
if not parts[5] then
publish(254, tonumber(parts[2]), tonumber(parts[3]), parts[4])
else
publishMeasurement(254, tonumber(parts[2]), tonumber(parts[3]), tonumber(parts[4]), tonumber(parts[5]))
end
else -- Hue message to MQTT
publishHue(parts[1], parts[2], parts[3], parts[4])
end
table.remove(cbusMessages, 1)
end
end
--[[
Main loop
Sends a heartbeat periodically to port 5433, listened to by the MQTT lastlevel script.
If execution is disrupted by any error or lockup then this script will be re-started.
If sending the heartbeat faults, then the loop is exited, which will also re-start this
script (it being resident/sleep zero)
--]]
while true do
-- Check for new messages from CBus
local stat, err = pcall(function ()
::check_again::
cmd = nil
cmd = server:receive()
if cmd and type(cmd) == 'string' then
n = #cbusMessages + 1; cbusMessages[n] = cmd -- Queue the new message
server:settimeout(0)
goto check_again -- Immediately check for more inbound messages with zero timeout
end
end)
if not stat then log('Socket receive error: '..err) end
server:settimeout(socketTimeout)
-- Process MQTT messages
client:loop(0)
-- Send outstanding messages to CBus
if #mqttMessages > 0 then outstandingMqttMessage() end
if mqttStatus == 1 then
-- When connected to the broker
if #cbusMessages > 0 and mqttStatus == 1 then outstandingCbusMessage() end -- Send outstanding messages to MQTT
if triggerPublish then triggerPublish = false; publishCurrent() end -- Queue publication of all discovery topics
if #unpublished > 0 then outstandingPublish() -- Publish outstanding items
else if os.time() - tagcaches >= 10 then crudCBusTopics() end end -- Add/update/delete MQTT items that change
elseif mqttStatus == 2 and os.time() - mqttDisconnected > 0 then -- MQTT is disconnected, so attempt a connection every second
-- When disconnected
client:connect(mqttBroker, 1883, 25) -- Requested keep-alive 25 seconds, broker at port 1883
mqttDisconnected = os.time()
end
if getHue then crudHue(true); getHue = false end -- Initial load of Hue devices, suppressing MQTT reconnect
-- Send heartbeat
local stat, err = pcall(function ()
if os.time() - heartbeat >= setHeartbeat then
heartbeat = os.time()
require('socket').udp():sendto('MQTTsend+'..heartbeat, '127.0.0.1', 5433)
end
end)
if not stat then log('A fault occurred sending heartbeat. Restarting...'); do return end end
end
Then 'MQTT receive', again resident zero sleep:
--[[
Push MQTT events to CBus. Used with Home Assistant.
--]]
logging = false
mqttBroker = '192.168.10.21'
mqttUsername = 'mqtt'
mqttPassword = 'password'
mqttClientId = 'shac-receive'
cbusTopic = { 'cbus/write/#' }
lastLevel = {}
lls = ''
function loadLastLevel()
local llsTest = storage.get('lastLevel', '')
if llsTest ~= lls then
lls = llsTest
local llt = string.split(lls, ',')
for _, v in ipairs(llt) do
parts = string.split(v, '=')
lastLevel[parts[1]] = tonumber(parts[2])
end
end
end
-- Load MQTT module and create new client
mqtt = require('mosquitto')
client = mqtt.new(mqttClientId)
status = 0
-- MQTT callbacks
client.ON_CONNECT = function(success)
if (success) then
log('MQTT receive connected')
status = 1
-- Subscribe to relevant topics
for _, v in ipairs(cbusTopic) do client:subscribe(v, 2) end
end
end
function contains(prefix, text) local pos = text:find(prefix, 1, true); if pos then return pos >= 1 else return false end end
client.ON_DISCONNECT = function(...)
log('MQTT receive disconnected')
status = 2
end
client.ON_MESSAGE = function(mid, topic, payload)
local parts = string.split(topic, '/')
if parts[1] ~= 'cbus' then do return end end
local net = 0
local app = tonumber(parts[4])
local group = tonumber(parts[5])
if logging then log(topic .. ' to ' .. payload) end
loadLastLevel()
if not parts[6] then
log('MQTT error: Invalid message format')
elseif parts[6] == 'getall' then
local mqttP = GetCBusByKW('MQTT', 'or')
local mqttPs = ''
for k, v in pairs(mqttP) do mqttPs = mqttPs..v['address'][2]..'/'..v['address'][3] end
local datatable = grp.all()
for key,value in pairs(datatable) do
parts = string.split(value.address, '/')
net = tonumber(parts[1])
app = tonumber(parts[2])
group = tonumber(parts[3])
if contains(parts[2]..'/'..parts[3], mqttPs) then -- only publish groups of interest (keyword 'MQTT')
if app == tonumber(parts[4]) and parts[3] then
level = tonumber(value.data)
state = (level ~= 0) and 'ON' or 'OFF'
if logging then log(parts[3], app, group, state, level) end
client:publish('cbus/read/' .. net .. '/' .. app .. '/' .. group .. '/state', state, 1, true)
client:publish('cbus/read/' .. net .. '/' .. app .. '/' .. group .. '/level', level, 1, true)
end
end
end
elseif parts[6] == 'switch' then
if payload == 'ON' then
if logging then log('Payload is ON') end
SetCBusLevel(0, app, group, 255, 0)
elseif payload == 'OFF' then
if logging then log('Payload is OFF') end
SetCBusLevel(0, app, group, 0, 0)
end
elseif parts[6] == 'measurement' then -- UNTESTED
SetCBusMeasurement(0, app, group, payload, 0)
elseif parts[6] == 'ramp' then
if payload == 'OPEN' then
if logging then log("Payload is OPEN, so using RAMP instead") end
payload = '255'
elseif payload == 'CLOSE' then
if logging then log("Payload is CLOSE, so using RAMP instead") end
payload = '0'
elseif payload == 'STOP' then
-- Once a blind level has been set for CBus it is set regardless of the current blind position, which is not updated like a ramp, so a stop command is nonsensical
if logging then log("Payload is STOP, which is incompatible with CBus... ignoring") end
do return end
elseif payload == 'ON' then
if contains('Fan', GetCBusGroupTag(net, app, group)) then
if logging then log("Payload is 'Fan' ON, so using RAMP instead") end
payload = '255'
else
if logging then log('Payload is ON') end
SetCBusLevel(0, app, group, 255, 0)
do return end
end
end
if payload == 'OFF' then
if logging then log('Payload is OFF') end
SetCBusLevel(0, app, group, 0, 0)
else
local key = '0'..'/'..app..'/'..group
parts = string.split(payload, ',')
local lev = tonumber(parts[1])
local num = math.floor(lev + 0.5)
if num and num < 256 then
if logging then log('Payload is RAMP '..payload) end
local toSet = 0
local ramp = 0
if logging and lastLevel[key] then log('Last level '..lastLevel[key]) end
if lastLevel[key] and num == 255 then
if contains('Blindzzz', GetCBusGroupTag(net, app, group)) then -- If blind fully open is desirable instead of lastlevel, then change to actually match a 'Blind' tag
if logging then log("Payload is 'Blind' ramp on, so ignoring lastlevel") end
toSet = num
else
toSet = lastLevel[key]
end
else
toSet = num
end
if parts[2] ~= nil then ramp = tonumber(parts[2]) else ramp = 0 end
SetCBusLevel(0, app, group, toSet, ramp)
end
end
end
end
if mqttUsername then
client:login_set(mqttUsername, mqttPassword)
end
client:connect(mqttBroker, 1883, 25)
client:loop_forever()
Then I use a resident 'MQTT lastlevel', zero sleep, to save the previously set level for lighting groups. This enables 'turn on' messages from Home Assistant to return a group to a prior level.
--[[
Maintain CBus 'lastlevels'
Used so that MQTT 'on' events can return a light/fan/blind to the last known set level.
Only monitors application 56 'lighting'.
Home Assistant with an MQTT integration and Google assistant only allows for on/off
("Hey Google, turn on the dunny light"), and not remembering the previous level set, so
this script, in conjunction with MQTT send works around that. Setting another level is
not affected - just "on" commands.
This script also monitors keepalive messages from 'MQTT send', and disables/enables that
script should it fail for whatever reason.
--]]
function loadLastLevel()
local lls = storage.get('lastLevel', '')
local llt = string.split(lls, ',')
for _, v in ipairs(llt) do
local parts = string.split(v, '=')
lastLevel[parts[1]] = tonumber(parts[2])
end
end
if not initialised then
logging = false
maxHeartbeat = 5 -- Max period without a received heartbeat (seconds)
server = require('socket').udp()
server:settimeout(0.5)
server:setsockname('127.0.0.1', 5433)
monitoring = {}
iterations = {}
last = {}
lastLevel = {}
loadLastLevel()
log('MQTT lastlevel initialised')
initialised = true
end
function saveLastLevel()
local ll = {}
for k, v in pairs(lastLevel) do
table.insert(ll, k..'='..v)
end
table.sort(ll)
local lls = table.concat(ll, ',')
local old = storage.get('lastLevel', '')
if lls ~= old then
storage.set('lastLevel', lls)
log('Saved last levels')
end
end
remove = {}
for g, ts in pairs(monitoring) do
parts = string.split(g, '/')
net = tonumber(parts[1])
app = tonumber(parts[2])
group = tonumber(parts[3])
current = GetCBusLevel(net, app, group)
-- If the monitored group stays at the same level for six iterations then it's a stable value and not ramping (0.5s per iteration)
if current == last[g] then
if iterations[g] > 6 then
table.insert(remove, g)
if current > 0 then -- If the stable value is non-zero then save it as the lastLevel
if current ~= lastLevel[g] then
lastLevel[g] = current
if logging then log('Set lastLevel to '..current..' for '..g) end
saveLastLevel()
end
end
else
iterations[g] = iterations[g] + 1
end
else
iterations[g] = 0 -- Level changed, so reset
last[g] = current
end
if os.time() - ts > 30 then -- Level has been changing for more than 30 seconds, so terminate the monitor
if logging then log('Terminate monitor for '..g..' (30 second timeout)') end
table.insert(remove, g)
end
end
for i=1,#remove do
if logging then log('Stop monitor for '..remove[i]..' after '..os.time() - monitoring[remove[i]]..' seconds') end
monitoring[remove[i]] = nil -- Clean up removed monitors
iterations[remove[i]] = nil
last[remove[i]] = nil
end
function contains(prefix, text)
pos = text:find(prefix, 1, true)
if pos then return pos >= 1 else return false end
end
-- Look for lastlevel values
cmd = server:receive()
if cmd and type(cmd) == 'string' then
-- If the command contains a slash it's to start monitoring a group
if contains('/', cmd) then
parts = string.split(cmd, '/')
if parts[2] == '56' then -- Only monitor the lighting application
if not monitoring[cmd] then
last[cmd] = -1
iterations[cmd] = 0
monitoring[cmd] = os.time()
if logging then log('Start monitor for '..cmd) end
else
iterations[cmd] = 0
end
end
-- If it contains a plus then it's a heartbeat
elseif contains('+', cmd) then
parts = string.split(cmd, '+')
if parts[1] == 'MQTTsend' then
MQTTsendHeartbeat = tonumber(parts[2])
end
end
end
--[[
Heartbeats:
The MQTT send script infinite loop can fail, stopping that script. To ensure it gets
restarted without intervention, this script listens for a heartbeat from it every
second. If heartbeats are not received within maxHeartbeat seconds then that script
is disabled and re-enabled.
--]]
-- If last heartbeat is nil (i.e. not yet received) then initialise it
if not MQTTsendHeartbeat then
MQTTsendHeartbeat = os.time()
end
MQTTsendSecondsSince = os.time() - MQTTsendHeartbeat
if MQTTsendSecondsSince > maxHeartbeat then -- No heartbeat received for a while, so re-start the MQTT send script
log('Missed MQTT send heartbeats (last received '..MQTTsendSecondsSince..' seconds ago) - Re-starting MQTT send')
script.disable('MQTT send')
script.enable('MQTT send')
MQTTsendHeartbeat = nil
end
And finally two small identical scripts that push MQTT and LUA group changes to the MQTT send script. These are event-based scripts, with the first set to fire on the keyword 'MQTT', and the second on the keyword 'HUE'.
'MQTT':
--[[
Pushes CBus events to MQTT resident scripts via internal sockets. Used with Home Assistant.
Tag required objects with the "MQTT" keyword and this script will run whenever one of those objects change.
It seems that this script must be disabled/enabled to enable it to be tied to new objects assigned the MQTT
keyword, and the MQTT send script does this automatically every time a change is detected.
--]]
-- Send an event to publish to broker
require('socket').udp():sendto(event.dst .. "/" .. event.getvalue(), '127.0.0.1', 5432)
-- Send an event to monitor for lastlevel (aplications other than lighting will be ignored)
require('socket').udp():sendto(event.dst, '127.0.0.1', 5433)
'HUE':
--[[
Pushes CBus events to MQTT resident scripts via internal sockets.
Tag required objects with the "HUE" keyword and this script will run whenever one of those objects change.
--]]
-- Send an event to publish to broker
require('socket').udp():sendto(event.dst .. "/" .. event.getvalue(), '127.0.0.1', 5432)
-- Send an event to monitor for lastlevel (aplications other than lighting will be ignored)
require('socket').udp():sendto(event.dst, '127.0.0.1', 5433)
Back to Home Assistant
In Home Assistant add an MQTT integration from Settings | Devices & Services | Integrations pointing to your Home Assitant server IP address. Then edit /config/configuration.yaml in the Terminal or File Editor add-in, add the following and re-start HA. This will tell Home Assistant MQTT client to listen to the discovery prefix 'cbus', which should result in all your configured MQTT objects in the NAC/SHAC/AC to be discovered.
The cloud settings will make sure everything needed will get published to Google Home (or your favourite digital assistant by modifying to suit). Comment/un-comment as required.
mqtt:
client_id: haos
keepalive: 20
discovery_prefix: cbus
cloud:
google_actions:
filter:
include_domains:
- switch
# - alarm_control_panel
- binary_sensor
- camera
# - climate
- cover
- fan
- group
# - input_boolean
# - input_select
- light
- lock
# - media_player
- scene
- script
- sensor
- switch
- vacuum
I hope I haven't forgotten to cover any step along the process. It's a non-trivial set up that honestly took way more hours than I envisaged.
There was much debugging of weird SHAC 'lock up' events due to finicky Mosquitto call-back functions that wouldn't tolerate any kind of delay when used in conjunction with an infinite loop in a script, unless the loop rate was backed way off (and even then the occasional hang occurred, indicating something wasn't thread-safe). But all that is fixed.
The result makes me smile every day, having achieved everything that I set out to.
Cheers.