Home Assistant, CBus, MQTT and Philips Hue

Home Assistant, CBus, MQTT and Philips Hue

This has been a coding adventure.

My aim? Simple, really: Get Home Assistant talking to CBus, replacing 'MisterHouse', which on my network dates from ten years ago (and is a non-maintained pig). And now dead to me.

My stetch goals? Add MQTT discovery for Home Assistant, to eliminate any manual config there. Plus return to previously set lighting levels like MisterHouse used to do when a 'turn on' command was issued. And work out how I can include Philips Hue hub devices in CBus scenes, like having 'All Off' from a button, instead of relying on barking "Hey Google, turn everying off", which would also turn off my electric blankets (plus more that I didn't want turned off...)

Sooo... My initial goal, and all stretch goals have been achieved!

The pieces of the puzzle include:

  • Home Assistant 'HAOS', running as a virtual machine. Home Assistant 'Core' as a container is not enough, as add-ins are required to get a MQTT broker and more going (but you could use a separately installed broker elsewhere on your network). HA Cloud talks to Google Assistant/Alexa.
  • Home Assistant plug-ins: The excellent 'Portainer', 'SSH & Web terminal' and 'Mosquitto broker'. File editor is also handy.
  • A container created with Portainer to run hue2mqtt.js
  • LUA code on a Clipsal SHAC/NAC or Wiser 5500AC2

If you don't care for integrating Philips Hue devices with CBus, then ignore Portainer and hue2mqtt.js (the LUA Hue code can stay there if you want, and will just be unused.) Plus SSH & Web terminal and File Editor is not required, just nice to have.


Article changes: (Like I said, it's been an adventure, which will probably continue as odd corner cases pop up, so I'll continue to evolve the content here.) Of note:

  • 28/5/22 In MQTT send script: Add Hue dimmable devices
  • 29/5/22 In MQTT send script: Switch away from loop_start() for Mosquitto client to stop crashes. This was a great stride forward in ensuring stability, as my previous code was evidently not thread safe for the MQTT callbacks despite best endevours (hence the watchdog code being there), which was exposed when trying to set many CBus groups at the same time. The code melted down every single time. This update eliminates threads, plus improves overall performance (up from running every tenth of a second to one hundredth). If you want to see all detailed logs though then lower the pace...
  • 30/5/22 In MQTT send script: Add create/remove/update objects with HUE keyword change

Prepare Home Assistant

I don't cover installing Home Assistant here. You probably wouldn't be reading this if you weren't already an avid user, but if you are new then you want 'HAOS' installed somewhere (RPi, NAC, VM, old laptop, etc.), and the how-to guide you want will depend on that 'somewhere'.

Install the official Mosquitto broker. First up, create a HomeAssistant user 'mqtt', and give it a password of 'password' (used below), and probably hide it so it doesn't appear on dashboards (it doesn't need to be admin). Then go to Settings, Add-ons, and from 'official' add-ons install and start Mosquitto. Any HA user can be used to authenticate to this Mosquitto instance, explaining the creation of the user 'mqtt'.

Portainer might be new to you though, and this allows the creation and maintenance of containers other than those intended to be run alongside Home Assistant. In short, you can do pretty well whatever you want, and all from the comfort of a GUI (just don't ask the HA folks for help if you get stuck).

Go to Settings, Add-ons, Store, and using the little three-dot menu at top right, add the repository https://github.com/MikeJMcGuire/HASSAddons. Once you do, you'll be able to install Portainer 2 from Mike's repository. After it's installed, turn off the option 'Protection mode', enable the sidebar entry and start it. That will enable you to configure a new container for hue2mqtt.js.

The first login to Portainer requires setting an admin password, and from there click on 'Volumes' in the blue bar at left, and click 'Add volume'. I created a volume called 'hue2mqtt'. Store it wherever you like. I used local storage. The reason a volume is needed is so that Hue bridge configuration (a.k.a "press the Hue button") only needs to be done once, and will survive re-creation of the hue2mqtt.js container.

Then click 'Containers' in the blue bar, and 'Add container'.

  • For the image, I used: jkohl/hue2mqtt:latest & always pull.
  • Under 'Command and logging' tab, for the command, I used: '-b' '192.168.10.15' '-m' 'mqtt://mqtt:password@192.168.10.21' '-i' '1' '--insecure'. See https://github.com/hobbyquaker/hue2mqtt.js/blob/master/README.md for details.
  • Under 'Volumes' tab, I added a new volume mounted in the container at '/root/.hue2mqtt' pointed at the volume hue2mqtt to allow persistent storage.

For me, 192.168.10.21 is the IP address of my Home Assistant server, which is now running a Mosquitto broker. 192.168.10.15 is my Philips Hue bridge (I set its IP address to fixed on my Unifi router by editing the client device, so DHCP always gives it the same address).

Once the container is running, go press the button on your Hue bridge, then drop to a HAOS terminal and execute 'docker logs hue2mqtt', and it should show 'bridge connected'. (Protection mode needs to be off in the SSH & Web Terminal info tab to be able to do this.)

If you want to, go grab MQTT Explorer by Thomas Nordquist at http://mqtt-explorer.com/, which is an excellent tool to gain visibility of what is going on behind the scenes. On second thought, definitely go grab it. If using Hue, then MQTT Explorer should show 'hue' topics after connection.

To the NAC / SHAC / AC

Honestly, this part of the puzzle was the hardest for me to get right. And I still don't know whether it's 100%, although it seems quite reliable.

On the reliability front, I've built in "dead man" switches that will re-start errant code, so transient issues will get sorted quickly and without intervention. To date, things are running most smoothly, without dead-man restarts, and with high performance and very low load on my SHAC.

Script names are important for the code below, given keep-alive and other re-starts, so adjust as necessary by examining thoroughly if you need to change the script names.

First up, 'MQTT send', as a resident, zero sleep, which handles pushing topics to MQTT:

The comments at the top of this script explain how to set up CBus objects with the right keywords to have them integrated. For Hue devices, create new CBus objects.

--[[
Push CBus and Philips Hue events to MQTT, and publish discovery topics. Used with Home Assistant.

Lighting, measurement and user parameter applications are implemented.

Add the keyword 'MQTT' to groups for CBus discovery, plus...

  One of  light, fan, cover, sensor or switch, plus...
  sa=     Suggested area
  img=    Image
  pn=     Preferred name (defaults to CBus tag)
  dec=    Decimal places
  unit=   Unit of measurement
  scale=  Multiplier / divider

Keyword examples:

MQTT, light, sa=Outside, pn=Outside Laundry Door Light, img=mdi:lightbulb, 
MQTT, switch, sa=Outside, img=mdi:gate-open, 
MQTT, fan, sa=Hutch, img=mdi:ceiling-fan, 
MQTT, cover, sa=Bathroom 2, img=mdi:blinds, 
MQTT, sensor, sa=Pool, pn=Pool Pool Temperature, unit= °C, dec=1, 
MQTT, sensor, sa=Pool, pn=Pool Level, unit= mm, dec=0, scale=1000, 

For Philips Hue devices, bi-directional sync with CBus occurs. Add the keyword 'HUE' to CBus objects, plus...
  pn= Preferred name (used as the MQTT topic, which needs to match exactly the name of the Hue device.)

Keyword examples:

HUE, pn=Steve's bedside light
HUE, pn=Steve's electric blanket

A useful result is that Philips Hue devices can then be added to CBus scenes, like an 'All off' function.

A 'hue2mqtt.js' instance is required, and for Home Assistant this could be run as a container using
Portainer, or run as a separate container / process on another VM. hue2mqtt is used to sync a Hue bridge
with the MQTT broker.

The CBus groups for Hue devices are usually not used for any purpose other than controlling their Hue device.
Turning on/off one of these groups will result in the Philips Hue hub turning the loads on/off. It is possible
that these CBus Hue groups could be used to also control CBus loads, giving them dual purpose.

Note: This script only handles on/off, as well as levels for dimmable Hue devices, but not colours/colour
temperature, as that's not a CBus thing. Colour details will return to previously set values done in the Hue
app.

Queues are used for publishing discovery topics, which allows incremental publication of newly tagged objects
discovered with the keyword MQTT. Queues are also used for MQTT and CBus message processing.

Changes:
1.00 - 25/05/22 Initial version
1.01 - 26/05/22 Add trim() when examining keywords to remove leading and trailing spaces
1.02 - 28/05/22 Add Hue dimmable devices, prevent CBus/MQTT publishing loops for Hue, and clean up comments
1.03 - 29/05/22 Ignore older 'ignore' flags in MQTT/CBus publish functions, and fix an issue where Hue loads
                that were initially first switched from CBus after script startup failed to set
1.04 - 29/05/22 Queuing of inbound CBus messages, and don't use loop_start() for Mosquitto client to stop crashes
1.05 - 30/05/22 Add create/remove/update objects with HUE keyword change, and fix a bug in Hue publish
--]]

logging = false

mqttBroker = '192.168.10.21'
mqttUsername = 'mqtt'
mqttPassword = 'password'
mqttClientId = 'shac-send'

mqttReadTopic = 'cbus/read/'
mqttDiscoveryTopic = 'cbus/'
hueTopic = 'hue/#'
hueSetTopic = 'hue/set/lights/'

mqttStatus = 2       -- Initially disconnected
mqttDisconnected = 0 -- Timestamp of MQTT disconnects, initially zero which will cause an immediate connection
notifyPublish = false

socketTimeout = 0.01 -- Essentially how frequently the main loop runs (lower = higher CPU) 0.01 = 1/100th of a second
setHeartbeat = 1     -- Send a heartbeat to 'MQTT lastlevel' every second
hue = {}             -- Hue device details (name, state, dimmable, etc) 
hueDevices = {}      -- Quick lookup to determine whether an object is a Hue device
cbusMessages = {}    -- Message queue
mqttMessages = {}    -- Message queue
ignoreCbus = {}      -- To prevent Hue message loops
ignoreMqtt = {}      -- To prevent Hue message loops
ignoreTimeout = 2    -- Timeout for old ignore messages (two seconds is a long time...)
tagcaches = 0        -- Timestamp when create/update/delete was last executed
cover = {}           -- Quick lookup to determine whether an object is a cover (blind)
userParameter = {}   -- Quick lookup to determine whether an object is a user parameter
publishAdj = {}      -- For user parameters, holds scale and decimals to apply
unpublished = {}     -- The current set of objects to publish discovery topics for
published = 0
triggerPublish = false
getHue = true

heartbeat = os.time()
allgrps = grp.all()


function contains(prefix, text) local pos = text:find(prefix, 1, true); if pos then return pos >= 1 else return false end end
function trim(s) return s:match "^%s*(.-)%s*$" end -- Remove leading and trailing spaces


--[[
UDP listener - receive messages from the event scripts 'MQTT' and 'HUE'
--]]

server = require('socket').udp()
server:settimeout(socketTimeout)
server:setsockname('127.0.0.1', 5432) -- Listen on port 5432 for CBus level changes

--[[
Mosquitto client and call-backs 
--]]

mqtt = require('mosquitto')
client = mqtt.new(mqttClientId)
if mqttUsername then client:login_set(mqttUsername, mqttPassword) end

client.ON_CONNECT = function(success)
  -- Fast, minimal code only in this callback
  if success then
    log('MQTT send connected')
    mqttStatus = 1
    client:subscribe(hueTopic, 2) -- Subscribe to relevant topics
    notifyPublish = true; triggerPublish = true -- Trigger a full publish
  end
end

client.ON_DISCONNECT = function(...)
  -- Fast, minimal code only in this callback
  log('MQTT send disconnected')
  mqttStatus = 2
end

client.ON_MESSAGE = function(mid, topic, payload)
  -- Fast, minimal code only in this callback
  n = #mqttMessages + 1
  mqttMessages[n] = { topic=topic, payload=payload } -- Queue the MQTT message
end


--[[
Publish lighting group and user parameter objects to MQTT 
--]]
function publish(net, app, group, level)
  local alias = net..'/'..app..'/'..group
  local state = ''
  if cover[alias] then
    state = 'stopped' -- For CBus blind controllers
  else
    state = (tonumber(level) ~= 0) and 'ON' or 'OFF'
  end
  if not userParameter[alias] then
    client:publish(mqttReadTopic..net..'/'..app..'/'..group..'/state', state, 1, true)
    client:publish(mqttReadTopic..net..'/'..app..'/'..group..'/level', level, 1, true)
    if logging then log('Publishing state and level '..mqttReadTopic..net..'/'..app..'/'..group..' to '..state..'/'..level) end
  else
    local adjust = publishAdj[net..'/'..app..'/'..group]
    if adjust then v = tonumber(string.format('%.'..adjust['dec']..'f', level * adjust['scale'])) else v = level end
    client:publish(mqttReadTopic..net..'/'..app..'/'..group..'/state', v, 1, true)
    if logging then log('Publishing value '..mqttReadTopic..net..'/'..app..'/'..group..' to '..v) end
  end
end

--[[
Publish measurement application objects to MQTT 
--]]
function publishMeasurement(net, app, group, channel, value)
  local adjust = publishAdj[net..'/'..app..'/'..group..'/'..channel]
  if adjust then v = tonumber(string.format('%.'..adjust['dec']..'f', value * adjust['scale'])) else v = value end
  client:publish(mqttReadTopic..net..'/'..app..'/'..group..'_'..channel..'/state', v, 1, true)
  -- if logging then log('Publishing measurement '..mqttReadTopic..net..'/'..app..'/'..group..'_'..channel..' to '..v) end
end

--[[
Publish Philips Hue objects to MQTT 
--]]
function publishHue(net, app, group, level)
  local state = (tonumber(level) ~= 0) and true or false
  local alias = net..'/'..app..'/'..group
  local hueState
  payload = { ['on'] = state }
  if hue[alias].dimmable then
    hueState = tonumber(level)
    if state then -- Only add 'bri' to the message when turning on
      if hueState == 255 then hueState = 254 end -- CBus is 0..255, but Hue is 0..254
      payload['bri'] = hueState
    end
  else
    hueState = state
  end
  if hue[alias].state ~= hueState then
    if ignoreMqtt[alias] and os.time() - ignoreMqtt[alias] > ignoreTimeout then -- Ignore older 'ignore' flags
      ignoreMqtt[alias] = nil
      if logging then log('Ignoring older MQTT ignore flag for '..alias) end
    end
    if not ignoreMqtt[alias] then
      local j = json.encode(payload)
      client:publish(hueSetTopic..hue[alias].name, j)
      -- Publishing to MQTT here will result in outstandingMqttMessage() below setting the CBus state for the group.
      -- This is undesired, so ignoreCbus[alias] is used to ensure that the MQTT change received does not set CBus.
      if hueState ~= hue[alias].state then ignoreCbus[alias] = os.time() end
      if logging then log('Published Hue state and level '..hueSetTopic..hue[alias].name..' to '..j) end
    else
      ignoreMqtt[alias] = nil
      if logging then log('Ignoring MQTT publish for '..alias) end
    end
    hue[alias].state = hueState
  end
end


--[[
Build and publish a MQTT discovery topic for light, switch, fan, cover or sensor 
--]]
function addDiscover(net, app, group, channel)
  local name = GetCBusGroupTag(0, app, group)
  if not name then if not channel then do return end else name = 'measurement' end end -- Need a name from tag lookup for everything but measurement app
  local pn = name
  -- All other keywords except MQTT are optional. Defaults:
  local sa = ''
  local img = ''
  local units = ''
  local scale = 1
  local decimals = 2
  local dType = 'light'
  local tagcache = ''
  
  -- Build an alias to refer to each group
  alias = '0'..'/'..app..'/'..group;  if channel then alias = alias..'/'..channel end

  -- Find the keywords (aka tagcache) for the group
  for _, v in ipairs(allgrps) do
    if v['address'] == alias then
      tagcache = v['tagcache']
      break
    end
  end

  -- Extract MQTT topic settings
  tags = string.split(tagcache, ',')
  for _, t in ipairs(tags) do
    tp = string.split(t, '=')
    tp[1] = trim(tp[1])
    if tp[2] then
      tp[2] = trim(tp[2])
      if tp[1] == 'sa' then sa = tp[2]
      elseif tp[1] == 'pn' then pn = tp[2]
      elseif tp[1] == 'img' then img = tp[2]
      elseif tp[1] == 'unit' then units = tp[2]
      elseif tp[1] == 'dec' then decimals = tonumber(tp[2])
      elseif tp[1] == 'scale' then scale = tonumber(tp[2])
      end
    else
      if tp[1] ~= 'MQTT' then dType = tp[1] end
    end
  end
  
  if logging then
    if sa == '' then dSa = 'no preferred area' else dSa = sa end
    log('Publish discovery '..name..' as '..dType..':'..pn..' in area '..dSa)
  end

  -- Build an OID (measurement application gets a channel as well)
  if not channel then
    oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group
  else
    oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group..'_'..channel
  end

  -- Build the type-specific payload to publish
  if dType == 'light' then
    payload = {
      ['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
      ['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/switch',
      ['bri_stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/level',
      ['bri_cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
      ['pl_off'] = 'OFF',
      ['on_cmd_type'] = 'brightness',
    }
  elseif dType == 'switch' then
    payload = {
      ['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
      ['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/switch',
      ['pl_on'] = 'ON',
      ['pl_off'] = 'OFF',
    }
  elseif dType == 'fan' then
    payload = {
      ['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
      ['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
      ['pl_on'] = 'ON',
      ['pl_off'] = 'OFF',
      ['pr_mode_cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
      ['pr_mode_cmd_tpl'] = '{% if value == "low" %} 86 {% elif value == "medium" %} 170 {% elif value == "high" %} 255 {% endif %}',
      ['pr_mode_stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/level',
      ['pr_mode_val_tpl'] = '{% if value == 0 %} OFF {% elif value == 86 %} low {% elif value == 170 %} medium {% elif value == 255 %} high {% endif %}',
      ['pr_modes'] = {'low', 'medium', 'high'}
    }
  elseif dType == 'cover' then
    payload = {
      ['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
      ['cmd_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
      ['pos_open'] = 255,
      ['pos_clsd'] = 0,
      ['pl_open'] = 'OPEN',
      ['pl_cls'] = 'CLOSE',
      ['pos_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/level',
      ['set_pos_t'] = 'cbus/write/'..net..'/'..app..'/'..group..'/ramp',
    }
  elseif dType == 'sensor' then
    if channel then
      payload = {
        ['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'_'..channel..'/state',
      }
    else
      payload = {
        ['stat_t'] = 'cbus/read/'..net..'/'..app..'/'..group..'/state',
      }
    end
  end

  -- Add payload common to all
  payload['name'] = pn
  payload['uniq_id'] = oid
  payload['dev'] = { ['ids'] = oid, ['sa'] = sa, ['mf'] = 'Schneider Electric', ['mdl'] = 'CBus' }
  if img ~= '' then payload['ic'] = img end
  if units ~= '' then payload['unit_of_meas'] = units end

  -- Publish to MQTT broker
  local j = json.encode(payload)
  local topic = mqttDiscoveryTopic..dType..'/'..oid..'/config'
  client:publish(topic, j, 1, true)

  -- Update the cover, user parameter and publish adjust lookups
  alias = '254'..'/'..app..'/'..group               -- Adjust alias for CBus local network
  if channel then alias = alias..'/'..channel end   -- Add channel if needed
  if dType == 'cover' then cover[alias] = true end
  if app == 250 then userParameter[alias] = true end
  if decimals ~= 2 or scale ~= 1 then publishAdj[alias] = { ['dec'] = decimals, ['scale'] = scale } end
end


--[[
Queue up initial discovery and current state CBus objects 
--]]
function publishCurrent()
  local mqttP = GetCBusByKW('MQTT', 'or')
  local n = 1
  for k, v in pairs(mqttP) do
    local app = tonumber(v['address'][2])
    local group = tonumber(v['address'][3])
    if v['address'][4] ~= nil then channel = tonumber(v['address'][4]) else channel = nil end
    unpublished[n] = {app=app, group=group, channel=channel}
    n = n + 1
  end
  n = n - 1
  published = 0
  log('Queued '..n..' objects with keyword MQTT for publication')
end


--[[
Create / update / delete Hue devices
--]]
function crudHue(initial)
  local hueP = GetCBusByKW('HUE', 'or')
  local found = {}
  local addition = false

  for k, v in pairs(hueP) do
    local app = tonumber(v['address'][2])
    local group = tonumber(v['address'][3])
    local pn = GetCBusGroupTag(0, app, group)
    local alias = '0'..'/'..app..'/'..group

    table.insert(found, alias)
    if not hue[alias] then hue[alias] = {}; addition = true end

    for _, v in ipairs(allgrps) do if v['address'] == alias then tagcache = v['tagcache'] break end end
    tags = string.split(tagcache, ',')
    for _, t in ipairs(tags) do
      tp = string.split(t, '=')
      tp[1] = trim(tp[1])
      if tp[2] then tp[2] = trim(tp[2]) if tp[1] == 'pn' then pn = tp[2] end end
    end
    hue[alias].name = pn; hueDevices[pn] = alias
  end
    
  -- Handle deletions
  for k, _ in pairs(hue) do
    local f = false; for _, v in ipairs(found) do if k == v then f = true; break end end
    if not f then
      kill = hue[k].name; hue[k] = nil; hueDevices[kill] = nil
    end
  end
  if addition and not initial then -- Re-connect MQTT to get status of newly added devices
    log('Hue object keyword added, re-subscribing to get retained status') 
    client:subscribe(hueTopic, 2) -- Re-subscribe
    -- Ensure that newly tagged/removed groups with the MQTT/HUE keywords send updates
    log('Restarting group change script: HUE')
    script.disable('HUE'); script.enable('HUE')
  end
end

--[[
Create / update / delete CBus MQTT discovery topics
--]]
function crudCBusTopics()
  if currTagcache then
    tagcaches = os.time()
    allgrps = grp.all()
    -- Check for differences in keywords set, and for new groups to publish
    newTagcache = {}; for _, v in ipairs(allgrps) do if v['tagcache'] and contains('MQTT', v['tagcache']) then newTagcache[v['address']] = v['tagcache'] end end
    unpublished = {}
    n = 1
    for k, v in pairs(newTagcache) do
      modified = false
      if not currTagcache[k] then modified = true
      elseif currTagcache[k] ~= v then modified = true end
      if modified then
        p = string.split(k, '/')
        app = tonumber(p[2]); group = tonumber(p[3])
        if p[4] ~= nil then channel = tonumber(p[4]) else channel = nil end
        unpublished[n] = {app=app, group=group, channel=channel}
        n = n + 1
      end
    end
    n = n - 1
    published = 0
    if n > 0 then log('Queued '..n..' objects with keyword MQTT for publication') end
    
    -- Handle deletions
    for k, v in pairs(currTagcache) do
      if not newTagcache[k] then
        tags = string.split(v, ', ')
        for _, t in ipairs(tags) do
          tp = string.split(t, '=')
          if not tp[2] and tp[1] ~= 'MQTT' then dType = tp[1]; break end
        end
        parts = string.split(k, '/')
        net = 254; app = parts[2]; group = parts[3];
        if parts[4] then oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group..'_'..parts[4] else oid = 'cbus_mqtt_'..net..'_'.. app..'_'..group end
        local topic = mqttDiscoveryTopic..dType..'/'..oid..'/config'
        log('Remove discovery topic '..topic)
        client:publish(topic, '', 1, true)
        local topic = mqttReadTopic..'254/'..app..'/'..group;
        client:publish(topic..'/state', '', 1, true)
        client:publish(topic..'/level', '', 1, true)
      end
    end

    -- Update the current tagcache
    currTagcache = newTagcache
  else
    -- Set the initial current tagcache
    allgrps = grp.all()
    currTagcache = {}; for _, v in ipairs(allgrps) do if v['tagcache'] and contains('MQTT', v['tagcache']) then currTagcache[v['address']] = v['tagcache'] end end
  end
  
  -- Update Hue devices
  crudHue()
end


--[[
Publish the next queued discovery topic
--]]
function outstandingPublish()
  if notifyPublish then
    log('Publishing discovery and current level topics')
    notifyPublish = false
  end
  addDiscover(254, unpublished[1].app, unpublished[1].group, unpublished[1].channel)
  -- Measurement application
  if unpublished[1].app == 228 then
    publishMeasurement(254, unpublished[1].app, unpublished[1].group, unpublished[1].channel, GetCBusMeasurement(0, unpublished[1].group, unpublished[1].channel))
  -- User parameters
  elseif unpublished[1].app == 250 then
    publish(254, unpublished[1].app, unpublished[1].group, GetUserParam(0, unpublished[1].group))
  -- Lighting and other
  else
    publish(254, unpublished[1].app, unpublished[1].group, GetCBusLevel(0, unpublished[1].app, unpublished[1].group))
  end
  table.remove(unpublished, 1)
  published = published + 1
  if #unpublished == 0 then
    log('Publishing completed for '..published..' discovery and current level topic'..(published ~= 1 and 's' or ''))
    -- Ensure that newly tagged/removed groups with the MQTT/HUE keywords send updates
    log('Restarting group change scripts: MQTT and HUE')
    script.disable('MQTT'); script.enable('MQTT')
    script.disable('HUE'); script.enable('HUE')
  end
end


--[[
Publish the next queued messages for CBus
--]]
function outstandingMqttMessage()
  while #mqttMessages > 0 do
    topic = mqttMessages[1].topic
    payload = mqttMessages[1].payload
    local parts = string.split(topic, '/')

    -- Messages from Philips Hue
    if parts[1] == 'hue' then
      if parts[2] == 'status' then
        device = parts[4]
        if hueDevices[device] then
          parts = string.split(hueDevices[device], '/')
          local net = parts[1]; local app = parts[2]; local group = parts[3]
          local stat, err = pcall(function ()
            j = json.decode(payload)
          end)
          if stat then
            local alias = net..'/'..app..'/'..group
            if ignoreCbus[alias] and os.time() - ignoreCbus[alias] > ignoreTimeout then -- Ignore older 'ignore' flags
              ignoreCbus[alias] = nil
              if logging then log('Ignoring older CBus ignore flag for '..alias) end
            end
            if not ignoreCbus[alias] then -- Only set the CBus status/level if this script did not initiate the change
              if j['hue_state']['bri'] ~= nil then
                -- Dimmable
                if not hue[alias].dimmable then hue[alias].dimmable = true end
                if j['val'] == 254 then j['val'] = 255 end -- Adjust for Hue having a dimming range of 0..254 / make CBus show 'on'.
                SetCBusLevel(net, app, group, j['val'], 0)
              else
                -- Switchable
                SetCBusState(net, app, group, j['val'])
              end
              oldState = hue[alias].state; if oldState == nil then oldState = -1 end
              -- Setting CBus here will result in the HUE event script requesting publication of the state using publishHue() above.
              -- This is undesired, so ignoreMqtt[alias] is used to ensure that the CBus change received does not publish to MQTT.
              if oldState ~= -1 and oldState ~= j['val'] then ignoreMqtt[alias] = os.time() if logging then log('Setting ignoreMqtt, oldState='..tostring(oldState)..' j.val='..tostring(j['val'])) end end
            else
              ignoreCbus[alias] = nil
              if logging then log('Ignoring CBus publish for '..alias) end
            end
            hue[alias].state = j['val']
          else
            log('MQTT unexpected message payload: '..payload)
          end
        end
      end
    end
    table.remove(mqttMessages, 1)
  end
end


--[[
Publish the next queued messages for MQTT
--]]
function outstandingCbusMessage()
  while #cbusMessages > 0 do
    cmd = cbusMessages[1]
    parts = string.split(cmd, '/')
    alias = parts[1]..'/'..parts[2]..'/'..parts[3]
    if not hue[alias] then -- CBus message to MQTT
      if not parts[5] then
        publish(254, tonumber(parts[2]), tonumber(parts[3]), parts[4])
      else
        publishMeasurement(254, tonumber(parts[2]), tonumber(parts[3]), tonumber(parts[4]), tonumber(parts[5]))
      end
    else -- Hue message to MQTT
      publishHue(parts[1], parts[2], parts[3], parts[4])
    end
    table.remove(cbusMessages, 1)
  end
end


--[[
Main loop

Sends a heartbeat periodically to port 5433, listened to by the MQTT lastlevel script.
If execution is disrupted by any error or lockup then this script will be re-started.
If sending the heartbeat faults, then the loop is exited, which will also re-start this
script (it being resident/sleep zero)
--]]

while true do
  -- Check for new messages from CBus
  local stat, err = pcall(function ()
    ::check_again::
    cmd = nil
	  cmd = server:receive()
    if cmd and type(cmd) == 'string' then
      n = #cbusMessages + 1;  cbusMessages[n] = cmd -- Queue the new message
      server:settimeout(0)
      goto check_again -- Immediately check for more inbound messages with zero timeout
    end
  end)
  if not stat then log('Socket receive error: '..err) end
  server:settimeout(socketTimeout)

  -- Process MQTT messages
  client:loop(0)

  -- Send outstanding messages to CBus
  if #mqttMessages > 0 then outstandingMqttMessage() end

  if mqttStatus == 1 then
    -- When connected to the broker

    if #cbusMessages > 0 and mqttStatus == 1 then outstandingCbusMessage() end -- Send outstanding messages to MQTT
    if triggerPublish then triggerPublish = false; publishCurrent() end        -- Queue publication of all discovery topics
    if #unpublished > 0 then outstandingPublish()                              -- Publish outstanding items
    else if os.time() - tagcaches >= 10 then crudCBusTopics() end end          -- Add/update/delete MQTT items that change

  elseif mqttStatus == 2 and os.time() - mqttDisconnected > 0 then -- MQTT is disconnected, so attempt a connection every second
    -- When disconnected

    client:connect(mqttBroker, 1883, 25) -- Requested keep-alive 25 seconds, broker at port 1883
    mqttDisconnected = os.time()
  end

  if getHue then crudHue(true); getHue = false end -- Initial load of Hue devices, suppressing MQTT reconnect

  -- Send heartbeat
  local stat, err = pcall(function ()
    if os.time() - heartbeat >= setHeartbeat then
      heartbeat = os.time()
      require('socket').udp():sendto('MQTTsend+'..heartbeat, '127.0.0.1', 5433)
    end
  end)
  if not stat then log('A fault occurred sending heartbeat. Restarting...'); do return end end
end

Then 'MQTT receive', again resident zero sleep:

--[[
Push MQTT events to CBus. Used with Home Assistant.
--]]

logging = false

mqttBroker = '192.168.10.21'
mqttUsername = 'mqtt'
mqttPassword = 'password'
mqttClientId = 'shac-receive'

cbusTopic = { 'cbus/write/#' }

lastLevel = {}
lls = ''

function loadLastLevel()
  local llsTest = storage.get('lastLevel', '')
  if llsTest ~= lls then
    lls = llsTest
    local llt = string.split(lls, ',')
    for _, v in ipairs(llt) do
      parts = string.split(v, '=')
      lastLevel[parts[1]] = tonumber(parts[2])
    end
  end
end


-- Load MQTT module and create new client
mqtt = require('mosquitto')
client = mqtt.new(mqttClientId)
status = 0


-- MQTT callbacks

client.ON_CONNECT = function(success)
  if (success) then
    log('MQTT receive connected')
    status = 1
    -- Subscribe to relevant topics
    for _, v in ipairs(cbusTopic) do client:subscribe(v, 2) end
  end
end

function contains(prefix, text) local pos = text:find(prefix, 1, true); if pos then return pos >= 1 else return false end end

client.ON_DISCONNECT = function(...)
  log('MQTT receive disconnected')
  status = 2
end

client.ON_MESSAGE = function(mid, topic, payload)
  local parts = string.split(topic, '/')

  if parts[1] ~= 'cbus' then do return end end

  local net = 0
  local app = tonumber(parts[4])
  local group = tonumber(parts[5])

  if logging then log(topic .. ' to ' .. payload) end

  loadLastLevel()

  if not parts[6] then
    log('MQTT error: Invalid message format')

  elseif parts[6] == 'getall' then
    local mqttP = GetCBusByKW('MQTT', 'or')
    local mqttPs = ''
    for k, v in pairs(mqttP) do mqttPs = mqttPs..v['address'][2]..'/'..v['address'][3] end

    local datatable = grp.all()
    for key,value in pairs(datatable) do
      parts = string.split(value.address, '/')
      net = tonumber(parts[1])
      app = tonumber(parts[2])
      group = tonumber(parts[3])
      if contains(parts[2]..'/'..parts[3], mqttPs) then -- only publish groups of interest (keyword 'MQTT')
        if app == tonumber(parts[4]) and parts[3] then
          level = tonumber(value.data)
          state = (level ~= 0) and 'ON' or 'OFF'
          if logging then log(parts[3], app, group, state, level) end
          client:publish('cbus/read/' .. net .. '/' .. app .. '/' .. group .. '/state', state, 1, true)
          client:publish('cbus/read/' .. net .. '/' .. app .. '/' .. group .. '/level', level, 1, true)
        end
      end
    end

  elseif parts[6] == 'switch' then
    if payload == 'ON' then
      if logging then log('Payload is ON') end
      SetCBusLevel(0, app, group, 255, 0)
    elseif payload == 'OFF' then
      if logging then log('Payload is OFF') end
      SetCBusLevel(0, app, group, 0, 0)
    end
    
  elseif parts[6] == 'measurement' then    -- UNTESTED
    SetCBusMeasurement(0, app, group, payload, 0)

  elseif parts[6] == 'ramp' then
    if payload == 'OPEN' then
      if logging then log("Payload is OPEN, so using RAMP instead") end
      payload = '255'
    elseif payload == 'CLOSE' then
      if logging then log("Payload is CLOSE, so using RAMP instead") end
      payload = '0'
    elseif payload == 'STOP' then
      -- Once a blind level has been set for CBus it is set regardless of the current blind position, which is not updated like a ramp, so a stop command is nonsensical
      if logging then log("Payload is STOP, which is incompatible with CBus... ignoring") end
      do return end
    elseif payload == 'ON' then
      if contains('Fan', GetCBusGroupTag(net, app, group)) then
        if logging then log("Payload is 'Fan' ON, so using RAMP instead") end
        payload = '255'
      else
        if logging then log('Payload is ON') end
        SetCBusLevel(0, app, group, 255, 0)
        do return end
      end
    end
    if payload == 'OFF' then
      if logging then log('Payload is OFF') end
      SetCBusLevel(0, app, group, 0, 0)
    else
      local key = '0'..'/'..app..'/'..group
      parts = string.split(payload, ',')
      local lev = tonumber(parts[1])
      local num = math.floor(lev + 0.5)
      if num and num < 256 then
        if logging then log('Payload is RAMP '..payload) end
        local toSet = 0
        local ramp = 0
        if logging and lastLevel[key] then log('Last level '..lastLevel[key]) end
        if lastLevel[key] and num == 255 then
          if contains('Blindzzz', GetCBusGroupTag(net, app, group)) then -- If blind fully open is desirable instead of lastlevel, then change to actually match a 'Blind' tag
            if logging then log("Payload is 'Blind' ramp on, so ignoring lastlevel") end
            toSet = num
          else
            toSet = lastLevel[key]
          end
        else
          toSet = num
        end
        if parts[2] ~= nil then ramp = tonumber(parts[2]) else ramp = 0 end
        SetCBusLevel(0, app, group, toSet, ramp)
      end
    end
  end
end

if mqttUsername then
  client:login_set(mqttUsername, mqttPassword)
end
client:connect(mqttBroker, 1883, 25)
client:loop_forever()

Then I use a resident 'MQTT lastlevel', zero sleep, to save the previously set level for lighting groups. This enables 'turn on' messages from Home Assistant to return a group to a prior level.

--[[
Maintain CBus 'lastlevels'

Used so that MQTT 'on' events can return a light/fan/blind to the last known set level.
Only monitors application 56 'lighting'.

Home Assistant with an MQTT integration and Google assistant only allows for on/off
("Hey Google, turn on the dunny light"), and not remembering the previous level set, so
this script, in conjunction with MQTT send works around that. Setting another level is
not affected - just "on" commands.

This script also monitors keepalive messages from 'MQTT send', and disables/enables that
script should it fail for whatever reason.
--]]

function loadLastLevel()
  local lls = storage.get('lastLevel', '')
  local llt = string.split(lls, ',')
  for _, v in ipairs(llt) do
    local parts = string.split(v, '=')
    lastLevel[parts[1]] = tonumber(parts[2])
  end
end

if not initialised then
  logging = false
  maxHeartbeat = 5 -- Max period without a received heartbeat (seconds)

  server = require('socket').udp()
  server:settimeout(0.5)
  server:setsockname('127.0.0.1', 5433)

  monitoring = {}
  iterations = {}
  last = {}

  lastLevel = {}
  loadLastLevel()

  log('MQTT lastlevel initialised')

  initialised = true
end

function saveLastLevel()
  local ll = {}
  for k, v in pairs(lastLevel) do
    table.insert(ll, k..'='..v)
  end
  table.sort(ll)
  local lls = table.concat(ll, ',')
  local old = storage.get('lastLevel', '')
  if lls ~= old then
    storage.set('lastLevel', lls)
    log('Saved last levels')
  end
end


remove = {}

for g, ts in pairs(monitoring) do
  parts = string.split(g, '/')
  net = tonumber(parts[1])
  app = tonumber(parts[2])
  group = tonumber(parts[3])
  current = GetCBusLevel(net, app, group)

  -- If the monitored group stays at the same level for six iterations then it's a stable value and not ramping (0.5s per iteration)
  if current == last[g] then
    if iterations[g] > 6 then
      table.insert(remove, g)
      if current > 0 then -- If the stable value is non-zero then save it as the lastLevel
        if current ~= lastLevel[g] then
          lastLevel[g] = current
          if logging then log('Set lastLevel to '..current..' for '..g) end
          saveLastLevel()
        end
      end
    else
      iterations[g] = iterations[g] + 1
    end
  else
    iterations[g] = 0 -- Level changed, so reset
    last[g] = current
  end
  
  if os.time() - ts > 30 then -- Level has been changing for more than 30 seconds, so terminate the monitor
    if logging then log('Terminate monitor for '..g..' (30 second timeout)') end
    table.insert(remove, g)
  end
end

for i=1,#remove do
  if logging then log('Stop monitor for '..remove[i]..' after '..os.time() - monitoring[remove[i]]..' seconds') end
  monitoring[remove[i]] = nil -- Clean up removed monitors
  iterations[remove[i]] = nil
  last[remove[i]] = nil
end


function contains(prefix, text)
  pos = text:find(prefix, 1, true)
  if pos then return pos >= 1 else return false end
end

-- Look for lastlevel values
cmd = server:receive()
if cmd and type(cmd) == 'string' then

  -- If the command contains a slash it's to start monitoring a group
  if contains('/', cmd) then
	  parts = string.split(cmd, '/')
    if parts[2] == '56' then -- Only monitor the lighting application
      if not monitoring[cmd] then
        last[cmd] = -1
        iterations[cmd] = 0
        monitoring[cmd] = os.time()
        if logging then log('Start monitor for '..cmd) end
      else
        iterations[cmd] = 0
      end
    end

  -- If it contains a plus then it's a heartbeat
  elseif contains('+', cmd) then
    parts = string.split(cmd, '+')
    if parts[1] == 'MQTTsend' then
      MQTTsendHeartbeat = tonumber(parts[2])
    end
  end
end


--[[
Heartbeats:

The MQTT send script infinite loop can fail, stopping that script. To ensure it gets
restarted without intervention, this script listens for a heartbeat from it every
second. If heartbeats are not received within maxHeartbeat seconds then that script
is disabled and re-enabled.
--]]

-- If last heartbeat is nil (i.e. not yet received) then initialise it
if not MQTTsendHeartbeat then
  MQTTsendHeartbeat = os.time()
end

MQTTsendSecondsSince = os.time() - MQTTsendHeartbeat
if MQTTsendSecondsSince > maxHeartbeat then -- No heartbeat received for a while, so re-start the MQTT send script
  log('Missed MQTT send heartbeats (last received '..MQTTsendSecondsSince..' seconds ago) - Re-starting MQTT send')
  script.disable('MQTT send')
  script.enable('MQTT send')
  MQTTsendHeartbeat = nil
end

And finally two small identical scripts that push MQTT and LUA group changes to the MQTT send script. These are event-based scripts, with the first set to fire on the keyword 'MQTT', and the second on the keyword 'HUE'.

'MQTT':

--[[
Pushes CBus events to MQTT resident scripts via internal sockets. Used with Home Assistant.

Tag required objects with the "MQTT" keyword and this script will run whenever one of those objects change.

It seems that this script must be disabled/enabled to enable it to be tied to new objects assigned the MQTT
keyword, and the MQTT send script does this automatically every time a change is detected.
--]]

-- Send an event to publish to broker
require('socket').udp():sendto(event.dst .. "/" .. event.getvalue(), '127.0.0.1', 5432)

-- Send an event to monitor for lastlevel (aplications other than lighting will be ignored)
require('socket').udp():sendto(event.dst, '127.0.0.1', 5433)

'HUE':

--[[
Pushes CBus events to MQTT resident scripts via internal sockets.

Tag required objects with the "HUE" keyword and this script will run whenever one of those objects change.
--]]

-- Send an event to publish to broker
require('socket').udp():sendto(event.dst .. "/" .. event.getvalue(), '127.0.0.1', 5432)

-- Send an event to monitor for lastlevel (aplications other than lighting will be ignored)
require('socket').udp():sendto(event.dst, '127.0.0.1', 5433)

Back to Home Assistant

In Home Assistant add an MQTT integration from Settings | Devices & Services | Integrations pointing to your Home Assitant server IP address. Then edit /config/configuration.yaml in the Terminal or File Editor add-in, add the following and re-start HA. This will tell Home Assistant MQTT client to listen to the discovery prefix 'cbus', which should result in all your configured MQTT objects in the NAC/SHAC/AC to be discovered.

The cloud settings will make sure everything needed will get published to Google Home (or your favourite digital assistant by modifying to suit). Comment/un-comment as required.

mqtt:
  client_id: haos
  keepalive: 20
  discovery_prefix: cbus

cloud:
  google_actions:
    filter:
      include_domains:
        - switch
#        - alarm_control_panel
        - binary_sensor
        - camera
#        - climate
        - cover
        - fan
        - group
#        - input_boolean
#        - input_select
        - light
        - lock
#        - media_player
        - scene
        - script
        - sensor
        - switch
        - vacuum

I hope I haven't forgotten to cover any step along the process. It's a non-trivial set up that honestly took way more hours than I envisaged.

There was much debugging of weird SHAC 'lock up' events due to finicky Mosquitto call-back functions that wouldn't tolerate any kind of delay when used in conjunction with an infinite loop in a script, unless the loop rate was backed way off (and even then the occasional hang occurred, indicating something wasn't thread-safe). But all that is fixed.

The result makes me smile every day, having achieved everything that I set out to.

Cheers.