Smart Home Panel (MQTT)

[Ne0-Hack3r]

Here is what I've come up with thus far in terms of linking SHP to HA via MQTT...

First, follow the instructions from @lwsrbrts in ISSUE #1

Second, configure your mosquito.conf like this:

Note: The format of 'topic' here is intended to obfuscate the serial number such that it only has to be entered once into the mosquito.conf and everything else can then be addresses as /ecoflow/SHP/... on the local broker. The same could be done for Delta Pro using ../DP/.. or ../DP1/.. ../DP2/.. for example.

Third, build automations for each params.id you want to monitor to place the messages for those into separate topics...

- id: '**random#**'
  alias: SHP - Circuits
  description: ''
  trigger:
  - platform: mqtt
    topic: ecoflow/SHP/data
    payload: 96
    value_template: '{{ value_json.params[''id''] }}'
  condition: []
  action:
  - service: mqtt.publish
    data:
      topic: ecoflow/SHP/circuits
      payload: '{{trigger.payload}}'
  mode: single

- id: '**random#**'
  alias: SHP - Info
  description: ''
  trigger:
  - platform: mqtt
    topic: ecoflow/SHP/data
    payload: 2
    value_template: '{{ value_json.params[''id''] }}'
  condition: []
  action:
  - service: mqtt.publish
    data:
      topic: ecoflow/SHP/info
      payload: '{{trigger.payload}}'
  mode: single

- id: '**random#**'
  alias: SHP - Limits
  description: ''
  trigger:
  - platform: mqtt
    topic: ecoflow/SHP/data
    payload: 30
    value_template: '{{ value_json.params[''id''] }}'
  condition: []
  action:
  - service: mqtt.publish
    data:
      topic: ecoflow/SHP/limits
      payload: '{{trigger.payload}}'
  mode: single

- id: '**random#**'
  alias: SHP - Voltage
  description: ''
  trigger:
  - platform: mqtt
    topic: ecoflow/SHP/data
    payload: 113
    value_template: '{{ value_json.params[''id''] }}'
  condition: []
  action:
  - service: mqtt.publish
    data:
      topic: ecoflow/SHP/voltage
      payload: '{{trigger.payload}}'
  mode: single

Fourth, create MQTT sensors for each topic above to pull key/value pairs into entity attributes in HA:

  - name: SHP-Circuits
    state_topic: "ecoflow/SHP/circuits"
    qos: 0
    value_template: "on"
    json_attributes_template: "{{ value_json.params | tojson}}"
    json_attributes_topic: "ecoflow/SHP/circuits"

  - name: SHP-Info
    state_topic: "ecoflow/SHP/info"
    qos: 0
    value_template: "on"
    json_attributes_template: "{{ value_json.params | tojson}}"
    json_attributes_topic: "ecoflow/SHP/info"

  - name: SHP-Limits
    state_topic: "ecoflow/SHP/limits"
    qos: 0
    value_template: "on"
    json_attributes_template: "{{ value_json.params | tojson}}"
    json_attributes_topic: "ecoflow/SHP/limits"

  - name: SHP-Voltage
    state_topic: "ecoflow/SHP/voltage"
    qos: 0
    value_template: "on"
    json_attributes_template: "{{ value_json.params | tojson}}"
    json_attributes_topic: "ecoflow/SHP/voltage"

Finally, create whatever custom sensors you want using the attributes from above...

Here are example sensors for the power on each circuit:
# SHP - Circuit Power
  - name: "SHP Circuit 1 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[0].chWatt | round(0) }}

  - name: "SHP Circuit 2 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[1].chWatt | round(0) }}

  - name: "SHP Circuit 3 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[2].chWatt | round(0) }}

  - name: "SHP Circuit 4 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[3].chWatt | round(0) }}

  - name: "SHP Circuit 5 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[4].chWatt | round(0) }}

  - name: "SHP Circuit 6 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[5].chWatt | round(0) }}

  - name: "SHP Circuit 7 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[6].chWatt | round(0) }}

  - name: "SHP Circuit 8 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[7].chWatt | round(0) }}

  - name: "SHP Circuit 9 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[8].chWatt | round(0) }}

  - name: "SHP Circuit 10 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[9].chWatt | round(0) }}

  - name: "SHP PRO1 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[10].chWatt | round(0) }}

  - name: "SHP PRO2 Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[11].chWatt | round(0) }}

  - name: "SHP ALL Circuit Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList') | sum(attribute='chWatt') | round(0) }}

Here are some binary sensors that might be useful:

# Binary Sensors
- binary_sensor:

# SHP - Status
  - name: "SHP Grid Available"
    state: >- 
      {{ state_attr('sensor.shp_info','gridSta') == 1 }}

  - name: "SHP Pro1 Connected"
    state: >- 
      {{ state_attr('sensor.shp_info','energyInfos')[0].stateBean.isConnect == 1}}

  - name: "SHP Pro1 Enabled"
    state: >- 
      {{ state_attr('sensor.shp_info','energyInfos')[0].stateBean.isEnable == 1}}

  - name: "SHP Pro2 Connected"
    state: >- 
      {{ state_attr('sensor.shp_info','energyInfos')[1].stateBean.isConnect == 1}}

  - name: "SHP Pro2 Enabled"
    state: >- 
      {{ state_attr('sensor.shp_info','energyInfos')[1].stateBean.isEnable == 1}}

Once you have the custom sensors you want you can use them to create dashboards with lists, gauges, etc.

[Ne0-Hack3r]

I am running into some issues with regard to the ../SET topic

If I use the following in mosquito.conf, I see payloads as I change settings in the app:
topic "" in 0 ecoflow/SHP/set /app/<EF USER ID #>/<UPPERCASE SN#>/thing/property/set
... but this, of course, does not allow for publishing settings changes back to the EF cloud MQTT

If, however, I use the following instead, I get strange/unpredictable results:
topic "" both 1 ecoflow/SHP/set /app/<EF USER ID #>/<UPPERCASE SN#>/thing/property/set
... changes made in the app will initially show up in mosquito and sometimes I can successfully publish a payload to make a change from that end but then prior changes, published by the app, seem to repeat and echo to mosquito over and over

I'm not sure if this two-way "bridged" setup is problematic or if there is something I'm not understanding about this configuration... but, at this point, it is not working for me in terms of both intercepting and publishing setting changes with MQTT Explorer pointed at the mosquito broker. Assuming I can get this kink worked out I should be able to create switches and configurable entities in HA to do things like toggle EPS mode and change charge/discharge limits on the SHP...

[lwsrbrts]

I've also noticed some oddities when attempting to configure a bridge connection for the SHP but I've not had any time to investigate further. My biggest issues was that energy usage practically doubles (in terms of what's recorded, not actual usage) when I use the both setting in my topic mappings.

I did a very small bit of investigation at the time and the only conclusion I had was that I was experiencing looping, where the same message is received and sent again, causing a duplication of the message. It must not be turning in to a loop storm that goes on forever but it obviously renders the values recorded by the SHP useless.

I do think that the /set topic has messages published to without you having to do anything in the app (perhaps try by turning off your bridge and shutting down your phone but monitor from MQTT Explorer). It's almost like it's done on the server itself as something similar to scheduled message publishing by them.

[Ne0-Hack3r]

My observation thus far is that ../set only sees messages when something changes in the app. When the app first opens it sends a ‘latestquotas’ message to ../get and the SHP places a large message with status of everything within ‘quotamap’ in ../get-reply but that exchange only appears to happen when the app starts (SHP sub-app opens). DP and D2 appear to operate similarly in terms of that ‘latestquotas’ ../get exchange at app open.

Have you seen this kind of ‘looping’ issue when doing ‘both’ on the ../set for DP? I have not tried that yet as I still have the hassio integration working for my DPs.

[lwsrbrts]

I've not observed any issues with using both on the set topic for my Delta Pros no - but then I've probably not paid close enough attention to them to confirm that to be the case.

I too still have both of my Delta Pros working with the HA integration and am not forced to use MQTT for them just yet so, like you, I am interested in having the SHP in HA. Unfortunately I'm not able to devote much time to tinkering at the moment.

[Ne0-Hack3r]

I noticed that my sensor to provide "SHP ALL Circuit Power" was also including the input/output of the Delta Pros when engaged as they are part of the same list (elements 10 & 11)... In Power Shell it's easy to select part of an array and send that along the pipeline but the syntax for that was not working in YAML... I'm still on a bit of YAML/Jinja 'learning curve' so I had to dig around a bit but this is what I came up with:

  - name: "SHP ALL Circuit Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[:10] | sum(attribute='chWatt') | round(0) }}

  - name: "SHP ALL PRO Power"
    unit_of_measurement: "W"
    device_class: power
    state_class: measurement
    state: >- 
      {{ state_attr('sensor.shp_circuits','infoList')[-2:] | sum(attribute='chWatt') | round(0) }}

As I said, syntax is a bit different than PowerShell where I might have done $X[0..9] and $X[-2..-1]
... but this gets the same job done in YAML/Jinja

[Ne0-Hack3r]

I've done a lot of refining the MQTT integration to HA for SHP... I think I have all the main settings and sensors I care about at this point and decent dashboard... I'm calling the beta complete :)

[Ne0-Hack3r]

I've gone back through and refined my configuration further eliminating the need to re-publish payloads from /app/device/property/SN (remapped to ecoflow/SHP/data in my case) to other topics... I was able to eliminate all SHP related automations except one:

# Automations for SHP
automation:

- id: 'EF_SHP_Limits'
  alias: SHP - Limits
  description: 'Re-Publish SHP Charge/Discharge Limits to new Topic'
  trigger:
  - platform: mqtt
    topic: ecoflow/SHP/data
    payload: 30
    value_template: '{{ value_json.params[''id''] }}'
  condition: []
  action:
  - service: mqtt.publish
    data:
      topic: ecoflow/SHP/limits
      payload: '{{trigger.payload}}'
      retain: true
  mode: single

Since the payload with the value of charge/discharge limits is sent to ../data only when/if changed my sensor would start in an unknown state and remain there until I changed the value in the app... with the above automation re-publishing to ../limits with a retained flag, the last known value is available on the broker when sensors are re-loaded in HA... The rest of the sensors were not an issue as the SHP publishes a 'heartbeat' every 1 second with most of the data I care about and publishes a payload with circuit power, etc. every 5 seconds...

Thus the MQTT sensors looks like this:

  - name: SHP Charge Level
    object_id: shp_charge_level
    unique_id: shp_charge_level
    unit_of_measurement: "%"
    device_class: battery
    state_class: measurement
    state_topic: "ecoflow/SHP/limits"
    qos: 0
    value_template: >-
      {% set d = value_json.params['forceChargeHigh'] %}
      {% if d is defined %}{{d}}{%else%}{{this.state}}{%endif%}

  - name: SHP Discharge Level
    object_id: shp_discharge_level
    unique_id: shp_discharge_level
    unit_of_measurement: "%"
    device_class: battery
    state_class: measurement
    state_topic: "ecoflow/SHP/limits"
    qos: 0
    value_template: >-
      {% set d = value_json.params['discLower'] %}
      {% if d is defined %}{{d}}{%else%}{{this.state}}{%endif%}

and are used by number templates like this:

  # Number Templates
  - number:

    - name: "SHP - Charge Limit"
      unique_id: shp_charge_limit
      state: "{{ states('sensor.shp_charge_level') }}"
      icon: mdi:battery-arrow-up
      min: 50
      max: 100
      step: 1
      set_value:
      - service: mqtt.publish
        data:
          topic: ecoflow/SHP/set
          payload: '{"from":"HA","id":"{{999900000+(range(10000,99999)|random)}}","moduleType":0,"operateType":"TCP","params":{"discLower":{{states("number.shp_discharge_limit")}},"forceChargeHigh":{{value}},"cmdSet":11,"id":29},"version":"1.0"}'

    - name: "SHP - Discharge Limit"
      unique_id: shp_discharge_limit
      state: "{{ states('sensor.shp_discharge_level') }}"
      icon: mdi:battery-arrow-down-outline
      min: 0
      max: 30
      step: 1
      set_value:
      - service: mqtt.publish
        data:
          topic: ecoflow/SHP/set
          payload: '{"from":"HA","id":"{{999900000+(range(10000,99999)|random)}}","moduleType":0,"operateType":"TCP","params":{"discLower":{{value}},"forceChargeHigh":{{states("number.shp_charge_limit")}},"cmdSet":11,"id":29},"version":"1.0"}'

[kidhasmoxy]

@Ne0-Hack3r Would you consider publishing your package yaml for the Smart Home Panel? Since it's already abstracted it's a great way to get the device for smart home panel created with all the sensors.

[Ne0-Hack3r]

I am working on completing the package for Delta Pro (MQTT) as well... once they are both done and working together I'll look at publishing the code... Probably need to setup my own project for that purpose...

[lk229]

As of a week or so ago, my Smart Home Panel started flooding MQTT with messages so often (multiple time a second) that the MQTT requests I generate with Home Assistant are usually ignored. My home-assistant.log file is expanding like crazy. The only way to stop the unwanted messages is to completely shut down the SHP. Deleting the shp_integration yaml does nothing so it's not an error in the integration. Is anyone else suddenly seeing this or do I maybe have malware in my SHP?

[lwsrbrts]

I'm not seeing the same - there's a possibility that you've got yourself in an echo chamber of sorts and messages are being echo'd between yours and their MQTT servers, like a message flood. I did see this when I was first setting up and mistakenly used both in the topic subscriptions.

Here's my current mosquitto.conf which works just fine. Note that in some cases, use of both in the topic subscription could be the cause of your issue so start there.

connection ecoflow-bridge
address mqtt.ecoflow.com:8883
remote_username [your username]
remote_password [your password]
cleansession true
remote_clientid [your device identifier]
try_private true

# Fix disconnects?
start_type automatic
restart_timeout 30 180
persistence true

bridge_insecure false
bridge_protocol_version mqttv311
bridge_tls_version tlsv1.2
bridge_cafile /etc/ssl/certs/ca-certificates.crt

# Smart Home Panel
topic [SHP SERIAL] in 0 ecoflow/ /app/device/property/
topic [USERID]/[SHP SERIAL]/thing/property/set out 1 ecoflow/ /app/

A word of warning, my topic subscriptions are different to what is described in the original post configuration at the top of this thread, so my topic subscriptions very likely won't look the same as yours, but the important bits are:

• In topic 1 where all the data arrives, I'm only accepting messages in - never sending them.
• In topic 2, I'm only ever sending them out - never receiving them.
• The other thing to note is that my message priority on the out (set) topic is 1 - not zero.

[lk229]

Thanks for the reply.
I just changed my mosquitto.conf topic 2 to use out instead of both, and priority 1 instead of zero.

I restarted my MQTT broker, then tried restarting HA, then did a complete reboot. My log file still floods but I think these might be different errors than I had before. I'm going to do a little more investigating and then will post again.

[lk229]

I'm receiving error messages similar to this one at the rate of 10 times per second.

2024-08-21 18:10:58.525 WARNING (MainThread) [homeassistant.components.mqtt.sensor] Invalid last_reset message '{"params":{"backupCmdChCtrlInfos":[{"powCh":0,"ctrlSta":1,"ctrlMode":0,"priority":0},{"powCh":0,"ctrlSta":0,"ctrlMode":0,"priority":0}],"gridDayWatth":409.49814,"backupFullCap":157412,"errorCodes":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"workTime":85430256,"backupBatPer":96,"backupDayWatth":191.96356,"loadCmdChCtrlInfos":[{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":0},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":1},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":2},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":3},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":4},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":5},{"powCh":1,"ctrlSta":1,"ctrlMode":1,"priority":6},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":7},{"powCh":1,"ctrlSta":0,"ctrlMode":0,"priority":8},{"powCh":1,"ctrlSta":1,"ctrlMode":1,"priority":9}],"cmdSet":11,"backupChaTime":817,"gridSta":1,"time":{"week":3,"year":2024,"sec":39,"min":55,"hour":17,"month":8,"day":21},"id":2,"energyInfos":[{"dischargeTime":697,"mulPackNum":1,"stateBean":{"isPowerOutput":1,"isConnect":1,"isEnable":1,"isGridCharge":0,"isMpptCharge":1,"isAcOpen":1},"outputPower":512,"lcdInputWatts":232,"fullCap":80000,"chargeTime":700,"emsChgFlag":0,"type":14,"emsBatTemp":35,"ratePower":3600,"batteryPercentage":96,"oilPackNum":0},{"dischargeTime":0,"mulPackNum":0,"stateBean":{"isPowerOutput":0,"isConnect":0,"isEnable":0,"isGridCharge":0,"isMpptCharge":0,"isAcOpen":0},"outputPower":0,"lcdInputWatts":0,"fullCap":0,"chargeTime":0,"emsChgFlag":0,"type":0,"emsBatTemp":0,"ratePower":0,"batteryPercentage":0,"oilPackNum":0}]},"version":"1.0","cmdId":0,"cmdFunc":0,"id":1799954293914088518,"addr":0,"timestamp":1724278258483}' from 'ecoflow/SHP/data'