Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

get_device_reading sometimes returns all readings instead of single reading #13

Closed
d1nd141 opened this issue Dec 26, 2018 · 4 comments
Closed

get_device_reading sometimes returns all readings instead of single reading #13

d1nd141 opened this issue Dec 26, 2018 · 4 comments

Comments

@d1nd141
Copy link

d1nd141 commented Dec 26, 2018

Hi,
If use f.ex.
temp_1_wday = fh.get_device_reading("myWeather", "fc1_day_of_week")
temp_1_wday = temp_1_wday['day_of_week']
temp_1_wday = temp_1_wday['Value']

or
temp_1_code = fh.get_device_reading("myWeather", "fc1_code")
temp_1_code = temp_1_code['fc1_code']
temp_1_code = temp_1_code['Value']

to get a specific reading.
Unfortunately sometimes (not always, and also not everytime the same device) i get all readings.
Not sure if it's FHEM related or python-fhem:

DEBUG:Fhem:Connected, sending...
INFO:Fhem:Sent msg, len=25
ERROR:Fhem:Failed to decode json, exception raised. {
  "Arg":"NAME~myWeather",
  "Results": [
  {
    "Name":"myWeather",
    "PossibleSets":"update",
    "PossibleAttrs":"alias comment:textField-long eventMap:textField-long group room suppressReading userReadings:textField-long verbose:0,1,2,3,4,5 disable event-aggregator event-min-interval event-on-change-reading event-on-update-reading oldreadings stateFormat:textField-long timestamp-on-change-reading DbLogExclude DbLogInclude SplunkLogExclude SplunkLogInclude SyslogLogExclude SyslogLogInclude alexaName alexaRoom cmdIcon devStateIcon devStateStyle fp_Keller fp_Keller_T fp_RDC fp_RDC_T fp_Upstairs fp_Upstairs_T genericDeviceType:security,ignore,switch,outlet,light,blind,thermometer,thermostat,contact,garage,window,lock homebridgeMapping:textField-long icon sortby webCmd webCmdLabel:textField-long widgetOverride userattr",
    "Internals": {
      "API": "YahooWeatherAPI",
      "APIOPTIONS": "transport:https,cachemaxage:600",
      "DEF": "979629",
      "INTERVAL": "3600",
      "LANG": "en",
      "LOCATION": "979629",
      "NAME": "myWeather",
      "NOTIFYDEV": "global",
      "NR": "358",
      "NTFY_ORDER": "50-myWeather",
      "STATE": "T: 1  H: 77  W: 2  P: 995",
      "TYPE": "Weather",
      "UNITS": "c"
    },
    "Readings": {
      "city": { "Value":"Howald", "Time":"2018-12-26 13:46:36" },
      "code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "country": { "Value":"Luxembourg", "Time":"2018-12-26 13:46:36" },
      "current_date_time": { "Value":"Wed, 26 Dec 2018 01:00 PM CET", "Time":"2018-12-26 13:46:36" },
      "day_of_week": { "Value":"Wed", "Time":"2018-12-26 13:46:36" },
      "description": { "Value":"Yahoo! Weather for Howald, Luxemburg, LU", "Time":"2018-12-26 13:46:36" },
      "fc10_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc10_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc10_date": { "Value":"04 Jan 2019", "Time":"2018-12-26 13:46:36" },
      "fc10_day_of_week": { "Value":"Fri", "Time":"2018-12-26 13:46:36" },
      "fc10_high_c": { "Value":"0", "Time":"2018-12-26 13:46:36" },
      "fc10_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc10_low_c": { "Value":"-2", "Time":"2018-12-26 13:46:36" },
      "fc1_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc1_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc1_date": { "Value":"26 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc1_day_of_week": { "Value":"Wed", "Time":"2018-12-26 13:46:36" },
      "fc1_high_c": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "fc1_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc1_low_c": { "Value":"-3", "Time":"2018-12-26 13:46:36" },
      "fc2_code": { "Value":"28", "Time":"2018-12-26 1 Unterminated string starting at: line 45 column 42 (char 2883)
temp_1_code:{}
Traceback (most recent call last):
  File "own.py", line 75, in <module>
    temp_1_code = temp_1_code['fc1_code']
KeyError: 'fc1_code'

or


DEBUG:Fhem:Sending: jsonlist2 NAME~myWeather
DEBUG:Fhem:Connected, sending...
INFO:Fhem:Sent msg, len=25
ERROR:Fhem:Failed to decode json, exception raised. {
  "Arg":"NAME~myWeather",
  "Results": [
  {
    "Name":"myWeather",
    "PossibleSets":"update",
    "PossibleAttrs":"alias comment:textField-long eventMap:textField-long group room suppressReading userReadings:textField-long verbose:0,1,2,3,4,5 disable event-aggregator event-min-interval event-on-change-reading event-on-update-reading oldreadings stateFormat:textField-long timestamp-on-change-reading DbLogExclude DbLogInclude SplunkLogExclude SplunkLogInclude SyslogLogExclude SyslogLogInclude alexaName alexaRoom cmdIcon devStateIcon devStateStyle fp_Keller fp_Keller_T fp_RDC fp_RDC_T fp_Upstairs fp_Upstairs_T genericDeviceType:security,ignore,switch,outlet,light,blind,thermometer,thermostat,contact,garage,window,lock homebridgeMapping:textField-long icon sortby webCmd webCmdLabel:textField-long widgetOverride userattr",
    "Internals": {
      "API": "YahooWeatherAPI",
      "APIOPTIONS": "transport:https,cachemaxage:600",
      "DEF": "979629",
      "INTERVAL": "3600",
      "LANG": "en",
      "LOCATION": "979629",
      "NAME": "myWeather",
      "NOTIFYDEV": "global",
      "NR": "358",
      "NTFY_ORDER": "50-myWeather",
      "STATE": "T: 1  H: 77  W: 2  P: 995",
      "TYPE": "Weather",
      "UNITS": "c"
    },
    "Readings": {
      "city": { "Value":"Howald", "Time":"2018-12-26 13:46:36" },
      "code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "country": { "Value":"Luxembourg", "Time":"2018-12-26 13:46:36" },
      "current_date_time": { "Value":"Wed, 26 Dec 2018 01:00 PM CET", "Time":"2018-12-26 13:46:36" },
      "day_of_week": { "Value":"Wed", "Time":"2018-12-26 13:46:36" },
      "description": { "Value":"Yahoo! Weather for Howald, Luxemburg, LU", "Time":"2018-12-26 13:46:36" },
      "fc10_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc10_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc10_date": { "Value":"04 Jan 2019", "Time":"2018-12-26 13:46:36" },
      "fc10_day_of_week": { "Value":"Fri", "Time":"2018-12-26 13:46:36" },
      "fc10_high_c": { "Value":"0", "Time":"2018-12-26 13:46:36" },
      "fc10_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc10_low_c": { "Value":"-2", "Time":"2018-12-26 13:46:36" },
      "fc1_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc1_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc1_date": { "Value":"26 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc1_day_of_week": { "Value":"Wed", "Time":"2018-12-26 13:46:36" },
      "fc1_high_c": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "fc1_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc1_low_c": { "Value":"-3", "Time":"2018-12-26 13:46:36" },
      "fc2_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc2_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc2_date": { "Value":"27 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc2_day_of_week": { "Value":"Thu", "Time":"2018-12-26 13:46:36" },
      "fc2_high_c": { "Value":"2", "Time":"2018-12-26 13:46:36" },
      "fc2_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc2_low_c": { "Value":"-1", "Time":"2018-12-26 13:46:36" },
      "fc3_code": { "Value":"30", "Time":"2018-12-26 13:46:36" },
      "fc3_condition": { "Value":"partly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc3_date": { "Value":"28 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc3_day_of_week": { "Value":"Fri", "Time":"2018-12-26 13:46:36" },
      "fc3_high_c": { "Value":"2", "Time":"2018-12-26 13:46:36" },
      "fc3_icon": { "Value":"partly_cloudy", "Time":"2018-12-26 13:46:36" },
      "fc3_low_c": { "Value":"-1", "Time":"2018-12-26 13:46:36" },
      "fc4_code": { "Value":"26", "Time":"2018-12-26 13:46:36" },
      "fc4_condition": { "Value":"cloudy", "Time":"2018-12-26 13:46:36" },
      "fc4_date": { "Value":"29 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc4_day_of_week": { "Value":"Sat", "Time":"2018-12-26 13:46:36" },
      "fc4_high_c": { "Value":"2", "Time":"2018-12-26 13:46:36" },
      "fc4_icon": { "Value":"cloudy", "Time":"2018-12-26 13:46:36" },
      "fc4_low_c": { "Value":"0", "Time":"2018-12-26 13:46:36" },
      "fc5_code": { "Value":"26", "Time":"2018-12-26 13:46:36" },
      "fc5_condition": { "Value":"cloudy", "Time":"2018-12-26 13:46:36" },
      "fc5_date": { "Value":"30 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc5_day_of_week": { "Value":"Sun", "Time":"2018-12-26 13:46:36" },
      "fc5_high_c": { "Value":"5", "Time":"2018-12-26 13:46:36" },
      "fc5_icon": { "Value":"cloudy", "Time":"2018-12-26 13:46:36" },
      "fc5_low_c": { "Value":"2", "Time":"2018-12-26 13:46:36" },
      "fc6_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc6_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc6_date": { "Value":"31 Dec 2018", "Time":"2018-12-26 13:46:36" },
      "fc6_day_of_week": { "Value":"Mon", "Time":"2018-12-26 13:46:36" },
      "fc6_high_c": { "Value":"5", "Time":"2018-12-26 13:46:36" },
      "fc6_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc6_low_c": { "Value":"2", "Time":"2018-12-26 13:46:36" },
      "fc7_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc7_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc7_date": { "Value":"01 Jan 2019", "Time":"2018-12-26 13:46:36" },
      "fc7_day_of_week": { "Value":"Tue", "Time":"2018-12-26 13:46:36" },
      "fc7_high_c": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "fc7_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc7_low_c": { "Value":"-1", "Time":"2018-12-26 13:46:36" },
      "fc8_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc8_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc8_date": { "Value":"02 Jan 2019", "Time":"2018-12-26 13:46:36" },
      "fc8_day_of_week": { "Value":"Wed", "Time":"2018-12-26 13:46:36" },
      "fc8_high_c": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "fc8_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc8_low_c": { "Value":"-1", "Time":"2018-12-26 13:46:36" },
      "fc9_code": { "Value":"28", "Time":"2018-12-26 13:46:36" },
      "fc9_condition": { "Value":"mostly cloudy", "Time":"2018-12-26 13:46:36" },
      "fc9_date": { "Value":"03 Jan 2019", "Time":"2018-12-26 13:46:36" },
      "fc9_day_of_week": { "Value":"Thu", "Time":"2018-12-26 13:46:36" },
      "fc9_high_c": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "fc9_icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "fc9_low_c": { "Value":"-2", "Time":"2018-12-26 13:46:36" },
      "humidity": { "Value":"77", "Time":"2018-12-26 13:46:36" },
      "icon": { "Value":"mostlycloudy", "Time":"2018-12-26 13:46:36" },
      "isConverted": { "Value":"0", "Time":"2018-12-26 13:46:36" },
      "lastError": { "Value":"", "Time":"2018-12-26 13:46:36" },
      "lat": { "Value":"49.584999", "Time":"2018-12-26 13:46:36" },
      "long": { "Value":"6.14139", "Time":"2018-12-26 13:46:36" },
      "pressure": { "Value":"995", "Time":"2018-12-26 13:46:36" },
      "pressure_trend": { "Value":"0", "Time":"2018-12-26 13:46:36" },
      "pressure_trend_sym": { "Value":"=", "Time":"2018-12-26 13:46:36" },
      "pressure_trend_txt": { "Value":"steady", "Time":"2018-12-26 13:46:36" },
      "pubDate": { "Value":"Wed, 26 Dec 2018 01:00 PM CET", "Time":"2018-12-26 13:46:36" },
      "pubDateComment": { "Value":"okay", "Time":"2018-12-26 13:46:36" },
      "pubDateRemote": { "Value":"Wed, 26 Dec 2018 01:00 PM CET", "Time":"2018-12-26 13:46:36" },
      "pubDateTs": { "Value":"1545825600", "Time":"2018-12-26 13:46:36" },
      "region": { "Value":" Luxemburg", "Time":"2018-12-26 13:46:36" },
      "state": { "Value":"T: 1  H: 77  W: 2  P: 995", "Time":"2018-12-26 13:46:36" },
      "temp_c": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "temperature": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "validity": { "Value":"up-to-date", "Time":"2018-12-26 13:46:36" },
      "visibility": { "Value":"16", "Time":"2018-12-26 13:46:36" },
      "wind": { "Value":"2", "Time":"2018-12-26 13:46:36" },
      "wind_chill": { "Value":"1", "Time":"2018-12-26 13:46:36" },
      "wind_condition": { "Value":"Wind: SE 2 km/h", "Time":"2018-12-26 13:46:36" },
      "wind_direction": { "Value":"135", "Time":"2018-12-26 13:46:36" },
      "wind_speed": { "Value":"2", "Time":"2018-12-26 13:46:36" Expecting object: line 125 column 63 (char 8687)
temp_1_wday:{}
Traceback (most recent call last):
  File "own.py", line 68, in <module>
    temp_1_wday = temp_1_wday['day_of_week']
KeyError: 'day_of_week'




DEBUG:Fhem:Connected, sending...
INFO:Fhem:Sent msg, len=25
ERROR:Fhem:Failed to decode json, exception raised. {
  "Arg":"NAME~myWeather",
  "Results": [
  {
    "Name":"myWeather",
    "PossibleSets":"update",
    "PossibleAttrs":"alias comment:textField-long eventMap:textField-long group room suppressReading userReadings:textField-long verbose:0,1,2,3,4,5 disable event-aggregator event-min-interval event-on-change-reading event-on-update-reading oldreadings stateFormat:textField-long timestamp-on-change-reading DbLogExclude DbLogInclude SplunkLogExclude SplunkLogInclude SyslogLogExclude SyslogLogInclude alexaName alexaRoom cmdIcon devStateIcon devStateStyle fp_Keller fp_Keller_T fp_RDC fp_RDC_T fp_Upstairs fp_Upstairs_T genericDeviceType:security,ignore,switch,outlet,light,blind,thermometer,thermostat,contact,garage,window,lock homebridgeMapping:textField-long icon sortby webCmd webCmdLabel:textField-long widgetOverride userattr",
    "Internals": {
      "API": "YahooWeatherAPI",
      "APIOPTIONS": "transport:https,cachemaxage:600",
      "DEF": "979629",
      "INTERVAL": "3600",
      "LANG": "en",
      "LOCATION": "979629",
      "NAME": "myWeather",
      "NOTIFYDEV": "global",
      "NR": "358",
      "NTFY_ORDER": "50-myWeather",
      "STATE": "T: -3  H: 93  W: 5  P: 994",
      "TYPE": "Weather",
      "UNITS": "c"
    },
    "Readings": {
      "city": { "Value":"Howald", "Time":"2018-12-26 20:46:45" },
      "code": { "Value":"29", "Time":"2018-12-26 20:46:45" },
      "condition": { "Value":"partly cloudy", "Time":"20 Unterminated string starting at: line 26 column 54 (char 1445)
Traceback (most recent call last):
  File "own.py", line 104, in <module>
    temp_2_low = temp_2_low['Value']
KeyError: 'Value'
@Andre0512
Copy link
Contributor

Hello,
the python interface always loads all data of a device, because FHEM offers no other possibility. The data is then filtered by python-fhem.
Therefore, from a performance perspective, it is better to filter the readings yourself if you need several readings from one device, like this:

data = fh.get_device_reading("myWeather", ["fc1_day_of_week", "fc1_code"], value_only=True)
temp_1_code = data["temp_1_code"]
temp_1_wday = data["temp_1_wday"]

Your problem is, that the json data coming from FHEM are not well-formed and can not be parsed by python. I assume that this error is in jsonlist2 of FHEM or in telnet transmission.
Did you test if the error also occurs with http?

Maybe this could be catched by requesting the data again when a decode error occours @domschl ? Maybe with a parameter that sets how often?

To catch this problem for the moment you could also do something like this:

# Try 5 times to get correct data
for x in range(5):
    data = fh.get_device_reading("myWeather", ["fc1_day_of_week", "fc1_code"], value_only=True)
    if data:
        break

@domschl
Copy link
Owner

domschl commented Dec 27, 2018 •
edited
Loading

This is most probably caused by a performance-problem with telnet. In telnet, there are no markers that indicate the actual end of a block, so timing has to be used to infer end-of-transmission. [I can reproduce this error, if I set fh.get_device_reading('myWeather', 'fc1_day_of_week', timeout=0.001), reading via telnet]

The errors indicate a JSON decode of an incomplete block, so somehow the system doesn't finish the transmission within timeout=0.1 default constraints.

I would try:

  • Set timeout=0.5 (experiment with the value):
    e.g. fh.get_device_reading('myWeather', 'fc1_day_of_week', timeout=0.5)
  • Use @Andre0512 recommendations to 'bundle' requests.
  • Consider switching to HTTP, which doesn't have those protocol limitations (and also performs better.)
  • Alternatively: If all you want to do is reading values, have a look at FhemEventQueue. You can set it up with server-side filters to deliver only the values you are interested in, only when Fhem actively sends them.

In the end, when using telnet, proper configuration of timeout is a requirement, which unfortunately cannot be avoided.

@d1nd141
Copy link
Author

d1nd141 commented Dec 27, 2018

Hi again,
i tried with http, and everything works fine.
Will probably stay on http connect.
As i'm reading at least 26 values, i will check with Queue. (to display on a 7.5 e-ink display)

Thanks 4 your help and explications!

@domschl
Copy link
Owner

domschl commented Dec 27, 2018

@d1nd141, you're welcome, and thanks @Andre0512 for the help and advice.

@domschl domschl closed this as completed Dec 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@domschl @d1nd141 @Andre0512