Support for Hue Entertainment

[mariusmotea]

Hi,

I need some help in order to create the dedicated service for hue entertainment. The requirements are these:

UDP port 2100 is used for DTLS handshaking and streaming. Only DTLS mode version 1.2 with Pre-Shared Key (PSK) Key exchange method with TLS_PSK_WITH_AES_128_GCM_SHA256 set as Cipher Suite is supported.

the challenge is to have this DTLS protocol implemented for steam decryption.

The api to create the Hue Entertainment area from mobile application seams to work with no changes.

[francescofact]

Please figure out how to implement this. I want so bad to use the new Razer Synapse 3 with my MiLights and it require Hue Entertainment

[mariusmotea]

i need to find some software that works with hue entertainment protocol in order to progress. I believe pydtls and sslpsk libraries are enough, they only need to be implemented.

[francescofact]

try Razer Synapse 3 to try the entertainment protocol

[francescofact]

It says that the bridge must be updated

[mariusmotea]

Issue is with secondary listening service that is not implemented. Now if you set apiversion to 1.22 in config.json you will be able to specify the lights and position for light in room, but this is not enough.

Marius.

[mariusmotea]

Not sure if available libraries for python can create the dtls server with psk authentication.

[francescofact]

I think that this library could help
https://pypi.python.org/pypi/Dtls/0.1.0

[mariusmotea]

i believe this must be used, but all examples i found are based on certificates authentication, not with psk. If you find any examples i will implement-it. Philips provide all the needed details for Entertainment protocol, just the handshake is the main issue.

[francescofact]

https://github.com/drbild/sslpsk

With this you can add psk autentication writing sslpsk.wrap_socket instead of ssl.wrap_socket

[mariusmotea]

This is for TLS, we need DTLS. Anyway i ask it here

[juanesf]

Will it work ?:
https://github.com/particle-iot/tinydtls/blob/master/README

[juanesf]

#ifdef DTLS_PSK
/* This function is the "key store" for tinyDTLS. It is called to

• retrieve a key for the given identity within this particular
• session. */

[juanesf]

Sorry

[francescofact]

they found the solution i think
kershner/screenBloom#66

[avinashraja98]

Can the dtls server be run using C/C++ and the data then piped into our python script?

[mariusmotea]

Yes, but it need to expose a method for sending the data to hue emulator (socket file ?)

[mariusmotea]

I believe this is one good option https://github.com/wolfSSL/wolfssl-examples/blob/master/dtls/server-dtls.c

[ghost]

Getting anywhere with this?

[mariusmotea]

I don't know c++, cannot start here. Only when sombody will write here that will help us there can be a progress.

[juanesf]

Hello, Again.
Searching or searching I found the following:

https://pypi.org/project/Dtls/#description

cipher: 0x00a8

[juanesf]

This repo has the installer corrected, you just have to download it and execute: python3 setup.py install

[mariusmotea]

check this comment from here,it looks like the required cipher is missing rbit/pydtls#20 (comment)

[juanesf]

In this post they use the same library but with a different cipher and apparently it works (kershner/screenBloom#66) n°7 PSK-AES128-CBC-SHA

[juanesf]

the next library is more updated and has the support of python3 corrected:
https://github.com/inpos/pydtls

[avinashraja98]

Hey Guys, I was able to create a server using the mbedtls ssl_server2 program. It supported the required dtls, cipher suites etc. I only had to disable the http response and create a loop to wait for more messages if the first character of the recv buf is 'H' of the huestream entertainment sdk message format. I am able to properly handshake and get the messages in a char buffer in the c program. I tried this with the example dtls_client.txt program provided by the entertainment sdk page. I am sure it should work for the other clients such as Razer Synapse too. I'll try confirming it later today. Now my question is what should I do with the light data? should I process it in the c program and send http msgs to the lights? should I forward it to the Hue-emulator python script using an unsecured udp channel? does the python script have implementation already done for entertainment? Thanks

[mariusmotea]

WOW, this is the news we expect to hear in this thread. For the moment we can do this: implementation with no security, meaning one static authentication string that will be provided to all users. Later we may find a way to fetch this via http request from bridge emulator. Light data must be passed mandatory to emulator because hue app configure xy data of the lights in every entertainment area so a believe it must be an http PUT or POST request in 127.0.0.1/entertainment (or any location).

[avinashraja98]

I understood the first part, which is getting the static auth string which i have hard coded into the server script for now, multiple username support is causing a slight problem, but I am sure I can figure it out later. I was just wondering if the python script has the implementation for parsing the light data and physically lighting up the bulbs and if yes can you please give me a line number in the file its done in. I am not very good at python sorry.

[avinashraja98]

So I just PUT/POST the light data bytes to a url hosted by the python emulator script. If I can see the implementation(if it exists) on the python script side I think I can understand it well.

[mariusmotea]

i create the service but currently it only prints the messages and don't do anything else. Once i will have some sample output i can go forward with the implementation.

[avinashraja98]

Sure, exactly what I was thinking. If you want sample data, you can create a script that sends dummy data to the udp port 2101 at these rates specified by the entertainment api.

• The bridge sends maximum at 25 Hz messages over ZigBee. Thus, the (fastest) effect rate should be 2 - 3 times slower than this 25 Hz, i.e < 12.5 Hz.
• Due to the lossy streaming channel (UDP, and ZigBee) keep the message rate higher than 25 Hz.

It is a little bit confusing but I'm sure you'll figure it out. I'll do the unencrypted data forward asap and report.

[mariusmotea]

If you have some sample output of python script with original data please paste it here so i can start the implementation

Thanks.

[avinashraja98]

Just finished the server. Should be seamlessly installed and started using easy_install.sh.
Try testing data reception using the hue app. Go to your entertainment area settings and press test area.
The light data should be dumped on the console.(Received at port 2101 by the python script)

[mariusmotea]

Hi,

Just to provide the current status:

• i implement the udp server and the parse for light data in rgb mode.
• because of the big number of requests per seconds i will create dedicated udp service in ESP lights firmware, but for other lights i need to perform some advanced filtering (ex: don't send requests to quick or send them only when there is a notable difference in the light color)
• Hue Sync app for Mac is disconnecting after few seconds. I saw this happens right after a GET request for configuration, so very likely it is inspecting some parameters and because are not the expected ones it stop sending data. In the few working seconds i saw the light data was changing according to the screen brightness .
• All 4 Hue app is triggering an error when i try the entertainment effects, i believe it work first time, but i'm not sure.

TO DO.

• extend the udp light service for all color lights (now is just the rgb-cct version)
• try to understand what values i need to change on the bridge so the Hue Sync will not stop the broadcast anymore.

[avinashraja98]

try to understand what values i need to change on the bridge so the Hue Sync will not stop the broadcast anymore.

This is the problem with Razer Synapse too. I've noticed requests to /api/username/groups/id when streaming show no changes. Is this the expected behavior? because I am pretty sure this is the issue as synapse requests a GET to this address and then immediately stops broadcast and shows stream busy error.

[mariusmotea]

I manage to made Hue Sync to work. Still i need a few changes because in hue application i still see a message that some lights cannot be controlled while hue entertainment is active despite i stop it. Hope today i will push a new release.

[mariusmotea]

With the last commit Hue Sync for Mac is working fine. I still have issues with All 4 Hue application for android that is not broadcasting or is broadcasting just one frame. Next step is to simplify the installer by providing precompiled binaries for arm and amd64 platfrorms.

[avinashraja98]

Latest commit works beautifully with Razer Synapse and Chroma apps(such as Overwatch).

[mariusmotea]

Today i add more improvements and with this i will close this issue.
@avinashraja98 i added you to contributors list, without you it would not have been possible to solve this issue.

[avinashraja98]

Thank You for this wonderful project. Hope everyone has fun with the new features.

[d8ahazard]

Hey there! I'm working on a project that aims to send color data directly to my hue system using the entertainment endpoint.

Just curious, did you ever manage to get a working solution for dtls-handshaking? If so, would you care to share the trick? I'm struggling to get the handshake bit to work...

[mariusmotea]

I manage to get entertainment working, the source code is present in the repo, but from my understanding you try to create the client, here we made the server.

[d8ahazard]

I manage to get entertainment working, the source code is present in the repo, but from my understanding you try to create the client, here we made the server.

I looked at the repo, but it seems as if you're using a compiled binary to do the transmission? I checked the entertainment.py file, but see nothing there for dtls. Is this implemented somewhere else?

[mariusmotea]

The compiled binary is generated from https://github.com/diyhue/diyHue/blob/master/BridgeEmulator/ssl_server2_diyhue.c

[d8ahazard]

The compiled binary is generated from https://github.com/diyhue/diyHue/blob/master/BridgeEmulator/ssl_server2_diyhue.c

Thank you. I'm not too familiar with C (although happy to try and learn).

Is there some way I could chat with you about the issue a bit more? i don't want to keep posting on a closed issue thread. ;)

[mariusmotea]

Unfortunately i did not made this, i just ask for help and i receive it.

[d8ahazard]

Do you know if I would be able to use the binary to accomplish my goal?

Current handshake request:

def do_handshake(self):
    print("Handshaking with ", self.bridge_ip)
    ssl_sock = False
    psk = binascii.unhexlify(self.bridge_key)
    print("PSK", psk)
    try:
        do_patch()  # dtls library doing its thing
        print("Patched")
        host = self.bridge_ip
        port = 2100
        print("Wrapping socket")
        tcp_socket = socket(AF_INET, SOCK_DGRAM)
        tcp_socket.connect((host, port))
        ssl_sock = sslpsk.wrap_socket(tcp_socket,
                                      ssl_version=258,
                                      ciphers='PSK-AES128-GCM-SHA256',
                                      psk=psk, hint=self.user)
        print("Socket wrapped")

        msg = "ping"
        ssl_sock.sendall(msg.encode())
        msg = ssl_sock.recv(4).decode()
        print('Client received: %s' % msg)
    except Exception as e:
        print("Socket exception: ", e)
    return ssl_sock

Current "send message" function:

def send_message(self, states):
    my_socket = self.do_handshake()
    udpmsg = bytearray()
    # Protocol
    udpmsg += bytes("HueStream",'utf-8')
    # Version
    udpmsg += bytes(10)
    # Sequence ID
    udpmsg += bytes(0)
    # Reserved
    udpmsg += bytes(00)
    # ColorSpace (RGB)
    udpmsg += bytes(0)
    # Reserved
    udpmsg += bytes(0)
    # Add individual lights
    for light_data in states:
        print("Sending state for id ", light_data)
        id = light_data[0]
        r = light_data[1]
        g = light_data[2]
        b = light_data[3]
        c = Converter()
        # Add light type
        udpmsg += bytes(0)
        udpmsg += bytes(id)
        udpmsg += bytes(int(r)) + bytes(int(g)) + bytes(int(b))
    print("Trying to send message: ", udpmsg)
    if type(my_socket) is not bool:
        my_socket.sendto(udpmsg, (self.bridge_ip, 2100))
    else:
        print("Invalid socket!")

[avinashraja98]

You are on the right track but this binary is the server and is used to receive the message. Something is probably wrong with your message configuration.

[d8ahazard]

You are on the right track but this binary is the server and is used to receive the message. Something is probably wrong with your message configuration.

I never get to the point of trying to send the message, the handshaking fails and I never get the "Socket wrapped" message printed.

[d8ahazard]

@avinashraja98 @mariusmotea

Two final questions:

1. Will the entertain-srv binary work as a relay from my application to send light data to my hue bridge? It looks like I'd just need to open a normal socket to localhost on port 2100 and send light data to that?

2. If so - may I have your permission to use the binary with my project? I will naturally give full credit...

[mariusmotea]

@avinashraja98 can you help us with an update to the ciphers? Hue Sync start to use new ciphers since PSK-AES128-CBC-SHA is considered week now.

[avinashraja98]

Oh I see. Its been a while since I looked at diyHue, I'll check it out and try getting a solution over the weekend. Lmk if you have any more information thanks.

[mariusmotea]

@avinashraja98 i think i found a solution to star the dtls server with openssl and get the data output from stdout in realtime. It seams openssl has the needed ciphers.

Thanks for your work again, you made a very important implementation for diyhue.

[avinashraja98]

That's great! No problem haha thanks.