MQTT Topics & Best Practices

MQTT Essentials Part 5: MQTT Topics & Best Practices

Topics A topic is a UTF-8 string, which is used by the broker to filter messages for each connected client. A topic consists of one or more topic levels. Each topic level is separated by a forward slash (topic level separator).

topic_basics

In comparison to a message queue a topic is very lightweight. There is no need for a client to create the desired topic before publishing or subscribing to it, because a broker accepts each valid topic without any prior initialization.

Here are a few example topics:

myhome/groundfloor/livingroom/temperature USA/California/San Francisco/Silicon Valley 5ff4a2ce-e485-40f4-826c-b1a5d81be9b6/status Germany/Bavaria/car/2382340923453/latitude Noticeable is that each topic must have at least 1 character to be valid and it can also contain spaces. Also a topic is case-sensitive, which makes myhome/temperature and MyHome/Temperature two individual topics. Additionally the forward slash alone is a valid topic, too.

Wildcards When a client subscribes to a topic it can use the exact topic the message was published to or it can subscribe to more topics at once by using wildcards. A wildcard can only be used when subscribing to topics and is not permitted when publishing a message. In the following we will look at the two different kinds one by one: single level and multi level wildcards.

Single Level: + As the name already suggests, a single level wildcard is a substitute for one topic level. The plus symbol represents a single level wildcard in the topic.

topic_wildcard_plus

Any topic matches to a topic including the single level wildcard if it contains an arbitrary string instead of the wildcard. For example a subscription to myhome/groundfloor/+/temperature would match or not match the following topics:

topic_wildcard_plus_example

Multi Level: # While the single level wildcard only covers one topic level, the multi level wildcard covers an arbitrary number of topic levels. In order to determine the matching topics it is required that the multi level wildcard is always the last character in the topic and it is preceded by a forward slash.

topic_wildcard_hash

topic_wildcard_hash_example

A client subscribing to a topic with a multi level wildcard is receiving all messages, which start with the pattern before the wildcard character, no matter how long or deep the topics will get. If you only specify the multilevel wildcard as a topic (#), it means that you will get every message sent over the MQTT broker. If you expect high throughput this is an anti pattern, see the best practices below.

Topics beginning with $ In general you are totally free in naming your topics, but there is one exception. Each topic, which starts with a $-symbol will be treated specially and is for example not part of the subscription when subscribing to #. These topics are reserved for internal statistics of the MQTT broker. Therefore it is not possible for clients to publish messages to these topics. At the moment there is no clear official standardization of topics that must be published by the broker. It is common practice to use $SYS/ for all these information and a lot of brokers implement these, but in different formats. One suggestion on $SYS-topics is in the MQTT GitHub wiki and here are some examples:

$SYS/broker/clients/connected $SYS/broker/clients/disconnected $SYS/broker/clients/total $SYS/broker/messages/sent $SYS/broker/uptime Summary So these were the basics about MQTT message topics. As you can see, MQTT topics are dynamically and give great flexibility to its creator. But when using these in real world applications there are some challenges you should be aware of. We collected our best practices, we learned the last year with excessively using MQTT in various projects. We are open to other suggestions or a discussion about these in the comments, so let us know your best practices or if you disagree with one of our best practices!

Best practices Don’t use a leading forward slash It is allowed to use a leading forward slash in MQTT, for example /myhome/groundfloor/livingroom. But that introduces a unnecessary topic level with a zero character at the front. That should be avoided, because it doesn’t provide any benefit and often leads to confusion.

Don’t use spaces in a topic A space is the natural enemy of each programmer, they often make it much harder to read and debug topics, when things are not going the way, they should be. So similar to the first one, only because something is allowed doesn’t mean it should be used. UTF-8 knows many different white space types, it’s pretty obvious that such uncommon characters should be avoided.

Keep the topic short and concise Each topic will be included in every message it is used in, so you should think about making them short and concise. When it comes to small devices, each byte counts and makes really a difference.

Use only ASCII characters, avoid non printable characters Using non-ASCII UTF-8 character makes it really hard to find typos or issues related to the character set, because often they can not be displayed correctly. Unless it is really necessary we recommend avoid using non ASCII character in a topic.

Embed a unique identifier or the ClientId into the topic In some cases it is very helpful, when the topic contains a unique identifier of the client the publish is coming from. This helps identifying, who send the message. Another advantage is the enforcement of authorization, so that only a client with the same ClientId as contained in the topic is allowed to publish to that topic. So a client with the id client1 is allowed to publish to client1/status, but not permitted to publish to client2/status.

Don’t subscribe to # Sometimes it is necessary to subscribe to all messages, which are transferred over the broker, for example when persisting all of them into a database. This should not be done by using a MQTT client and subscribing to the multi level wildcard. The reason is that often the subscribing client is not able to process the load of messages that is coming its way. Especially if you have a massive throughput. Our recommended solution is to implement an extension in the MQTT broker, for example the plugin system of HiveMQ allows you to hook into the behavior of HiveMQ and add a asynchronous routine to process each incoming message and persist it to a database.

Don’t forget extensibility Topics are a flexible concept and there is no need to preallocate them in any kind of way, regardless both the publisher and subscriber need to be aware of the topic. So it is important to think about how they can be extended in case you are adding new features to your product. For example when your smart home solution is extended by some new sensors, it should be possible to add these to your topic tree without changing the whole topic hierarchy.

Use specific topics, instead of general ones When naming topics it is important not to use them like a queue, for example using only one topic for all messages is a anti pattern. You should use as specific topics as possible. So if you have three sensors in your living room, you should use topics myhome/livingroom/temperature, myhome/livingroom/brightness and myhome/livingroom/humidity, instead of sending all values over myhome/livingroom. Also this enables you to use other MQTT features like retained messages, which we cover in one of the next posts.

So that’s the end of part five in our MQTT Essentials series. We hope you enjoyed it. In the next post we cover the often mention Quality of Service (QoS) in MQTT. We’ll explain why this is an essential feature and how you can leverage it.

Have a great week and we’ll hope to see you on the next MQTT Monday!

If you still haven’t signed up for our newsletter in order to get each new post delivered directly to your inbox, you can do so below. If you prefer RSS, you can subscribe to our RSS feed here.