Getting started with MQTT

Tweet about this on TwitterShare on LinkedIn3Share on Google+0Share on Reddit0Buffer this pageFlattr the authorEmail this to someonePrint this page

In this post, we will take a look at a communication protocol named Message Queue Telemetry Transport a.k.a. MQTT. MQTT is a lightweight, secure, battery friendly and a machine-to-machine (M2M)/”Internet of Things” connectivity protocol.

Andy Stanford-Clark and Arlen Nipper from Cirrus Link Solutions authored the first version of the protocol in 1999.

What is MQTT?

MQTT (formerly Message Queue Telemetry Transport) is a publish-subscribe based “light weight” messaging protocol for use on top of the TCP/IP protocol. It is designed for connections with remote locations where a “small code footprint” is required and/or network bandwidth is limited. The Publish-Subscribe messaging pattern requires a message broker. The broker is responsible for distributing messages to interested clients based on the topic of a message.

The specification does not specify the meaning of “small code foot print” or the meaning of “limited network bandwidth”. Thus, the protocol’s availability for use depends on the context. In 2013 IBM submitted MQTT v3.1 to the OASIS specification body with a charter that ensured only minor changes to the specification could be accepted. MQTT-SN is a variation of the main protocol aimed at embedded devices on non-TCP/IP networks, such as ZigBee.

Historically, the ‘MQ’ in ‘MQTT’ came from IBM’s MQ message queuing product line. However, queuing per se is not required to be supported as a standard feature in all situations.

Real world applications

Facebook Messenger : Facebook has used aspects of MQTT in Facebook Messenger. However, it is unclear how much of MQTT is used or for what; Moreover it is to be noted that this is a phone application not a sensor application.

Understanding OSI Model

To gain a better understanding of where MQTT fits, you need to have an understanding of the OSI model. If you already know about it, you can skip the below video.

MQTT is in the same layer as HTTP, Layer 7.

Why MQTT?

  1. It is a publish/subscribe protocol
  2. It has Multiple Quality of Service levels (QOS)
  3. It has at-least-once and exactly-once semantics
  4. It has a low overhead (2 bytes at minimum)
  5. It supports offline messaging
  6. It retains messages, like a key/value store

Matteo Collina, the author of Mosca, gave a nice talk on what MQTT is and how it works well with Node.

You can find a copy of his presentation on MQTT and Node.js here.

Where will MQTT fit in?

Imagine the below environment

mqtt_arch

where the status/sensor readings of a single machine/device needs to be transmitted to another machine/device, and a real time dashboard and a tablet or a wearable. Hooking up and maintaining data across so many devices may be tricky. This is where MQTT fits in well.

Hands on

Now that we have a fair understanding of MQTT, we will try out a simple example. We will be implementing a server/broker which will take in data and distribute it to all its clients using Mosca. And then we will implement a MQTT client using MQTT.js that publishes and subscribes to topics via the broker.

Create a new folder named mqttApp. We will create 2 new folders inside it named server and client.

Open a new terminal/prompt inside the server folder. Create a new Node.js project by running

npm init

Fill it up as applicable. Next, we will install Mosca as a dependency and save it. Run

npm install mosca --save

Create another file named server.js inside the server folder. Update it as below

A simple MQTT broker. Things to notice

Line 1 : Include Mosca

Line 3 : Configure the server

Line 8 : Start the server

Line 12 : The call back when the server is started

Line 17 : clientConnected : when a client is connected; the client is passed as a parameter.

Line 22 : published : when a new message is published; the packet and the client are passed as parameters.

Line 27 : subscribed : when a client is subscribed to a topic; the topic and the client are passed as parameters.

Line 32 : unsubscribed : when a client is unsubscribed to a topic; the topic and the client are passed as parameters.

Line 37 : clientDisconnecting : when a client is being disconnected; the client is passed as a parameter.

Line 42 : clientDisconnected : when a client is disconnected; the client is passed as a parameter.

Simple pub/sub API.

If you want to enable the offline mode, we need to store the data into one of the Key value pair storages. Out of the box Mosca supports most of the popular key value pair datastores. Below is a quick example how you can use it with MongoDB.

Updated server.js.

The only thing that changed is line 3. We have added the MongoDB settings here. (You can use a service like Mongo Lab to store the data on the cloud.) And then we include these settings on line 12.

Before we start the server, we need to start the MongoDB instance. If you are new to MongoDB, take a look at this.

Once MongoDB is started, we will start the server by running

node server.js

If MongoDB is started and running, you should see the success message. Else you will see a stack of error.

This is a simple broken server with offline capabilities. Before we go further, we will update our server to work as a real time broker. Update server.js  as below

Here we hook a callback to the server published event and then build a new packet on every message received and then send the new packet to the client.

Now, we will build a MQTT client that can pub/sub to topics. Open a new terminal/prompt inside the client folder.

Our client will be another Node.js application. In this example, both are running on the same machine. But in real time, the client application will be deployed on an actual device like a Pi or a Arduino. And the server would be on the cloud.

Now, run

npm init

Fill it up as applicable. MQTT.js would be a dependency here. Run

npm install mqtt --save

Create a new file named client1.js inside the client folder. We will emulate 2 clients on our local machine to see how things work. Update client1.js as below

Things to notice

Line 1 : We require MQTT

Line 3 : We connect to the server

Line 5 : We subscribe to a topic named “presence”

Line 8 : We publish a message to “presence”

Line 10 : We close the close the client

We will create another client that will listen for changes in the “presence” topic. Create a new file inside client folder and name it as client2.js. Update it as below

Same as client1.js except this client only listens to changes.

To test this, We will first start the server. Run

node server.js

And you should see the server started log. Next, client 2, Run

node client2.js

You should see the client started log. And finally client 1, Run

node client1.js

And the server console should reflect

Screen Shot 2014-10-22 at 7.54.29 pmThe last packet is our presence topic. And the client2 console should look like

Screen Shot 2014-10-22 at 7.55.07 pmSimple and easy communication with MQTT protocol!

You can find the complete code in the example here.

Also do checkout this awesome video by Matteo Collina where he Live Codes a Robot and control it from the web – LXJS 2014

Hope this post gave you an ideas as how to implement MQTT in Node.js

Special thanks to Matteo Collina for reviewing this article


Thanks for reading! Do comment.
@arvindr21

Tweet about this on TwitterShare on LinkedIn3Share on Google+0Share on Reddit0Buffer this pageFlattr the authorEmail this to someonePrint this page
  • Sunil Gudivada

    i want to work this code in the browser. what shoud i do ?

  • Clemens

    Hi, thank you so much for this awesome tutorial!

    When i log the client’s message to the console, I always get a buffer format instead of an ordinary string. Do i have to use the mqtt.js parser here?

    Thanks in advance for any advice on this issue!

    • Clemens

      I found that I had messed up with the .toString() function, sorry for that post. Thanks anyway!

  • Iosu R

    Client.js line 3
    createClient is deprecated, use connect instead

  • Behram Buhariwala

    Sir, I have followed the steps as per the tutorial. When i run the file client2.js and the client1.js it detects a change to the “presence” topic. As per the tutorial, the message to be displayed on client2.js after runing client1.js should be “Client 1 is alive.. Test Ping………..”. But i get data in a buffer format i.e. . Please help me rectify the same.

    • Agrim

      The message is in buffer format as per MQTT spec. call toString() on it.

  • java_an

    mqtt works on sub and pub concept see for more details

    http://www.javaproficiency.com/2015/09/how-to-install-mosquitto-mqtt-on-ubuntu.html

  • Niral

    How to publish message from another javascript file other than server.js?

    • / Arvind Ravulavaru

      Can you elaborate your question please?

      • Niral

        As you can see you’re doing Server.publish() in server.js to publish messages to client. How can I use publish method in other javascript file rather than server.js as object server is not in scope of other file. I don’t know much of Javascript as I am Electronics guy.

        • / Arvind Ravulavaru

          So, if I understood the requirement correctly, when the client reaches out to the server, you want to run a processing logic from some other file and return the packet.

          For that you can use Node’s require() and module.exports.

          You can create a file named myLogic.js at the same level as server.js. The contents of myLogic.js would be

          And now your server.js would be

          If my understanding is correct, above would be a solutions, else please share your requirement.

          You can know more about Nodejs design patterns from here : /node-js-design-patterns/

          Thanks.

          • Niral

            Thanks for the replay..

            Above code snippet and that post is really informative.

            Let me tell you exact scenario in my project. I am using express in my project for routing.

            in my app.js I am doing..

            var data = require(‘./routes/data.js’);

            app.get(‘/data/:a/:b’, data.data);

            in data.js (which is in routes directory) I have..

            exports.data = function (req, res) {
            var data = {
            “Data”: {
            “a”: req.params.a,
            “b”: req.params.b
            }
            };

            res.send(data);
            };

            What I’m trying to do is to publish same data to MQTT client after sending json response to HTTP client (res.send(data);).

          • / Arvind Ravulavaru

            If the request is coming to express, how will mosca know the client? Express understands its own clients and mosca its. You cannot send a response to a mosca client from a request to Express (As far as I know, I could be totally wrong!) You can raise an issue here : https://github.com/mcollina/mosca and check with Matteo.

            Do let me know how it works out!

  • Pedro Jimenez

    Hi, nice tutorial. I hope this code help, I just wrote some code to complement the clients, the new client has the capability to re-subscribe to the topics when the client reconnect to the mqtt server. https://github.com/pedrojimenezp/mqtt-client-server

    • / Arvind Ravulavaru

      Hello Pedro, Thanks!. And Thanks for sharing the code.