This post provides an update to our previous post and video describing The OpenNMS entry to the Eclipse IoT challenge 2017

Project Description

The aim of this project is to use IoT pollution measurements as a compelling use case to prove a design for an MQTT plugin for OpenNMS.
This will position OpenNMS as a highly capable platform for distributed IoT monitoring applications. IoT projects will immediately be able to leverage OpenNMS’s existing very powerful event correlation and performance data visualization in the IoT space. Further more, OpenNMS can be used both collect the IoT data and also to manage the IoT infrastructure itself.
The MQTT plugin allows OpenNMS to connect to multiple MQTT brokers and parse any compressed or uncompressed JSON, CSV, XML (and in the future Protobuf) MQTT payload into OpenNMS events or OpenNMS Performance data. Example configurations are provided for several MQTT data sources including Eclipse Kura.
Performance data can either be stored in OpenNMS’s internal RRD files or alternatively stored in Cassandra using the OpenNMS Newts technology which offers a highly scalable cloud storage solution.
Similarly events can be processed and stored internally in OpenNMS’s Postgres database or forwarded to Elastic Search for a large scale data archive.

A complete power-point presentation of the project which serves as a detailed report is available at Airwatch I0T Challenge Report March 2018

The code associated with this project is available on github at opennms mqtt plugin

Open technology and standards

This project seeks to support the adoption of open technologies and standards for IoT projects by providing an alternative Open Source solution to the cloud hosted IoT environments offered by Google, Amazon , Microsoft etc. It seeks to compliment the work of the Eclipse IoT project and in particular to support the deployment of solutions built on Eclipse Kura. OpenNMS is an open source network and infrastructure management platform written in Java and distributed under the  Affero General Public License (AGPL). Commercial support is available from the OpenNMS Group. OpenNMS supports and/or implements or integrates with the following standards and technologies

• Karaf OSGi platform
• Cassandra
• Elastic Search
• Postgresql
• Grafana
• Kibana
• ActiveMQ
• Jboss Rules (Drools)
• Management protocols supported include SNMP, WMI, JMX, SYSLOG and now MQTT
• All external integration can be achieved using ReST XML or Json calls.
• Eclipse is the recommended IDE for working on OpenNMS code.

Applicability of a solution to a specific industry

The specific use case for this implementation has been in support of an emerging project called Solent Airwatch. This project seeks to support  citizen science / crowd sourced data collection of air pollution measurements across a city. Although we are initially targeting the measurement of Air Quality, the same technology could be applied for any other IoT measurement application.

Key to our approach is the leverageing of an existing  data collection platform which has been proven in large scale data collection and processing of network and data centre statistics. Many of the large telecoms players who already use OpenNMS are actively investigating business opportunities for IoT. So this allows them to extend rather than replace an existing platform. In addition, the solution allows them to manage the IoT infrastructure as well as collect data using OpenNMS

Lessons learned

Participating in this IoT challenge has helped us accelerate our development of the Solent Airwatch solution. It has allowed us to learn about the practicalities of implementing an MQTT based solution. Specifically we have learned;

1. Core data collection and connectivity is only part of the problem of Citizen Science. We need to be able to integrate back end data collection with front end web enabled communities to deliver a complete solution. We are in the process of designing a front end web site which will explain and excite interest in the measurements begin collected.

2. The Data science around measurements collection and analysis is very important. We need to be able to support complex cross correlation between data sets in order to leverage the data we are collecting from our IoT infrastructure. This is more complex and less structured than the data we normally deal with from networks or data centres. We are presently investigating integrations with Cassandra and OpenNMS which can support more complex data analytic s using R or Matlab.

3. We had hoped to re-implement our initial Raspberry Pi based Sniffy in Eclipse Kura. We still intend to do this but the documentation and example code surrounding Kura is incomplete and complex. We think the platform will need to become much more accessible if it is to get wider adoption.

We are hoping to get several Master’s students to work on taking this initial proof of concept forwards.

Overall the project has been a very interesting challenge and we will certainly want to be involved with the Eclipse IoT team in the future.

In December Ronny, David and I went to TM forum Digital disruption in San Jose. OpenNMS has been involved in the TM Forum for many years but this was the first time we had taken a stand at the show. We have always had a bit of a love/hate relationship with the TM Forum because from a business perspective it is hard to justify the membership based upon the natural demographic of our customers.

The TM Forum  have traditionally only addressed the needs of Telecoms service providers but as the industry has been converging, they have realised the need to expand their membership to included wider IT/Web industry players.

This change in focus is potentially very good for OpenNMS since we have made a lot of headway in the wider IT space but have found it difficult to penetrate Telecoms. As part of their efforts to  consciously move from the Telco space into the wider Cloud/IT/Web infrastructure providers market the TM Forum have moved their North American show to Silicon Valley in order to attract this audience. We wanted to use the show to demonstrate the new STUI display and features of OpenNMS 14 which were particularly interesting to a cloud provider.

Evaluation

We have quite a large number of fairly senior decision makers to the stand but unfortunately the worst storm to hit San Francisco in 4 years happened half through the week and most of the delegates left early !!. Overall I think the show was worthwhile but given the footfall on the stand, I think the project could have had better value for the money we spent. In future I think if we continue to engage with the TM Forum the most cost effective way will be to participate in catalysts rather than take a stand. Catalysts are real projects in which we can demonstrate OpenNMS technology along side other vendors equipment and systems. I think overall it is a better window for OpenNMS than just a stand.

STUI

If you want to know more about STUI, see the youtube video which David Hustace has created here.

http://www.youtube.com/watch?v=2oudyfiwijw