Processing WeatherStation data using Azure IoT Hub and Stream Analytics

Posted on Saturday, January 2, 2016

Recently I received the Microsoft IoT Pack for Raspberry Pi 2
Using this starter pack you can create the WeatherStation sample that can be found over here
With this sample and combining it with the Azure IoT Hub and Stream Analytics it is very easy to display the data from the WeatherStation in a Bing Maps website.

The complete sourcecode for this solution, can be found here

WeatherStation application

First of all, lets start with a new Windows IoT Core Background Application. If you don't have this template then first follow the steps here to set up your pc.

In this project, add the class that's reading the barometric sensor. The class BMP280.cs is the class that can be found in the WeatherStation sample.
Then add NuGet references to Microsoft.Azure.Devices.Client and Newtonsoft.Json. These references are used to communicate with the Azure IoT Hub and for serializing object to Json before we send them to the IoT Hub.

With this done, update the StartupTask class with the following code.

public sealed class StartupTask : IBackgroundTask
{
    private const float seaLevelPressure = 1013.25f;

    private BackgroundTaskDeferral _deferred;
    private BMP280 _bmp280;
    private DeviceClient _deviceClient;
    private Geoposition _location;
    private bool _isCancelled;
    private DateTime _lastReport;
    private IList<float> _tempuratures;
    private IList<float> _pressures;
    private IList<float> _altitudes;

    public async void Run(IBackgroundTaskInstance taskInstance)
    {
        _deferred = taskInstance.GetDeferral();
        try
        {
            taskInstance.Canceled += TaskInstance_Canceled;

            _bmp280 = new BMP280();
            await _bmp280.Initialize();

            var connectionString = "<IoT Hub device Connection String>";
            var connectionStringBuilder = IotHubConnectionStringBuilder.Create(connectionString);
            _deviceClient = DeviceClient.CreateFromConnectionString(connectionString);
            await _deviceClient.OpenAsync();

            var geoLocator = new Geolocator();
            _location = await geoLocator.GetGeopositionAsync();

            _lastReport = DateTime.UtcNow.AddMinutes(5 * -1);
            _tempuratures = new List<float>();
            _pressures = new List<float>();
            _altitudes = new List<float>();

            while (!_isCancelled)
            {
                await ProcessSensorData();

                if (_lastReport.AddMinutes(5) < DateTime.UtcNow)
                {
                    await SendSensorDataToIoTHub(connectionStringBuilder.DeviceId);
                    _lastReport = DateTime.UtcNow;
                }
                
                await Task.Delay(5000);
            }
        }
        catch (Exception ex)
        {
            Debug.WriteLine($"Error during run WeatherStation: {ex.Message}");
        }
        finally
        {
            _deferred.Complete();
        }
    }

    private async Task ProcessSensorData()
    {
        _tempuratures.Add(await _bmp280.ReadTemperature());
        _pressures.Add(await _bmp280.ReadPreasure());
        _altitudes.Add(await _bmp280.ReadAltitude(seaLevelPressure));
    }
    
    private async Task SendSensorDataToIoTHub(string deviceId)
    {
        var temperature = _tempuratures.Average();
        var pressure = _pressures.Average();
        var altitude = _altitudes.Average();
        var weatherStationMessage = new WeatherStationMessage
        {
            DeviceId = deviceId,
            PreciseTime = DateTime.UtcNow,
            Temperature = _tempuratures.Average(),
            Pressure = _pressures.Average(),
            Altitude = _altitudes.Average(),
            Latitude = _location.Coordinate.Latitude,
            Longitude = _location.Coordinate.Longitude
        };
        await _deviceClient.SendEventAsync(
            new Message(
                Encoding.UTF8.GetBytes(
                    JsonConvert.SerializeObject(weatherStationMessage))));
        _tempuratures.Clear();
        _pressures.Clear();
        _altitudes.Clear();
    }

    private void TaskInstance_Canceled(IBackgroundTaskInstance sender, BackgroundTaskCancellationReason reason)
    {
        _isCancelled = true;
    }
}

Lets explain what this code is doing. When the background task is started, it first initializes the barometric sensor and then creates the Iot Hub client.
For the IoT Hub we need to have the connection string for a specific device. The quickest way to add a device to the IoT Hub is by using the Device Explorer
After that, the geolocation of the device is stored, so in the Bing Map website, this location can be used to display the temperature of this device.
Finally the barometric sensor data is read every 5 seconds and every 5 minutes the average is calculated and this is send to the IoT Hub.

Stream Analytics

With this in place, a Stream Analytics job can be created that saves the WeatherStation data in an Azure Storage Account.
In the Stream Analytics job an Input for the IoT Hub must be created.

For the Output two Table Storage outputs will be created, one output (WeatherStationDeviceData) which will store all the data and a second output (WeatherStationDevices) that will only store the geolocation and the last know temperature of a device.

With these input and outputs in place, the query to combine them can be created.

SELECT
    *
INTO
    [WeatherStationDeviceData]
FROM
    [WeatherStationData]

SELECT
    'Location' as [PartitionKey],
	DeviceId,
	Latitude,
	Longitude,
    Temperature,
    Altitude,
    Pressure
INTO
    [WeatherStationDevices]
FROM
    [WeatherStationData]

The second part of the query uses a fixed PartitionKey 'Location', this makes sure that the WeatherStationDevices output (which uses 'Location' as PartitionKey and DeviceId as rowKey) will only have a single record for a device.
New records will be updated based upon the PartitionKey and RowKey and therefor the last temperature for a device will always be stored in the WeatherStationDevices table.

Bing Maps Website

Now that there is data stored inside a Table Storage, a simple website can be created to display this data in a Bing Map.
A new ASP.Net 5 website can be added to the existing solution. The website will use Mvc to display the page and will use WebApi to query the devices data.
In the Index.cshtml file, add the following markup.

<div class="row">
    <div id="map"></div>
</div>

@section scripts{
    <script type="text/javascript" src="http://ecn.dev.virtualearth.net/mapcontrol/mapcontrol.ashx?v=7.0"></script>
    <script type="text/javascript">
        $(function () {
            map.initialize();
        });
    </script>
}

The map.initialize method initializes the Bing Map and retrieves the devices data and adds a pushpin to the map for each device.

var map = function () {
    var
        bingMap = null,
        pinInfobox = null,
        initialize = function () {
            // Initialize the map
            try {
                bingMap = new Microsoft.Maps.Map(document.getElementById("map"),
                    {
                        credentials: '<Bing maps credentials>',
                        mapTypeId: Microsoft.Maps.MapTypeId.road
                    });
                pinInfobox = new Microsoft.Maps.Infobox(new Microsoft.Maps.Location(0, 0), { visible: false });
                Microsoft.Maps.Events.addHandler(bingMap, 'viewchange', hideInfobox);
                bingMap.entities.push(pinInfobox);

                var geoLocationProvider = new Microsoft.Maps.GeoLocationProvider(bingMap);
                geoLocationProvider.getCurrentPosition({ successCallback: ZoomIn });

                function ZoomIn(args) {
                    bingMap.setView({
                        zoom: 5,
                        center: args.center
                    });
                }

                $.getJSON('api/values', null, function (locations) {
                    $.each(locations, function (index, location) {
                        var pushpin = new Microsoft.Maps.Pushpin(
                            new Microsoft.Maps.Location(location.Latitude, location.Longitude));
                        pushpin.deviceId = location.DeviceId;
                        pushpin.temperature = location.Temperature;
                        pushpin.setOptions({
                            visible: true
                        });
                        Microsoft.Maps.Events.addHandler(pushpin, 'click', displayInfobox);
                        bingMap.entities.push(pushpin);
                    });
                });
            }
            catch (err) {
                alert(err.message);
            }
        },
    displayInfobox = function (e) {
        pinInfobox.setOptions({
            offset: new Microsoft.Maps.Point(0, 25),
            visible: true
        });
        pinInfobox.setLocation(e.target.getLocation());
        var html = '<div id="infoboxText"';
        html = html + 'style="background-color:White; border-style:solid;border-width:medium; border-color:DarkOrange; min-height:100px; ';
        html = html + 'position:absolute;top:0px; left:23px; width:240px;">';
        html = html + '<b id="infoboxTitle" style="position:absolute; top:10px; left:10px; width:220px;">';
        html = html + e.target.deviceId + '</b>';
        html = html + '<a id="infoboxDescription" style="position:absolute; top:30px; left:10px; width:220px;">';
        html = html + e.target.temperature + ' deg C</a></div>';
        pinInfobox.setHtmlContent(html);
    },
    hideInfobox = function (e) {
        pinInfobox.setOptions({ visible: false });
    }

    return {
        initialize: initialize
    }
}();

The last thing to add, is the WebApi controller for retrieving the devices data. This controller simply queries the Table Storage WeatherStationDevices table and returns all the devices data.

[Route("api/[controller]")]
public class ValuesController : Controller
{
    private readonly IConfigurationRoot _configuration;

    public ValuesController(IConfigurationRoot configuration)
    {
        _configuration = configuration;
    }

    // GET: api/values
    [HttpGet]
    public IEnumerable<Location> Get()
    {
        var storageAccount = CloudStorageAccount.Parse(_configuration["Storage:ConnectionString"]);
        var tableClient = storageAccount.CreateCloudTableClient();
        var table = tableClient.GetTableReference("WeatherStationDevices");
        var devices =
            (from device in table.CreateQuery<WeatherStationDevice>()
                where device.PartitionKey == "Location"
                select device)
            .ToList();

        return devices.Select(d =>
            new Location
            {
                DeviceId = d.RowKey,
                Latitude = d.latitude,
                Longitude = d.longitude,
                Altitude = d.altitude,
                Pressure = d.pressure,
                Temperature = d.temperature
            });
    }
}

Disclaimer: Any views or opinions expressed on this blog are my own personal ones and do not represent my employer in any way.