Getting sensor data via API using Node Assistant
In this walkthrough demo, we will show how we can get realtime dat from an API and bring this into Twinview. We will perform a HTTP GET request to get the information we need.
What we will need
Postman API tool (Free version is fine)! - We use this to test our API calls
An API Key for Openweather (Free version is fine!)
⏰ Time: About 30 mins Difficulty Level: 🍺 🍺 📺 Video Tutorial Available
In this tutorial we are going to make a query to the Openweather API to get the external temperature for our buildings location and bring this into Twinview and place on the project dashboard.
Step 1 - Review the API Documentation
We first need to check and confirm thE the API can provide the data we need as well as the following understanding:
The Datatype what is returned for the 'temperature' attribute.
What info we need to provide the API so that it can return the temperature for our location
How we Authenticate with the API
Any rate limits or Poll limits
We can review the documentation here:
https://openweathermap.org/currentWe can see that the API uses basic authentication, and all we need to do is pass our API key {API key) into the URL as a parameter along with the Longitude {Lon} and Latitude{Lat} of our building,
We need to update the URL above placing the {Lon} and {Lat} placeholders with the Longitude and Latitude values for our actual building location. You can use a website like https://www.latlong.net to get this.
We also need to replace the {API Key} placeholder with our Openweathermap API key. To do this we need to login to Openweathermap and goto the 'My API keys' page. You can access this here:
Once you have replaced all 3 placeholders in the URL, you new URL should be something like below (your API Key and Lon Lat values will be different thqn those shown.
Step 2 - Testing the Endpoint with Postman
Before we create the IOT Data Stream in Twinview it is good practice to test the endpoint we have just created using a free tool like Postman . Postman is the industry standard tool for testing and playing around with APIs.
In simple terms we will use Postman to make a HTTP GET call to the API URL endpoint and view the response.
Ensure that you use the URL thaat contains your API Key and location values. Do NOT copy and use the example above as the API Key is for info only and has been been deleted.
Step 2.1 - Paste the Endpoint URL onto the URL Field
Step 2.3 - Change the HTTP method to GET
We need to make sure that the HHTP request (left of the URL) iis GET.
We know this because we are requesting (getting) data from the server and not posting data to the server (this would be. a POST request). Many API' will confirm what request type to use with each endpoint.
Step 2.4 - Press 'Send' to make the request and await the response.
If the request was successful we should see the weather information in the body section. The data object that is returned is called the 'payload'. The data is structured in an standard schema called JSON.
We can also see that the server sent a 200 response back when we made the web request. This means that request was 'Successful, and we can now create the device in Twinview.
Step 3 - Create a IOT Data Stream Device in Twinview
To add a new IOT Data Stream in Twinview, we must first:
Create a new IOT integration Interface
Add a IOT Data Stream to the interface.
Create a device security group and add the newly created IOT Data Stream to the Device group, then give ourselves permission to access the Device security group.
Step 3.1 - Create a new IOT Integration Interface
When in the project, click on IOT Integrations in the left hand main-menu. If you dont see this menu them your user role does not have permission to access IOT integrations or your Twinview subscription does not include the IOT and Analytics package,
In the IOT Integrations modal click 'Add' and give your IOT Integration a name, for example 'External API Services'. We must also select a Reference and those should set to Twinview Direct Integration.
Click Save, and a new field will appear called 'Subdomain'. Enter "Ingestion" into this interface them click Save again.
While your project can just contain one IOT integration interface, and you can add all your data streams to the single interface, we recommend creating a new IOT integration per sub system, for example you could have a IOT integrations called 'BMS Integration, LoRaWAN Sensors and Metering systems etc.
The "Ingestion" subdomain is the default Twinview data Ingestion service and by entering this as the subdomain we are instruction Twinview to use the default data ingestion service. Some enterprise customers who utilise their own Datalake may have their own ingestion service and subdomain.
Step 3.2 - Create a new IOT Data Stream Device
Now we have created our 'IOT Integration Interface' we can add the IOT Data Streams to the integration interface. Each IOT Data Stream in a single point of data, e.g. 'Temperature, Humidity, CO2 etc'.
When in the project click the 'IOT Integrations' in the left hand side menu, and when the model opens, you should see the IOT interface we created in step 3.1. Click 'Devices' to show all 'IOT Data Stream 'devices setup on the interface. At the moment we dont have any, so we must create one by clicking the 'Add'' button.
Give your IOT Data Stream Device a name and make sure the 'Type' is set to 'Sensor', then click 'Save'
We strongly recommend that you define a standard naming schema for your IOT Data Stream devices as this will make them easier to recognise in the future when creating dashbords or reports that consume this data.
We recommend that the name should make it possible to know the Devices location in the building by incorporating the floor level, room and asset that the IOT Data Stream is applicable to.
The IOT Data Stream Device will be created and a new model will appear that allow you to specify the IOT Data Stream Device details. For this training exercise the only field we need to check is the 'Data Type' field. and the 'Suffix' field.
We cab review the Opeweather API to find both the Data Type and optionally any Suffix we may want to add.
We can see from the API document that the Temperature Datatype the API will return is a Numeric value, so we must ensure that we set the 'Data Type' to 'Number' (See 1 below).
We can also see from the documentation that the Temperature value been returned in in 'Kelvin', so we can add the symbol 'K' in the optional suffix field (See 2 Below).
You will also notice that the Edit Device modal also shows 'Endpoint' URL along with the request type POST. You will also see that Twinview is showing and example payload structure. We will need this information in the next few steps, however we can access this infomation anytime we want from the IOT Data Stream Devices list, so for now we can click 'Save" to create our IOT Data Stream Device.
Even though we have just added a new IOT Data Stream Device successfully if we tried to add the data stream to a dashboard widget the device would not show in the list when we try do so, this is because when you create a new IOT Data Stream Device it does not belong to any security groups.
This behaviour is by design as making the device accessible before its been applied to a security group or adding it automatically to an existing security group would mean everyone would have access to the IOT Data Stream device.
If you have 1000'd of devices, rather than manually add them one-by-one through the interface as above, they can be created using the Twinview IOT Device API endpoint, or using the Node Assistant visual scripting 'Create Device' node.
For a video tutorial on how to add many IOT Devices in built using the Node Assistant visual scripting see the tutorial here.
Step 3.3 - Creating a new Device Security Group and associating the IOT Data Stream we just created and ourselves to the group.
To create a new security group, while in the project goto the 'Settings ' which is accessible from the main left-hand menu.
In the Setting dialogue, click the 'Device Group' tab and then click 'Add' to create a new Device Group.
Give your device group a name and click 'Save' to create the Device Group. Once the Group has been created click the 'View' button to open the device group.
On the Device group window we fist need to go and add the IOT Data Stream Device we created earlier to our new 'Device Group'. To do this we click on the 'Devices' tab then click the 'Use' bottom against the IOT Data Stream Device we created earlier.
We have successfully assigned the IOT Data Stream Device to the Device Group. Next we need to assign ourselves to the Device Group so that the IOT Data Stream Device will be available for us to use on widgets, dashboarding and reporting. To do this click on the 'Users' tab and click 'Use' on your user, and any other users you want to be able access this IOT Data Stream Device.
We recommend that you think about how you want to use Device Groups on your project and plan these into your onboarding and infrastructure strategy.
Some people have a single device group, and people have only a handful but use then carefully, for example "High Security Devices" and "Standard Devices".
Some clients use device groups to control access to devices by their tenants and or staff and have a device group for each Tennant/Apartment.
Step 4 - Publishing the Data from Openweather API to Twinview Ingestion service.
Now that you have successfully created the IOT Data Stream Device, and added this to a device group, the next step is to start publishing data to the Data Stream from the Openweather API.
The issue is that Twinview Ingestion service wants us to POST data to it (as shown above), and the open weather API wants us to make a GET Request to get the data. What we need is some middleware service to do the following
On a set heartbeat/frequency:
GET the data from Openweather API
Transform the data and extract the value we need
Format the the results into the JSON object required by Twinview.
POST this data to the Twinview Ingestion service.
This is where the Twinview Node Assistant visual scripting interface can help. Node Assistant runs on Twinview Physical Gateways as well as Twinview virtual (cloud hosted) gateways. Becasuse we are getting the IOT data from a web API (openweather) and not a local network, a physical gateway is not needed and a virtual gateway would work in this scenario.
Step 4.1. - Load up Node Assistant and Login
If your Twinview subscription included IOT Analytics, you will have been assigned a Node Assistsnt device login.
Navigate to your gateway URL https://{YOUR_GATEWAY_URL}, and you should see a login screen (as below). Enter your username and password as provided by Twinview., then click 'Login'
Step 4.2. - Once logged in we first create a new flow
Once you are logged in, you should add a new flow by clicking on the "+" to the far right of the flow tabs.
Once the "+" is clicked it will create a new tab. Double click this new tab and a nodal will slide out from the right. Give the tab a name by completing the Name field, and then clicking 'Done'
We now have a clean canvas which to create our flow. We are going to replicate the HTTP get request we did in step 2 of this article, but instead of using Postman software we are going to make a flow in Node Assistant to make an API call every 30 minutes, extract the value we need and send this to Twinview's ingestion service.
Below is a a simplicity diagram of this process.
Fist search for a Inject node in the left node browser and drag this onto the Flow Canvas as shown.
Next we need a HTTP Request Node. Search for this in the node browser and drag this onto the canvas, behind the inject node, and wire output the inject node into the inout of the http request node.
Now lets configure some setting in the 'Inject node'. To do this double click the inject node* and the edit modal screen should slide in from the right.
*The inject node may have changed its label to be called 'timestamp'
We need to click the 'Inject once after' check, and also change the dropdown to 'every 30 minutes'.
This is telling the flow to run straight away and then every 30 minutes thereafter.
Click 'Done' to save the settings.
Next we have to configure the HTTP request node. To do this we double click the node and configure it as below.
The Method should be set to 'GET' as we are getting information from the API.
The URL is the endpoint as where we are making the request as per the Openweater documentation. This is the same URL we used in section 1 and 2 to make the postman request and must include or API key and the Lon/Lat passed as parameters in the URL.
The <- Return should be set to 'a Parsed JSON Object' as we know that this API returns JSON.
Click 'Done'
Before we run the flow we need to add a debug node so that we can check that its working and see the data payload been returned from the API. To do this we need to search for the 'debug' node in the node pallet and drag this onto the canvas after the http request node.
We now need to open the 'debug' console so we can see the output of the debug node ans see if it is working. The debug console is on the right hand side and can be turned on by clicking the 'ladybug' icon in the top left.
To start the script and fire the flow we need to click the blue save button in the top right. Once saved the flow ill start and then fire evert 30 minutes automatically. If successful you should see a payload of data be received by Node assistant.
Tip: You can manually fire the flow by clicking the blue button on the left side of the Inject node, rather than wait for it to fire. This is good for debugging and manually firing the flow rather than wait for the next scheduled poll.
If successful you should see the response in the debug Windows as shown below. The response is in JSON which is a Key:Value schema. You can expand the tree to see the full data object.
Congratulations we have successfully created an automated script that calls the Openweather API every 30 minutes and gets the response into Node Assistant.
The next step is we need to extract the piece of data we need and send this to the IOT Data Stream Device called 'Openweather-Sensor-Temprature' we created in Twinview (in Step 3.2) every 30 minutes.
Create a function node to format send the data to Twinview Ingestion service in the JSON format required
Search for the 'function' node in the node browser and drag this onto your canvas. Double click the node to edit and paste the following into the code area:
Make sure you change the UUID to match the UUID of the IOT Data Stream Device you created in Section 3.2. The UUID tells Twinview which device this data should be associated with.
It should look like this, if looks okay press 'Done'
How to get diffrent data from the response
The 'State' is just the value we are sending. The msg.payload.main.temp is saying place the vale we get from the payload here. If you wanted a different value to the temperature you just need to make sure that this is pointing to the right path.
An easy way to get a path for a value is to grab the value from the debug response and use the 'copy path button'. You can then paste the path in place of the msg.payload.main.temp - just make sure it begins with msg. and you may need to add this.
If using a diferent data, you need to make sure that the data type aligns with the data type you set in your IOT Data Stream device in Section 3.2. If data types do not align twinview will throw an error.
Give the Function a name and connect it to the output of your http request node.
Giving your function nodes to explain what their function is, is good practice.
Send the data to the Twinview Ingestion Service
In the last few steps we have setup an automatic flow that fires every 30 minutes and gets dat from the Openweather API. We then extract the data we need and format this into a paylaod that the Twinview Ingestion services requires, including adding our devide UUID.
Now we need to send the payload to the Twinview 'Ingestion' service. To do this we will use another HTTP request node and we will place this after the function node.
We need to edit the http request node we just added and change the Method to POST (as we will be posting data to Twinview). We also need to specify where to send the data by entering the Twinview Data ingestion service URL in the URL field. Either of the Twinview Ingestion URL's below will work.
Congratulations your IOT Data Steam Devices are now setup and receiving data!
The next steps is to go and create a dashboard and add a widget to show the data,
Hopefully this Tutorial was useful to walk through the process of connecting data from a 3rd party source into Twinview, and the power/potential of Node Assistant and the part it plays in this process.
This tutorial was a basic introduction and has not even begun to scratch the surface of what is possible. More advanced API authentication methods are supported including OAuth2 and we highly recommend receiving advanced training, as some of the more advanced functions such as looping and extraction can improve efficiencies and processes, especially when working on larger projects with 10,000 of devices
Any questions, feedback or comments on this tutorial, please let us know at support@twinview.com or help@twinview.com
Last updated
Was this helpful?