8 months ago

Big data and IoT

It shouldn’t be wrong to state that the future of technology lies in data and its analysis. More and more devices and objects are today connected to the internet, making it easier for transmitting the information gathered originally for analysis. The agenda is to harness this information so as to learn about trends and patterns which can be used to deliver a positive impact on energy conservation, health, lifestyle and transportation. However, it is not the data that produces these objectives rather it is the solutions that you get from analysing these to find the answers that we need.  

Before we head further there are two terms which need to be discussed in relation to this future – The Internet of Things (IoT) and big data. It is almost impossible to talk about one without the other. Though the two are not same, both these practices are closely related. 

So, before we examine the connection between the two, let’s take a closer look at it.

Big Data

Much before the IoT burst out into the scene to perform analytics big data used to be in existence. Information is regarded as big data when it presents the 4 Vs – Veracity, Volume, Variety and Velocity. Together these equate to a massive quantity of data which can be either structured or unstructured. Veracity determines the very uncertainty of data and velocity refers to the speed of data processing.

Big Data is not the new oil." - Jer Thorp, the co-founder of the Office For Creative Research, a multi-disciplinary research group exploring new modes of engagement with data.


The Internet of Things

The concept of Internet of Things takes into account a vast variety of things. So, the process involves conversion of these things into smart objects. It could be anything from cars to watches, train tracks or fridges. Products which would be connected to the internet normally will be able to acquire and process data. For seamless functioning, these are equipped with computer chips and sensors for data gathering. However, unlike the chips that are used in mobile devices, PCs or smartphones, these chips are largely used for gathering data which indicates customer product performance and usage patterns. IoT is basically the means which collects and sends data. Information from the IoT devices reside in big data and is measured against it. It is no surprise that IoT might soon touch every aspect of our lives – manufacturing, transportation, smart homes and definitely consumer goods like wearables, smartphones among others.

If you think that the internet has changed your life, think again. The IoT is about to change it all over again!” — Brendan O’Brien, Chief Architect & Co-Founder, Aria Systems.

Bringing the two practices together

Internet of Things is a disruptive technology which requires new infrastructure – software and hardware applications and the operating systems. Enterprises then will have to deal with the influx of data which starts flowing in to analyse it in real time as it expands by the minute. That’s how the big data comes in. The big data analytics tools have the potential of handling the masses of data transmitted from the IoT devices which reproduce a vast stream of information. However, to differentiate between the two – the IoT delivers the information from where the big data analytics have the potential to draw the information so as to produce the insights required of it.  The Internet of Things collect data on different scale. Thus the analytics solution must accommodate the needs of rapid processing and ingestion followed by a fast and accurate extraction. 

There are certain solutions such as SQream technologies which happen to deliver near real time analytics on even the humongous sized datasets. It also essentially condenses an absolute rack database into a small server processing up to about 100 TB so about a minimal hardware is mandatory. Further the gen next analytics database leverage GPU technology which allows further downsizing of the hardware i.e. about 5 TB on a laptop. This lets the IoT companies to correlate the increasing amount of data sets that helps it get real time responses and accommodate the changing trends by working on the size challenges without overlooking the performance.   

The challenges

Given the continual expansion of Internet of Things, it is projected that by 2020, over 20.8 billion things will be used globally. As a result of this, we may experience massive cyber security issues. Probably, cyber criminals might also break into the power grid, traffic system or any other connected network which may hold sensitive data. Security platforms such as Zscaler offer IoT devices a proper protection against security breaches with a cloud based solution. You can route the traffic via the platform and incorporate policies for the devices so that do not communicate with unnecessary servers.  

In the end, big data and internet of things do share a closely knitted future. There is absolutely no doubt that the two fields will bring forth new opportunities and solutions which might have a long lasting impact. 

Knowledge Center | General
What is IoT