These are the ways you can use big data in your business

In the age of digital technology and social media, the amount of information generated is increasing exponentially. Currently, the world produces 2.5 quintillion bytes of data daily (there are 18 zeros in a quintillion). The explosion of data continues in the roaring '20s, both in terms of generation and storage — the amount of stored data is expected to continue to double at least every four years. 

If a company has a website, a smartphone app, and receives customer requests and feedback via email or messengers, it already has data that can be used for analysis. But how will it benefit the business?

Large companies began asking themselves this question seven years ago, but at the time, few saw the benefit of big data analytics. In 2015, only 17% of companies around the world used big data in their work. IT, banking, and telecommunications companies were the pioneers in implementing big data. This is not surprising. These sectors accumulate the largest amount of data: banks use transactions, telecoms use geodata, and search engines use query histories.

All large companies now use big data analytics. In the U.S., more than 55% of companies from various spheres work with this technology. In Europe and Asia, the demand for big data is slightly lower – about 53%. It turns out that in the last five years, businesses have been using big data three times more.

The growing interest in big data on the part of businesses is simple to explain. Companies that ignore big data technologies have begun to notice their own lost profits. For example, Caterpillar, one of the world's leading special equipment corporations, admitted this. In 2014, its distributors were missing out on $9 to 18 billion in profits each year just because they did not adopt big data technologies. Caterpillar has more than 3.5 million pieces of hardware equipped with sensors that collect data on the condition of key components, their operating modes, and wear rates. This helps owners optimize the use of the equipment and manage maintenance costs.

Union Pacific Railroad, the largest U.S. railroad company, improved its risk management system with the help of "big data" – the number of derailments was reduced by 75%. To do this, the company began collecting data from thermometers, acoustic and visual sensors installed on the bottom of each locomotive, information about weather conditions, brake system conditions, and GPS coordinates of trains. Based on this information, predictive models are built to monitor the condition of the wheels and the railroad and predict train derailment several days or even weeks before a possible incident. This is enough time to promptly eliminate problems, avoid damage to the train and delays to other trains.

While railroads are struggling with the risks of train derailments, one of the world's largest and busiest airports in Dubai is using big data to improve its logistics. If there are many passengers transferring from one flight to another on two flights, big data will help assign their exits next to each other.

What are the characteristics of Big Data?

Let's name the main characteristics of Big Data: 

Volume – from 150 GB per day;

Velocity – The speed of accumulation and processing of data sets. Big data is updated regularly, so intelligent technologies are needed to process it online;

Variety of data types. Data can be structured, unstructured, or partially structured. 

For example, in social networks, the data flow is unstructured: it can be text posts, photos, or videos.

Today three more features are added to these three:

Veracity – the validity of both the data set itself and the results of its analysis;

Variability –  data streams have their peaks and troughs under the influence of seasons or social phenomena. The more unstable and variable the data stream is, the harder it is to analyze it;

Value – value or significance. Like any information, big data can be simple or complex to understand and analyze. An example of simple data is posted on social networks; complex data is bank transactions.

How is Big Data collected and stored?

Big data is needed to analyze all relevant factors and make the right decision. Big Data is used to build simulation models for a decision, an idea, a product.

The main sources of Big Data:

• Internet of Things (IoT) and connected devices;
• social networks, blogs, and media;
• company data: transactions, orders for goods and services, cab and carsharing trips, customer profiles;
• instrument readings: weather stations, air, and water gauges, data from satellites;
• city and state statistics: data on movements, births, and deaths;
• medical data: tests, diseases, diagnostic scans.

Modern computing systems provide instant access to massive amounts of big data. Special data centers with the most powerful servers are used for their storage.

Big Data and Blockchain.

I am inspired by the fact that I am involved in cutting-edge technology.  The most amazing thing about this job for me is the opportunity to have a very strong influence on global processes. It's kind of like detective work. You determine what happened, where, and why. Owning and properly using big data allows companies to understand why they are losing money and clients, and how to avoid this in the future and increase profits.

Using blockchain adds another data layer to the Big Data analytics process. Blockchain is a direct collaboration without intermediaries, not only between you and your customers but also between companies in the supply chain, for example. The possibilities of blockchain are limitless. So, you create a distributed database that is: fully secure; easy to verify; easy to track.

For example, there are 10-20 organizations and millions of clients placed in one system. All of the data that is within this system is secure, impossible to fake, and always traceable. This means that all members of this chain have a single source of truth, a kind of one big repository of truth. This phenomenon will fundamentally change the way we do business. 

In essence, it creates what we call "transparency".  Transparency for everyone involved in the process, including customers. And it makes a huge difference in business.  Through this, you create trust, and for organizations, trust is the main asset, the key resource. 

Dmitry Tsyplakov 

Serial Entrepreneur/Digital Product Manager