Big Data is a hot topic today. Although the concept isn’t new, it has become increasingly important. In the information age, data is everywhere, and society continues to store it for future use. Companies like Google, Microsoft, and others are building data centers around the world, including in The Netherlands, to store this growing volume of data.
This raises several important questions:
Is all data actually information?
Is all data valuable?
Is it really necessary to store everything?
There is no simple answer to these questions, because data only becomes meaningful when a data analyst examines it to find insights. At that moment, it turns into information that matters. A skilled data analyst must be able to explore Big Data effectively, identify valuable information, and separate it from junk data.
Big Data refers to all collected data that does not have meaning yet. What sets Big Data apart from other types of data collection are its key characteristics:
Volume: Everything is recorded, kept, and stored.
Velocity: In the information age, data is generated rapidly and stored immediately, often without time to decide what is worth keeping.
Variety: All types of data are stored. For example, the exact time a survey is completed, which often has little relevance when analyzing the survey results.
There are currently three main types of Big Data sources. As society moves further into the information age, more sources may emerge. The three existing sources are:
Data collected by people: This includes all digital information people gather themselves. For example, I have a collection of tweets stored online and on my computer. Not everything I save will be useful to me in the future, which makes it part of Big Data.
Process-generated data: As mentioned earlier, when I complete a survey, the exact time I submit it is recorded. This timestamp is usually irrelevant for the research the survey supports. However, if something goes wrong, it can help trace or verify the survey process.
Automatically collected machine data: For example, my household electricity usage is recorded automatically. If I use Toon (Eneco), it tracks which appliance is used and where in the home. This happens without any manual input, simply by using Toon. Such data becomes valuable when I want to know which appliance consumes the most electricity.