The ecourse Big Data I co-developed with Laudius and another teacher in 2018 is about what Big Data is, the ethics in Big Data and the analysis of Big Data. Big Data is mostly analyzed with statistics because it is mostly quantitative data.
The book Small Data written by Martin Lindstrom tells us about what Small Data is using real life cases. Lindstrom argues in the book that because of our contemporary preoccupation with digital data we endangers high quality insights and observations. Therefore society needs Small Data.
Small Data are the rituals, habits, gestures, preferences, the small details, likes, dislikes, hesitations, speech patterns, decors, password, tweets, status updates, etc. Small Data are for example a family's shoe collection and the photo album on Facebook.
Big Data has the following attributes accoording to the book 'Succes met Big Data' used in my Big Data ecourse:
Large amounts of data
No checks on the completeness of each component
Absence of any organization
The amount of data can change at any time.
In the ecourse the students learn two definitions for Big Data. The student learn that 'Big Data is such a large amount of unstructured and sometimes incomplete set of data that processing it with conventional database systems is not possible'. And the second definition the students learn is 'Big Data are collections of data that are characterized by large quantity (volume), the high speed at which new data is created (velocity) and the large mutual variation (variety)'. The combination of these two definitions and the attributes make that Big Data is about quantity and hypotheses and not details.
It is necessary to point out the difference between Big Data and Small Data to know why both is necessary during research. Not both are used in all kinds of research. Depending on the research that is going to be performed the researchers can choose between Big Data or Small Data or for a combination of both. Big Data’s problem is the mismatch between technology and human. Humans don’t always do what is predicted, technology is programmed to always do the same. Big Data lives in databases that are defined too narrowly to create insight. Insight is provided by enriching Big Data with Small Data. Small Data is about emotions, Big Data is not.
A single piece of Small Data is not enough to create a hypothesis or a foundation for a business strategy. Big Data is a foundation for business strategy, but if companies want to understand consumers Big Data can offer valuable, but incomplete solutions. The marriage between Big Data and Small Data is essential for success. Big Data and Small Data are partners in a dance, a share quest for balance between them. Big Data helps us cut corners and automate our lives, humans will evolve simultaneously to address and pivot around the changes technology creates. Big Data might find it hard to find meaning, relevance, or insights. The insights are provided by collecting Small Data. Small Data is about the details, sometimes it is personal data like a grocery list.