WHAT IS BIG DATA?
Embedded Files
Why we collect Data
The impact of users and organisations when using data for decision making
Roles & Responsibilities & Challenges faced by Data Specialist
Strategies used by Data Specialists to ensure Data Compliance
Data-Driven culture, responsibilities and challenges
The history of Big Data
The history of Big Data
The idea of collecting data is started a long time a go, in the 1600s , John Graunt used early statistical analysis to study the bubonic plaque in London. Since then, humans have collected data to understand the world better.
However, in the modern world, the concept of Big Data began in 1958 when IBM Researcher H. P. Luhn introduced the term Business Intelligence. He described it as the ability to understand connections between facts to support better decisions. This idea later become the foundation for the Big Data Tools and systems we use today.
In the 1960s and 70s, computers was used more in companies and governments, allowed to store and process data faster than ever before. Soon after, databases were created to organise information better.
in the 1980s, personal computers and client- server system started to appear, making data more accessible to businesses. Around this time, IBM Developed SQL(Structured Query Language), a new way to interact with databases using simple commands. SQL became popular because it helped users search, organise, and manage data more easily, without needing complex programming skills.
By the 1990s, the rise of the internet and e-commerce brought an explosion of data. Websites, emails, online shopping, and search engines has generate more information than ever before. Traditional tools like relational databases, data warehouses or those using SQL couldn't keep up with the speed, variety, and volume of data. Companies started to realise they needed new ways to store and analyse data at a much bigger scale. As the need grew, many new tools and technologies were developed, such as Hadoop, MapReduce, Apache Spark, and NoSQL databases to handle larger and more complex datasets. This marked the beginning of a new era in data processing and analytics.
In the 2000s, the digital world started to change fast, and people began using internet more than ever, uploading videos, using smartphones, chatting on social media, and shopping online. This new approach has generated content and created a massive amount of data every second.
To manage all this , Hadoop allowed data to be sored and processed across many computers.
MapReduce, created by Google helped to make big tasks into smaller ones that could run in parallel. Later, Apache Spark made data processing even faster by using memory instead of just storage. NoSQL databases like MongoDB or Cassandra were introduced to handle unstructured data such as images, videos, and texts.
Thanks to these tools, Big Data become more that just a storage, it became a way to discover patterns, predict outcomes, and make smart decisions.
Today, Big Data is used in many industries, from helping Doctors to detect diseases earlier, to help companies improve products or a service.
As technology continues to grow, so does the power of Big Data. The future will likely bring even more advanced ways to use data, including AI, automation and real time insights to improve our lives.(Phillips, 2021)
Big Data
Big Data refers to the huge volume of digital information created every second, from phones, websites, apps, social media, and business systems. Because this Data grows rapidly and comes in many formats, organisations need powerful tools and technologies to collect, store, and analyse it effectively.(Simplilearn, 2019) (Mucci and Stryker, 2024)
LLM (Large Language Model)
LLM (Large Language Model)
A LLM is type of artificial intelligence trained on large volumes of text , to understand and generate language that sounds like it was written by human.
A LLM is type of artificial intelligence trained on large volumes of text , to understand and generate language that sounds like it was written by human.
The video by (3Blue1Brown, 2024) explains that an LLM doesn't truly understand words like a person does, but it learns to recognise patters and probabilities. it basically guesses the next word in a sentence based on everything it has seen before.
The video by (3Blue1Brown, 2024) explains that an LLM doesn't truly understand words like a person does, but it learns to recognise patters and probabilities. it basically guesses the next word in a sentence based on everything it has seen before.
These models are used in many real-life tools like chatbots, virtual assistants, and translation apps. In online shops, for example and LLM can automatically reply to customer questions or create product descriptions based on previous data and conversations.(IBM, 2023)
These models are used in many real-life tools like chatbots, virtual assistants, and translation apps. In online shops, for example and LLM can automatically reply to customer questions or create product descriptions based on previous data and conversations.(IBM, 2023)
Internet of Things
The internet of things (IoT) is the idea of connecting everyday object like lights, fridges, cars or security systems to the internet so they can collect and share Data. These devices use sensors to work automatically and help people save time or make better decisions.
For example , a smartwatch can track our heart rate and send updates to our phones. A smart home system can turn off the lights, and adjust the heating, or lock the doors automatically. These things happen without us even being there.
The video by (Simplilearn, 2020) explains that IoT is growing rapidly and helps us improve our daily lives. But IoT it also important in industries. In retail, for example, sensors can track stock levels in real time, while in farming, IoT can monitor soil moisture or weather conditions to help crops grow better (IBM, 2024)
Page updated
Report abuse