Hive is a data warehouse infrastructure tool to process structured data in Hadoop. Its allows users to write queries in SQL -like language called HiveQL or HQL. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.It is a platform used to develop SQL type scripts to do MapReduce operations.
A relational database
A design for OnLine Transaction Processing (OLTP)
A language for real-time queries and row-level updates
It stores schema in a database and processed data into HDFS.
It is designed for OLAP.
It provides SQL type language for querying called HiveQL or HQL.
It is familiar, fast, scalable, and extensible.
Hive efficiently converts your queries into Map reduce task at the backend.
All the data types in Hive are classified into four types, given as follows:
Column Types
Literals
Null Values
Complex Types
Hive organizes tables into partitions. It is a way of dividing a table into related parts based on the values of partitioned columns such as date, city, and department. Using partition, it is easy to query a portion of the data.
What is Partitions
Hive Partitions is a way to organizes tables into partitions by dividing tables into different parts based on partition keys.
Partition is helpful when the table has one or more Partition keys. Partition keys are basic elements for determining how the data is stored in the table
What is Buckets
Buckets in hive is used in segregating of hive table-data into multiple files or directories. it is used for efficient querying.
The data i.e. present in that partitions can be divided further into Buckets
The division is performed based on Hash of particular columns that we selected in the table.
Buckets use some form of Hashing algorithm at back end to read each record and place it into buckets
In Hive, we have to enable buckets by using the set.hive.enforce.bucketing=true;
https://www.guru99.com/hive-partitions-buckets-example.html
When the partitions directories still exist in the HDFS, simply run this command:
MSCK REPAIR HIVE EXTERNAL TABLES
Run metastore check with repair table option
hive> Msck repair table <db_name>.<table_name>