chapte 04 : type of tables in hive

Types of table in hive

1. Managed Table

   these are defualt table of hive which are managed by warehouse system.
   Managed tabular data will get stored in ware house.
   
   HDFS PATH : /user/hive/warehouse/<<tablename>>/tabledata
   How to create managed table 
   
   Syntax : Create table <<tablename>> <schema def>>
   
   row format delimited
   fields terminated by '|'
   lines terminated by '\n'
   stored as filename; 
   
   
   There are two way to load the data 
   
   
   1. local path (load data local inpath  'local path' into table tablename )
   2. HDFS path (load data  inpath 'HDFS PATH' into table tablename)
   
   
   Once the data loaded from any way it will be a part of HDFS only
   
   
   /user/hive/warehouse
   
   
   

   
   

Chapter 03 : Hive Achitecture

Hive Architecture

Chapter 02 :Metastore and how to configure it

METASTORE

Internal database of hive which is primarially responsible for keeping the metadata information of the table like table name, table schema, table column information, partitioned key information if any.


Note
By default hive will come up with the DERBY database , however we can also configure mysql like THRIFT SERVER extra as hive metastore.


HOW TO CONFIGURE METASTORE IN HIVE

There are two section :-

1. Connection URL
   
   Hive-site.xml
   <property>
   <name>javax.dbo.option.connectionURL</name>
   <value>jdbc.derby; databasename =/var/lib/hive/metastore/metastore_db; create =true;</value>
   <description>Connectionstring for JDBC metastore</description>
   </property>
   
   
   
2. Driver Class Details
   
   Hive-site.xml
   <property>
   <name>javax.dbo.option.ConnectionDriverName</name>
   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
   <description>Driver Class name for JDBC metastore</description>
   </property>
   
   
   
   
   metastore  is generallly one of the configuaration file.
   
   

Apache Hive Introduction

Note : Even though we are querying in hive each and every job is internally converted into MR job only.

Hive is not come with default installation of hadoop we need to install it exclusively on hadoop boxes.

Script Mode : Join Script in Apache Pig

A = LOAD 'data1.txt' using PigStorage('\t') as (cid:int,name:chararray,place:chararray);

B = LOAD 'data2.txt' using PigStorage('\t') as (cid:int,name:chararray,place:chararray);

joindata = JOIN A By cid,B By cid;

rjoin = JOIN A By cid RIGHT,B By cid;

ljoin = JOIN A By cid LEFT,B By cid;

Fjoin = JOIN A By cid FULL,B BY cid;

DUMP rjoin;DUMP ljoin;DUMP Fjoin;