Big Data - Hadoop

Introduction to Hadoop

What is Big Data
Need and significance of innovative technologies
what is Hadoop
3 Vs (Characteristics)
History of Hadoop and its Uses
Different Components of Hadoop
Various Hadoop Distributions

HDFS ( Hadoop Distributed File System)

Significance of HDFS in Hadoop
HDFS Features
    a) NameNode
    b) DataNode
    c) JobTracker
    d) TaskTrack
    e) Secondary NameNode


HDFS ( Hadoop Distributed File System)

Data Storage in HDFS
    a) Blocks
    b) Heartbeats
    c) Data Replication
    d)HDFS Federation
    e) High Availability


HDFS ( Hadoop Distributed File System)

Accessing HDFS
    a) Blocks CLI (Command Line Interfac e) Unix and Hadoop Commands
    b) Java Based Approach
Data Flow
    a) Anatomy of a File Read
    b) Anatomy of a File Write
Hadoop Archives

MapReduce

Introduction to MapReduce
MapReduce Architecture
MapReduce Programming Model
MapReduce Algorithm and Phases
Data Types
Input Splits and Records
Blocks Vs Splits
Basic MapReduce Program
    a) Blocks
    b) Driver Code
    c) Mapper Code
    d) Reducer Code
    e) Combiner and Shuffler
Creating Input and Output formats in MapReduce Jobs
    a) File Input / Output Format
    b) Text Input / Output Format
    c) Sequence File Input / Output Format,etc.
How to Debug MapReduce Jobs in Local and Pseudo cluster mode
The MapReduce Web UI
Data Localization in MapReduce
Distributed Cache
Compression Mechanisms
Joins
    a) Map-Side Joins
    b) Compression Mechanisms

Pig

Introduction to Apache Pig
MapReduce Vs. Apache Pig
SQL Vs. Apache Pig
Different Data types in Apache Pig
Modes of Execution in Apache Pig
    a) Local Mode
    b) Map Reduce or Distributed Mode
Execution Mechanism
    a) Grunt shell
    b) Scripta
    c) Embedded
Data Processing Operators
    a) Loading and Storing Data
    b) Filtering Data
    c) Grouping and Joining Data
    d) Sorting Data
    e) Combining and Splitting Data
How to write a simple PIG Script
UDFs in PIG

Sqoop

Introduction to Sqoop
Sqoop Architecture and Internals
MySQL client and server installation
How to connect relational database using Sqoop
Sqoop Commands
    a) Different flavors of imports
    b) Export
    c) HIVE imports

Hive

The Metastore
Comparison with Traditional Databases
    a) Schema on Read Versus Schema on Write
    b) Updates, Transactions, and Indexes
HiveQL
    a) Data Types
    b) Operators and Functions
Tables
    a) Managed Tables and External Tables
    b) Partitions and Buckets
    c) Storage Formats
    d) Importing Data
    e) Altering Tables
    f) Dropping Tables
Querying Data
    a) Sorting and Aggregating
    b) MapReduce Scripts
    c) Joins
    d) Subqueries
    e) Views
User-Defined Functions
    a) Writing a UDF
    b) Writing a UDAF

HBase

Introduction to Hbase
HBaseVs HDFS
Use Cases
Basics Concepts
    a) Column families
    b) Scans
Hbase Architecture
Zoo Keeper
Clients
    a) REST
    b) Thrift
    c) Java Based
    d) Avro
MapReduce integration
MapReduce over Hbase
Schema definition
Basic CRUD Operations