Apache Hadoop™ YARN · Moving Beyond MapReduce and Batch Processing With Apache Hadoop™ 2 (Fahad Batla's Library) by Murthy, Arun C. -- Read -- Imperial Library of Trantor

Index

About This eBook Title Page Copyright Page Contents Foreword by Raymie Stata Foreword by Paul Dix Preface

Focus of the Book Book Structure Book Conventions Additional Content and Accompanying Code

Acknowledgments About the Authors 1. Apache Hadoop YARN: A Brief History and Rationale

Introduction Apache Hadoop Phase 0: The Era of Ad Hoc Clusters Phase 1: Hadoop on Demand Phase 2: Dawn of the Shared Compute Clusters Phase 3: Emergence of YARN Conclusion

2. Apache Hadoop YARN Install Quick Start

Getting Started Steps to Configure a Single-Node YARN Cluster Run Sample MapReduce Examples Wrap-up

3. Apache Hadoop YARN Core Concepts

Beyond MapReduce Apache Hadoop MapReduce Apache Hadoop YARN YARN Components Wrap-up

4. Functional Overview of YARN Components

Architecture Overview ResourceManager YARN Scheduling Components Containers NodeManager ApplicationMaster YARN Resource Model Managing Application Dependencies Wrap-up

5. Installing Apache Hadoop YARN

The Basics System Preparation Script-based Installation of Hadoop 2 Script-based Uninstall Configuration File Processing Configuration File Settings Start-up Scripts Installing Hadoop with Apache Ambari Wrap-up

6. Apache Hadoop YARN Administration

Script-based Configuration Monitoring Cluster Health: Nagios Real-time Monitoring: Ganglia Administration with Ambari JVM Analysis Basic YARN Administration Wrap-up

7. Apache Hadoop YARN Architecture Guide

Overview ResourceManager NodeManager ApplicationMaster YARN Containers Summary for Application-writers Wrap-up

8. Capacity Scheduler in YARN

Introduction to the Capacity Scheduler Capacity Scheduler Configuration Queues Hierarchical Queues Queue Access Control Capacity Management with Queues User Limits Reservations State of the Queues Limits on Applications User Interface Wrap-up

9. MapReduce with Apache Hadoop YARN

Running Hadoop YARN MapReduce Examples MapReduce Compatibility The MapReduce ApplicationMaster Calculating the Capacity of a Node Changes to the Shuffle Service Running Existing Hadoop Version 1 Applications Running MapReduce Version 1 Existing Code Advanced Features Wrap-up

10. Apache Hadoop YARN Application Example

The YARN Client The ApplicationMaster Wrap-up

11. Using Apache Hadoop YARN Distributed-Shell

Using the YARN Distributed-Shell Internals of the Distributed-Shell Wrap-up

12. Apache Hadoop YARN Frameworks

Distributed-Shell Hadoop MapReduce Apache Tez Apache Giraph Hoya: HBase on YARN Dryad on YARN Apache Spark Apache Storm REEF: Retainable Evaluator Execution Framework Hamster: Hadoop and MPI on the Same Cluster Wrap-up

A. Supplemental Content and Code Downloads

Available Downloads

B. YARN Installation Scripts

install-hadoop2.sh uninstall-hadoop2.sh hadoop-xml-conf.sh

C. YARN Administration Scripts

configure-hadoop2.sh

D. Nagios Modules

check_resource_manager.sh check_data_node.sh check_resource_manager_old_space_pct.sh

E. Resources and Additional Information F. HDFS Quick Reference

Quick Command Reference

Index