hadoop知识点

Exam Sections and Blueprint

1. HDFS (17%)

Describe the function of HDFS daemons

描述hdfs的功能性deamon，

Describe the normal operation of an Apache Hadoop cluster, both in data storage and in data processing

描述hadoop集群在数据存储和数据处理方面正常的操作流程。

Identify current features of computing systems that motivate a system like Apache Hadoop

？？

Classify major goals of HDFS Design

分类hdfs设计的主要目标

Given a scenario, identify appropriate use case for HDFS Federation

给定场景，确认hdfs federation合适的应用。

Identify components and daemon of an HDFS HA-Quorum cluster

确认hdfs ha-quorum集群的的组件和常驻程序。

Analyze the role of HDFS security (Kerberos)

分析hdfs的安全性，hdfs。

Determine the best data serialization choice for a given scenario

判断给定场景最好的数据序列号选择

Describe file read and write paths

描述文件读写路径

Identify the commands to manipulate files in the Hadoop File System Shell

确认hdfs 操作文件的命令行命令

2. YARN and MapReduce version 2 (MRv2) (17%)

Understand how upgrading a cluster from Hadoop 1 to Hadoop 2 affects cluster settings

理解将集群从hadoop1升级到hadoop2对集群配置的影响。

Understand how to deploy MapReduce v2 (MRv2 / YARN), including all YARN daemons

理解如何部署mapreduce2（mr2/yarn），包括yarn的常驻程序。

Understand basic design strategy for MapReduce v2 (MRv2)

理解mr2基本的设计策略。

Determine how YARN handles resource allocations

掌握yarn分配资源的方法。

Identify the workflow of MapReduce job running on YARN

了解yarn上mr任务的工作流程。

Determine which files you must change and how in order to migrate a cluster from MapReduce version 1 (MRv1) to MapReduce version 2 (MRv2) running on YARN

了解如何将集群从mr1升级到mr2，知道哪些文件必须修改。

3. Hadoop Cluster Planning (16%)

Principal points to consider in choosing the hardware and operating systems to host an Apache Hadoop cluster

构架hadoop集群时硬件和操作系统的选择要点。

Analyze the choices in selecting an OS

分析操作系统的不同选择。

Understand kernel tuning and disk swapping

理解内核调节和磁盘交换区。

Given a scenario and workload pattern, identify a hardware configuration appropriate to the scenario

给定一个负载模型和场景，说明适合该场景的硬件配置。

Given a scenario, determine the ecosystem components your cluster needs to run in order to fulfill the SLA

给定场景，判断满足指定延迟所需的集群生态组件。

Cluster sizing: given a scenario and frequency of execution, identify the specifics for the workload, including CPU, memory, storage, disk I/O

给定集群规模，给定执行频率，说明工作负载，包括cpu，内存，存储，磁盘I/O

Disk Sizing and Configuration, including JBOD versus RAID, SANs, virtualization, and disk sizing requirements in a cluster

磁盘大小及配置，包括JBOD(磁盘簇),RAID(磁盘阵列),SANs(局域网存储)，虚拟化和磁盘大小需求。

Network Topologies: understand network usage in Hadoop (for both HDFS and MapReduce) and propose or identify key network design components for a given scenario

 

4. Hadoop Cluster Installation and Administration (25%)

Given a scenario, identify how the cluster will handle disk and machine failures’

给定情况，确认集群如何处理磁盘和机器失败

Analyze a logging configuration and logging configuration file format

分析log配置和log文件格式

Understand the basics of Hadoop metrics and cluster health monitoring

理解hadoopmetric基础和集群健康情况监控

Identify the function and purpose of available tools for cluster monitoring

确认集群监控工具的功能和目的

Be able to install all the ecoystme components in CDH 5, including (but not limited to): Impala, Flume, Oozie, Hue, Cloudera Manager, Sqoop, Hive, and Pig

能够安装cdh5生态的组件，包括但不限于impala，flume，oozie，hue，cloudera manager，sqoop，hive，pig

Identify the function and purpose of available tools for managing the Apache Hadoop file system

确认hadoop文件系统管理工具的功能和目的。

5. Resource Management (10%)

Understand the overall design goals of each of Hadoop schedulers

理解hadoop 各种scheduler的设计目标

Given a scenario, determine how the FIFO Scheduler allocates cluster resources

给定场景，判断fifo scheduler如何分配集群资源

Given a scenario, determine how the Fair Scheduler allocates cluster resources under YARN

给定场景，判断fair scheduler如何分配集群资源

Given a scenario, determine how the Capacity Scheduler allocates cluster resources

给定场景，判断capacity scheduler如何分配集群资源

6. Monitoring and Logging (15%)

Understand the functions and features of Hadoop’s metric collection abilities

理解hadoop metric收集的方法和特点

Analyze the NameNode and JobTracker Web UIs

分析namenode和jobtracker的web ui

Understand how to monitor cluster daemons

理解如何监控集群常驻程序。

Identify and monitor CPU usage on master nodes

识别并能监控主节点上的cpu使用情况

Describe how to monitor swap and memory allocation on all nodes

描述如何监控物理内存和交换内存的在节点上的分配情况。

Identify how to view and manage Hadoop’s log files

了解如何管理和监控hadoop日志文件。

Interpret a log file

理解日志文件。

 

 

Disclaimer: These exam preparation pages are intended to provide information about the objectives covered by each exam, related resources, and recommended reading and courses. The material contained within these pages is not intended to guarantee a passing score on any exam. Cloudera recommends that a candidate thoroughly understand the objectives for each exam and utilize the resources and training courses recommended on these pages to gain a thorough understand of the domain of knowledge related to the role the exam evaluates.