《大数据系统运维》课件 第4章 性能管理.pdf

大数据系统运维大数据应用人才培养系列教材第四章性能管理4.1 性能分析4.2 性能监控工具4.3 性能优化习题4.1性能分析第四章性能管理性能因子影响Hadoop大数据作业性能的因子有以下几点：Hadoop配置：酉己置对Hadoop集群的性能是非常重要的；不合理的配置会产生CPU 负载、内存交换、I。等的额外开销问题。文件大小：特别大和特别小的文件都会影响Map任务的性能。Mapper,Reducer的数量:会影响M叩、Reduce的任务和Job的性能。硬件:节点的性能、配置规划及网络硬件的性能会直接影响到作业的性能。代码：质量差的代码会影响M叩/Reduce性能。4.1性能分析第四章性能管理性能指标包寸舌：Elapsed time、Total Allocated Containers.Number of maps,Launched map tasks.Number of reduces,Launched reduce tasks、Job state、Total time spent by all map tasks(ms)、Total time spent by all reduce tasks(ms)、Total vcore-seconds taken by all map tasks.Total vcore-seconds taken by all reduce tasks、Map input records、Map output records.Map output bytes、Map output materialized bytess Input split bytes、Combine input records、Combine output records.Reduce input groups.Reduce shuffle bytes、Reduce input records、Reduce output records.Spilled Records.Shuffled Maps、Failed Shuffles、Merged Map outputs.GC time elapsed(ms)、CPU time spent(ms)、Physical memory(bytes)snapshot.Virtual memory(bytes)snapshot.Total committed heap usage(bytes)等。大数据应用人才培养系列教材第四章性能管理4.1 性能分析4.2 性能监控工具4.3 性能优化 习题4.2性能监控工具第四章性能管理通过浏览器查看Hadoop NameNode开放的50070端口，可以了解至Uhadoop 集群的基本配置信息和监控到hadoop集群的状态。|10.1.89.17:50070/dfshealth.htmltab-overview|Hado o pOverviewDatanodes Datanode Volume Failures Snapshot Startup Progress UtilitiesOverView master:8020(active)Started：Version：Compiled：Cluster ID：Block Pool ID：Tue Apr 18 11:29:10 CST 20172.7.1,rl5ecc87ccf4a0228f35af08fc56de536e6ce657a2015-06-29T06：04Z by jenkins from(detached from 15ecc87)CID-fl814741-d9a2-4d94-8aal-adcff8cdfcc6BP-1688754380-10.1.89.17-1492486135710集群基本信息（1）4.2性能监控工具第四章性能管理 10.1.89,17:50070/dfshealth.html#tab-overviewSummarySecurity is off.Safemode is off.1 files and directories,0 blocks=1 total filesystem object(s).Heap Memory used 271.32 MB of 958.5 MB Heap Memory.Max Heap Memory is 958.5 MB.Non Heap Memory used 38.17 MB of 38.94 MB Commited Non Heap Memory.Max Non Heap Memory is 130 MB.Configured Cacity：29.95 GBDFS Used：24 K B(0%)Hon DFS Used：5.73 GBDFS Remaining：24.22 GB(80.86%)Block Pool Used:24 K B(0%)Datallodes usages%(Min/Median/Wax/stdDev)：0.00%/0.00%/0.00%/0.00%Live Nodes3(Decommissioned:0)Dead Nodes0(Decommissioned：0)DecoBRi ssioning Nodes0Total Datanode Vo lune Failures0(0 B)Number of Undei-Replicated Blocks0集群基本信息（2）4.2性能监控工具第四章性能管理 10.1.89,17:50070/dfshealth.html#tab-overviewTotal Datanode Vo lune FailuresHumber of IMdei-Replicated BlocksIhmbex of Blocks Pending Deletion Block Deletion Start Time0(0 B)002017/4/18 上午 11：29：10NameNode Journal StatusCurrent transaction ID：3Journal Manager StateFileJ ouxnalManag er(root=/usr/cstor/hado op/cl oud/df s/name)EditLogFil eOutput Stream(/usr/cstor/hado op/cl oud/df s/name/cur r ent/edit s_inpr ogres s_0000000000000000003)NameNode StorageStorage Directory/usr/cstor/hadoop/cloud/dfs/nameTypeIMAGE_AND_EDITSStateActive集群基本信息（3）4.2性能监控工具第四章性能管理 10.1.89,17:50070/dfshealth.html#tab-datanodeDatanode InformationIn operationNodeLast cont actAdnin StateCapacityUsedHon DFS UsedRemainingBlocksBlock pool usedFailed VolmesVers i onslave2:50010(10.1.89.18:50010)0In Service9.98 GB8 K B1.91 GB8.07 GB08 K B(0%)02.7.1slavel：50010(10.1.71.16:50010)0In Service9.98 GB8 K B1.91 GB8.07 GB08 K B(0%)02.7.1slave3：50010(10.1.71,17:50010)0In Service9.98 GB8 K B1.91 GB8.07 GB08 K B(0%)02.7.1DecomissioningNodeLast cont actUnder replicated blocksBlocks wi th no live repli casUnder Replicated Blocks In files under construction集群基本信息（4）4.2性能监控工具第四章性能管理(1)运行了哪些作业，每个作业的类型、执行时间、起始时间、结束时间、执行时间、当前状态、最终状态等。1030.100.4:8088/cluster函All ApplicationsLogged in as:dr.whoCluster MetricsApps SubmittedApps PendingApps RunningApps CompletedContainers Runningleaioiy UsedMemory TotalMemory ReservedVCor es UsedVCores TotalVCores ReservedActive NodesDecommissioned NodesLost NodesUnhealthy NodesRebooted Nodes200200 B24 GB0 B0240nQ0QScheduler Metrics To o lsShowing 1 to 2 of 2 entriesScheduler TypeScheduling Resource TypeMmiaun AllocationMaximum AllocationOpacity Scheduler MEMORYmemory：8192,vCores：8_Show 20 entriesSearch:ID，%u ApplicationName 十 人Type a a StartTiae FinishTiwe StateQueue v,FinalStatus 人 Trackingc Hess v UI CaDolication 1495010639895 0002 rootQuasiMonteC axlo MAPREDUCEdefault Sat May 20 Sat May 20 FAILEDFAILED Historv16:28:33 16:28:36+0800 2017+0800 2017aDDlication 1495010639895 0001 rootDistributedShell YARNdefault Sat May 20 Sat May 20 FAILEDFAILED|History15:58:06 15:58:11+0800 2017+0800 20174.2性能监控工具第四章性能管理(2)作业运行在集群的哪些计算节点上。Kill ApplicationAppli cati on OvervUser：rootffame：Quasi Mont eCarloApplication Type：MAPREDUCEAppli cation Tags：Y amAppli c ati onStat e：FAILEDFinalS tatus Reported by FAILED AM：Started:Sat May 20 16:28:33+0800 2017Elqsed：2secTracking URL：HistoryDiagnostics：Application application_l495010639895_0002 failed 2 times due to AM Container for appattempt_l495010639895_0002_000002 exited with exitCode：_1000 For more detailed output,check application tracking page：http：/master：8088/cluster/app/application_l495010639895_0002Then click on links to logs of each attempt.Application MetricsTotal Resource Preempted:Total Number of Non-AM Containers Preempted：0Total Number of AM Containers Preeiqpted:0Resource Preempted from Current Attempt：Number of Non-AM Containers Preempted from Current Attempt:0Aggregate Resource Allocation：4130 MB-seconds,2 vcore-secondsShow 20 entries Search：Attempt ID Started 0 Node 0 Logs 0aDDattemDt 1495010639895 0002 000002 Sat Mav 20 16:2835-0800 httD：/slave3:8042 Loes2017 aDPatteniDt 1495010639895 0002 000001 Sat May 20 16:28:33-0800 httD:slave 1:8042 Loss2017Showing 1 to 2 of 2 entries4.2性能监控工具第四章性能管理(3)HDFS文件信息，包括Size、Replication.Block SizeoHado op Overview Dat anodes Snapshot Starts Progress UtilitiesBrowse Directory/use r/r oot/QuasiMont eCar lo_1496131133155_811889410/inGo!PermissionOwier GroupSizeLast libdifiedReplicationBlodc SizeHane-rw-r-r-rootsupergroup118 B2017/5/30 下午3:58:553128 MBpartO-rw-r-r-rootsupergroup118 B2017/5/30 下午3:58:553128 MBpartiHadoop 2015.4.2性能监控工具第四章性能管理集群CLI通过yarn、m叩red等CLI工具，也可监控作业的运行。操作系统自带工具 通过操作系统自带的工具，如vmstat,可以监控到节点的物理运行性能，可 以监控每个节点的资源占用信息。4.2性能监控工具第四章性能管理Ganglia工作原理图如下：Ganglia Data FlowThe Ganglia Monitoring System(simplified)4.2性能监控工具第四章性能管理Ganglia监控画面cpu_aidle-CPU a（dle出苗 Hide/Show EventsTimeshiftcoj c e-CPU Idletyjl（三丁Q3E3 Hide/Show Events TimeshiftCPU Idle08:50 69:00 09:10 69:26 09:30 09:40 s z q yj z bigdat a-01 last hour Now:96.71 Min:95.56Avg:96.62 Max:97.40CPU Nicecpu.steal-CPU steal Hide/Show Events Timeshiftcpu n ce-CPU N cc程立（2Z3 Hide/Show Events TimeshiftCPU steal08:5009:0009:1008:5609:0009:10画 szq-yjz-bigdata-01 last hour szq-yjz-bigdata-01 last hour0.0009:20Now:0920Nov/：09:30 09:400.00 Min：09:30 09:406.60 Min:e.eeAvg:o.ooMax:0.00Avg:0.00Max:0.004.2性能监控工具第四章性能管理其他监控工具其它常用监控工具还有Dr.Elephant、nagioss eBay Eagle等,如下图显示的 Dr.Elephant监控画面。大数据应用人才培养系列教材第四章性能管理4性能分析4.2 性能监控工具4.3 性能优化习题4.3性能优化第四章性能管理Hadoop集群配置规划优化0 Hadoop硬件配置规划优化3操作系统规划优化Hadoop集群配置规划优化4.3性能优化第四章性能管理Hadoop性能优化内存优化配置多个M叩Re d u c e工作目录 压缩M叩Reduce中间结果 调整虚拟CPU个数 其他优化常用技巧4.3性能优化第四章性能管理作业优化在经过以上Hadoop性能优化后，如果对作业运行还有加快的需求，则采用以下优化方 法可以进一步提升作业运行性能。大数据应用人才培养系列教材第四章性能管理4性能分析4.2 性能监控工具4.3 性能优化习题习题:L请列出3个以上主要性能因子。2,请列出5个以上主要性能指标并说明其代表的含义。3,请列出3个以上主要性能监测工具并说明它的运用方法。4.Hadoop集群配置规划优化可以采取哪些措施？5.请说明H a d。p集群优化的5个技巧。6 如何调整Map任务数目？请比较调整Map任务数的运行效果。7 如何修改调度策略？感谢聆听