2Exadatasetupandconfiguration.pptx

Click to edit Master text styles,Second level,Third level,Fourth level,Fifth level,Click to edit Master title style,合作伙伴技术培训，,2012,年,2,月,#,Exadata set up and configuration,Agenda,What customers need to do,What needed before Exadata set up,How to do Exadata installation,Post Exadata reimage,OneCommand step-by-step introduction,typical issues in Exadata set up and configuration,monitoring tools deployment,What customers need to do,机房环境,Exadata,规格,高,1.998,米,宽,0.6,米,厚,1.2,米,注意：,由于,Exadata,对于倾斜角度比较敏感，通往机房的门高必须大于,2,米,What customers need to do,Exadata,型号,Full Rack,Rack,Rack,1/8 Rack,X4-2,852.8 kg,525.3 kg,365.6 kg,365.6 kg,X3-2,871.4 kg,543.9 kg,368.8 kg,368.8 kg,X2-2,966.6 kg,602.8 kg,409.1kg,机房环境,机房地板承重,What customers need to do,机房环境,制冷、湿度和海拔,温度：摄氏,532,湿度：,10%90%,海拔：,3048,米,制冷空气流向,从前向后,What customers need to do,机房环境,电源,低压（,220V),单相电：,低压（,220V),三相电：,高压（,380V),单相电：,高压（,380V),三相电：,注意：需要两路电,What customers need to do,机房环境,需要准备的配套设备,生产网接入交换机,(,以满配为例，最少需要八个端口）,管理网接入交换机（最少需要,1,个端口）,DNS,服务器（理论上和,NTP,服务器共用一台机器）,NTP,服务器（理论上和,DNS,服务器共用一台机器）,What customers need to do,Exadata,型号,Full Rack,Rack,Rack,1/8 Rack,X4-2,68,IP,地址,M,：,49,C,：,19,38,IP,地址,M,：,27,C,：,11,22 IP,地址,M,：,15,C,：,7,22 IP,地址,M,：,15,C,：,7,X3-2,69 IP,地址,M,：,50,C,：,19,39 IP,地址,M,：,28,C,：,11,22 IP,地址,M,：,15,C,：,7,22 IP,地址,M,：,15,C,：,7,X2-2,70,IP,地址,M,：,51,C,：,19,40,IP,地址,M,：,29,C,：,11,23 IP,地址,M,：,16,C,：,7,-,机房环境,IP,地址预留,机房环境,防火墙需打开的端口：,20/21,，,FTP over TCP,，数据库,/,存储等管理,22,，,SSH over TCP,，数据库,/,存储,/IB/KVM,等管理,23,，,Telnet over TCP,，,KVM Telnet,操作,25,，,SMTP,，数据库,/,存储,/IB/KVM/PDU,等管理,53,，,DNS,，数据库,/,存储,/IB/KVM/PDU,等管理,69,，,TFTP over TCP,，数据库,/,存储,/IB/KVM,等管理,80,，,HTTP over TCP,，数据库,/,存储,/IB/KVM/PDU,等,Web,管理,123,，,NTP over TCP,，数据库,/,存储,/IB/KVM/PDU,等管理,161,，,SNMP over UDP,，数据库,/,存储,/IB/KVM/PDU,等管理,162,，,SNMP over UDP,，,IPMI over UDP,，数据库,/,存储,/IB/KVM/PDU,389,，,LDAP over TCP/UDP,，用户管理,What customers need to do,机房环境,防火墙需打开的端口：,443,，,HTTPS over TCP,，数据库,/,存储,/IB/KVM/PDU,等,Web,管理,465,，,SMTP,，数据库,/,存储,/IB/KVM/PDU,等管理,（,SSL,）,514,，,Syslog over UDP,，数据库,/,存储,/IB/KVM/PDU,等管理,623,，,IPMI over UDP,，数据库,/,存储,/IB/KVM/PDU,等管理,1158,，,TCP,，,Oracle EM Grid Control,1159,，,TCP,，,Oracle EM Grid Control,1521,，,TCP,，,Database listener,1812,，,RADIUS over UDP,，数据库,/,存储,/IB/KVM/PDU,管理,4889,，,TCP,，,Oracle EM Grid Control,5120,，,TCP,，,ILOM,（,CD,）,5121,，,TCP,，,ILOM,（,Keyboard&Mouse,）,5123,，,TCP,，,ILOM,（,diskette,）,What customers need to do,机房环境,防火墙需打开的端口：,5555,，,TCP,，,ILOM,（加密）,5556,，,TCP,，,ILOM,（认证）,6481,，,TCP,，,ILOM,（,Servicetag,）,7578,，,TCP,，,ILOM,（,Video,）,7579,，,TCP,，,ILOM,（,Serial,）,7777,，,TCP,，,Oracle EM Grid Control,数据库和存储管理,7799,，,TCP,，,Oracle EM Grid Control,数据库和存储管理,What customers need to do,Agenda,What customers need to do,What needed before Exadata set up,How to do Exadata installation,Post Exadata reimage,OneCommand step-by-step introduction,typical issues experienced in Exadata set up and configuration,monitoring tools deployment,What needed before Exadata set up,安装前确认,按照上节所述，逐一确认安装前机房准备情况,Exadata SN,（机柜后左上角）,检查,Exadata,各,PDU,电源开关,明确网络拓扑结构,使用,config,程序生成参数文件,What needed before Exadata set up,IP,地址要求（,X4-2,四分之一配为例）,管理（,15,个，,NET0/ILOM,）,2,个,Database Server,（,10.10.1.1-2,）,3,个,Storage Cell,（,10.10.1.3-5,）,2,个,Database Server ILOM,（,10.10.1.6-7,）,3,个,Storage Cell ILOM,（,10.10.1.8-10,1x CISCO,（,10.10.1.12,）,2x InfiniBand,（,10.10.1.14-15,）,2,个,PDU,（,10.10.1.16-17,）,客户访问（,7,个，,BONDETH0,）,2,个,Database Server public,（,192.168.1.150-151,）,2,个,Database Server VIP,（,192.168.1.152-153,）,3,个,Database Server SCAN,（,192.168.1.154-156,）,InfiniBand,私网（,10,个，,BONDIB0,）,5,个,Database Server/Storage Cell,（,192.168.10.1-5,）,What needed before Exadata set up,IP,地址要求（,X3-2,半配为例）,管理（,28,个，,NET0/ILOM,）,4,个,Database Server,（,10.200.200.32-35,）,7,个,Storage Cell,（,10.200.200.36-42,）,4,个,Database Server ILOM,（,10.200.200.43-46,）,7,个,Storage Cell ILOM,（,10.200.200.47-53,）,1x CISCO,（,10.200.200.55,）,3x InfiniBand,（,10.200.200.56-58,）,2,个,PDU,（,10.200.200.59-60,）,客户访问（,11,个，,BONDETH0,）,4,个,Database Server public,（,172.17.8.240-243,）,4,个,Database Server VIP,（,172.17.8.244-247,）,3,个,Database Server SCAN,（,172.17.8.248-250,）,InfiniBand,私网（,11,个，,BONDIB0,）,11,个,Database Server/Storage Cell,（,192.168.10.1-11,）,What needed before Exadata set up,系统加电检测,PDU B,（右面）先开,Sun Fire X4270M2 PS1,（上面）亮,Sun Fire X4170M2 PS1,（右面）亮,CISCO PSU,一绿一红,InfiniBand,交换机,PS0,亮,PDU A,（左面）后开,各系统,OK,灯闪（,Standby,状态，,1/3,，,ILOM up/host off,）,否则需连串口检查（,115200,，,8N1,）,What needed before Exadata set up,登录,MOS,，由,Exadata,主,notes,（,1187674.1.,）查看更新状态,What needed before Exadata set up,链接到,Exadata,支持文档（,888828.1,），查看最新,image,、,database,，,bundle patch,和,Onecommand,版本,及对应,Exadata,补丁,What needed before Exadata set up,查看下载所需软件介质,What needed before Exadata set up,登录,edelivery,，查看,Exadata image,最新软件包，并下载,What needed before Exadata set up,Agenda,What customers need to do,What needed before Exadata set up,How to do Exadata installation,Post Exadata reimage,OneCommand step-by-step introduction,typical issues in Exadata set up and configuration,monitoring tools deployment,将,onecommand,的,patch,包上载到,DB,节点,1,上，解压后运行,config.sh,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,按照提示，一步步填写相关信息，并生成,reimage,所需相关文件,How to do Exadata installation,获取各,DB/Cell NET0 MAC,地址，用于修改,XXX-dm01-preconf_rack_0.csv,文件,rootsfexdb01 network-scripts#cd/etc/sysconfig/network-scripts,rootsfexdb01 network-scripts#vi ifcfg-eth0,DEVICE=eth0,BOOTPROTO=static,ONBOOT=yes,IPADDR=10.200.100.32,NETMASK=255.255.255.0,NETWORK=10.200.100.0,BROADCAST=10.200.100.255,GATEWAY=,HWADDR=00:21:28:3e:20:30,How to do Exadata installation,获取各,DB/Cell NET0 MAC,地址（举例）,Hostname,MAC(eth0,）,00:21:28:3E:20:30,00:21:28:3D:D7:B8,00:21:28:3E:21:2C,00:21:28:3E:20:F0,00:21:28:E7:8F:D8,00:21:28:E7:B2:88,00:21:28:E7:D2:62,00:21:28:E7:8E:5E,00:21:28:E7:8C:8A,00:21:28:E7:99:3E,00:21:28:E7:B4:C8,How to do Exadata installation,获取各,DB/Cell NET0 MAC,地址，修改,XXX-dm01-preconf_rack_0.csv,文件并检查,How to do Exadata installation,将下载的,Image,包传到,DB,节点,1,上（如,/images,目录，,exadata1,）,Cell Image,会解压到,dl180,（,X4270M2,）目录,DB Image,会解压到,dl360,（,X4170M2,）目录（约,2GB,）,rootdm01db01 image#unzip/root/image/ImageMaker/V29925-01-cell-11.2.2.4.2-linux.zip,rootdm01db01 image#unzip/root/image/ImageMaker/V29924-01-db-11.2.2.4.2-linux.zip,rootdm01db01 image#unzip cellImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar.zip,Archive:cellImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar.zip,inflating:cellImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar,rootdm01db01 image#unzip computeImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar.zip,Archive:computeImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar.zip,inflating:computeImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar,rootdm01db01 image#tar-pxvf cellImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar,rootdm01db01 image#tar-pxvf computeImageMaker_11.2.2.4.2_LINUX.X64_111221-1.x86_64.tar,How to do Exadata installation,将,U,盘插到,DB,节点,1,上，并确认自动识别,rootdm01db01 ImageMaker#cd/var/log,rootdm01db01 log#tail-f me*s,Mar 1 15:15:01 dm01db01 kernel:sdd:assuming drive cache:write through,Mar 1 15:15:01 dm01db01 kernel:SCSI device sdd:15633408 512-byte hdwr sectors(8004 MB),Mar 1 15:15:01 dm01db01 kernel:sdd:Write Protect is off,Mar 1 15:15:01 dm01db01 kernel:sdd:assuming drive cache:write through,Mar 1 15:15:01 dm01db01 kernel:sdd:sdd1,Mar 1 15:15:01 dm01db01 kernel:sd 7:0:0:0:Attached scsi removable disk sdd,Mar 1 15:15:01 dm01db01 kernel:sd 7:0:0:0:Attached scsi generic sg3 type 0,Mar 1 15:20:01 dm01db01 auditd6769:Audit daemon has no space left on logging partition,Mar 1 15:20:01 dm01db01 auditd6769:Audit daemon is suspending logging due to no space left on logging partition.,How to do Exadata installation,使用,U,盘,reimage Exadata,创建,Cell U,盘,rootdm01db01 image#cd dl180,sh-3.2#./makeImageMedia.sh-preconf././preconf.csv-nodisktests-stit-dualboot no,Done.Pre config verification OK,Please wait.Calculating md5 checksums for cellbits.,Please wait.Making initrd.,199489 blocks,Please wait.Calculating md5 checksums for boot.,Choose listed USB devices to set up the Oracle CELL installer,sdd Approximate capacity 7816 MB,Enter the comma separated(no spaces)list of devices or word ALL for to select all:ALL,sdd will be used as the Oracle CELL installer,All data on sdd will be erased.Proceed y/n?Y,How to do Exadata installation,使用,U,盘,reimage Exadata,创建,DB U,盘,sh-3.2#cd dl360,sh-3.2#./makeImageMedia.sh-preconf././preconf.csv-nodisktests-stit-dualboot no,Done.Pre config verification OK,Please wait.Calculating md5 checksums for cellbits.,Please wait.Making initrd.,199369 blocks,Please wait.Calculating md5 checksums for boot.,Choose listed USB devices to set up the Oracle CELL installer,sdd Approximate capacity 7816 MB,Enter the comma separated(no spaces)list of devices or word ALL for to select all:ALL,How to do Exadata installation,使用,U,盘,reimage Exadata,将制作好的,Compute U,盘,image,插入到,Compute,节点机，然后,reboot,，然后按住,F8,，选择从,USB,进行引导，默认进行下去，过程中会自动进行重启，然后选择从硬盘启动，继续进行，然后会提示拔出,U,盘，敲回车，计算机自动关机，然后手动开开计算机电源。,将制作好的,Cell U,盘,image,插入到,Cell,节点机，然后,reboot,，然后按住,F8,，选择从,USB,进行引导，默认进行下去，过程中会自动进行重启，然后选择从硬盘启动，继续进行，然后会提示拔出,U,盘，敲回车，计算机自动关机，然后手动开开计算机电源。,How to do Exadata installation,使用,U,盘,reimage Exadata,通过,ilom,和,iso,镜像,reimage Exadata,How to do Exadata installation,这个,reimage,方法不制作,usb,启动盘，而是制作一个,iso,镜像，不使用,PXE,和,NFS,等复杂配置，只是使用,ilom redirection,等机器自带功能。,制作,usb,启动盘的命令稍加修改就可以将启动文件重定向生成,iso,文件。而这个,iso,文件可以位于我们自己的笔记本电脑上，然后各个节点通过,ilom redirection,的功能加载,iso,文件为虚拟光驱，之后各个节点可以共享这个启动盘文件（,iso,文件），这样同时,reimage,机器就成为可能。,通过,ilom,和,iso,镜像,reimage Exadata,How to do Exadata installation,制作,CELL ISO,文件,cd/reimage/dl180,./makeImageMedia.sh-preconf/reimage/preconf.csv-stit-nodisktests-notests-dualboot=no/reimage/cell.iso,制作,DB ISO,文件,cd/reimage/dl360,./makeImageMedia.sh-preconf/reimage/preconf.csv-stit-nodisktests-notests-dualboot=no/reimage/db.iso,通过,ilom,和,iso,镜像,reimage Exadata,How to do Exadata installation,登录各个节点的,ilom web interface,修改,ilom,的,ip,地址,修改方法：,点击,configuration-,network,通过,ilom,和,iso,镜像,reimage Exadata,How to do Exadata installation,修改,ilom,所在节点的启动方式,修改方法：,点击,remote control,host control,选择,cdrom,通过,ilom,和,iso,镜像,reimage Exadata,How to do Exadata installation,Mount,虚拟光驱,先要从,web interface,打开,console,的界面,点开,console,后，点击菜单里面的,devices,CD-ROM image,选择制作好的,iso,文件,选择完后，,console,应该是这样的回应：,通过,ilom,和,iso,镜像,reimage Exadata,How to do Exadata installation,使用,CDROM,方式重启机器,计算节点,reimage,计算节点在完成上面动作后，不需要任何的人工干预，所以可以,8,个节点并行的刷。（对于,iso,文件的竞争并不激烈）,Cell,节点,reimage,Cell,节点相对计算节点有些麻烦，在完成上面的动作以后，需要手动将节点的启动顺序调整成为,”default hard driver”,，以确认下一次重启是从硬盘启动，否则,cell,节点就会反复,reimage,。（该情况不会在计算节点出现）,Agenda,What customers need to do,What needed before Exadata set up,How to do Exadata installation,Post Exadata reimage,OneCommand step-by-step introduction,typical issues in Exadata set up and configuration,monitoring tools deployment,Post Image-Exadata InfiniBand Switch,重新配置,配置管理网络,IP,等,（,root/welcome1,）,以用户名,root,登陆交换机,ILOM,，修改,ILOM,配置（,IP/DNS/NTP,等）,Post Exadata reimage,核心,InfiniBand,交换机（,ib1,）,-,系统信息,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,核心,InfiniBand,交换机（,ib1,）,-IB,端口,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,核心,InfiniBand,交换机（,ib1,）,-SM,设置,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,边缘,InfiniBand,交换机（,ib2/ib3,）,-,系统信息,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,边缘,InfiniBand,交换机（,ib2/ib3,）,-,端口信息,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,边缘,InfiniBand,交换机（,ib2/ib3,）,-SM,设置,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,记录交换机,Firmware,版本,rootdm01sw-ib1#nm2version,(Note:The command nm2version is deprecated and will be removed from future,releases.Please use version instead),SUN DCS 36p version:1.3.3-2,Build time:Apr 4 2011 11:15:19,SP board info:,Manufacturing Date:2011.05.10,Serial Number:NCD6I0236,Hardware Revision:0 x0006,Firmware Revision:0 x0000,BIOS version:SUN0R100,BIOS date:06/22/2010,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,记录交换机优先级设定（,IB1,核心交换机为,8,，,IB2/IB3,边缘交换机为,5,）,rootdm01sw-ib1#setsmpriority list,Current SM settings:,smpriority 8,controlled_handover TRUE,subnet_prefix 0 xfe80000000000000,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,记录交换机,Subnet Manager master,设定（,IB1,核心交换机）,rootdm01sw-ib1#getmaster,Local SM enabled and running,20120301 11:01:25 Master SubnetManager on sm lid 2 sm guid 0 x2128469a13a0a0:SUN DCS 36P QDR dm01sw-,or,rootdm01sw-ib,2,#,disablesm,rootdm01sw-ib,2,#,enablesm,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户,root,登陆交换机,Linux,操作系统，修改,Linux,配置（,IP/netmask,等）,rootdm01sw-ib1#cd/etc,rootdm01sw-ib1 etc#vi/etc/hosts,.,10.200.100.56 dm01sw-dm01sw-ib1,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户,root,登陆交换机,Linux,操作系统，修改,Linux,配置（,IP/netmask,等）,rootdm01sw-ib1 network-scripts#cd/etc/sysconfig/network-scripts,rootdm01sw-ib1 network-scripts#vi ifcfg-eth0,DEVICE=eth0,BOOTPROTO=static,ONBOOT=yes,TYPE=Ethernet,IPADDR=10.200.200.56,NETMASK=255.255.255.0,BROADCAST=10.200.200.255,GATEWAY=10.200.200.1,HWADDR=00:E0:4B:38:71:8A,NETWORK=10.200.200.0,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户,root,登陆交换机,Linux,操作系统，修改,Linux,配置（,IP/netmask,等）,rootdm01sw-ib1 sysconfig#cd/etc/sysconfig,rootdm01sw-ib1 sysconfig#vi network,NETWORKING=yes,NETWORKING_IPV6=no,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户,root,登陆交换机,Linux,操作系统，修改,Linux,配置（,DNS/NTP,等）,rootdm01sw-ib1 sysconfig#cd/etc,rootdm01sw-ib1 etc#vi resolv.conf,search localhost,nameserver 133.37.253.119,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户,root,登陆交换机,Linux,操作系统，修改,Linux,配置（,DNS/NTP,等）,rootdm01sw-ib1 etc#cd/etc,rootdm01sw-ib1 etc#vi ntp.conf,#DO NOT EDIT THIS FILE#,server 133.37.253.102 prefer,#server none,driftfile/var/lib/ntp/drift,pidfile/var/run/ntpd.pid,#END,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户,root,登陆交换机,Linux,操作系统，修改,Linux,配置（,DNS/NTP,等）,rootdm01sw-ib1 etc#chkconfig ntpd on,rootdm01sw-ib1 etc#,service ntpd-full-restart,rootdm01sw-ib1 etc#reboot,Post Image-Exadata InfiniBand Switch,重新配置,Post Exadata reimage,配置管理网络,IP,等,以用户名,root,登陆交换机,Linux,操作系统，观察网络配置,rootdm01sw-ib1#ifconfig eth0,eth0 Link