Mellanox-InfiniBand技术和方案汇报教学教材.pptx

2015 Mellanox Technologies,Click to edit Master title style,Click to edit Master text styles,Second level,Third level,Fourth level,Fifth level,#,-,Mellanox Confidential,-,2015,年,4,月,InfiniBand,技术和方案汇报,提纲,当前网络有哪些难以克服的问题,面,向未来数据中心，如何构建新型网络以解决当前问题,InfiniBand,网络有哪些技术特点,InfiniBand,网,络使用场景分析,以太网的固有问题,1,可用性问题,问,题：,极易引起广播风暴,占,用大量网络带宽,甚至网,络彻底瘫,痪,缓解对策：,使用,VLAN,，缩小广播域，减小影响范围。,但无法彻底解决。,鱼与熊,掌：,是否有网络技术可自由组网，又彻底不会带来广播风暴？,以太网的固有问题,2,效率问题,问,题：,多次数据内存拷贝，大幅增加延迟,中,断处理响应需要,CPU,处理，极为,影,响性能,协议栈处理是软件操作，性能低下，且大幅占用,CPU,鱼与熊掌,：,能否既可靠传输，又高效处理，且不需要占用,CPU,资源？,RACK 1,OS,NIC,Buffer 1,Application,1,Application,2,OS,Buffer 1,NIC,Buffer 1,TCP/IP,RACK 2,Buffer 1,Buffer 1,Buffer 1,Buffer 1,Buffer 1,以太网的固有问,题,3,多网融合,长期以来，计算网和存储网使用两种不同的网络,计算,网使用以太网,尝试使用,iSCSI,来发展,IP SAN,，,但性,能提升遇到巨大瓶颈,存,储网使用,FC,FC,网无法向计算网延伸,能否使用一种网络融合计算和存储？,新,型高速网络的需求,业务发展需要,小型机,单,机存在性能极限，然而扩展困难,x86,机,群易扩展，低成本，替换小机是大趋势,如,果要,x86,机群更好发挥整体性能，就需要有高性能的网络，把,x86,机群有机整合，以发挥整体性能,需要高带宽,实现节点间数据实现大吞吐，比如数据装载阶段,需,要低延迟,需要节点间应用快速信息交换，比如数据查询阶段,构,建新型网络结构,实现自,由组网,，且彻底解决广,播风,暴,实现网络协议的完全卸载，既,可靠传输，又高效处理，且,不占,用,CPU,资,源,要发挥网络的极致性能，提供最高的带宽，最低的延迟,需,要指定公开网络规范，,不能使用某个厂商的私有技术,需要独立的第三方机构维护网络规范，,需,要有稳健的路标，可应对数据中心的未来挑战,InfiniBand,应运而生,什么是,InfiniBand,网络,InfiniBand,贸易协,会,(,IBTA),成立于,1999,年,InfiniBand,是由,InfiniBand,行业协会所倡导的,。,InfiniBand,代表了两种计算潮流的融合：下一代,I/O（NGIO）,和未来的,I-O（FIO）。,大部分,NGIO,和,FIO,潮流的成员都加入了,InfiniBand,阵营,IBTA,制,定和维护,InfiniBand,标准,InfiniBand,是一个基于交换网格的体系结构,实现,CPUs,和,I/O,之间的互联通道,超高的性能,高带宽,(,从,10Gbps,起步，直至,60Gbps,),扩展空间很大,!,低延迟,机群中跨节点应用程序间的快速反应,.,低的,CPU,占用及,RDMA(,远程直接内存访问,),打破以太网的,CPU,和,操,作系统介入通讯,.,增加应用程序性能,单网融合,LAN,SAN,和应用程序通讯,高可靠的子网管理器,InfiniBand,是一个为大规模易扩展机群而设计的网络技术,第一个高速互联工业标准,!,InfiniBand,设,计路标,SDR-Single Data RateDDR-Double Data RateQDR-Quad Data RateFDR-Fourteen Data RateEDR-Enhanced Data RateHDR-High Data RateNDR-Next Data Rate,InfiniBand,开放资源,InfiniBand,软件主要由,OpenFabrics,开源联盟所开发,www.openfabrics.org/index.html,InfiniBand,标准由,InfiniBand,贸易协会制定维护,www.infinibandta.org/home,业界,InfiniBand,产品发展趋势,延时,5usec,2.5usec,1.3usec,0.7usec,0.5usec,160/200Gb/s,100Gb/s,56Gb/s,40Gb/s,20Gb/s,10Gb/s,2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017,2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017,带宽,Same Software Interface,0.5usec,技术路线,领先竞争对手,一,代产品,2000,2020,2010,2005,20Gbs,40Gbs,56Gbs,100Gbs,“Roadrunner”,Mellanox Connected,1,st,3,rd,TOP500,2003,Virginia Tech(Apple),2015,2,00Gbs,Terascale,Petascale,Exascale,Mellanox,InfiniBand,的技术特点,PCI,串行高带宽连接,SDR:10Gb/s HCA,连接,DDR:20Gb/s HCA,连接,QDR:40Gb/s HCA,连接,FDR:56Gb/s HCA,连接,现在,EDR:100Gb/s HCA,连接,2015,年,极低的延迟,低于,1,微妙（纳妙级）的应用延迟,可靠、无损、自主管理的网络,基于链路层的流控机制,先进的拥塞控制机制可以防止阻塞,完全的,CPU,卸载功能,基于硬件的传输协议,可靠的传输,内核旁路技术,内存可提供远程节点访问,RDMA,读和,RDMA,写,服务质量控制,（,QoS,）,Mellanox,网卡,:,提供多个独立的,I/O,通道,在链路层提供多条虚拟通道,集群可扩展性和灵活性,一个子网可支持,48,000,个节点，一个网络可支持,2,128,个节点,在节点之间为平行路由,提供多种集群拓扑方式,:,胖树，,3D,，,简化集群管理,集中路由管理,支持带内网络诊断和升级,RDMA,技术实现内核旁路，为,HPC,应,用提供最好的网络通讯性能,在,InfiniBand,上运行,RDMA,KERNEL,HARDWARE,USER,RACK 1,OS,NIC,Buffer 1,Application,1,Application,2,OS,Buffer 1,NIC,Buffer 1,TCP/IPRACK 2,HCA,HCA,Buffer 1,Buffer 1,Buffer 1,Buffer 1,Buffer 1,Arbitration,De-mux,Mux,链路控制,数据包,信,用反馈,Link Control,Receive,Buffers,包传输,基于信,用的链路层流控,链路流控确保即使在大压力的网络负载下，网络中也没有任何包丢失,链路接收端对每个虚拟链路根据,缓冲,区状况独立分发信用,流控信用以,64,字节为单位,每个虚拟链路提供独立的流控,:,不,会造成队首阻塞牺牲应用的情况,虚拟网络,虽然是共享同一个物理通道，在有了,QoS,和虚拟链路之后，在一个虚拟链路上的拥塞和延迟,不,会对另外一个虚拟链路造成任何影响,链路层流控,服务质量,QOS,管,理,低优先级,虚拟链路属性,报文分类识别,源端口组,目标端口组,分区,QOS,分类,应用服务,ID,SL 0-3,SL 4,SL 6,SL 7,SL 8,SL 10,SL 12,Default,GPFS,IP_O_IB,SDP,RDS,Native Multicast,Clock Sync,W 32,W 32,W 32,W 32,W 64,W 64,网,络节点用户,业,务包类型,服务级别,W 64,心跳通讯,存储业务,私有云业务,交易业务,政府业务,虚拟链路,VL-0,VL-1,VL-2,VL-3,VL-4,VL-5,VL15,高优先级,虚拟链路属性,子网管理器,物理子网建立,子网发现,信息收集,LID,分发,建立通路,端口配置,交换机配置,启动子网,SM,Switch-2,Switch-3,Switch-4,Switch-5,Host,Host,G-11,G-22,G-23,G-42,G-41,G-32,G-33,G-51,G-52,Switch-1,G-01,G-21,G-12,G-13,G-31,G-53,H-11,G-01,H-21,LID-1,LID-2,LID-3,LID-7,LID-8,LID-10,LID-32,LID-34,Switch-6,G-61,G-62,LID-8,G-43,中央视图资源占,用和状态,拥塞图,Top 10,报警节点,事件通知板,Top10,带宽占用,Top10,拥塞,带宽消耗,全局网络管理,监控分析,中央设备管理,自动化提供,Unified Fabric Management,功能,Automatic,Discovery,Central Device Mgmt,Fabric Dashboard,Congestion Analysis,Health&Perf Monitoring,Advanced Alerting,Fabric Health Reports,Service Oriented,Provisioning,监控优化系统性能和使用,更快识别解决问题,分析度量网络使用,用户应用的设备供给,Unified Fabric Management,优势,InfiniBand,如何组网,InfiniBand,组件,主机通道适配器,(,HCA),终结,IB,链路的设备，实现传输层功能，支持,verbs,接口,交换机,在一个子,网内实现数,据包转发的设备,路由,在,IB,子网间实现包路由的设备,网关,实现,InfiniBand,和以太网的互通,ProcessorNode,InfiniBandSubnet,Gateway,HCA,ProcessorNode,ProcessorNode,HCA,HCA,StorageSubsystem,Consoles,RAID,Ethernet,Gateway,FibreChannel,HCA,Subnet Manager,Switch,Switch,Switch,Switch,典型拓扑,胖树,26,全胖树,半,胖树,1000,节点机群拓扑案例,MSX6536,Configure 648 ports,MSX6025,MSX6025,MSX6025,9,9,9,9,9,18,18,18,Total 1296 IB nodes,MSX6536,Configure 648 ports,胖树案例,使,用,3,台交换机做聚合交换机,使,用多台交换机作为页交换机，连接服务器,使用,2,台网关，实现,InfiniBand和以太网的互联互通,两万个,InfiniBand,节点,Mellanox end-to-end scalable FDR and QDR InfiniBand,Supports variety of scientific and engineering projects,Coupled atmosphere-ocean models,Future space vehicle design,Large-scale dark matter halos and galaxy evolution,Leveraging InfiniBand backward and future compatibility,系统案例,:NASA,阿莫斯研究中心,High-Resolution Climate Simulations,Mellanox InfiniBands,系统的有效性,TOP500 systems listed according to their efficiency,InfiniBand is the key element responsible for the highest system,efficiency;in average 30%higher than 10GbE,Mellanox delivers efficiencies of,up to 99.8%with InfiniBand,Average Efficiency,InfiniBand:87%,Cray:79%,10GbE:67%,GigE:40%,InfiniBand,虚拟化技术,ESXi 5.1 10/40GbE,认证,在,Vmware,的硬件兼容列表中,(HCL),40GbE,实现更高的效率,ESXi 5.1 InfiniBand,认证,通过合作伙伴认证支持产品,(PVSP),发布了,VMware&Mellanox,联合白皮书,VM,到,VM,通讯,:IPoIB,存储,:,基于,RDMA,的,SRP/iSER,ESXi OP,Mellanox,40GbE,驱动将直接集成到,ESXi,中,基,于,Vmware,的方案和路线图,ConnectX-3,VirtualMachine 1,HCA Driver,DMA Remapping,Memory,VirtualMachine 1,HCA Driver,VirtualMachine 1,HCA Driver,Physical Server,Virtual Machine Monitor,Port,VMM OffloadFunctions,VMWare,软件拓扑图,Mellanox,单根虚拟化,(SR-IOV),卡向,OS/Hypervisor,提供多个,PCIe,设备,实,现应用直接访问,(,Application Direct Access),减,少,CPU,开销，提升应用程序性能,使用,RDMA&ADA,消除虚拟化带来的性能损失,在虚拟化平台上享用低延迟技术,VF Device Driver,VM,VF Device Driver,VM,VF Device Driver,VMn,Virtual NIC,VM,Physical Function Device Driver,PF,VF,VF,VF,先进的,SRIOV,虚拟化技术实现,Hypervisors,旁,路，为用户提供了最佳性能,旁路,Hypervisors,技术需要在,网,卡中嵌入交换机提供以下功能,服务质量,(,QoS),拥塞控制,安全过滤,(,ACLs,反欺骗,),本地虚机网络通讯,二层隧道,(,L2 Tunneling),高可用,性,(,基于硬件的链路聚合,),优点,在虚拟化环境中提供更高的性能,嵌入,式交换机,(eSwitch),High Availability,(“,LAG,”),SRIOV I/O Device,Hypervisor,eSwitch,Trusted,VM,VM,VM,VM,PF,VF,VF,VF,NIC,NIC,NIC,NIC,SR-IOV R,DMA,在虚拟化环境中达到原生,R,DMA,性能,SR-IOV,大幅提升虚机网络性能,即使,在虚拟化环境中也没有性能,损失,Mellanox,产,品介绍,IB TOR,Interconnect,IB,核心交换机,IB,远程交换机,Interconnect,IB,适配器,Interconnect,IB,适配器,Interconnect,以太网交换机,Interconnect,以太网适配器,Interconnect,线缆,直连铜缆,光缆,光模块,混合铜缆,进入,100Gb/s,的时代,Copper(Passive,Active),Optical,Cables(VCSEL),Silicon Photonics,36,EDR(100Gb/s),Ports,9,0ns Latency,Throughput,of 7.2Tb/s,Interconnect,IB Switch,Adapters,100Gb/s Adapter,0.7us latency,150 million messages per,second,10/25/40/50/56/100Gb/s,Eth Switch,32 100G Ethernet Ports,300ns Latency,1,0/25/40/50/56/100Gb/s,世界最高的性能,ConnectX-4 EDR 100G InfiniBand,Uni-Directional Throughput,100 Gb/s,Bi-Directional Throughput,195 Gb/s,Latency,0.61 us,Message Rate,149.5 Million/sec,InfiniBand,网卡性能比较,Mellanox Adapters,Single Port,Performance,ConnectX-4,EDR 100G,Connect-IB,FDR 56G,ConnectX-3 Pro,FDR 56G,Uni-Directional Throughput,100 Gb/s,54.24 Gb/s,51.1 Gb/s,Bi-Directional Throughput,195 Gb/s,107.64 Gb/s,98.4 Gb/s,Latency,0.61 us,0.63 us,0.64 us,Message Rate,149.5 Million/sec,105 Million/sec,35.9 Million/sec,ConnectX-4 InfiniBand Message Rate,InfiniBand,在数据仓库行,业,应用,甲骨,文使用,InfiniBand,技术实现,10,倍性能提升,Exadata,(,内存数据库一体机,),Exalogic,(,云计算一体机,),降低,5,倍的硬件投资,提高,10,倍的性能,使,用,InfiniBand,端到端方案实现互联和加速,Exalytics,(,大数据,),ZFS(,InfiniBand,存储,),SuperCluster,(,盒子数据中心方案,),Superior Database and Cloud Interconnect Performance,Database,Cluster,8,计算节点服务器,(,1U),64 C,ores,Storage,Cluster,14,storage,servers(2U),112 C,ores,in storage,100 TB SAS,盘,or336 TB SATA,盘,5 TB PCI Flash,Data mirrored across storage servers,可扩展的工业标准服务器实现统一的计算和存储,InfiniBand Network,冗余,40Gb/s,IB,交换机,统一,服务器和存储网络,甲骨文,Exadata,数据库计算,存储融合设备,ExaData,系统架构：内部,InfiniBand,互联,Oracle RAC:,全局共享缓存,54,缓存融合利用可,扩展,的共享缓存实现缓存同步，消除磁盘,IO,瓶颈,55,X86,标准服务器,+InfiniBand,+,SSD,：,“,Exadata,替代方案”,华为,FusionCube,融合数据计算和存储应用设备,Storage,Blade,Compute Blade,Compute Blade,CPU,CPU,CPU,CPU,InfiniBand QDR/FDR,CPU,PCI-e,SSD,Storage,Blade,CPUPCI-e,SSD,Storage,Blade,CPU,PCI-e,SSD,Storage,Blade,CPU,PCI-e,SSD,典型市场,数据库，私有云，数据仓库,高性能,High Performance,IO,提升,10,倍,IOPS,3,到,5,倍提升,高可靠性,节点间复制,高速数据恢复,(1TB,数据，从,12,小时到,30,分钟,),高扩展性能,可达,2000,节点,容量和性能几乎线性扩展,Mellanox InfiniBand Removes,Huawei FusionCube,IO Bottleneck,RDMA,实,现了数据仓库性能的大幅提升,Microsoft,全面利用,RDMA,进行优化,和,TCP/IP,相,比的性能提升,SQL,机柜到机柜的查询性,能,提升,2,倍,数据装载性能提升,4,倍,应用性能提升,10,倍,InfiniBand,和,RoCE,Mellanox,VPI,接口卡,Mellanox,交换机,Mellanox,网络管理,InfiniBand,在存储的应用,基于,InfiniBand,的存储互联方案,SMB Direct,使用,RDMA,互联技术,实现,业界领先的性能,使用,RDMA,解决存储,IOPs,瓶,颈,100usec,200usec,6000usec,25,usec,1,us,20,usec,10,usec,传统网络,(,6msec),软件,磁盘,使用,SSDs,(0.5msec),使用快速网络,(,0.2msec),使用,RDMA,(0.05msec),网络,100usec,200usec,200usec,25,usec,25,usec,180 IOPs,3000 IOPs,4300 IOPs,20,000 IOPs,Synchronous(back to back),使用完整的,操作系,统旁路以及,NV-Dimm/Cache,(0.007msec),1,us,6us,3,us,100,000,IOPs,Synchronous,InfiniBand,超强存储性能,2300K IOPs,5-10%the latency under 20 x the workload,我们提供更快的,IO,速率和更低的访问时间,!,only 131K IOPs,基,于,InfiniBand,的存储市场动向,RDMA,技术在存储行业持续受到,推广,2013,年,11,月,19,日,发,布新,NetApp EF550,闪存阵列,更新,E5500,阵列,在,StorageR,介绍,InfiniBand,前,端方案,EF550E5500,In-Box,With,Havana,2013,年,11,月,7,日,InfiniBand,融入,OpenStack Havana,发布版,唯一被列入红帽认证方案的网卡,iSER,InfiniBand,RoCE,2013,年,11,月,14,日,EMC,发布顶级存储,XtremIO GA,使用,InfiniBand,构建互联方案,EMC Isilon,白皮书推荐使用,Mellanox InfiniBand,网关,2013,年,11,月,19,日,昆腾选择,Mellanox IB,最为正式,InfiniBand,合作伙伴，发布最新,StorNext 5,扩展文件系统,Dec 9,2013,Symantec,支持,I,nfiniBand,CFS,支持,Mellanox InfiniBand,和,RoCE,Scale-out File Systems,Ceph,日渐流行,开,源,软件定义,向外扩,展存储,Scale-out block and object to many petabytes,Many enterprise storage features,Software-Defined:build your own or pre-integrated bundles,Integrated with OpenStack,Mellanox,使,Ceph,更快,10/40GbE accelerates Ceph today,20 x Higher Throughput,4x Higher IOPs with 40Gb Ethernet Clients!,(http,:/,Testing results based on fio benchmark,8m block,20GB file,128 parallel jobs,RBD Kernel Driver with Linux Kernel 3.13.3 RHEL 6.3,Ceph 0.72.2.IOPs,Testing results based on fio benchmark,4k,block,20GB file,128 parallel jobs,RBD Kernel Driver with Linux Kernel 3.13.3 RHEL 6.3,Ceph,0.72.2,VMware vSphere,存储,两种实施方案,独立的存储阵列,TCP/IP(NFS or iSCSI),With RDMA,iSER on RoCE or InfiniBand,SRP on InfiniBand,基于以太网,TCP/IP,的分布式存储,VMware VSAN&VVOL,Cisco,Dell,Fujitsu,HP,Hitachi,Huawei,IBM,Supermicro,,ScaleIO,Used in VMware VDI Ref.Design,with ConnectX-3,40GE,RDMA but benefits from fast 10/40GbE network,ConnectX-3,通过认证支持,VMware EVO:RAIL,Compute:4 x ESXi Servers,Interconnect per server,Certified 10GbE,NIC,with 2 ports,Server storage,VSAN,Virtual Desktop Infrastructure,60 Virtual Desktop per server(Baseline),VMware EVO:RAIL Appliance,+,Mellanox ConnectX-3 10GE,最大化,VDI,的有效性,Virtual desktops:120 per server,Boot:0.7 seconds per desktop,Deployment:7.5,s,econds per desktop,Recompose:9.2,s,econds per desktop,,VDI Improvement vs.EVO:Rail Baseline,EMC VDI Appliance with VMware Horizon View over 40GE,1,000,desktops with average user response times of 1,seconds,Deploy 1,000,virtual desktops in,1.5 hours,Simultaneously,boots 1,000 virtual desktops in 12,minutes,Support,up to 167 desktops per server,node,Price,per desktop of an entry-level business,PC,Low-cost,project implementation and efficient management and maintenance,Powered by,Mellanox 40GbE E2E Solution*,ESXi 5.5 with ConnectX-3 Inbox driver,SX1036 40GE SwitchConnectX-3 40GE NICs&Cables,ScaleIO is an EMC company,More Virtual Desktops per Server with Linear Scalability,*,http,:/,Delivers 5X,Performance Improvement Over 10GbE,Microsoft Windows Server 2012 R2,存储方案,SMB Direct,SMB 3.0 over RDMA,New class of enterprise file storage,更好的性能,更低的,CPU,开销,替换,Fibre Channel,获得更低的成本和更高的性能,改变,Windows Server 2012 R2,的机内驱动,InfiniBand&RoCE,加速,Microsoft Hyper-V and SQL Server,应用无需改变,File Client,File Server,SMB Client,User,Kernel,Disk,RDMA,Adapter,Network w/RDMA support,RDMA,Adapter,SMB Server,Network w/RDMA support,NTFSSCSI,Application,SMB Direct-SAN,成本消减,50%,Delivers SAN-like,functionality,directly from,the Windows,Stack,Utilize,inexpensive,industry-standard,commodity,hardware,E,liminate,the cost of proprietary hardware and software,SAN solutions,Source:,Microsoft White Paper“Windows,Server 2012 R2 Storage-Technical Scenarios and,Solutions,”,Windows Server 2012 R2,存储方案,Application data storage on cost,effective,continuously,available,high performance,SMB 3.0,file shares backed by Storage Spaces,Disaggregated compute and storage for independent management and,scale,Performance,Scale:,SMB 3.0,File Storage network,Continuous Availability and Seamless Scale Out with File Server Nodes,Elastic,Reliable,Optimized,Tiered Storage Spaces,Standard volume hardware for low cost,设计参考,:MS SQL over Violins WFA*,,Windows Flash Array,Sources:,Customer Story:Australian DoD Consolidates SQL Server with SMB Direct,Violin Memory,and Mellanox InfiniBand,2 Physical Servers8 sockets of MS SQL Server,licenses,WFA enables lower TCO,45 Physical Servers,90 sockets of MS SQL Server licenses,96%reduction in server count,92%reduction in SQL Server licenses,Windows Roadmap:Storage Spaces Share Nothing,Enabling cloud hardware designs,Support DAS(shared nothing)storage hardware,Prescriptive configurations,Scalable large pools simplify storage expansion,Fault tolerance&3-copy Mirror,Handles disk,enclosure and node failures,Centralized Management,Converged Fabric(RDMA+TCP)With QoS,SDN,NIC teaming,Packet-Direct,Key use cases,Hyper-V IaaS,Storage for Backup and Replication targets,Doesnt need shared JBODs and SAS fabric behind Scale Out File Server nodes,Scale-Out File Server,Hyper-V Clusters,SMB Direct Storage,Network Fabric,Shared JBODStorage,InfiniBand Dominates the Enterprise Application Appliances,RDMA is Pervasive in Enterprise Application Appliances,SAP HANA,方案,High-Performance analytics,a,ppliance,Processes the data in memory,delivers results from memory,Fulfills the need for,real-time decisions,the”Need for Speed”,HANA will support RDMA,Ported over RoCE with Accelio,GA,a,t June15,Well publish CloudX HANA over IB,SAP Go-to-Market Is through Certified OEM Solutions,BusinessObjects Solutions,Other Applications,SAP ERP,SAP NetWeaver BW,Third-Party Data Sources,In-Memory Computing Engine,Calculation and Planning Engine,Modeling Studio,Real-Time Replication Services,Data Services,SAP HANA,“5,大,”,可向外扩张数据库方案,Oracle,IBM,Teradata,Microsoft,SAP,Database,Oracle RAC,DB2 pureScale,Teradata,Database,MS SQL,HANA,Vendors,Appliance,Exadata,PureData,DWA:,27xx;67xx,NA,NA,OEMs Certified Appliances,NA,NA,NA,PDW:HP,Dell,Quanta,HP,Lenovo,Fujitsu,Cisco,Dell,Hitachi,EMC,NEC,SGI,Huawei,VCE,Data,Warehouse,Appliance/Active Enterprise Data Warehouse Parallel Data Warehous