《深度学习》课件 第6章 深度学习开源框架.pdf

深度学习BIGDATA全国高校标准教材云计算姊妹篇，剖析大数据核心技术和实战应用第六章深度学习开源框架6.1 Caffe6.2 TensorFlow6.3 其它开源框架 习题 6.1Caffe第六章深度学习开源框架 6.LlCaffe框架 Caffe是一个清晰、高效、开源的深度学习框架，很适合初学者。它由加州大学伯克利的PHD贾扬清开发是应用最广泛的深度学习框架之一。贾扬清Caffe 6.1Caffe第六章深度学习开源框架 6.LlCaffe框架 Caffe是一个清晰、高效、开源的深度学习框架，很适合初学者。它由加州大学伯克利的PHD贾扬清开发是应用最广泛的深度学习框架之一。代码组织好。可读性强，例子多。支持C+、Python 和Matlab0 采用MKL、OpenBLAS等 计算库，支持 GPU力口速。国内外都有较 活跃的社区、且有很多衍生 项目。6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe1、操作系统为cent enos72、安装Nvidia GPU驱动(1)添力口ELRep。源#rpm import https:/www.elrepo.org/RPM-GPG-KEY-elrepo.org#rpm-Uvh http:/www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpmrootlocalhost sudo rpm-Uvh http:/www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpmRetrieving http:/www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpmPreparing.#100%Updating/installing.1:elrepo-release-7.0-2.el7.elrepo#*#100%6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(2)安装显卡驱动#yum install nvidia-xll-drvnvidia-xll-drv-32bitn symbol nvUvmlnterfaceDupMemorydepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvUvmlnterfaceCheckEccErrorSlowpathdepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvUvmlnterfaceAddressSpaceCreatedepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfaceCopyEngineAllocatedepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfacellnregisterGpudepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvUvmlnterfaceAddressSpaceDestroydepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfaceRegisterUvmCallbacksdepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfaceChannelAllocatedepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfaceGetP2PCapsdepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfaceDupAllocationdepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfacePmaAllocPagesdepmod:WARNING:/lib/modules/3.10.0-229.el7.x86_64/weak-updates/nvidia/nvidia-uvm.ko needs n symbol nvllvmlnterfaceSessionCreateDone.unknowunknowunknowunknowunknowunknowunknowunknowunknowunknowunknowVerifying:kmod-nvidia-367.27-1.el7.elrepo.x86_64 1/6Verifying:nvidia-xll-drv-32bit-367.27-1.el7.elrepo.x86_64 2/6Verifying:nvidia-xll-drv-367.27-1.el7.elrepo.x86_64 3/6Verifying:nvidia-xll-drv-361.45.11-1.el7.elrepo.x86_64 4/6Verifying:nvidia-xll-drv-32bit-361.45.11-1.el7.elrepo.x86_64 5/6Verifying:kmod-nvidia-361.45.11-1.el7.elrepo.x86_64 6/6Updated:nvidia-xll-drv.x86_64 0:367.27-1.el7.elrepo nvidia-xll-drv-32bit.x86_64 0:367.27-1.el7.elrepoDependency Updated:kmod-nvidia.x86_64 0:367.27-1.el7.elrepoComplete!6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(3)重启#reboot(4)查看GPU信息#nvidia-smirootlocalhost mnist#nvidia-smi Tue Jun 14 15:20:01 2016I NVIDIA-SMI 361.45 Driver Version:361.45.11|I-+-+-+I GPU Name Pers1stence-M|Bus-Id Disp.A|Volatile Uncorr.ECC II Fan Temp Perf Pwr：Usage/Cap|Memory-Usage I GPU-Util Compute M.|I=+=+=|I 0 GeForce GTX TIT.Off|0000:82:00.0 Off|N/A|I 0%50C P0 52W/2SOW|23M1B/12287M1B|0%Default|+-+-+-+-+I Processes:GPU Memory|I GPU PID Type Process name Usage|I No running processes found 6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(5)安装CUDA 7.5 Toolkit:CUDA 7.5 Toolkit为Nvidia工具包。该工具包包括GPU加速库，调试和优化工具，C/C+编译器和运行库等。#cd/root/cDeep/Nvidia/#sh cuda_7.5.18_linux.runDo you accept t he previously read EULA?(accept/decline/quit):acceptInst a ll NVIDIA Accelerat ed Graphics Driver for L1nux-x86_64 352.39?(y)es/(n)o/(q)uit):nInst a ll t he CUDA 7.5 Toolkit?(y)es/(n)o/(q)uit):yEnt er Toolkit Locat ion defa ult is/usr/local/cuda-7.5:Do you want t o inst all a symbolic link at/usr/local/cuda?(y)es/(n)o/(q)uit):yInst a ll t he CUDA 7.5 Samples?(y)es/(n)o/(q)uit):yEnt er CUDA Samples Locat ion default is/root:Inst alling t he CUDA Toolkit in/usr/1ocal/cuda-7.5.Inst alling t he CUDA Sanpies in/root.Copying samples t o/root/NVIOIA_CUDA-7.5_Samples now.Finished copying sanples.MW-Summary-Driver：Not Select edToolkit:Inst a lled in/usr/local/cuda-7.5Samples:Inst a lled in/rootPlease nake sure t hat-PATH Includes/usr/1ocal/cuda-7.S/bin-LD_LIBRARY_path includes/usr/1ccal/cuda-7.5/1ib64,or.add/usr/local/cuda-7.5/lib64 t o/et c/ld.so.conf and run Idconfig as rootTo uninst all t he CUDA Toolkit,run t he uninst all script in/usr/1ocal/cuda-7.5/binTo uninst all t he NVIDIA Driver,run nvidia-uninst a llPlease see CUDZInst a11at ion_Guide_Limjx.pdf in/usr/1ocal/cuda-7.5/doc/pdf for det ailed informat ion on set t ing up CUDA.WARNING:Incomplet e inst allat ion?This inst a llat ion did not inst all t he CUDA Driver.A driver of version at least 352.00 is required for CUDA 7.5 funct ionalit y t o work.To inst all t he driver using t his inst aller,run t he following connand.replacing wit h t he name of t his run file:sudo.run-silent-driverLogfile is/t nip/cudacinst al 1.22204.1 og 6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(6)安装cuDNN v4:针对深度学习框架的一套GPU加速方案#cd/root/cDeep/Nvidia/#tar xvzf cudnn-7.0-linux-x64-v4.0-prod.tgz#cp cuda/include/cudnn.h/usr/local/cuda-7.5/include#cp cuda/lib64/libcudnn*/usr/local/cuda-7.5/lib64#chmod a+r/usr/local/cuda-7.5/lib64/libcudnn*(7)配置环境变量#vim/etc/profile在文件最后添加export LD_LIBRARY_PATH=,$LD_LIBRARY_PATH:/usr/local/cuda-7.5/lib64export CUDA_HOME=/usr/local/cuda-7.5更新环境变量#source/etc/profile 6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe3、Caffe 安装(1)安装依赖库#source/etc/profile#yum install protobuf-devel snappy-devel opencv-devel boost-devel#cd/root/cDeep/Caffe/#unzip leveldb-master.zip#cd leveldb-master#./build_detect_platform build_config.mk./#make#cp-r include/leveldb/usr/local/include#cp/root/cDeep/Caffe/leveldb-master/out-static/libleveldb.a/usr/local/lib 6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(2)安装GCC和GCC+#yum-y install gcc gcc-c+(3)安装GIT和CMAKE#yum-y install git#yum-y install cmake(4)安装GLOG#cd/root/cDeep/Caffe/#tar zxvf glog-0.3.3.tar.gz#cd glog-0.3.3#./configure#make#make install(5)安装 GFLAGS#cd/root/cDeep/Caffe/#unzip gflags-master.zip#cd gflags-master#mkdir build&cd build#export CXXFLAGS=-fPIC&cmake.&make VERBOSE=1#make&make install(6)安装LMDB#cd/root/cDeep/Caffe/#unzip lmdb.zip#cd Imdb-mdb.master/libraries/liblmdb#make&make install 6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(7)安装 HDF5#cd/root/cDeep/Caffe/#tar-xf hdf5-1.8.17.tar#cd hdf5-1.8.17#./configure prefix=/usr/local/hdf5#make&make install(8)安装OpenBlas#cd/root/cDeep/Caffe/#unzip OpenBLAS.zip#cd xianyi-OpenBLAS-3f6398a#make#make PREFIX=/usr/local/openblas install(9)安装OpenCV#cd/root/cDeep/Caffe/#unzip opencv-2.4.13.zip#cd opencv-2.4.13#mkdir release&cd release#cmake-D CMAKE_BUILD_TYPE=RELEASE-D CMAKE_INSTALL_PREFIX=/usr/local-D BUILD_PYTHON_SUPPORT=ON-D BUILD_EXAMPLES=ON.#make#make install 6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(10)修改配置文件#cd/root/cDeep/Caffe/#unzip caffe-master.zip&cd caffe-master#cp Makefile.config.example Makefile.config#vim makefile.config修改makefile,config文件里的信息1)USE_CUDNN:=1 使用cuda进行加速2)OPENCV_VERSION:=2 对应OpenCV版本是2.4.1 2.4.133)CUDA_DIR:=/usr/local/cuda-7.5 cuda头文件及库目录4)BLAS:=open 使用openbiasBLASJNCLUDE:=/usr/local/openblas/include 设置头文件路径BLAS_LIB:=/usr/local/openblas/lib 设置动态库路径5)INCLUDE_DIRS:=/usr/local/include/usr/local/hdf5/include/usr/include/python2.76)设置其它依赖库头文件路径LIBRARY_DIRS:=/usr/local/lib/usr/lib/usr/local/hdf5/lib 设置其它依赖库动态库路径6.1Caffe第六章深度学习开源框架 6.L2 安装 Caffe(11)Caffe编译#cd/root/cDeep/Caffe/#cd caffe-master#make all-J4#make test#make runtestConvolutionLayerTest/O.TestSimpleConvolutionGroupConvolutionLayerTest/O.TestSimpleConvolutionGroup(0 ms)ConvolutionLayerTest/O.TestGradientGroupConvolutionLayerTest/G.TestGradientGroup(1 8 ms)Con volutionLayerTest/G.TestOCXLon volutionConvolutionLayerTest/O.TestODConvolution(0 ms)ConvolutionLayerTest/O.TestlxIGradientConvolutionLayerTest/G.TestlxIGradient(470 ms)1 5 tests from ConvolutionLayerTest/0(1 1 59 ms total)1 test from ConstantFiUerTest/1,where TypeParam=doubleConstantFillerTest/l.TestFillConstantFillerTest/1.TestFill(0 ms)1 test from ConstantFillerTest/1(0 ms total)3 tests from SplitLayerTest/O,where TypeParam=caffe：:CPUDeviceSplitLayerTest/O.TestGradientSplitLayerTest/O.TestGradient(4 ms)SplitLayerTest/O.TestSetupSplitLayerTest/0.TestSetup(0 ms)SplitLayerTest/O.TestSplitLayerTest/O.Test(0 ms)3 tests from SplitLayerTest/0(4 ms total)Global test environment tear-down2069 tests from 277 test cases ran.(41 7904 ms total)2069 tests.6.1Caffe第六章深度学习开源框架 6.1.3案例：基于Caffe的目标识别1.获取数据集#cd./data/cifarlO#./get_cifarlO.sh1 21 561 21 561 21 561 21 561 21 561 21 561 root1 21 561 21 561103 61 Jun1 1 03 30730000 Jun1 1 03 30730000 Jun1 1 03 30730000 Jun1 1 03 30736006 Jun1 1 03 30730000 Junroot 504 Jun1 1 03 88 Jun1 1 03 30730000 Jun5 5 5 5 5 55 52009 batches.meta.txt2009 data_batch_l.bin2009 data_batch_2.bin2009 data_batch_3.bin2009 data_batch_4.bin2009 data_batch_5.bin15:56 get_cifarl0.sh2009 readme.html2009 test batch.bincifarlO数据集文件示意图airplane automobile birdcatdeerdog frog horseship truckCifarlO图像小意图 6.1Caffe第六章深度学习开源框架 6.1.3案例：基于Caffe的目标识别2.成训练及测试数据，将样本转为LMDB格式#cd./data/cifarlO#./get_cifarlO.sh3,训练及测试drwxr-r-.drwxr-r-.22root rootroot root36 Jun36 Jun2424-rwxrwxrwx.1rootroot3677 Jun2-rwxrwxrwx.1rootroot508 Jun24-rw-r-r-1rootroot12299 Jun24-rwxrwxrwx.1rootroot5243 Jun217:18 17:1815:56 17:18cifarl0_test_lmdb cifarlGtrainlradb convertcifardata.cpp create cifarie.sh17:18 mean.binaryproto15:56 readme.md#cd./examples/cifarlO&,/train_full.shII0627106271062710627106271062710627106271062719627106271962710627106271062710627106271062710627106271062710627106271062711:54:55.960958 11:54:56.297761 11:54:56.297785 11:54:56,301308 11:54:56.381329 11:54:56.361349 11:54:58.606628 11:54:58,606654 11:54:58.606663 11:55:00.914706 11:55:00.914757 11:55:00,914767 11:55:03.210448 11:55:03.210533 11:55:03.210544 11:55:05.472537 11:55:05,472561 11:55:05.472570 11:55:07.708394 11:55:07,732182 11:55:08.037986 11:55:08.038010 11:55:08.041465 11:55:08,041487 110414OA13600 solver.cpp：337 Iteration 67000,Testing net(#0)13600 solver.cpp：40413600 solver.cpp：464Test net output#0：accuracy 0.815Test net output#1：loss-0.534026(*1-0.534026 loss)13600 solver.cpp:228 Iteration 67000,loss-0.30726413600 solver.cpp：244 Train net output#0：loss G.307264(*13600 sgd_solver.cpp：lO6 Iteration 67000,lr=le-0513600 solver.cpp：228 Iteration 67200,loss-0.42048413600 solver.cpp：244 Train net output#0：loss=6.420484(*13600 sgd_solver.cpp：106 Iteration 6720G,lr le-0513600 solver.cpp：228 Iteration 67400,loss=0.30912413600 solver.cpp：244 Train net output#0:loss 0.309124(*13600 sgd_solver.cpp：106 Iteration 67400,lr=le-0513600 solver.cpp：228 Iteration 676G0,loss 0.3184G413600 solver.cpp：244 Train net output#0：loss=0.318404(*13600 sgd_solver.cpp：106 Iteration 67600,lr-le-0513600 solver.cpp：228 Iteration 67800,loss=0.30816113600 solver.cpp:244)Train net output#0：loss=0.308161(13600 sgd_solver.cpp：106 Iteration 67800,lr=le-0513600 solver.cpp：337 Iteration 68000,Testing net(#0)13600 blocking_queue.cpp:50 Data layer prefetch queue empty13600 solver.cpp：404 Test net output#0:accuracy 0.8151 1-G.307264 loss)1=0.420484 loss)1-0.309124 loss)1=0.318404 loss)1=0.308161 loss)13600 solver.cpp：404 Test net output#1:loss=0.534015(*1=0.534015 loss)13600 solver.cpp：228 Iteration 68000,loss-0.30615513600 solver.cpp：244 Train net output#0：loss=0.306155(*1=0.306155 loss)全国高校标准教材云计算姊妹篇，剖析大数据核心技术和实战应用第六章深度学习开源框架6.1 Caffe6.2 TensorFlow6.3其它开源框架习题 6.2TensorFlow第六章深度学习开源框架 6.2.lTensorFlow*匡架 GoogleTensorFlow是Google开源的一个深度学习框架。实现了本地和分布式两种接口机制。本地实现机制的client端、master端和worker均运行在同一个机器中;分布式实现机制它与 本地实现的代码基本相同，但是client端、master端和worke进程一般运行在不同的机器中，所包含的不同任务由一个集群调度系统进行管理。具备如下优点：1多样化部署；2.可被基于梯度的机器学习算法借鉴;3.灵活的Python接 ；4.可映射到不同硬件平台；5.支持分布式训练。6.2TensorFlow第六章深度学习开源框架 6.2.2安装 TensorFlow1、安装pip#cd/root/cDeep/Tensorflow/#python get-pip.pyrootdlocalhost cDeep#python get-pip.pyCollecting pip八叩八SNIMissingwarning：An HTTPS request has beer platfora.This may cause the server to present an incorrect TLS certificate,which can cause validation failures.You can i llibS.readthedocs.org/en/latest/security.htBlifsni tni ssi ngwarning./tmp/tmpK OTPGk/pip.zip/pip/_vendor/requests/packages/url11 b3/uti1/ssl_.py:1 22:InsecurePlatformWaming：A true SSLContext(y cause certain SSL connections to fail.You can upgrade to a newer version of Python to solve this.For more infomation.Downloading pip-8.1.2-py2.py3-none-any.whl(1.2MB)100%I|1.2MB 20kB/SCollecting wheel100%|71 kB 1 5kB/SInstalling collected packages：pip.wheelSuccessfully installed pip-8.1.2 wheel-0.29.0/tmp/tmpK OTPGk/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl.py:1 22:InsecurePlatformWaming:A true SSLContext(v cause certain SSL connections to fail.You can UDarade to a newer version of Pvthon to solve this.For more information.pip安装示意图 6.2TensorFlow第六章深度学习开源框架 6.2.2安装 TensorFlow2.安装TensorFlow#cd/root/cDeep/Tensorflow/#pip install upgrade tensorflow-0.8.0-cp27-none-linux_x86_64.gpu.whlProcessing./t ensorflow-O.8.0-cp27-none-linux_x86_64.gpu.whlCollect ing prot obuf-3.0.0b2(from t ensorf 1 ow-0.8.0)/usr/1ib/pyt hon2.7/si t e-packages/pi p/vendor/request s/packages/url1ib3/ut i1/ssl.py:318:SNlMissingwaming:An HTTPS request has I e on t his plat form.This may cause t he server t o present an incorrect TlS cert ifica t e,which can cause validat ion failures.You c.ht t ps:/urllib3.readt hedocs.org/en/lat est/securit y.ht alt sninissingwarmng.SNIMissingwarning/usr/1ib/pyt hon2.7/slt e-packages/pip/_vendor/request s/packages/urllib3/ut il/ssl.py:122:InsecurePlat forawaming:A t rue SSLCont e:ely and may cause cert ain SSL connect ions t o fa il.You can upgrade t o a newer version of Pyt hon t o solve t his.For sore inforaat i(arning.InsecurePlat formWarningDownloading prot obuf-3.0.0b2-py2.py3-none-any.wt)l(326kB)100%327kB 18kB/sRequirement already up-t o-dat e:wheel in/usr/11b/pyt hon2.7/sit e-packages(from t ensorflow-0.8.0)Collect ing six-1.10.0(from t ensorflow.8.0)Downloading six-1.10.0-py2.py3-none-any.whlCollect ing nuapy1.8.2(from t ensorflow-0.8.0)Downloading nuffpy-l.ll.0-cp27-cp27mu-manylinuxl x86 64.whl(IS.3MB)100%15.3MB 27kB/SCollect ing set upt ools(from prot obuf-3.0.0b2-t ensorflow-0.8.0)Downloading set upt ools-23.0.0-py20.9.8:Successfully uninst alled set upt ools-0.9.8Found exist ing inst a llat ion:six 1.3.0DEPRECATION:Uninst alling a dist ut ils inst alled project(six)has been deprecat ed and will be removed in a fut ure version.Th ninst all t he project.Uninst alling six-1.3.0:Successfully uninst alled six-1.3.0Found exist ing inst allat ion：numpy 1.7.1DEPRECATION：Uninst alling a dist ut ils inst alled project(numpy)has been deprecat ed and rill be removed in a fut ure version,uninst all t he project.Uninst alling nwnpy-1.7.1:Successfully uninst alled numpy-1.7.1Successfulinst alled numov-l.11.0 orot ohuf-.O.Ob?spt unt ools-2.0.0 six-1.10.0 t pnsorflow-0.8.0左图为 tensorflow安装示意图 6.2TensorFlow第六章深度学习开源框架 6.