10-数字图像处理-光度立体视觉(课堂PPT).ppt

单击此处编辑母版文本样式,第二级,第三级,*,单击此处编辑母版标题样式,Image&Vision Lab,信息视觉处理,光度立体视觉,Photometric Stereo,Materials from:,www.cs.cmu.edu/afs/cs/academic/class/15385-s06/lectures/ppts/,2,DC&CV Lab.,CSIE NTU,双向反射分布函数（,BRDF,）,The bidirectional reflectance distribution function,is the fraction of incident light emitted in one direction when the surface is illuminated from another direction.,ratio of the scene radiance to the scene irradiance,Bi,-,directional Reflectance Function,3,:Polar angle,:Azimuth angle,4,双向反射分布函数（,BRDF,）,:polar angle between surface normal and lens center,:azimuth angle of the sensor,:emitting from,:incident to,:irradiance of the incident light at the illuminated surface,:radiance of the reflected light,:ratio of the scene radiance to the scene irradiance,D,ifferential reflectance model:,5,双向反射分布函数（,BRDF,）,定义：输出方向的辐射度与输入方向的辐照度的比率；,BRDF,可以从零（在该方向没有反射光）变化至无穷大（输出方向的单位辐射度来自于输入方向任意小的辐射度）；,BRDF在输入和输出方向是对称的。（著名的,Helmholtz,互易原理）；,尽管对于某些输入与输出的角度，,BDRF,可以较大，但对大多数来说它不能大。事实上平均值必须相当小。,6,source,surface,reflection,surface,incident,direction,body,reflection,Body Reflection:,Diffuse Reflection,Matte Appearance,Non-Homogeneous Medium,Clay,paper,etc,Surface Reflection:,Specular Reflection,Glossy Appearance,Highlights,Dominant for Metals,Image Intensity =Body Reflection+Surface Reflection,Mechanisms of Reflection,7,Body Reflection:,Diffuse Reflection,Matte Appearance,Non-Homogeneous Medium,Clay,paper,etc,Surface Reflection:,Specular Reflection,Glossy Appearance,Highlights,Dominant for Metals,Many materials exhibit,both Reflections:,Example Surfaces,8,viewing,direction,surface,element,normal,incident,direction,Lambertian BRDF is simply a constant:,albedo,Surface appears equally bright from ALL directions!(independent of ),Surface Radiance:,Commonly used in Vision and Graphics!,source intensity,source intensity,I,Diffuse Reflection and Lambertian BRDF,9,Diffuse Reflection and Lambertian BRDF,10,CANT perceive the shape of the snow covered terrain!,CAN perceive shape in regions,lit by the street lamp!,WHY?,White-out:Snow and Overcast Skies,11,Assume Lambertian Surface with Albedo=1(no absorption),Assume Sky radiance is constant,Substituting in above Equation:,Radiance of any patch is the same as Sky radiance!(white-out condition),Diffuse Reflection from Uniform Sky,12,source intensity,I,surface,element,normal,incident,direction,viewing,direction,specular/mirror,direction,Mirror BRDF is simply a double-delta function:,Valid for very smooth surfaces.,All incident light energy reflected in a SINGLE direction (only when =).,Surface Radiance:,specular albedo,Specular Reflection and Mirror BRDF,13,Observed Image Color =a x Body Color+b x Specular Reflection Color,R,G,B,Klinker-Shafer-Kanade 1988,Color of Source,(Specular reflection),Color of Surface,(Diffuse/Body Reflection),Does not specify any specific model for,Diffuse/specular reflection,Combing Specular and Diffuse:Dichromatic Reflection,14,Diffuse and Specular Reflection,diffuse,specular,diffuse+specular,15,Photometric Stereo,16,Image Intensity and 3D Geometry,Shading as a cue for shape reconstruction,What is the relation between intensity and shape?,Reflectance Map,17,Surface Normal,surface normal,Equation of plane,or,Let,Surface normal,18,Surface Normal,19,Gradient Space,Normal vector,Source vector,plane is called the Gradient Space(,pq,plane),Every point on it corresponds to a particular surface orientation,20,Reflectance Map,Relates image irradiance,I(x,y),to surface orientation,(p,q),for given source direction and surface reflectance,Lambertian case:,:source brightness,:surface albedo(reflectance),:constant(optical system),Image irradiance:,Let,then,21,Lambertian case,Reflectance Map,(Lambertian),cone of constant,Iso-brightness contour,Reflectance Map,22,Lambertian case,iso-brightness,contour,Note:is maximum when,Reflectance Map,23,Glossy surfaces(Torrance-Sparrow reflectance model),diffuse term,specular term,Diffuse peak,Specular peak,Reflectance Map,24,Shape from a Single Image?,Given a single image of an object with known surface reflectance taken under a known light source,can we recover the shape of the object?,Given,R(p,q),(,(p,S,q,S,),and surface reflectance)can we determine,(p,q),uniquely for each image point?,NO,25,Solution,Take more images,Photometric stereo,Add more constraints,Shape-from-shading,26,Photometric Stereo,27,We can write this in matrix form:,Image irradiance:,Lambertian case:,Photometric Stereo,28,Solving the Equations,inverse,29,More than Three Light Sources,Get better results by using more lights,Least squares solution:,Solve for,as before,Moore-Penrose pseudo inverse,30,Color Images,The case of RGB images,get three sets of equations,one per color channel:,Simple solution:first solve for,n,using one channel,Then substitute known,n,into above equations to get,Or combine three channels and solve for,n,31,Computing light source directions,Trick:place a chrome sphere in the scene,the location of the highlight tells you the source direction,32,For a perfect mirror,light is reflected about,N,Specular Reflection-Recap,We see a highlight when,Then,s,is given as follows:,33,Computing the Light Source Direction,Can compute,N,by studying this figure,Hints:,use this equation:,can measure,c,h,and r in the image,N,r,N,C,H,c,h,Chrome sphere that has a highlight at position,h,in the image,image plane,sphere in 3D,34,Depth from Normals,Get a similar equation for,V,2,Each normal gives us two linear constraints on z,compute z values by solving a matrix equation,V,1,V,2,N,35,积分方法,积分的方法可以分为,局部,方法,(local approach),和,全局,方法,(global approach),两类。局部方法比较容易操作，计算效率也比较高，可是它会传递误差，因此对光度立体得到的法向数据要求很精确。,36,局部和全局积分方法,局部方法,：,Coleman,和,Jain,提出了一种扫描算法，是从点,P,相邻两点的法向矢量来计算,P,点的深度。,Healey,和,Jain,是从,P,点的相邻,8,个点的法向来恢复,P,点的深度的。,Wu,和,Li,用格林理论和多路径积分去恢复相对高度。,最简单的局部积分,方法就是利用梯度从不同路径积分，然后去平均,全局方法,：把表面积分当作一个优化问题。因为他把梯度数据当成全局数据，所以这种算法比较抗噪声。,Ikeuchi,用最小二乘法从针图来估计表面深度。,Horn,、,Frankot and Chellappa,、,Simchony,等也提出了一些相应的全局算法。,37,积分方法的比较,真实形状,38,不同方法积分结果,Possion,方程优化方法对尺度的保持比较好,Frankot Chellapp,频域优化的方法对形态保持的比较好,39,不同方法积分结果,仿射变换方法,加权能量方程,40,Limitations,Big problems,Doesn,t work for shiny things,semi-translucent things,Shadows,inter-reflections,Smaller problems,Camera and lights have to be distant,Calibration requirements,measure light source directions,intensities,camera response function,41,Trick for Handling Shadows,Weight each equation by the pixel brightness:,Gives weighted least-squares matrix equation:,Solve for,as before,42,由多个视图获取表面法线与发射率,一个球面组成的,5,幅图像，是使用正交投影从一个视角得到的。,43,由多个视图获取表面法线与发射率,表面的反射系数,44,由多个视图获取表面法线与发射率,法线向量场,45,由法线获取表面形状,通过积分求形状,46,实际例子,乒乓球图像：,(a)(b)(c)(d),(e)(f)(g)(h),47,实际例子,恢复结果：,48,Original Images,49,Results-Shape,Shallow reconstruction,(effect of interreflections),Accurate reconstruction,(after removing interreflections),50,Results-Albedo,No Shading Information,51,Original Images,52,Results-Shape,53,Results-Albedo,54,Results,Estimate light source directions,Compute surface normals,Compute albedo values,Estimate depth from surface normals,Relight the object(with original texture and uniform albedo),55,成像系统,56,传统的光度立体视觉视觉需要光源从多个方向依次对物体照明，并拍摄多幅图像，才能完成光度立体的计算需要。近些年来，有学者提出了彩色光度立体的概念，即分别用红、绿、蓝三色光从三个方向对物体进行光照，在一幅彩色图像中分别进行三色光的光度计算，进而恢复立体信息。,最新进展,57,最新进展,三色光同时记录的光度信息是彼此分离的，因为照相机对三色光的光度记录是分开的，对于,Bayer,相机来说，虽然没有分别记录，但是,CCD,上每一个像素点的只对一种色光感应，因此也能够进行光度彩色光度立体。,58,进一步学习材料,David A.Forsyth,等著,林学訚 等译,.,计算机视觉,一种现代方法,M.,北京：电子工业出版社，,2004.6.,www.cs.unc.edu/vision/comp256/,Dense Photometric stereo,http,:/www.cse.ust.hk/cktang/sample_pub/dense_ps_pami06.pdf,59,