虚拟交互式光线追踪电视演播室的应用--google翻译网页.docx

虚拟交互式光线追踪电视演播室的应用
[ PDF ] [ BIBTEX ] [ Reference ] Andreas Pomi and Philipp Slusallek安德烈亚斯Pomi和菲利普Slusallek
Computer Graphics Lab,计算机图形学实验室
Saarland University,萨尔州大学，
Im Stadtwald 36.1, D-66123 Saarbrücken, Germany,进出口Stadtwald 36.1个D - 66123 Saarbrücken，德国，
email: {apomi, slusallek}@graphics.cs.uni-sb.de,电子邮件：{apomi，slusallek} @ graphics.cs.uni - sb.de，
www: http://graphics.cs.uni-sb.de/MR万维网： http://graphics.cs.uni-sb.de/MR
urn:nbn:de:0009-6-2480金塔：NbN纳米：德:0009 - 6 - 2480
Abstract 摘要
In the last years, the well known ray tracing algorithm gained new popularity with the introduction of interactive ray tracing methods.在过去的几年，著名的光线追踪算法获得了新的流行射线追踪方法的引进与互动。 The high modularity and the ability to produce highly realistic images make ray tracing an attractive alternative to raster graphics hardware.高模块化和能力生产高度逼真的图像使光线追踪一个有吸引力的替代光栅图形硬件。
Interactive ray tracing also proved its potential in the field of Mixed Reality rendering and provides novel methods for seamless integration of real and virtual content.交互式光线追踪渲染现实也证明了其潜力的领域中混合，并提供真实的虚拟内容和方法新颖的无缝集成。 Actor insertion methods, a subdomain of Mixed Reality and closely related to virtual television studio techniques, can use ray tracing for achieving high output quality in conjunction with appropriate visual cues like shadows and reflections at interactive frame rates.演员插入方法，一个子域混合和现实密切相关的虚拟电视演播室技术，能利用光线追踪率在互动框架为实现这样的阴影和反射高品质的输出配合适当的视觉线索。
In this paper, we show how interactive ray tracing techniques can provide new ways of implementing future virtual studio applications.在本文中，我们展示了交互光线追踪技术可以应用的新途径提供工作室执行未来虚拟。
Keywords: Mixed Reality, Rendering, Interactive Ray Tracing, OpenRT, Streaming Video Textures, In-Shader Rendering, Actor Insertion, 3D Compositing, Video Billboards, Image-Based Visual Hull, Virtual TV Studio关键词：混合现实，渲染，交互式光线追踪，OpenRT，流式视频素材，在着色渲染，演员插入，3D合成，视频广告牌，基于图像的视觉外型，虚拟电视演播室
Subjects: Ray Tracing, Computer Graphics, Mixed Reality, Realistic Computer Graphics主题：射线追踪，计算机图形学，混合现实，真实的电脑绘图
1. 1。 Introduction简介
Novel approaches to the classic ray tracing algorithm made it possible to achieve interactive frame rates [ Wal04 , WPS03 ].新的方法，以经典的光线跟踪算法使我们能够实现互动帧速率[ Wal04 ， WPS03 ]。 Interactive ray tracing provides new ways of implementing interactive applications that gain from its benefits like modularity and rendering quality.交互式光线跟踪提供互动应用的新途径得到落实，质量和渲染得到任何好处喜欢的模块化。 In application areas like interactive Mixed Reality (MR) rendering, ray tracing can be an alternative to traditional approaches [ PS04 ].在应用领域，如混合现实交互（MR）的渲染，光线跟踪可以是[替代传统的方法PS04 ]。
For convincing rendering results in virtual television studio applications, an appropriate rendering method is needed. It should provide the possibility to easily enhance the composite scene by visual cues.对于结果令人信服的渲染应用虚拟电视演播室，一个合适的渲染方法是必要的。它应该提供的可能性，很容易地提高了视觉线索综合现场。 Visual cues, reflecting the visual interaction of the real actors with the artificial scene, can be provided by shadows, reflection, and occlusion effects.视觉线索，反映了人工视觉场景与演员互动的现实，可以由阴影，反射和遮挡效果。
Interactive ray tracing ideally suits this demands.交互式光线追踪非常适合这一要求。 Its modularity allows for a completely independent implementation of object representation and visual effects.其模块化允许一个完全独立的视觉效果和实施的对象表示。 GPU (Graphics Processing Unit) based approaches often lack this modularity and suffer from restrictions when a combination of rendering methods (eg hardware-supported 3D reconstruction of an actor combined with a shadow casting method and reflections) is needed. GPU（图形处理单元）的方法往往缺乏这种模块化和限制患有相结合的方法时，呈现（如硬件支持的三维重建方法的思考和铸造一个演员结合了阴影）是必要的。
Ray tracing also facilitates the combination process of the actor and the virtual scene (compositing).光线跟踪也促进合成）的结合过程中的演员与虚拟场景（。 The demand-driven concept of ray tracing in combination with the unrestricted shading possibilities allows new ways for implementing compositing.阴影的可能性无限制的需求与概念驱动的射线追踪组合允许实施合成新途径。
The concept of in-shader compositing moves compositing from a post-process directly into the shading calculations and thus allows the closest visual interaction between actor and scene.着色器概念中，从合成动作合成后处理计算直接进入阴影，从而使演员和场景之间的互动最亲密的视觉。
In the remainder of this paper, we give an overview of the state-of-the-art in interactive, distributed ray tracing within the field of MR rendering.在本论文的其余部分，我们给出一个互动概述国家最先进的在-，分布在光线追踪渲染先生的领域。 The advantages of using ray tracing for virtual studio applications are pointed out and ray tracing based methods (video billboards and a 3D reconstruction shader based on visual hull rendering) for seamless actor insertion into virtual scenes are described.演播室应用虚拟的优势为用射线跟踪，指出了和射线追踪的方法（视频广告牌和着色渲染三维重建基于视觉船体）的演员到虚拟场景的无缝插入描述。
Figure 1. 图1。 A typical setup for an OpenRT interactive ray tracing system with streaming video textures. The video texture servers stream the video content via multicast networking to the rendering client cluster. 一个典型的设置，纹理视频流OpenRT一个具有交互光线追踪系统。视频纹理流的客户端服务器群集通过多播网络视频内容的呈现。 The OpenRT server runs the rendering application, accomplishes texture synchronization and provides the video output. 该OpenRT应用服务器上运行的渲染，纹理同步完成，并提供视频输出。

2. 2。 Interactive Ray Tracing交互式光线追踪
There are several approaches to interactive ray tracing: Software ray tracing has been accelerated by new algorithmic approaches by more than a factor of 30 [ Wal04 ] on a single CPU.有几种方法射线追踪互动：软件射线追踪方法加速了新算法由[30以上因素Wal04 ]在一个单一的CPU。 Ray tracing easily parallelizes linearly on shared-memory machines as well as on PC clusters. The OpenRT system [ WD03 ] is an example of such an interactive ray tracing framework.线性并行射线跟踪轻松共享内存机以及基于PC集群。系统的OpenRT [ WD03 ]是一个框架的例子这种交互式光线追踪。
Another approach would be to use the programmability of modern raster graphics chips (GPUs).另一种方法是使用）可编程现代光栅图形芯片（GPU的。 However, this approach is restricted by the limited programming model [ PBMH02 ].但是，这种方法是受制于有限的编程模型[ PBMH02 ]。 A better alternative is provided by a graphics hardware architecture especially designed to perform ray tracing, like the SaarCOR [ SWS02 , WSS05 ] system.提供一个更好的选择是由一个图形硬件[建筑设计来完成，特别是光SaarCOR追踪，如SWS02 ， WSS05 ]体系。
Software ray tracing offers a number of advantages over traditional, GPU based rendering techniques [ MH99 ].软件提供了基于射线追踪[渲染技术的数字比传统的优势，GPU的MH99 ]。 The high modularity of a ray tracer allows to combine rendering effects in a very straightforward way.高度模块化的射线追踪的渲染相结合的方式允许在一个非常直接的影响。 Resource conflicts and limitations as experienced with GPUs do not occur.资源冲突和经验的局限性与GPU不发生。 Since shading is performed in software, arbitrary complex shading algorithms (eg global illumination methods [ BWS03 ]) can be used.由于遮阳是软件执行中，任意复杂的着色算法（如全局照明方法[ BWS03 ]）都可以使用。 The modularity also allows to use other geometry representations than triangles like spline surfaces (eg [ BWS04 ]) or volumetric representations (eg [ MFK04 ]).该模块还允许使用（如[声明以外的其他表面几何形状像三角形样条BWS04 ]）或如[体积申述（ MFK04 ]）。 The image-based visual hull shader presented below is an example for a non-triangle data representation.在图像的视觉呈现船体着色下面是一个代表性的例子为一个非三角形数据。
Ray tracing is output-sensitive, ie the computing power that is necessary to render an output frame depends mainly on the frame content.光线跟踪输出敏感，即计算能力是必要的渲染输出帧的内容，主要取决于框架。 Expensive shading calculations need only be computed for the visible parts of a scene.昂贵的阴影计算只需要计算的一个场景中可见部分。 Ray tracing is thus a demand-driven algorithm.因此，光线跟踪需求驱动的算法。
Another advantage of using ray tracing is its ability to handle far more complex models than are possible with raster graphics methods.另一种利用射线追踪优势是它能够处理更复杂的模型可能比一些方法与光栅图形。 Compared to the linear GPU approach, ray tracing scales logarithmically in the number of triangles [ Gla95 , Hav00 ].相较于线性GPU的方法，光线追踪三角形[尺度的对数的数量Gla95 ， Hav00 ]。 Highly detailed models with more than 10 9 triangles can be rendered even with soft shadows [ DWS04 ]. 10 9三角形非常详细的车型多，甚至可以提供软阴影[ DWS04 ]。
2.1. 2.1。 The OpenRT Framework该OpenRT框架
Since the power of a typical PC is often not sufficient for high-performance applications, it is necessary to combine the processing capabilities of several hosts.由于PC电源的典型应用，往往不足够高的性能，必须要结合多个主机的处理能力。 OpenRT [ DWBS03 , WD03 ] provides a framework for distributed interactive ray tracing. OpenRT [ DWBS03 ， WD03 ]提供了一个跟踪框架的分布式交互式光线。 The OpenRT library features an interface very similar to OpenGL and hides the distribution and networking aspects from the application.该OpenRT图书馆的特点是界面非常相似，OpenGL和隐藏网络方面，从应用程序的分布和。
The host with the rendering application is referred to as the OpenRT server.应用程序主机与渲染称为OpenRT服务器。 A number of rendering clients (a cluster) provide the necessary computing power.一个客户数量呈现（一组）提供必要的计算能力。 Beside the application API, OpenRT also features a shader programming API for surface, light-source, and camera shaders.除了应用程序API，OpenRT还具有和照相机着色着色器编程的API表面光源。
In [ PMWS03 ], we introduced two extensions to OpenRT to support interactive Mixed Reality rendering methods.在[ PMWS03 ]中，我们引入了两个扩展OpenRT支持混合现实交互渲染方法。 Streaming video textures allow for using live video in the shading process.流媒体视频纹理着色工艺允许使用的实况录像。 Since shading is performed on the clients, the video content is streamed by dedicated texture servers (see Figure 1 ).由于遮阳是客户端上执行时，视频内容是由专门的纹理流服务器（见图1 ）。 Multicast networking [ Ste98 ] and a synchronization mechanism ensure scalability in the number of hosts. Figure 2 shows how a virtual scene can be enhanced by live video.多播网络[ Ste98 ]和可扩展性的主机同步机制确保在人数。图2显示了如何在虚拟场景可以增强视频直播。
Figure 2. 图2。 A streaming video texture used on the screen of a TV set. 流媒体视频纹理用于设置屏幕电视。 The video content is also used for dynamic lighting of the room. 视频内容也可用于照明的房间动态。

The second extension is an Augmented Reality view compositing method [ PS04 ].第二个扩展是一个增强现实的看法合成方法[ PS04 ]。 Here a live video background is streamed to the rendering clients and included into the rendering process (in-shader compositing, [ Pom05 ]).在这里，一个实时视频传输到后台渲染客户和[包括进入渲染过程（着色合成， Pom05 ]）。 This enables the easy implementation of differential rendering methods [ Deb98 ] for achieving convincing visual cues.这使得渲染方法易于实施的差别[ Deb98线索]实现有说服力的视觉效果。 Both extensions are implemented as plugins to OpenRT without changes to the core system.这两个扩展的插件来实现为OpenRT没有系统变化的核心。
2.2. 2.2。 Interactive Ray Tracing for Virtual Studio Applications交互式光线跟踪的虚拟演播室的应用
Interactive ray tracing can provide a number of advantages over traditional GPU-based rendering for virtual studio applications.交互式光线追踪可以提供一个工作室的申请数目较传统GPU的虚拟优势为基础的渲染。 The high visual quality can be used to provide visual cues of the positions of the inserted persons.高品质的视觉可以用来插入人士提供视觉线索的位置。 Lighting effects can be used to cast shadows and reflections of the real actor onto the virtual objects.灯光效果，可用于实物演员投上阴影和反射的虚拟。 Live video can be included into the scene for implementing virtual video displays.可实时视频显示列入实施虚拟视频场景。
There are also a number of technical advantages: Ray tracing inherently supports video field rendering, ie the necessary computing power is exactly the same for rendering 25 frames per second or 50 interlaced fields [ Poy03 ].也有一些优势技术：光线跟踪渲染本身支持视频领域，即计算能力是完全必要的[相同的每秒25帧的渲染或隔行扫描场50 Poy03 ]。
The software frame-buffer of a ray tracer allows easy connection to professional video frame-buffer boards that support the demands of a video studio infrastructure like video synchronization and digital video (SDI) outputs.该软件帧缓冲示踪光线可以方便地连接到专业视频帧缓冲板支持）输出的要求视频（SDI的视频工作室的基础设施，如视频同步和数字化。
Ray tracing also supports easy implementation of arbitrary camera models (eg via the OpenRT camera shader).光线跟踪也支持Shader）轻松实现任意摄像机OpenRT模型（例如，通过摄像头。 This makes the matching of studio cameras to the virtual camera easily possible.这使得相机匹配的虚拟演播室摄像机可能易行。 Camera effects like depth-of-field can be correctly rendered and need not be approximated [ WGDD98 ].场相机的效果，如景深可以正确渲染，不需要近似[ WGDD98 ]。
Figure 3. 图3。 An OpenRT video billboard application. 一个OpenRT视频广告牌的申请。 The video image of a person in front of a greenscreen is streamed as a video texture. 在一个绿屏视频图像的前一个人的质地作为视频流。 The scene description contains two billboard rectangles with an associated video billboard shader. Chroma keying is performed in-shader. 现场描述包含两个长方形广告牌广告牌着色与相关视频。色度键控着色演出，。 The resulting video output shows two persons inserted into the scene. 由此产生的视频输出显示了两个人到现场插入。 Lighting effects and reflections provide the necessary visual cues. 灯光效果和思考提供必要的视觉线索。

2.3. 2.3。 The Concept of In-Shader Compositing着色器概念中，合成
The technique of combining two images (or video streams) into a single image is referred to as compositing [ Bri99 , Kel00 ].在图像技术或视频流）合并成一个单一的两幅图像（被称为合成[ Bri99 ， Kel00 ]。 Traditional compositing methods are based on the formal 2D image compositing algebra introduced by Porter and Duff [ PD84 ] and are implemented as a post-process.传统的合成方法是基于正式二维[图像合成代数介绍了波特和达夫PD84 ]过程中实施，并作为一个后。
The compositing process is controlled by auxiliary information, for example an alpha-channel or depth information.在合成过程由辅助信息例如，对于一个alpha通道或深入的信息。
A traditional video compositing system for virtual studio purposes would thus consist of a live video input from the studio, showing the actor in front of a background that can be used to generate the auxiliary information, eg by chroma keying. The second image is provided by a renderer, and in a 2D post-process the compositing of both images is performed by special hardware (a video keyer [ Poy03 ]).传统的视频合成为虚拟演播室系统的目的将因此包括从演播室现场视频输入显示色度键控演员在前面的一个背景，可用于生成如辅助信息。第二个图像是由一个渲染器，并在二维后处理的硬件合成两种特殊的图像执行（视频键控器[ Poy03 ]）。
The concept of in-shader compositing [ Pom05 ] provides an alternative to the traditional approach: the compositing is performed directly inside the shading process.在[概念中合成着色Pom05 ]提供了一种替代传统的方法：在合成过程中直接执行里面的阴影。 Here, the full 3D information is still available (eg occlusion/depth, surface normals, etc.).在这里，全三维信息仍然是可用（例如闭塞/深度，表面法线等）。 There is no need to carry additional auxiliary information to a post-process.没有必要进行额外的辅助信息到后处理。 This ideally suits the demands of a ray tracing framework.这非常适合跟踪框架要求的光线。 As a byproduct, the compositing is performed on demand since a ray tracer is demand-driven.作为副产品，在合成上执行的需求，因为光线追踪是需求驱动的。 The actor insertion methods presented in the following section can be seen as applications of in-shader compositing.插入部分演员在下列方法可以看作是合成应用在着色。
3. 3。 The Actor Insertion Problem插入的演员问题
The basic rendering problem of virtual studios can be referred to as the actor insertion problem: one ore more persons (actors) are placed into a virtual scenario.虚拟演播室的基本渲染问题可以称为演员插入的问题：一矿多的人（演员）的情景放置到一个虚拟的。 This is a special case of the general compositing problem of combining virtual and synthetic parts.这是一种人工合成的部分特殊案件的问题和一般合成相结合的虚拟。
For a convincing result, it is necessary to provide the feeling of interaction between the virtual and synthetic components.对于一个令人信服的结果，它需要提供虚拟元件和合成的感觉之间的相互作用。 The composite image is not accepted by the audience if it lacks visual cues of interaction.复合图像不接受观众的互动，如果它缺乏视觉线索。 These cues can for example be shadows, cast by the actor onto the virtual scene or a shiny floor, showing a glossy reflection of the person.例如可以将这些线索阴影，由演员演员在虚拟场景或地板光泽，呈现出光泽反射的人。 Also occlusion (depth) of the actor by scene parts plays an important role, especially when walking around.此外闭