吉利在2024GTC上分享2024.pdf

陈勇 博士Dr.Yong Chen吉利汽车 GEELY AUTO驭模有道 智胜未来Guide the model with wisdom surpassing the future大模型助力智能化变革F o u n d a t i o nm o d e lf a c i l i t a t et h ei n t e l l i g e n tt r a n s f o r m a t i o n由粗犷的硬件驱动体验转向数据算法驱动体验Shifting from a raw hardware-driven experience to a data algorithm-driven experience.基于用户体验驱动技术价值创造，使智能化设计回归理性Create technological value driven by user experience,bringing intelligent design back to rationality.技术驱动创新Technology-driven innovation系统集成化System integrationAI算法迭代AI algorithm iteration数据闭环Data Closure Loop大模型Foundation model1R1V3R1V5R5V5R9V5R10V5R11V+1L/3L5R12V+2L5R13V+3L30T100-200T200-500T500-800T1000T+1000+TOPSComputational PowerHighBeginnerMiddle硬件驱动体验Hardware-driven experience数据算法驱动体验Data-driven experience algorithmsBEV去硬件Hardware reduction轻地图Reduce high-precision mapsE2EODD increaseMore大模型发展核心四要素:3+1The four core elements of the development of foundation models:3+1人工智能时代：涌现式+继承式A r t i f i c i a lI n t e l l i g e n c eE r a：E m e r g e n t+I n h e r i t e d大数据平台Big Data Platform星睿智算中心Xingrui Intelligence Computing CenterLLM/Multimodal ModelAI-DRIVE智能驾驶大模型AI-DRIVE Intelligent Driving Foundation Model汽车行业的知识积累Knowledge Accumulation in the Automotive Industry数据Data算力Computational Power算法Algorithm先 验 知 识Previous Knowledge大模型助力智能化变革Foundation models boost the intelligent transformation大语言模型LLM多模态模型Multimodal Model GUI+MUIVUI+NUI代码生成Code Generation摘要生成/知识问答Abstract generation/Knowledge Q&A+LMLM+丰富的生成内容Rich Generative Content全新的交互体验Completely new interactive experience先进的生产力工具Advanced productivity tools新的开发范式New development paradigm百 模 大 战Battle of a Hundred Molds产品需要有市场和价值定位The product needs to have a market and value positioning新技术是来解决问题的或创造价值增量的New technologies are developed to solve problems or to create incremental value.用户场景决定技术价值User scenarios determine the value of technologyWeallneedfoundationmodelsD ow ea l ln e e df o u n d a t i o nm o d e l s?智能驾驶核心要素Key Elements of Intelligent Driving 在复杂道路和拥堵的交通流条件下，接管率高High takeover rate in complex road and congested traffic conditions.智能驾驶体验未实现全驾驶场景覆盖，体验不连贯Incomplete coverage of driving scenarios in intelligent driving experience,leading to inconsistent experiences.大量冗余传感器及技术，系统成本居高不下High system costs due to redundant sensors and technologies.大规模的数据采集标注、软硬件设计开发Large-scale data collection,annotation,software,and hardware design and development.智能驾驶长尾效应带来的安全困境Safety Dilemma Caused by the Long Tail Effect of Intelligent Driving.感知大多数还停留在标注阶段，缺少认识能力Most perception remains in the annotation stage,lacking cognitive ability.安全：安全安全感Safety:Safety Sense of Safety体验：有没有好不好Experience:Presence Quality成本：去冗余去体验Cost:Redundancy Experience数据驱动模型迭代和体验升级Data-drivenmodeliterationandexperienceenhancement智能驾驶大模型应用Applications of the foundation model on autonomous driving解 决 关 键 核 心 问 题 并 创 造 价 值Addressing Key Core Issues and Creating Value数据量不足Insufficient data volume数据采集、标注成本高High costs of data collection and annotation局部优化、无认识博弈Local optimization,no cognitive game数据合成技术Data synthesis technologyAIGC风格迁移AIGC style transfer虚拟资产开发Virtual asset development语义分割 Semantic segmentationFree Space同类聚合 Similarity aggregation数据标签Data labeling视频理解Video understanding自动标注 Automatic annotationE2E大模型技术E2E foundation model technology多感知融合技术（车内外）Multi-perception fusion technology(inside and outside the vehicle)解决安全性和可解释性等问Addressing issues such as safety and interpretability智能驾驶大模型应用Applications of the foundation model on autonomous driving感知行车场景Perception of driving scenarios泊车 场景Parking scenes智能驾驶CornerCaseIntelligent driving CornerCase大模型赋能数据合成技术：两手都要抓 质+量Empowering Data Synthesis Technology with Foundation Model:Grasping Both Quality and Quantity智能驾驶大模型应用Applications of the foundation model on autonomous driving数字孪生Digital Twin场景重建Scene Reconstruction合作伙伴Partnerships平台功能Platform FunctionalityLLM/GAIL/SUMO 第三方标准库Third-party standard library路采数据转换Road data conversion3D场景3D scenes高精地图High-precision maps交通信号控制Traffic signal control环境控制Environment controlAI交通流AI traffic flow标准法规案例场景Standard regulatory scenario cases路采数据驱动Roadside data-driven智驶功能设计案例Intelligent driving function design cases案例泛化Case generalization仿真控制内核Simulation control kernel仿真数据协议层Simulation data protocol layer车辆动力学Vehicle dynamics物理传感器(视觉)Physical sensors(vision)物理传感器(GPS、IMU、Lidar、Radar)Physical sensors数据合成内核Data synthesis kernel云服务器部署Cloud server deployment标注功能模块Annotation function module场景建设Scene Construction仿真内核主车控制静态场景动态场景LLM Sim2Real视觉图像AIGC风格迁移Visual image AIGC style transfer合成数据集传感器数据标注数据Sim2Real图像数据标注数据集非标注真值数据激光点云数据其它传感器数物理传感器模型引擎渲染技术Engine rendering technology将数字孪生技术应用在自动驾驶研发测试上，在虚拟空间中建立物理世界模型，还原真实世界道路场景、交通流，构建元宇宙智驾仿真技术平台，应用车辆动力学建模和物理级传感器建模关键技术和自动标注功能模块，高效合成标注数据，实现自动驾驶算法数据训练，让数据驱动更安全的自动驾驶。Applying digital twin technology to autonomous driving research and testing,establishing physical world models in virtual space,reconstructing real-world road scenes and traffic flow,building a meta-universe intelligent driving simulation technology platform,and applying key technologies such as vehicle dynamics modeling and physical-level sensor modeling and automatic labeling function modules,efficiently synthesizing annotated data,achieving data-driven safer autonomous driving.Physical sensor modelAnnotated datasetSim2Real image dataUnlabeled ground truth dataLaser point cloud dataOther sensor dataAnnotated dataSensor dataMain vehicle controlStatic scenesDynamic scenesSynthetic datasetSimulation kernel智能驾驶大模型应用Applications of the foundation model on autonomous drivingSim2Real风格迁移效果Performance of Sim2Real style transfer technology真实数据real data虚拟数据Virtual data迁移数据Sim2Real data 实验g相对a提升3.17%；加入迁移数据后，16个类别中的15个类别得到提升；Experiment g improved by 3.17%compared to a;after adding the Sim2Real data,15 out of 16 categories were improved.智能驾驶大模型应用Applications of the foundation model on autonomous driving预标注大模型技术框架Pre-labeling foundation model technologyBackbone选用图文多模态模型，大大增强了模型的理解和泛化能力；The backbone adopts the multimodal model of text and images,which greatly enhances the understanding and generalization ability;结合多方数据源，及数据强化策略使模型更好地泛化业务场景;Combining multiple data sources and data enhancement strategies allows the model to fit better in different scenarios;最先进的多尺度特征、去噪训练等策略的引入使得Transformer架构性能优越;The introduction of multi-scale features,denoising training,and other strategies makes the Transformer superior in performance;可同时处理语义分割、物体检测等2D图像感知任务，可实现标注数据互通。It can processing 2D image perception tasks such as semantic segmentation and object detection simultaneously,and realize annotation data interoperability.智能驾驶大模型应用Applications of the foundation model on autonomous driving预标注大模型效果Performance of Pre-labeling foundation model国际数据集International data set吉利某车型production return data以上是在模型在国际数据集Cityscapes、ACDC以及量产回传数据上的推理效果图；The above shows the inference results on international datasets such as Cityscapes,ACDC,and production return data.智能驾驶大模型应用Applications of the foundation model on autonomous driving基于AI大模型与虚拟数据合成技术，吉利汽车在国际知名数据集CityScapes（语义风格的标杆数据集）及ACDC数据集（极端天气场景数据集）上，取得了实时榜单全球第一的成绩。Based on AI foundation model and virtual data synthesis technology,Geely Automobile ink.ranked the first place in the real-time list on the internationally renowned dataset CityScapes and ACDC.CityScapes：语义风格的标杆数据集CityScapes：The benchmark dataset for semantic segmentationACDC：极端天气场景数据集ACDC：Adverse Conditions Dataset with Correspondences智能驾驶大模型应用Applications of the foundation model on autonomous driving大模型赋能数据合成技术Foundation model empowers data synthesis technology 自 然 语 言 对 话+更 多 N a t u r a ll a n g u a g ed i a l o g u e+m o r e.银河E8大模型音乐律动GEELY Galaxy E8 Foundation Model-Musical Rhythm音 乐 不 仅 可 以 被 听 见，还 可 以 被 看 见Musiccannotonlybeheardbutalsoseen智能座舱大模型应用Foundation models applied to intelligent cockpit八大创新 缔造极致智算基座大模型研发底座：吉利星睿智算中心基 于 英 伟 达 算 力 工 具，构 建 全 球 车 企 首 个“云-数-智”一 体 化 超 级 云 计 算 平 台Foundation model R&D base：Geely Xingrui lntelligent Computing CenterBuilding the worlds first cloud digital intelligence integrated super cloud computing platform for automotive companies based on NVIDIAs computing power and toolchainEight major innovations create the ultimate intelligent computing base吉利星睿智算中心1,020,000,000,000,000,000Geely Xingrui lntelligent Computing Center总算力102亿亿次/秒 中国车企第一Total computing power 102 Billions per secondChinas No.1 carmaker大模型研发底座：吉利星睿智算中心基 于 英 伟 达 算 力 工 具，构 建 全 球 车 企 首 个“云-数-智”一 体 化 超 级 云 计 算 平 台Foundation model R&D base：Geely Xingrui lntelligent Computing CenterBuilding the worlds first cloud digital intelligence integrated super cloud computing platform for automotive companies based on NVIDIAs computing power and toolchain“当今企业之间的竞争，不是产品之间的竞争，The competition between companies today is not between products而是商业模式之间的竞争。”彼得 德鲁克but between business models.Peter Drucker人工智能产业竞争不是仅仅是商业模式的竞争The competition in the artificial intelligence industry is not merely a competition of business models更应该关注价值闭环模式：商业价值和学术价值双闭环。but should focus more on the value closed-loop mode:dual closed loops of business value and academic value.人工智能产业将会是产学研协同发展的最佳实践案例 The artificial intelligence industry will be the best practice case of industry-academia-research collaboration.以战养战：不能高估大模型现在的能力War against War:Do not overestimate the current capabilities of foundation models见所未见：更不能低估大模型未来的能力See the unseen:Never underestimate the future capabilities of foundation modelsThanks