最优控制问题求解方法综述(中英双语).doc

最优控制问题求解方法综述
Summary of approaches of optimal control problem
摘要：最优控制问题就是依据各种不同的研究对象以及人们预期达到的目的，寻找一个最优控制规律或设计出一个最优控制方案或最优控制系统。解决最优问题的主要方法有变分法、极小值原理和动态规划法，本文重点阐述了各种方法的特点、适应范围、可求解问题的种类和各种方法之间的互相联系。
Abstract：Optimal control problems are to find an optimal control law or design a optimal control program or system according to various kinds of different research objects and the aim people want. The approaches to solve optimal control problems generally contain variational method, the pontryagin minimum principle and dynamic programming. This paper mainly states characteristics, range of application, kinds of the solvable problems of each approach and the association between these three methods.

关键词：最优控制、变分法、极小值、动态规划
Keywords: optimal control , classical variational method , the pontryagin minimum principle , dynamic programming

正文：
最优控制理论是现代控制理论的一个主要分支，着重于研究使控制系统的性能指标实现最优化的基本条件和综合方法。最优控制理论是研究和解决从一切可能的控制方案中寻找最优解的一门学科。它所研究的问题可以概括为：对一个受控的动力学系统或运动过程，从一类允许的控制方案中找出一个最优的控制方案，使系统的运动在由某个初始状态转移到指定的目标状态的同时，其性能指标值为最优。这类问题广泛存在于技术领域或社会问题中。
Optimal control theory is a main branch of modern control theory, which focuses on studying basic conditions and synthetic approaches of optimizing systematic performance index. Optimal control theory is a subject studying and solving for the optimal solution from all possible control solutions. What it study can be summarized in this way: given a manipulated dynamic system or motor process, we are supposed to find a optimal control solution from allowable solutions of the same category, making the systematic movement transfer to the appointed state from a original state and getting a optimal performance index at the same time. And this kind of problems exist in technology field or social problems.
为了解决最优控制问题，必须建立描述受控运动过程的运动方程，给出控制变量的允许取值范围，指定运动过程的初始状态和目标状态，并且规定一个评价运动过程品质优劣的性能指标。通常，性能指标的好坏取决于所选择的控制函数和相应的运动状态。系统的运动状态受到运动方程的约束，而控制函数只能在允许的范围内选取。因此，从数学上看，确定最优控制问题可以表述为：在运动方程和允许控制范围的约束下，对以控制函数和运动状态为变量的性能指标函数（泛函）求取极值（极大值或极小值）。解决最优控制问题的主要方法有古典变分法、极小值原理和动态规划。
For suppose of solving optimal control problems, we must build motion equations describing the manipulated motor process, give allowable value range of the control variables, designate the original state and target state of the motor process and stipulate a performance index to evaluate merits of the quality in the motor process. In general, the merits of a performance index depend on the control function and homologous motion state that we choose. Thus, the optimal control problems can be formulated from mathematical point of view as follows: solving for extremum (maximum or minimum) of the performance index function (functional) based on control function and the motion state under the constraint of motion equation and the allowable control range. The main approaches of solving optimal control problems includes classical variational method, the pontryagin minimum principle and dynamic programming.
一、 变分法
First. Variational method
变分法是处理泛函的数学方法，和处理函数的普通微积分相对，譬如，这样的泛函可以通过未知函数的积分和它的导数来构造。变分法最终寻求的是极值函数：它们使得泛函取得极大或极小值。有些曲线上的经典问题采用这种形式表达：一个经典的例子是最速降线，在重力作用下一个粒子沿着该路径可以在最短时间从点A到达不直接在它地下的一点B。在所有从A到B的曲线中必须极小化的是下降时间的表达式。变分法的关键定理是欧拉——拉格朗日方程，它对应于泛函的临界点。在寻找函数的极大和极小值时，在一个解附近的微小变化的分析给出一阶的一个近似。它不能分辨是找到了最大值或最小值（或者都不是）。
Variational method is a mathematical method to conduct functional, just as the ordinary calculus dealing with functions. For instance, such functional can be constructed by unknown functional calculus and its differential. Variational method aims at findin extreme functions that make functional obtain maximum or minimum. Some classical problems on curved lines always adopt this kind of expression: a classical example is brachistochrone, along which a granule can get to B (not under A directly) from A in the minimum duration under the effect of gravity. In brachistochrone, what is supposed to be minimum is the expression of fall time among all these curved lines from A to B. The key theorem of variational method is Euler——Lagrangian equation, which is correspondent to the functional critical point. While we can’t distinguish the maximum or minimum (or neither) when we are finding the functional extremum through giving a first order approximation of a small change around a solution.
用变分法求解连续系统最优控制问题，实际上就是具有等式约束条件的泛函极值问题，只要把受控系统的数学模型看成是最优轨线应满足的等式约束条件即可。变分法中的三类基本问题：拉格朗日（Lagrange）问题、梅耶（Mayer）问题、波尔扎（Bolza）问题。
Solving the optimal control problems of continuous systems with variational method is the functional extremum problem with conditions of equality constraint.We just need to regard the methematical models of manipulated systems as conditions of equality constraint which the optimal trajectory follows. Three essential problems of variational method: Lagrange problems, Mayer problems and Bolza problems.
但是，变分法作为一种古典的求解最优控制的方法，只有当控制向量不受任何约束，其容许控制集合充满整个m维控制空间，用古典变分法来处理等式约束条件下的最优控制问题才是行之有效的。在许多实际控制问题中，控制函数的取值常常受到封闭性的边界限制，如方向舵只能在两个极限值范围内转动，电动机的力矩只能在正负的最大值范围内产生等。因此，古典变分法对于解决许多重要的实际最优控制问题是无能为力的。
However, as a classical method of solving optimal control problems, variational method can be used to deal with optimal control problems under conditions of equality constraint only when the control vector has no constraint and its allowable control group fills the whole control space of “m” dimensions. While in most practical control problems, values of the control function always are limited by closure boundary. For instance, the rudder can only be twirled in the range between two extremum and the moment of force of electromotor must have a range with a positive maximum and minus minimum. As a result, classical variational method is helpless to solve many important practical optimal control problems.

二、 极小值原理
Second. The pontryagin minimum principle
在古典变分法求解最优控制问题时，假定控制变量不受任何限制，即容许控制集合可以看成整个m维控制空间开集，这时控制变分可以任取，同时还要求哈密尔顿函数H对u连续可微。在这种情况下，应用变分法求解最优控制问题是行之有效的。但是，实际工程问题中，控制变量往往是受到一定限制的，容许控制集合是一个m维有界闭集，这时，控制变分在容许集合边界上就不能任意选取，最优控制的必要条件便不存在了。若最优控制解（如时间最小问题落在控制集的边界一般便不满足，就不能再用古典变分法来求解最优控制问题了。
When solving optimal control problems with classical method, we assume that the control variable has no constraint, meaning that the control set can be regarded as a open set in the whole control space of “m” dimensions, then the control variation is arbitrary, and the Hamilton function is required to be continuously differentiable. In this case, variational method woks well. Nonetheless, the control variation is always restricted in practical engineering problems and the allowable control set is a bounded closed set of “m” dimensions so that the necessary conditions of optimal control don’t exist because the control variation isn’t arbitrary at the boundary of allowable control set. If the optimal control solution (such as minimum time problems) is at the boundary of control set, then we can’t adopt classical variational method to solve optimal control problems.
极小值原理是控制变量受限制的情况下求解最优控制问题的有力工具。极小值原理是由庞德里亚金提出来的，它对于解决受约束的最优控制问题是很有效的。当不受约束时，可以用变分法成功地解决最优控制的求解问题。实际上，一般都是有约束的。当要求在一个m维的密闭集中取值时，变分法就不再适用了。这如同要求闭区间上连续可微函数的极值一样，令其倒数为零，求解时可能无解，但这不是真正意义上的无解，而是解可能出现在边界上。例如，在闭区间上存在最大值和最小值，得不到有关最值的任何信息，问题是最值出现在边界上。与此类似，用变分法求解带有约束的最优控制，有时也是行不通的，因为最优控制往往要求在闭集的边界上取值。
The pontryagin minimum principle is a powerful tool to solve optimal control problems with constraint of control variation. The pontryagin minimum principle is proposed by Pontryagin, and it’s helpful to solve optimal control problems with constraint. Variational method can solve optimal control problems successfully when has no constraint. While actually, generally has constraint. Variational method will be useless when is restricted in a closed set of “m” dimensions. It is just like solving for the extremum of a continuously differential function in a closed interval, when its reciprocal is zero, then there may be no solution if we solve for its solution, yet it does not have no solutions really, but has solution at boundaries. For example, has maximum and minimum in the closed interval and there is no information about extremum because the extremum is at the boundaries. Therewith, sometimes variational method is not able to solve the optimal control problems with constraint because the optimal control is generally obtained at the boundary of the closed set.
极小值原理的突出优点是可用于控制变量受限制的情况，能给出问题最优控制所必须满足的条件。虽然最小值原理为解决带有闭集约束的最优控制问题提供了有效的方法，但遗憾的是它只是一个必要条件。
The obvious advantage of the pontryagin minimum principle is that it is suitable to solve problems with constraint of control variation and can give the conditions which optimal control should follows. Though the pontryagin minimum principle provides valid approaches for solving optimal control problems with constraint in a closed set, it is just a necessary condition.

三、 动态规划
Third. Dynamic programming
动态规划是运筹学的一个分支，是求解决策过程最优化的数学方法。20世纪50年代初美国数学家R.E.Bellman等人在研究多阶段决策过程的优化问题时，提出著名的最优化原理，把多阶段过程转化为一系列单阶段问题，利用各阶段之间的关系，逐个求解，创立了解决这类过程优化问题的新方法——动态规划。
Dynamic programming is a branch of operational research and a mathematical method of solving optimal decision-making process problems. In early 1950s, American mathematicians R.E.Bellman and so on came up with the famous principle of optimality when studying optimal problems of multistage-decision process. They transferred multistage-decision process to a series of single stage problems and solve them one by one making use of the connections between each stage and then found the new method solving this kind of process optimization problems——dynamic programming.
动态规划又称为多级决策理论，是贝尔曼提出的一种非线性规划方法。动态规划的核心是贝尔曼的最优化原理，它将一个多级决策问题化为一系列单级决策问题，从最后一级状态开始到初始状态为止，逆向递推求解最优决策。
Dynamic programming, also named multistage-decision theory, is a nonlinear programming method proposed by R.E.Bellman. The core of dynamic programming is the principle of optimality of R.E.Bellman, which transfers a multistage-decision problem to a series of single stage decision problems and reverse recursion for optimal decision from the final stage state to the original state.
动态规划是求解最优化问题的重要方法，在应用动态规划时，有一个前提条件是系统的状态变量必须满足“无后效性”。因此，在应用动态规划方法时，要注意状态变量的选取，使之满足“无后效性”的条件。例如，讨论物体在空间运动时，不仅选用物体的空间位置作为状态变量，而且要将速度变量也包括在状态变量之内，以便满足“无后效性”的条件。动态规划法的局限性还表现在所谓的“维数灾难”问题：当状态变量的维数增加，要求计算机内存成指数倍增长，计算工作量也大大增加。
Dynamic programming is an important approach solving optimal problems. There is a precondition that state variation of the system must have no aftereffect. Thus, we should pay attention to choose state variations to make sure it has no aftereffect when applying dynamic programming. For instance, when discussing space movement of substance, we should not only choose substance’s space position as state variation, but also take the velocity variation into state variation to ensure it has no aftereffect. The limitation of dynamic programming also manifested as so-called “curse of dimensionality” problem: computer memory is required to increase exponentially as the dimension of state variation increase so that the computational effort greatly increases.
动态规划问世以来，在经济管理、生产调度、工程技术和最优控制方面得到了广泛的应用。例如最短路线、库存管理、资源分配、设备更新、排序、装载等问题，用动态规划方法比用其他方法求解更为方便。
After dynamic programming came out, it has been broadly applied in economic administration, production dispatching, engineering technology and optimal control. Problems like shortcut, inventory management, resource allocation, equipment replacement, ranking, loading and so on, are more easier to solve with dynamic programming than using other methods.
虽然动态规划主要用于求解以时间划分阶段的动态过程的优化问题，但是一些与时间无关的静态规划（如线性规划与非线性规划），主要人为地引进时间因素，把它视为多阶段决策过程，也可以用动态规划方法方便地求解。
Though dynamic programming is mainly used to solve optimal problems of multistage dynamic process divided by time, it also can be used to solve some static state programming (such as linear programming and nonlinear programming) which has nothing to do with the time if we introduce time factor factitiously and regard it as multistage decision process.
动态规划程序设计是对解最优化问题的一种途径、一种方法，而不是一种特殊算法。不像前面所述的那些搜索或数值计算那样，具有一个标准的数学表达式和明确清晰的解题方法。动态规划程序设计往往针对一种最优化问题，由于各种问题的性质不同，确定最优解的条件也互不相同，因而动态规划的设计方法对不同的问题，有各具特色的解题方法，而不存在一种万能的动态规划算法，可以解决各类最优控制问题。因此在学习动态规划时，除了要对基本概念和方法正确理解外，以丰富的想象力去建立模型，用创造性的技巧去求解。我们也可以通过对若干有代表性的问题的动态规划算法进行分析、讨论，逐渐学会并掌握这一设计方法。
Dynamic programming is an approach and a method for solving optimal problems rather than a special arithmetic. It is not like those search or numerical calculation stating above, which have a standard mathematical expression and a clear and definite solution approach. Dynamic programming generally aims at one kind of optimal problems, design approach of dynamic programming has distinctive solution methods for different problems and there is no universal dynamic programming algorithm to solve all kinds of optimal control problems as the conditions of confirming the optimal solution distinct because of their different property. So besides correct understanding of basic conception and method, we should build model with abundant imagination and solve problems with creative skill when learning dynamic programming. We also can gradually grasp this design approach through analyzing and discussing some typical problems with dynamic programming algorithm.

四、三种方法之间的联系与区别
Fourth. The connection and distinction between these three approaches
变分法、最小值原理和动态规划三者都是研究优化问题的，而且也是求解最优控制的有力工具。从数学上来讲，最优就是寻找函数的极值（极小或极大）问题 。动态规划法、极小值原理和变分法，都是求解最优控制问题的重要方法。由动态规划的哈密顿——雅克比方程，可以推得变分法中的欧拉方程和横截条件，也可以推得极小值原理的必要条件。
Variational method, the pontryagin minimum principle and dynamic programming are all studying optimal problems and powerful tools of solving optimal control problems. From the mathematical point of view, optimum is problem to find extremum (minimum or maximum) of functions. Dynamic programming, the pontryagin minimum principle and variational method are all important approaches to solve optimal control problems. From the Hamilton-Jacobi equation in dynamic programming, we can yield the Euler equation and transversal condition in variational method and the necessary conditions in the pontryagin minimum principle.
研究最优控制问题有力的数学工具是变分理论，而经典变分理论只能够解决控制无约束的问题，但是工程实践中的问题大多是控制有约束的问题，因此出现了现代变分理论。现代变分理论中最常用的两种方法是动态规划法和极小值原理。它们都能够很好地解决控制有闭集约束的变分问题。值得指出的是，动态规划法和极小值原理实质上都属于解析法。此外，变分法、线性二次型控制法也属于解决最优控制问题的解析法。最优控制问题的研究方法除了解析法外，还包括数值计算法和梯度型法。
The powerful tool for studying optimal control problems is variational method, while it is only suitable for solving problems with no constraint and most practical engineering problems have constraint of control. Thus, modern variational theory arose. And the frequently-used methods in modern variational theory is dynamic programming and the pontryagin minimum principle, which both can solve variational problems with constraint of control in a closed set well. What should be indicated is that dynamic programming and the pontryagin minimum principle both belong to analytical method essentially. In addition, variational method