Abstract:
We consider the classical Bolza problem under fairly general assumptions on its components (the integrand and off-integral function).
The main results obtained are: conditions for the semicontinuity of the value functions, a characterization of the subdifferential of the value function and a partial conversion of the latter result.
Bibliography: 15 titles.
Keywords:
value function, Hamiltonian, subdifferential, convex function, semicontinuity, pseudo-Lipschitz mapping.
but under substantially more general assumptions on $g$ and $L$.
It is an easy matter to see that, under these assumptions, the problem includes many optimal control problems (see, for example, [7]). Apart from the simplicity of formulation, this is the main factor that makes the problem quite attractive. The term ‘generalized Bolza problem’ was introduced by Clarke [8], although the problem of the minimization of $J$ with the integrand $L$ convex in $(x,y)$, but otherwise under assumptions substantially weaker than in the classical calculus of variations was previously considered by Rockafellar [12].
As follows from the title, our work aims at extending the Hamilton–Jacobi theory to such problems under fairly general assumptions on $L$ and $g$. Specifically, we assume that
In the course of further discussions we can add more assumptions on $L$.
We must observe that the generalization of the Hamilton–Jacobi theory to a certain class of optimal control problems was initiated by Vinter (see [15] and [1]). A key assumption in those works was that the set of feasible controls was bounded. This excludes possible applications of results to problems in the calculus of variations. It seems that assumptions similar to ours were for the first time made by Rockafellar [13] and Clarke [8] (although a similar convex problem was considered by Rockafellar a few years earlier [12]). The first of these papers was basically devoted to the existence of solutions, while the second — to necessary conditions for a minimum. The main result in [13] was a theorem on the lower semicontinuity of $J$ in the weak topology of $W^{1,1}$. This is a key property in our paper too. It must also be said that sufficient conditions for lower semicontinuity of the functional were first obtained by Olech [10] in 1976. Necessary and sufficient conditions for this property were obtained by this author in [3], shortly after the appearance of Rockaffelar’s work.
I dedicate this paper to V.M. Tikhomirov, my teacher and then my closest friend for almost 60 years.
§ 2. Preliminary results
2.1. Integral functionals
To begin with, we consider the simplest integral functional with variable lower limit
A proof of the following result can be found in [7], § 9.1, Theorem 1.
Proposition 2.1. Consider a function $\varphi(t,x)$ on $[0,T]\times\mathbb{R}^n$ that is measurable with respect to the $\sigma$-algebra generated by the products $\Delta\times Q$, where $\Delta$ is a measurable subset of $[0,T]$ and $Q$ is a Borel subset of $\mathbb{R}^n$. If $\varphi$ satisfies the growth condition, then for each $C\in \mathbb{R}$ the set
coincides with $\displaystyle\int_0^T\varphi^*(t,p)\,dt$ (see [7], § 8.3, Theorem 2), no matter whether or not $\varphi$ satisfies the growth condition. However, as follows from Proposition 2.1, if $\varphi$ satisfies the growth condition, then the function $x(\,{\cdot}\,)\to \displaystyle\int_0^T\varphi(t,x(t))\,dt$ is bounded below on $L^1$ and assumes a finite value at least at one point. In what follows we assume that the function $\varphi(t,0)$ is summable. A slightly more precise version of Proposition 2.1 is stated below.
Proposition 2.2. Again, let $\varphi (t,x)$ satisfy the assumptions of Proposition 2.1. Also let $\beta(\,{\cdot}\,)$ be a nonnegative summable function on $[0,T]$. Then for any $C\in \mathbb{R}$ and $r\geqslant 0$ the set
and the ball of radius $k$ around $p$ lies in the convex hull of the $2n$ points $p+\pm k\sqrt{n}\,e_i$, where the $e_i$ are elements of some orthonormal basis of $\mathbb{R}^n$. Now the proof follows easily from Proposition 2.1. The proposition is proved.
In what follows we assume that the following condition is satisfied:
on the set $X(t,x)$ of absolutely continuous functions on $[t,T]$ such that $x(t)=x$.
Proposition 2.3. In addition to conditions ($\mathrm A_1$)–($\mathrm A_3$), assume that $L(t,x,\,\cdot\,)$ is a convex function for all $t$ and $x$. Then for any $t$ the functional $J_t$ is lower semicontinuous in the weak topology of the space $W^{1,1}[t,T]$ and attains its minimum on every nonempty set $X(t,x)$.
Proof. By ($\mathrm A_1$) the first statement holds true if the functional
is lower semicontinuous. But this is a well-known fact (see, for instance, [8] or [3], Theorem 5). In turn, the second statement follows from the first and Proposition 2.2.
The proposition is proved.
We must emphasize that the convexity of $L(t,x,\,\cdot\,)$ is a necessary condition for the last statement to hold. The verification of this fact is elementary. On the other hand, under very weak assumptions the lower closure of $I_t$ (that is, the functional whose epigraph is the closure of the epigraph of $I_t$) coincides with the integral functional associated with the convexification of $L(t,x,\,\cdot\,)$.
2.2. Subdifferentials
There are several types of subdifferentials studied in nonsmooth analysis. We need the simplest one known as the Dini–Hadamard subdifferential or directional subdifferential [5], [11]. So let $X$ be a Banach space and $f$ a function on $X$ which is finite at $x\in X$. The subdifferential we need (of $f$ at $x$) is the set $\partial^{-}f(x)$ of linear functionals $x^*\in X^*$ such that
It immediately follows from the definition that the Dini–Hadamard subdifferential (if it is nonempty) is a convex set closed in the weak$^*$ topology of the dual space. We need the following properties of the Dini–Hadamard subdifferential:
Consider a set-valued mapping $Q(t)$ from $[0,T]$ to $\mathbb{R}^n$, and let $Q$ be the graph of $Q(t)$.
Definition 2.4. $L$ is pseudo-Lipschitz on $Q$ if there are $\beta>0$ and a strictly positive summable function $k(t)$ such that for any $t\in [0,T]$ and $x\in Q(t)$ and any $y\in \mathbb{R}^n$ for which $L(t,x,y)<\infty$ there exist $x'\in Q(t)$ and $y'$ such that
The latter is, in particular, true if for all $t\in [0,T]$ and $y$ the function $L(t,\,{\cdot}\,,y)$ is $(k(t)+\beta|y|)$-Lipschitz on $Q(t)$.
An easily verifiable consequence of this property is that the set-valued mapping $(t,x)\to F(t,x):= \operatorname{epi} L(t,x,\,\cdot\,)$ has the global pseudo-Lipschitz property introduced in [4]: if $x,x'\in Q(t)$, then the inclusion
holds for all $N>0$. (An earlier version of this property was introduced in [9].) Here $\operatorname{epi} f=\{(\alpha,u)\colon \alpha\geqslant f(u)\}$ is the epigraph of the function $f$.
The origins of the following result (sufficiently well known for the standard problem of the classical calculus of variations) go back to the 1930 work of Bogolyubov [2].
Proposition 2.5. Assume that $L$ satisfies ($\mathrm A_2$) and ($\mathrm A_3$) and is, in addition, pseudo-Lipschitz in a neighbourhood of the graph of a mapping $x(\,{\cdot}\,)\in W^{1,1}$. Let $\widehat L(t,x,\,\cdot\,)$ be the convexification of $L(t,x,\,\cdot\,)$, and let
Then for each $t$ there is a sequence of $x_m(\,{\cdot}\,)\in X(t,x)$ (that is, such that $x_m(t)=x(t)$) weakly converging to $x(\,{\cdot}\,)$ and such that
Proof. Our proof follows basically the proof of Theorem 4.1 in [4]. The theorem itself cannot be used here because of an additional assumption (($\mathrm A_4$) in [4]) which is not needed here.
Clearly, it is sufficient to prove the proposition only for $t=0$. Given any $\varepsilon>0$, we can find measurable $y_i(\,{\cdot}\,)$ and $\alpha_i(\,{\cdot}\,)$, $i=1,\dots,n+1$, such that for each $t$
Now we can find a set $\Delta\subset [0,T]$ with measure not smaller than $T-\varepsilon$ and such that the values of the functions $|y_i(t)|$ and $L(t,x(t),\dot x(t))$ do not exceed certain $\rho$ almost everywhere on $\Delta$. Once $\Delta$ is chosen, we can modify the $y_i(\,{\cdot}\,)$ outside $\Delta$ by setting them equal to $\dot x(t)$. Finally, we can find sequences of functions $\alpha_{im}(\,{\cdot}\,)$ assuming values $0$ and $1$, satisfying $\sum_i\alpha_{im}(t)=1$ almost everywhere and converging weakly in $L^{\infty}$ to functions coinciding with $\alpha_i(\cdot)$ on $\Delta$ and equal to $(n+1)^{-1}$ outside $\Delta$. It is clear that the functions $z_m(\,{\cdot}\,)$ defined by
converge to $x(\,{\cdot}\,)$ in the weak topology of $W^{1,1}$ and therefore uniformly.
As $L$ is pseudo-Lipschitz near $x(\,{\cdot}\,)$, the distance of $(y_i(t),L(t,x(t),y_i(t)))$ to $\operatorname{epi} L(t,z_m(t),\,\cdot\,)$ (as a function of $t$) tends to zero in the $L^1$-metric. By Theorem 3.1 in [4] (for $z_m(\,{\cdot}\,)$ playing the role of $x(\,{\cdot}\,)$) there exist $a_m(\,{\cdot}\,)\in L^1$ and $x_m(\,{\cdot}\,)\in W^{1,1}$ such that $a_m(t)\geqslant L(t,x_m(t),\dot x_m(t))$ almost everywhere and $a_m(\,{\cdot}\,)- L(\,\cdot\,,x(\,{\cdot}\,),\dot z_m(\,{\cdot}\,))\to 0$ and $x_m(\,{\cdot}\,)-z_m(\,{\cdot}\,)\to 0$ in $L^1$ and $W^{1,1}$, respectively. This completes the proof of the proposition.
on the set $X(t,x)$ of absolutely continuous functions on $[t,T]$ such that $x(t)=x$. Let $V(t,x)$ be the value function, that is, the minimum value of the functional in the problem. By Proposition 2.5 the value function in the problem associated with the convexified integrand $\widehat L$ coincides with $V$, at least if $L$ is pseudo-Lipscitz on $\mathbb{R}^n$. Hence without any loss of generality we may assume in what follows that $L(t,x,\,\cdot\,)$ is a convex function for all $t$ and $x$.
Theorem 3.1. In addition to ($\mathrm A_1$)–($\mathrm A_3$), assume that $L(t,x,\,\cdot\,)$ is a convex function. Then the value function $V(t,x)$ is lower semicontinuous.
Proof. Note first that by ($\mathrm A_3$) the inequality
holds for all $t$, $x$ and $y$, and the function $\varphi^*(t,0)$ is summable as $\varphi$ satisfies the growth condition. By Proposition 2.3, for any $t\in [0,T]$ the functional $J_t$ is lower semicontinuous in the weak topology of $W^{1,1}$. Furthermore, since the function $t\to \varphi^*(t,p)$ is summable for any $p$, the function $\varphi(s,y(s))$ must be summable for some $y(\,{\cdot}\,)\in L^1$. This follows from the simple fact mentioned above that the function $\displaystyle p\to \int \varphi^*(t,p)\,dt$ is conjugate to
We may assume without any loss of generality that the function $\varphi(s,0)$ is summable.
Let $(t_i,x_i)\to (t,x)$. If $V(t_i,x_i)\to \infty$, then, of course, $\liminf V(t_i,x_i)\geqslant V(t,x)$. So we may assume that $V(t_i,x_i)\leqslant C<\infty$. By Proposition 2.3, $J_{t_i}$ attains its minimum on $X(t_i,x_i)$ at some $x_i(\,{\cdot}\,)$, that is $J_{t_i}(x_i(\,{\cdot}\,))= V(t_i,x_i)$. The sequence $\{g_i(x_i(T))\}$ is bounded, which is immediate from Proposition 2.3. Setting $x_i(s)= x_i(t_i)$ for $s\in [0,t_i]$, we extend every $x_i(\,{\cdot}\,)$ to the whole of $[0,T]$. Then the integrals
are uniformly bounded, so that the sequence $(x_i(\,{\cdot}\,))$ is weakly compact in $W^{1,1}$. Therefore, we can assume that this sequence converges weakly in $W^{1,1}$ to some $x(\,{\cdot}\,)$. We can also be sure that the functions $x_i(\,{\cdot}\,)$ are uniformly bounded, that is, there is $r>0$ such that $|x_i(s)|\leqslant r$ for all $s$.
Consider first the case when $t_i\leqslant t$ for all $i$. In this case
The first term in the right side of this inequality is nonnegative. This follows from (3) and the fact that the $x_i(\,{\cdot}\,)$ are uniformly bounded. Now the inequality $\liminf J_{t_i}(x_i(\,{\cdot}\,))\geqslant J_t(x(\,{\cdot}\,))$ follows from Proposition 2.3.
Now let $t_i>t$ for all $i$. By Proposition 2.3, in this case
for any $\tau>t$. So to prove the inequality $\lim_{\tau\searrow t} J_{\tau}(x(\,{\cdot}\,))\geqslant J_{t}(x(\,{\cdot}\,))$ we only need to refer to Fatou’s lemma, having in mind (3) along with the fact that $x(\,{\cdot}\,)$ is a bounded mapping. This completes the proof of the theorem.
called the Hamiltonian of $L$. In our case $H(t,x,\,\cdot\,)$ is everywhere finite for all $t$ and $x$. This follows from the obvious inequality $H(t,x,p)\leqslant\varphi^*(t,p)+\beta|x|$, which, in turn, immediately follows from (3).
Theorem 3.2. Assume that ($\mathrm A_1$) and ($\mathrm A_2$) hold. Also assume that $L(t,x,\,\cdot\,)$ is convex for all $t$ and $x$, and let $(\alpha,p)\in\partial^{-} V(t,x)$.
(a) If $L(\,{\cdot}\,{,}\,{\cdot}\,,u)$ is continuous at $(t,x)$ from the left in the sense that $L(s,w,u)\to L(t,x,u)$ as $(s,w)\to (t,x)$, $s<t$ (where the case $L(t,x,u)=\infty$ is not ruled out), then
where $r(\lambda)\to 0$ as $\lambda\to 0$. Since $L(\,{\cdot}\,{,}\,{\cdot}\,,u)$ is a continuous function, we get that $-\alpha\leqslant L(t,x,u)- \langle -p,u\rangle$. Hence
(b) Now let $0\leqslant t<T$. We choose some $\overline x(\,{\cdot}\,)$ defined on $[t,T]$ such that $\overline{x}(t)=x$ and $V(t,x)= J_t(\overline{x}(\,{\cdot}\,))$. By Proposition 2.3 such a function $\overline{x}(\,{\cdot}\,)$ does exist. Set $\overline{u}(s)= \overline{x}(t+s) - \overline{x}(t)$. Then for any sufficiently small $\lambda>0$
(where $r(\lambda)\to 0$ as $\lambda\to 0$), which immediately implies the required inequality.
The theorem is proved.
§ 4. Characterization of the value function
In this section we plan to prove a result partially converse to Theorem 3.2 in the case when $L$ does not depend on $t$. Assume first that a set-valued mapping $\mathcal F\colon \mathbb R^n\rightrightarrows \mathbb R^m$ is fixed. Set
Lemma 4.1 ([6], Proposition 4.8). Let $\mathcal{F}$ be a convex-valued mapping, and let $\overline{y}\in\mathcal{F}(\overline{x})$. Also assume that there are $\varepsilon>0$ and a nondecreasing strictly positive function $\rho(t)$ on $\mathbb{R}_+$ such that the relation
holds for any $x,x'\in B(\overline{x},\varepsilon)$ and $N>0$.
If the set $\mathcal{U}(\overline{x},\overline{q})$ is nonempty and bounded, that is to say, there exists $r>0$ such that $\mathcal{U}(\overline{x},\overline{q})\subset B(\overline{y},r)$, then for any $\eta>0$ there exists $\delta>0$ such that $|y-\overline{y}|\leqslant r+\eta$ whenever $y\in \mathcal{U}(x,q)$, provided that $|x-\overline{x}|\leqslant\delta$ and $|q-\overline{q}|<\delta$. In other words, $\mathcal{H}$ is a Lipschitz mapping in a neighbourhood of $(\overline{x},\overline{q})$.
The following assumption allows us to apply this lemma to our problem:
and the norms of $x$, $x'$ and $y$ do not exceed $m$. Note that, unlike in ($\mathrm A_4$), if this condition is satisfied, then the domains of definition of the mappings $F(x,\,\cdot\,)$ are independent of $x$.
If $L$ is a convex function of the last argument, then applying the lemma to
Corollary 4.2. Assume that ($\mathrm A_1$)–($\mathrm A_4$) hold. Then $H(\cdot,\,\cdot\,)$ satisfies the Lipschitz condition in a neighbourhood of any point $(x,p)$ at which it is finite.
Proof. By definition $\mathcal{F}$ is a set-valued mapping from $\mathbb{R}^n$ into $\mathbb{R}^{n+1}$ whose values are convex closed sets. Set $u= (\alpha,y)$ and $q=(\xi,p)$. Then $\xi\leqslant 0$ if $\mathcal{H}(x,q)<\infty$, and for $\xi=-1$ we obtain
However, $|y|^{-1}L(x,y)\to \infty$ as $|y|\to\infty$ by ($\mathrm A_3$). Hence $\mathcal{U}(x,q)$ is a bounded set and, as follows from (4), $\mathcal{F}(x')\subset \mathcal{F} (x) + k|x-x'|B$ for some $k>0$.
The corollary is proved.
The main result in this section is contained in the following statement.
Theorem 4.3. Let conditions ($\mathrm A_1$)–($\mathrm A_4$) be satisfied, and let $\varphi(t,x)$ be a lower semicontinuous function on $[0,T]\times\mathbb{R}^n$ with values in $(-\infty,\infty]$ and such that
holds for all absolutely continuous mappings $x(\,{\cdot}\,)\colon [t,T]\to \mathbb{R}^n$ with bounded derivative.
Proof. It is obviously sufficient to prove the theorem only for $t=0$. Note also that $\varphi(t,\,\cdot\,)$ is differentiable almost everywhere on $B(\overline{x}(t),\rho)$.
Let $\mathcal{N}_m$ be the system of functions $\eta(t,x)\geqslant 0$ on $[0,T]\times\mathbb{R}^n$ that are measurable in $t$, continuously differentiable with respect to $x$ and such that for each $t$
the functions $\psi_m(t,x)$ are continuously differentiable and converge uniformly to $\varphi(t,x)$ on compact sets Here $\psi_m'(t,x)$ is the derivative of the function $\psi_m(t,\,\cdot\,)$ at $x$. Note that by our assumptions $H(x,\varphi'(t,x))\leqslant 0$ whenever $(\lambda,\varphi'(t,x))\in\partial^{-}\varphi(t,x)$.
According to our assumptions, the inequality $|\varphi'(t,x)|\leqslant k(t)$ holds for any $t\in [0,T]$ and $x\in B(\overline{x}(t),\rho)$ such that $\varphi(t,\,\cdot\,)$ is differentiable at $x$. As $H$ is a locally Lipschitz function (Corollary 4.2), we can be sure that for any $t$ there exists $k_1(t)>0$ such that $H$ satisfies the Lipschitz condition with constant $k_1(t)$ on the set of pairs $(x,p)$ such that $x\in B(\overline x(t),\rho)$ and $|p|\leqslant rk(t)$.
Note next that every vector $\psi'((t,x))$ belongs to the closed convex hull of the set $\{\varphi'(t,x-w)\}$, $|w|\leqslant 1/m$. In other words, for any $t$ and any $x\in B(\overline x,\rho)$ there exist $w_i$ satisfying $|w_i|\leqslant 1/m$ and $\alpha_i\geqslant 0$, $i=1,\dots,n+1$, such that $\sum\alpha_i=1$, $\varphi(t,\,\cdot\,)$ is differentiable at $x-w_i$ and
P. Bettiol and R. B. Vinter, Principles of dynamic optimization, Springer Monogr. Math., Springer, Cham, 2024, xviii+781 pp.
2.
N. Bogoliouboff (Bogolyubov), “Sur quelques méthodes nouvelles dans le calcul des variations”, Ann. Math. Pura Appl. (4), 7 (1930), 249–271
3.
A. D. Ioffe, “On lower semicontinuity of integral functionals. I”, SIAM J. Control Optim., 15:4 (1977), 521–538
4.
A. Ioffe, “Existence and relaxation theorems for unbounded differential inclusions”, J. Convex Anal., 13:2 (2006), 353–362
5.
A. D. Ioffe, Variational analysis of regular mappings. Theory and applications, Springer Monogr. Math., Springer, Cham, 2017, xxi+495 pp.
6.
A. D. Ioffe, “Maximum principles for optimal control problems with differential inclusions”, SIAM J. Control Optim., 62:1 (2024), 271–296
7.
A. D. Ioffe and V. M. Tihomirov, Theory of extremal problems, Stud. Math. Appl., 6, North-Holland Publishing Co., Amsterdam–New York, 1979, xii+460 pp.
8.
F. H. Clarke, “The generalized problem of Bolza”, SIAM J. Control Optim., 14:4 (1976), 682–699
9.
P. D. Loewen and R. T. Rockafellar, “Optimal control of unbounded differential inclusions”, SIAM J. Control Optim., 32:2 (1994), 442–470
10.
C. Olech, “Weak lower semicontinuity of integral functionals”, J. Optim. Theory Appl., 19:1 (1976), 3–16
11.
J.-P. Penot, Calculus without derivatives, Grad. Texts in Math., 266, Springer, New York, 2013, xx+524 pp.
12.
R. T. Rockafellar, “Existence and duality theorems for convex problems of Bolza”, Trans. Amer. Math. Soc., 159 (1971), 1–40
13.
R. T. Rockafellar, “Existence theorems for general control problems of Bolza and Lagrange”, Adv. Math., 15:3 (1975), 312–333
14.
R. T. Rockafellar and R. J. B. Wets, Variational analysis, Grundlehren Math. Wiss., 317, Springer-Verlag, Berlin, 1998, xiv+733 pp.
15.
R. B. Vinter, Optimal control, Systems Control Found. Appl., Birkhäuser Boston, Inc., Boston, MA, 2000, xviii+507 pp.
Citation:
A. D. Ioffe, “On the Hamilton–Jacobi theory for nonsmooth variational problems”, Sb. Math., 216:3 (2025), 357–367