Newton’s Method and Secant Method for Set-Valued Mappings

. For ﬁnding zeros or ﬁxed points of set-valued maps, the fact that the space of convex, compact, nonempty sets of R n is not a vector space presents a major disadvantage. Therefore, ﬁxed point iterations or variants of Newton’s method, in which the derivative is applied only to a smooth single-valued part of the set-valued map, are often applied for calculations. We will embed the set-valued map with convex, compact images (i.e. by embedding its images) and shift the problem to the Ba-nach space of directed sets. This Banach space extends the arithmetic operations of convex sets and allows to consider the Fr´echet-derivative or divided diﬀerences of maps that have embedded convex images. For the transformed problem, Newton’s method and the secant method in Banach spaces are applied via directed sets. The results can be visualized as usually nonconvex sets in R n .


Introduction
In this article we illustrate how Newton's method and the similar secant method can be applied to solve two standard problems for set-valued mappings with convex, compact images. Our key method is to embed these problems into a Banach space setting, which allows to reformulate them as zero-finding problems. For references on uses of Newton's method in Banach spaces see e.g. [12,6]. We would like to point out that the ideas presented here differ from the pioneering works (e.g. [10,1]) based on homotopy methods or mutational equations. We start this article with a brief introduction to the Banach space of directed sets. In Sections 2 and 3, we apply this to introduce the Fréchet-derivative as well as Newton's method and the secant method for set-valued maps. We conclude with several numerical examples in Section 4.

Directed sets
Directed sets provide a way to interpret compact, convex, nonempty sets of R n , i.e. sets in C(R n ), as elements of a Banach space. They were introduced in [3] (see also references therein), their visualization is studied in [3,Part II]. For C, D ∈ C(R n ) and M ∈ R m×n , we write C + D for the Minkowski sum, λ · C for the multiplication with non-negative scalars and M · C for the linear image. Definition 1. Directed intervals resp. sets − → A ∈ D(R n ) are defined recursively. For n = 1, a directed interval is a pair − → A = (a 1 (l)) l=±1 of two real numbers. (We will also use the notation given by a map − −− → A n−1 : S n−1 → D(R n−1 ) that is uniformly bounded with respect to · n−1 , and a continuous real-valued function a n (·). The norm is defined by As a space of functions on S n−1 with values in a linear space, with the usual pointwise definitions of addition and scalar multiplication, D(R n ) is a linear space. With the norm defined above, D(R n ) is a Banach space that can be equipped with a partial ordering that represents inclusion of sets (see [3,Part I]). Sets in C(R n ) can be embedded in this Banach space via J n : C(R n ) → D(R n ) as Here, δ * (l, C) denotes the support function of C in direction l ∈ S n−1 , Y (l, C) the corresponding supporting face (i.e. the maximizers of l, · on the set C), and Π n−1,l is a linear projection which maps the orthogonal complement of span{l} into R n−1 and l to the origin in R n−1 , see [3, Part I] for details.

Proposition 1 ([3]
). The embedding J n : C(R n ) → D(R n ) is positively linear: In contrast to formulations using pairs of sets (see [3] for references), a directed set (and thus a difference of embedded convex sets) can be visualized as a subset in R n which is usually nonconvex. The visualization of the directed set is plotted with arrows (outer normals for embedded convex sets, inner ones for their inverses) on the boundaries (see [3, Part II] for details).

Fréchet-Derivative for Set-Valued Maps
As D(R m ) forms a Banach space, we use the usual definition of the Fréchetderivative of a function − → F : R n → D(R m ). To lift a set-valued map F : R n ⇒ R m with images in C(R m ) to the space D(R m ), we will use the notation This notion of the derivative generalizes the one of set-valued maps for one variable studied in [8,4].
Proof. (i) A constant map in a Banach space has 0 as Fréchet-differential.
Clearly, the sum and the product of two functions are Fréchet-differentiable, if the operands have this property.
We now study explicit formulas for the Fréchet-derivative.

Newton's Method and Secant Method
For the sake of brevity, we refer to the literature for convergence results about iterative methods in Banach spaces. For Newton's method see e.g. [12,5,6], for the secant method e.g. [9] and for Gauß-Newton method e.g. [7].

Newton's Method for Directed Sets
Problem 1. Let F : R n → C(R m ) be given. We are looking for a solutionx ∈ R n of 0 ∈ F (x). Using the partial order "≤" on D(R m ) and the embedded map To apply Newton's iteration for directed sets to Problem 1 we choose a starting point x 0 ∈ R n and iteratively obtain x k+1 = x k + d k , where d k ∈ R n has to solve the linear inequality Checking this inequality essentially means checking the positivity of a function, which is easily realized algorithmically.
Remark 1. In many publications, e.g. in the study of necessary optimality conditions in non-smooth optimization or optimal control problems, the set-valued map F is given in the form F (x) = g(x) + H(x) with a smooth function g : R n → R m and an u.s.c. map H : R n ⇒ R m (representing subdifferentials or normal cones in the mentioned applications), see e.g. [5,6]. The set-valued derivative of F is avoided and an inclusion step of the form is studied. While in this case the computation of d k typically requires a nonlinear inclusion to be solved, for this splitting of F (with Fréchet-differentiable − → H ) Newton's method as suggested above leads to an inclusion problem linearly in d k , since Newton's method in (1) demands a solution of Problem 2. Let G, H : R n → C(R m ) be given. We are looking for a solution H , Newton's iteration with directed sets for this problem proceeds similarly to the one for Problem 1, with the difference that the updates d k have to solve the linear equation

Secant Method Based on Directed Sets
To apply the secant method as studied e.g. in [11,9], we need a notion of divided differences. Set-valued divided differences in one variable are studied e.g. in [8,4]. Following [11], we introduce multivariate divided differences for directed sets.
In the limit for u → v, this definition yields the Fréchet-differential − → F (u).
This approach generalizes the divided differences in one variable and satisfies the characteristic equation To apply the secant method to Problem 1, one chooses two starting points x −1 , x 0 ∈ R n and iteratively computes To treat Problem 2 with the secant method, we again set An advantage of (5) is that the derivative − → F is avoided and replaced by differences of function values. It is obvious that the iterations (2) and (5) generalize their pointwise analogues.

Examples
In this section we present numerical examples for the procedures proposed above. Example 1. Let U 1 , . . . , U M ∈ C(R n ) be such that the vectors J n (U i ) are linearly independent in D(R n ). Consider an unknown convex combination In order to apply Newton's method for Problem 2 to compute the vector λ ∈ R M , we define so that the Newton step results in From the assumption of linear independence we obtain d k i − λ i + x k i = 0 for i = 1, . . . , M , and hence d k = −x k + λ. This means that already the first iteration step leads to the solution x 1 =x = λ. We obtain the same picture for the secant method. Since the left-hand side of (5) equals the one of the Newton step (2).
From now on, we consider Problem 2.
In Example 3, we solve Problem 2 using the Gauß-Newton method in [7] and the Gauß-secant method for two-dimensional directed sets. Similar to the approximation of convex sets with finitely many supporting hyperplanes, we approximate a set-valued map − → F : R n → D(R 2 ) by choosing a finite number of unit vectors l µ ∈ S 1 , µ = 1, . . . , M and evaluating Recalling the ideas behind the Gauß-Newton method, we replace − → F M (x) = 0 by the minimization problem min x∈R n − → F M (x) 2 2 which is solved iteratively by minimizing as iterative steps. The (i, j)-th entry of the matrix of divided differences is given for j = 1, 2 by the expressions which are much simpler to compute than the corresponding partial derivatives.
In the next nonlinear test example we are looking for parameters in the determining matrix of an ellipsoid centered in the origin which coincides with the unit ball.
H(x) = B 1 (0) in Problem 2. In this example, it is rather complicated to evaluate the derivative of The derivative of − → F 1 (x; l) within the Gauß-Newton method is complicated, since it involves the projection to R, whereas the calculation of − → F [u, v](w) in the Gauß-secant method only requires the weighted sum of differences of function values. Table 1 demonstrates that the Gauß-secant method converges rather rapidly, but slower than the Gauß-Newton method. In Fig. 1(b) the iterates − → X k := J 2 (A(x k )·B 1 (0)) of the secant method are depicted for the starting values x −1 = (4, 3) , x 0 = (3, 2) . Depending on the starting values, convergence to one of the solutions (±1, ±1) can be observed.
To sketch the outline of further research, we mention [2]. In this publication, Newton's method for set-valued maps is used to approximate invariant sets, i.e. to find sets P for which g(P ) = P . This task is similar to Problem 2, if the set is parametrized e.g. as P (x) = x·B 1 (0) (see Example 2) and G(x) = g(P (x)), H(x) = h(P (x)), h(x) = x, but considerably more complicated, as the map − → G depends on a directed set. This leads to a zero-finding problems of the form 0 = − → F ( − → X ) = J n (g(V n ( − → X )) − − → X , the solution of which requires much more advanced Banach space techniques than in the present paper.  Table 1. Iterates x k of the Gauß-Newton and the secant method for Example 3