Square roots of the eigenvalues of the self-adjoint operator
In
mathematics , in particular
functional analysis , the singular values of a
compact operator
T
:
X
→
Y
{\displaystyle T:X\rightarrow Y}
acting between
Hilbert spaces
X
{\displaystyle X}
and
Y
{\displaystyle Y}
, are the square roots of the (necessarily non-negative)
eigenvalues of the self-adjoint operator
T
∗
T
{\displaystyle T^{*}T}
(where
T
∗
{\displaystyle T^{*}}
denotes the
adjoint of
T
{\displaystyle T}
).
The singular values are non-negative
real numbers , usually listed in decreasing order (σ 1 (T ), σ 2 (T ), …). The largest singular value σ 1 (T ) is equal to the
operator norm of T (see
Min-max theorem ).
Visualization of a
singular value decomposition (SVD) of a 2-dimensional, real
shearing matrix M . First, we see the
unit disc in blue together with the two
canonical unit vectors . We then see the action of M , which distorts the disc to an
ellipse . The SVD decomposes M into three simple transformations: a
rotation V * , a
scaling Σ along the rotated coordinate axes and a second rotation U . Σ is a (square, in this example)
diagonal matrix containing in its diagonal the singular values of M , which represent the lengths σ 1 and σ 2 of the
semi-axes of the ellipse.
If T acts on Euclidean space
R
n
{\displaystyle \mathbb {R} ^{n}}
, there is a simple geometric interpretation for the singular values: Consider the image by
T
{\displaystyle T}
of the
unit sphere ; this is an
ellipsoid , and the lengths of its semi-axes are the singular values of
T
{\displaystyle T}
(the figure provides an example in
R
2
{\displaystyle \mathbb {R} ^{2}}
).
The singular values are the absolute values of the
eigenvalues of a
normal matrix A , because the
spectral theorem can be applied to obtain unitary diagonalization of
A
{\displaystyle A}
as
A
=
U
Λ
U
∗
{\displaystyle A=U\Lambda U^{*}}
. Therefore,
A
∗
A
=
U
Λ
∗
Λ
U
∗
=
U
|
Λ
|
U
∗
{\textstyle {\sqrt {A^{*}A}}={\sqrt {U\Lambda ^{*}\Lambda U^{*}}}=U\left|\Lambda \right|U^{*}}
.
Most
norms on Hilbert space operators studied are defined using singular values. For example, the
Ky Fan -k -norm is the sum of first k singular values, the trace norm is the sum of all singular values, and the
Schatten norm is the p th root of the sum of the p th powers of the singular values. Note that each norm is defined only on a special class of operators, hence singular values can be useful in classifying different operators.
In the finite-dimensional case, a
matrix can always be decomposed in the form
U
Σ
V
∗
{\displaystyle \mathbf {U\Sigma V^{*}} }
, where
U
{\displaystyle \mathbf {U} }
and
V
∗
{\displaystyle \mathbf {V^{*}} }
are
unitary matrices and
Σ
{\displaystyle \mathbf {\Sigma } }
is a
rectangular diagonal matrix with the singular values lying on the diagonal. This is the
singular value decomposition .
Basic properties
For
A
∈
C
m
×
n
{\displaystyle A\in \mathbb {C} ^{m\times n}}
, and
i
=
1
,
2
,
…
,
min
{
m
,
n
}
{\displaystyle i=1,2,\ldots ,\min\{m,n\}}
.
Min-max theorem for singular values . Here
U
:
dim
(
U
)
=
i
{\displaystyle U:\dim(U)=i}
is a subspace of
C
n
{\displaystyle \mathbb {C} ^{n}}
of dimension
i
{\displaystyle i}
.
σ
i
(
A
)
=
min
dim
(
U
)
=
n
−
i
+
1
max
x
∈
U
‖
x
‖
2
=
1
‖
A
x
‖
2
.
σ
i
(
A
)
=
max
dim
(
U
)
=
i
min
x
∈
U
‖
x
‖
2
=
1
‖
A
x
‖
2
.
{\displaystyle {\begin{aligned}\sigma _{i}(A)&=\min _{\dim(U)=n-i+1}\max _{\underset {\|x\|_{2}=1}{x\in U}}\left\|Ax\right\|_{2}.\\\sigma _{i}(A)&=\max _{\dim(U)=i}\min _{\underset {\|x\|_{2}=1}{x\in U}}\left\|Ax\right\|_{2}.\end{aligned}}}
Matrix transpose and conjugate do not alter singular values.
σ
i
(
A
)
=
σ
i
(
A
T
)
=
σ
i
(
A
∗
)
.
{\displaystyle \sigma _{i}(A)=\sigma _{i}\left(A^{\textsf {T}}\right)=\sigma _{i}\left(A^{*}\right).}
For any unitary
U
∈
C
m
×
m
,
V
∈
C
n
×
n
.
{\displaystyle U\in \mathbb {C} ^{m\times m},V\in \mathbb {C} ^{n\times n}.}
σ
i
(
A
)
=
σ
i
(
U
A
V
)
.
{\displaystyle \sigma _{i}(A)=\sigma _{i}(UAV).}
Relation to eigenvalues:
σ
i
2
(
A
)
=
λ
i
(
A
A
∗
)
=
λ
i
(
A
∗
A
)
.
{\displaystyle \sigma _{i}^{2}(A)=\lambda _{i}\left(AA^{*}\right)=\lambda _{i}\left(A^{*}A\right).}
Relation to
trace :
∑
i
=
1
n
σ
i
2
=
tr
A
∗
A
{\displaystyle \sum _{i=1}^{n}\sigma _{i}^{2}={\text{tr}}\ A^{\ast }A}
.
If
A
⊤
A
{\displaystyle A^{\top }A}
is full rank, the product of singular values is
det
A
⊤
A
{\displaystyle {\sqrt {\det A^{\top }A}}}
.
If
A
A
⊤
{\displaystyle AA^{\top }}
is full rank, the product of singular values is
det
A
A
⊤
{\displaystyle {\sqrt {\det AA^{\top }}}}
.
If
A
{\displaystyle A}
is full rank, the product of singular values is
|
det
A
|
{\displaystyle |\det A|}
.
Inequalities about singular values
See also.
[1]
Singular values of sub-matrices
For
A
∈
C
m
×
n
.
{\displaystyle A\in \mathbb {C} ^{m\times n}.}
Let
B
{\displaystyle B}
denote
A
{\displaystyle A}
with one of its rows or columns deleted. Then
σ
i
+
1
(
A
)
≤
σ
i
(
B
)
≤
σ
i
(
A
)
{\displaystyle \sigma _{i+1}(A)\leq \sigma _{i}(B)\leq \sigma _{i}(A)}
Let
B
{\displaystyle B}
denote
A
{\displaystyle A}
with one of its rows and columns deleted. Then
σ
i
+
2
(
A
)
≤
σ
i
(
B
)
≤
σ
i
(
A
)
{\displaystyle \sigma _{i+2}(A)\leq \sigma _{i}(B)\leq \sigma _{i}(A)}
Let
B
{\displaystyle B}
denote an
(
m
−
k
)
×
(
n
−
l
)
{\displaystyle (m-k)\times (n-l)}
submatrix of
A
{\displaystyle A}
. Then
σ
i
+
k
+
l
(
A
)
≤
σ
i
(
B
)
≤
σ
i
(
A
)
{\displaystyle \sigma _{i+k+l}(A)\leq \sigma _{i}(B)\leq \sigma _{i}(A)}
Singular values of A + B
For
A
,
B
∈
C
m
×
n
{\displaystyle A,B\in \mathbb {C} ^{m\times n}}
∑
i
=
1
k
σ
i
(
A
+
B
)
≤
∑
i
=
1
k
(
σ
i
(
A
)
+
σ
i
(
B
)
)
,
k
=
min
{
m
,
n
}
{\displaystyle \sum _{i=1}^{k}\sigma _{i}(A+B)\leq \sum _{i=1}^{k}(\sigma _{i}(A)+\sigma _{i}(B)),\quad k=\min\{m,n\}}
σ
i
+
j
−
1
(
A
+
B
)
≤
σ
i
(
A
)
+
σ
j
(
B
)
.
i
,
j
∈
N
,
i
+
j
−
1
≤
min
{
m
,
n
}
{\displaystyle \sigma _{i+j-1}(A+B)\leq \sigma _{i}(A)+\sigma _{j}(B).\quad i,j\in \mathbb {N} ,\ i+j-1\leq \min\{m,n\}}
Singular values of AB
For
A
,
B
∈
C
n
×
n
{\displaystyle A,B\in \mathbb {C} ^{n\times n}}
∏
i
=
n
i
=
n
−
k
+
1
σ
i
(
A
)
σ
i
(
B
)
≤
∏
i
=
n
i
=
n
−
k
+
1
σ
i
(
A
B
)
∏
i
=
1
k
σ
i
(
A
B
)
≤
∏
i
=
1
k
σ
i
(
A
)
σ
i
(
B
)
,
∑
i
=
1
k
σ
i
p
(
A
B
)
≤
∑
i
=
1
k
σ
i
p
(
A
)
σ
i
p
(
B
)
,
{\displaystyle {\begin{aligned}\prod _{i=n}^{i=n-k+1}\sigma _{i}(A)\sigma _{i}(B)&\leq \prod _{i=n}^{i=n-k+1}\sigma _{i}(AB)\\\prod _{i=1}^{k}\sigma _{i}(AB)&\leq \prod _{i=1}^{k}\sigma _{i}(A)\sigma _{i}(B),\\\sum _{i=1}^{k}\sigma _{i}^{p}(AB)&\leq \sum _{i=1}^{k}\sigma _{i}^{p}(A)\sigma _{i}^{p}(B),\end{aligned}}}
σ
n
(
A
)
σ
i
(
B
)
≤
σ
i
(
A
B
)
≤
σ
1
(
A
)
σ
i
(
B
)
i
=
1
,
2
,
…
,
n
.
{\displaystyle \sigma _{n}(A)\sigma _{i}(B)\leq \sigma _{i}(AB)\leq \sigma _{1}(A)\sigma _{i}(B)\quad i=1,2,\ldots ,n.}
For
A
,
B
∈
C
m
×
n
{\displaystyle A,B\in \mathbb {C} ^{m\times n}}
[2]
2
σ
i
(
A
B
∗
)
≤
σ
i
(
A
∗
A
+
B
∗
B
)
,
i
=
1
,
2
,
…
,
n
.
{\displaystyle 2\sigma _{i}(AB^{*})\leq \sigma _{i}\left(A^{*}A+B^{*}B\right),\quad i=1,2,\ldots ,n.}
Singular values and eigenvalues
For
A
∈
C
n
×
n
{\displaystyle A\in \mathbb {C} ^{n\times n}}
.
See
[3]
λ
i
(
A
+
A
∗
)
≤
2
σ
i
(
A
)
,
i
=
1
,
2
,
…
,
n
.
{\displaystyle \lambda _{i}\left(A+A^{*}\right)\leq 2\sigma _{i}(A),\quad i=1,2,\ldots ,n.}
Assume
|
λ
1
(
A
)
|
≥
⋯
≥
|
λ
n
(
A
)
|
{\displaystyle \left|\lambda _{1}(A)\right|\geq \cdots \geq \left|\lambda _{n}(A)\right|}
. Then for
k
=
1
,
2
,
…
,
n
{\displaystyle k=1,2,\ldots ,n}
:
Weyl's theorem
∏
i
=
1
k
|
λ
i
(
A
)
|
≤
∏
i
=
1
k
σ
i
(
A
)
.
{\displaystyle \prod _{i=1}^{k}\left|\lambda _{i}(A)\right|\leq \prod _{i=1}^{k}\sigma _{i}(A).}
For
p
>
0
{\displaystyle p>0}
.
∑
i
=
1
k
|
λ
i
p
(
A
)
|
≤
∑
i
=
1
k
σ
i
p
(
A
)
.
{\displaystyle \sum _{i=1}^{k}\left|\lambda _{i}^{p}(A)\right|\leq \sum _{i=1}^{k}\sigma _{i}^{p}(A).}
History
This concept was introduced by
Erhard Schmidt in 1907. Schmidt called singular values "eigenvalues" at that time. The name "singular value" was first quoted by Smithies in 1937. In 1957, Allahverdiev proved the following characterization of the n th singular number:
[4]
σ
n
(
T
)
=
inf
{
‖
T
−
L
‖
:
L
is an operator of finite rank
<
n
}
.
{\displaystyle \sigma _{n}(T)=\inf {\big \{}\,\|T-L\|:L{\text{ is an operator of finite rank }}<n\,{\big \}}.}
This formulation made it possible to extend the notion of singular values to operators in
Banach space .
Note that there is a more general concept of
s-numbers , which also includes Gelfand and Kolmogorov width.
See also
References
^
R. A. Horn and
C. R. Johnson . Topics in Matrix Analysis. Cambridge University Press, Cambridge, 1991. Chap. 3
^ X. Zhan. Matrix Inequalities. Springer-Verlag, Berlin, Heidelberg, 2002. p.28
^ R. Bhatia. Matrix Analysis. Springer-Verlag, New York, 1997. Prop. III.5.1
^
I. C. Gohberg and
M. G. Krein . Introduction to the Theory of Linear Non-selfadjoint Operators. American Mathematical Society, Providence, R.I.,1969. Translated from the Russian by A. Feinstein. Translations of Mathematical Monographs, Vol. 18.