# Vector Norms

The norm of a vector , denoted by , can be intuititvely interpretated as its “size”. For example, the norm of a real number in the 1-D real space is its absolute value , or its distance to the origin, and the norm of a complex number is its modulus , its Euclidean distance to the origin. Here is the most general definition of a vector norm:

Definition: The norm of a vector in vector space is a real non-negative value representing the length or magnitude of the vector. Specifically, the norm of satisfy the following three conditions:

• Positivity:
(208)
• Homogeneity:
(209)

• Triangle inequality:
(210)

The triangle inequality can also be expressed in alternative forms. If we define , the triangle inequality becomes

(211)
If we further define , i.e., , then the above becomes
(212)
Combining the two results above, we get
(213)
The first equality holds if and are in the same direction, the second equality holds if they are in opposite direction. More specially when , both equalities hold.

The p-norms of an n-D vector and a function are defined as:

(214)
The p-norm satisfies the three requirements in the definition of vector norm. The first two are trivially obvious, while the third one happens to be Minkowski's inequality (see appendix):
(215)
The p-norms corresponding to , , and are most commonly used:
• , the absolute sum of all elements:
(216)
• , the Euclidean norm:
(217)
• , the maximum absolute value of all elements:
(218)

Out of the three p-norms, the Euclidean 2-norm is the only one that is unitary invariant, i.e., it is conserved or invariant under any unitary transform , where is a unitary matrix satisfying :

(219)
i.e., the length of the vector is not changed by any unitary transform (such as rotation, when is a rotation matrix).

Definition: Two norms and are equivalent if there exist two positive real constants and so that

(220)
Note that this relationship can also be written as
(221)

Theorem: All different norms are equivalent.

Proof

• We first show that equivalence is transitive, i.e., if both and are equivalent to :
(222)
then they are equivalent to each other.

From the first equation we get and . Substituting these into the right and left hand sides of the second equation respectively yields:

(223)
i.e., and are equivalent.

• If we can further show that an arbitrary norm is equivalent to , i.e.,
(224)
then according to the transitivity property shown above, all norms are equivalent.

If , it is obvious that the equalities above hold . If , , we can define a normalized vector with , so that the relationship for the equivalence above can be written in terms of as

(225)
i.e., to prove that all norms are equivalent all we need to show is that is bounded both from above and below, as we will do below.

• We now prove that an arbitrary norm is a continuous function over , i.e., for any , there exists a so that
(226)
Both and can be expressed in terms of a basis that spans the space:
(227)
then we have
(228)
Now consider the alternative form of the triangle inequality of :
(229)
If we let , the above becomes
(230)
indicating that for any given , we can choose so that if , then , i.e., the norm is indeed continuous over .

• Finally, according to the extreme value theorem, a continuous function, such as , defined over a compact (closed and bounded) set, such as the unit sphere in the n-D space, must have its maximum and minimum values:
(231)
i.e., , which is what we need to prove.
Q.E.D.

The three p-norms are equivalent:

The distance between two vectors is defined as the norm of their difference . In particular, when , the distance between and the origin of the space is the norm of . Specifically, can be defined based on the p-norm:

(232)
Specially, when
• , is the city block (Manhattan) distance:
(233)
• , is the Euclidean distance:
(234)
• , is the Chebyshev distance:
(235)

The three unit “circles” or “spheres”, are formed by all points of unity norm with unity distance to the origin (blue, black, and red for , , and , respectively).