A vector space over field is a set which is closed under the following two operations of addition and scalar multiplication defined for its members (called vectors), i.e., the results of the operations are also members of .
A subset of is a subspace of , denoted by , if it is also a vector space, i.e., it is closed under the same operations defined in :
Listed below is a set of typical vector spaces:
This space contains all -D vectors expressed as an -tuple, an ordered list of elements (or components):
A subspace of can be a () that passes origin (zero). For example, any 2-D plane passing through the origin of a 3-D space is its subspace. However, if the 2-D plane does not pass through the origin, it is not a subspace. Also, as 3-D cube or sphere centered at the origin is not a subspace, as it is not closed under the operations of addition and scalar multiplication.
The concept of an -D space or can be generalized by allowing the dimension to become to infinity so that a vector in the space becomes a sequence for or . If all vectors are square-summable, the space is denoted by . All discrete energy signals are vectors in .
A vector space can also be a set of real or complex valued continuous functions defined over either a finite range such as , or an infinite range . If all functions are square-integrable, the space is denoted by . All continuous energy signals are vectors in .
Note that the term “vector”, generally denoted by in the following, may be interpreted in two different ways. First, in the most general sense, it represents a member of a vector space, such as any of the vector spaces considered above; e.g., a function . Second, in a more narrow sense, it can also represent a tuple of elements, an -D vector , where may be infinity. It should be clear what a vector represents from the context.
A set of vectors are linearly independent if none of them can be represented as a linear combination of the others.
Theorem: For any set of linearly independent vectors , if their linear combination is zero
Proof: Assume , then we would get
In particular, in the n-D Euclidean space, the vectors can be written as (), and their linear combination
The convex combination of vectors is their sum weighted by coefficients that add up to 1:
An inner product in a vector space is a function that maps two vectors to a scalar or and satisfies the following conditions:
An inner product space is a vector space with inner product defined. In particular, when the inner product is defined, is called a unitary space and is called a Euclidean space.
Some examples of the inner product are listed below:
In general, the norm of a vector is a non-negative real scalar that measures its size or length. if and only if . There exist different definitions for vector norm, as shown later. A vector is normalized (becomes a unit vector) if . Any given vector can be normalized when divided by its own norm: .
The most widely used 2-norm of a vector is defined as
The Cauchy-Schwarz inequality holds for any two vectors and in an inner product space :
Proof: If either or is zero, , the theorem holds (an equality). Otherwise, we consider the following inner product:
The distance between two vectors and is a real constant that satisfies:
A vector space is a metric space if a distance (or metric) between any two vector (two points) and is defined.
The angle between two vectors and is defined as
Two vectors and are orthogonal or perpendicular to each other, denoted by , if their inner product is zero , i.e., the angle between them is .
The orthogonal projection of a vector onto another vector is defined as a vector
A sequence of points in a metric space is a Cauchy sequence if it converges, i.e., for any , there exists an integer so that the following is true for any :