This note is a brief presentation of vectors from the same perspective as Halmos’ excellent small volume: “Finite-Dimensional Vector Spaces” (ISBN: 0387900934). I add nothing here.

I dedicate this page to Marvin Epstein who taught linear algebra at Berkeley in 1953. He was perhaps the best math teacher I have ever known. He made this stuff seem simple, elegant and important.

The Vector Space

A vector space is always associated with some field. If you don’t know what a field is read “real numbers” for “field” and you wont miss anything in this note. We call the elements of the field “scalars” for no better reason than to distinguish them from vectors. We use Greek letters α, β, γ to designate scalars and small Latin letters u, v, w to denote vectors. Aside from the operators that come with the field there are two others in a vector space, addition of two vectors yielding a vector and scalar multiplication of a scalar by a vector yielding a vector. Thus αu is a vector resulting from multiplying the scalar α by the vector u, and u+v is the vector sum of vectors u and v.
There are several axioms governing these new operations and relating them to the operations in the field.

There is a vector, 0 such that for all x, x+0 = x.
(u+v)+w = u+(v+w)
u+v = v+u
α(u+v) = αu+αv
(α + β)u = αu + βu
α(βu) = (αβ)u
0u = 0
1u = u

The “0” in “0u” is a scalar, of course, because you can not put a vector there. Ditto for “1” in “1u”. The right sides are vectors because that is what this kind of multiplication yields. It should be noted that all of these equations are true if we interpreted u, v and w as scalars. Vectors thus behave much like numbers.
Perhaps the simplest thing about vectors that is not true of scalars is that we may have two vectors u and v so that αu+βy = 0 only when α=β=0. If vectors were merely numbers then we could let α=v and β=–u to solve the equation. This not generally possible in vector spaces.
Note that if we let the vectors be the field then all of the axioms are satisfied. This is a very simple example of a vector space. An even simpler one is the vector space that consists merely of the single vector 0. The simplest non trivial vector space may be illustrated by pairs of scalars <α, β>—but we must give the rule for adding and scalar multiplication before we can claim to have presented a vector space.

<α, β> + <γ, δ> = <α + γ, β + δ>
α<β, γ> = <αβ, αγ>

It is easy and only slightly tedious to verify that the vector axioms are satisfied by these definitions. It may help to note that if the field is the reals then <α, β> may be taken as the vector from the origin to the point with Cartesian coördinates α, β. If we let u = <0, 1> and v = <1, 0> then we indeed cannot solve αu+βv = 0 except for α=β=0.

n-tuples of field elements provide an n dimensional vector space when analogous definitions for addition and scalar multiplication are provided.

A less widely known vector space is the space of real valued functions of some variable ranging over some fixed domain. The domain of the functions does not matter. Vector addition and scalar multiplication are defined thus:

(f+g)(x) = f(x) + g(x)
(αf)(x) = α(f(x))

Again the vector axioms are easily verified.

Linear Operators

A linear operator is a function that operates on a vector in one vector space, its domain, and yields a vector in another vector space, its range. The two spaces must have the same field. We denote linear operators by large Latin letters A, B, C. Linear operators obey the following axioms:

A(u+v) = Au + Av
A(αu) = α(Au)

In the rest of this note we mean “linear operator” by “operator”. The meanest operator is Au=0 for all u. Au=u is slightly more interesting. These two operators are called 0 and I respectively. For our Cartesian vector space we may define an operator A<α, β> = <β, –α>. This is a rotation of 90 degrees. A full fledged operator here might be A<α, β> = <3.7α + 6β, 2.1α + 5β>. Operators on function space include the derivative and

(Af)(x) = f(x+3).

We now make a simple, powerful but confusing observation—given two vector spaces D and R over the same field, the set of all operators with domain D and values in R is itself a vector space over the same field! We must define the addition of and scalar multiplication of operators first:

(A+B)u = Au+Bu
(αA)u = α(Au)

Note that in each case we have defined a new operator by providing a formula that defines the result of applying that operator, (A+B) or (αA) to some arbitrary vector u. We also decree that two operators A and B are equal if Au=Bu for all u. Now it is both confusing and tedious to verify the axioms as we have three vector spaces, the domain, the range and the tentative vector space that the operators themselves compose. We illustrate the axiom α(u+v) = αu+αv. To show that two operators A and B (which we view as vectors) are equal we must show that Au = Bu for all u.

(αA+αB)u = (αA)u + (αB)u = α(Au) + α(Bu) = α(Au+Bu) = α((A+B)u) = (α(A+B))u

We thus see that (αA+αB) = (α(A+B)) since u was arbitrary.
Similarly (α+β)u = αu+βu is proven for operators thus:

((α+β)A)u = (α+β)(Au) = α(Au) + β(Au) = (αA)u + (βA)u = ((αA) + (βA))u

and α(βA) = (αβ)A is proven:

(α(βA))u = α((βA)u) = α(β(Au)) = (αβ)(Au) = ((αβ)A)u

The others are even easier.
When the domain and range are the same space the space of operators has a natural multiplication defined: (AB)u = A(Bu). The following are easily derived:

(AB)C = A(BC)
A(B+C) = AB + AC
(A+B)C = AC + BC
(αA)B = A(αB) = α(AB)

Notably absent is AB = BA for this is not generally true.

Models

An apocryphal anecdote concerns a mathematician who studied some sort of algebraic structures for some years before discovering that there were none. In short his axioms were inconsistent! We have already provided a few examples of vector spaces. How many non-isomorphic vector spaces are there? We will hold the field constant. To study this question we introduce the notion of independence of a set of vectors.
A finite set {u_i} of vectors is independent whenever Σα_iu_i = 0 implies α_i = 0. If B is the set of all of the u_i then the value Σα_iu_i is called a linear combination of elements of B and B is said to span the set of all such values.
Consider the following process for discovering an independent vector set B in a space X: Let B be initially empty. While possible do:

Choose a vector in X that cannot be represented as Σα_iu_i with u_i in B. Add the vector to B.

When any point in X can be represented by Σα_iu_i with u_i in B, it is impossible to continue this process. The dimension of the space is the number of vectors in B and the resulting set B is a basis for the space and B spans X. Note that there is much latitude in choosing vectors in this process. For example any vector but 0 will serve as the first element of B. It turns out that the number of elements in B does not depend on these choices for elements for B. For some vector spaces this process of choosing base elements for B does not terminate. The functional vector spaces above are like this when the function’s domain is infinite. If this process is applied to two spaces over the same field, X and Y and the respective independent sets are the same size then an obvious isomorphism has been found between X and Y; for each u in X may be represented as Σα_iu_i. If the same α_i are used to represent a vector in Y, using Y’s independent set, then it is easy to see that we have established an isomorphism. Indeed this produces all isomorphisms between two such spaces.

We speak above of constructing a vector space by merely inventing basis vectors out of thin air and then just assuming that there must be some vector space, satisfying the axioms, and for which these vectors form a basis. This is a common practice and here is the excuse for such thinking:
As we already mentioned the field of a vector space itself satisfies the vector space axioms by taking the multiply from the field for scalar multiplication, and taking addition in the field for vector addition. This results in V₁, a one dimensional vector space for which {1} (1 of the field) is the natural basis. We can now form a new 2D vector space V₂ = V₁⊕V₁ as a direct sum with elements <α, β> as elements of V₂. One must verify that the axioms of a vector space are inherited by direct sums, but this is fairly easy. That <α, β> + <γ, δ> = <α + γ, β + δ> is the normal assumption in taking direct sums.

Now if we have invented three vectors p, q, r from thin air and want to claim them as a basis for some new vector space, then we take V₃ = V₁⊕V₁⊕V₁ as our new vector space and take <1, 0, 0>, <0, 1, 0>, <0, 0, 1> respectively as p, q and r.

Dual Spaces
Warning (admission): The proofs here become slightly non trivial and I am too lazy to fill them in, where they are most needed.

An especially interesting class of linear operators are those that map a vector space into the scalars. Collectively this class composes another vector space which is called the “dual” of the first. The dual of space X is written X*. We shall prove that X** is not only isomorphic to X but that there is a natural isomorphism and we shall therefore usually confuse X with X**.

∀x (x ∈ X → ∃b (b ∈ X** & ∀p (p ∈ X* → bp = pv))) and further the b is unique.
Suppose u, v ∈ X and A, B ∈ X*. We first locate a special element b of X** for any particular element v of X. The following equation defines b:

for every vector p in X*:
bp = pv

It must of course be verified that this transformation is indeed linear. We must still show

• every element of X** comes this way from some element of X,
• distinct elements of X lead to distinct elements of X**.

We shall wander a bit first and discover some new scenery first.

Choose an element b of X**. Choose two elements A and B of X*. bA and bB are two scalars.

Quadratic forms

Infinite dimensional spaces