Robert Eisele
Engineer, Systems Architect and DBA

Dot Product

The dot product of two vectors \(\mathbf{a}\) and \(\mathbf{b}\) is a value expressing the angular relationship between two vectors and is defined to be the product of the vector lengths times the cosine of the angle \(\theta\) between the vectors:

\[\mathbf{a}\cdot\mathbf{b} = |\mathbf{a}|\cdot|\mathbf{b}|\cos\theta\]

If the vectors are orthogonal \(\left(\theta=90^°\right)\) it follows that

\[\mathbf{a}\cdot\mathbf{b} = 0\]

If \(\theta<90^°\) it follows that

\[\mathbf{a}\cdot\mathbf{b} > 0\]

If \(\theta>90^°\) it follows that

\[\mathbf{a}\cdot\mathbf{b} < 0\]

If the vectors are contradirectional \(\left(\theta=180^°\right)\) it follows that

\[\mathbf{a}\cdot\mathbf{b} = -|\mathbf{a}|\cdot|\mathbf{b}|\]

If the vectors are codirectional \(\left(\theta=0^°\right)\) it follows that

\[\mathbf{a}\cdot\mathbf{b} = |\mathbf{a}|\cdot|\mathbf{b}|\]

We can use this property to see that the dot product with itself is

\[\mathbf{a}\cdot\mathbf{a} = |\mathbf{a}|\cdot|\mathbf{a}|=|\mathbf{a}|^2\]

It turns out the vector length can therefore be defined as

\[|\mathbf{a}|=\sqrt{\mathbf{a}\cdot\mathbf{a}}\]

Algebraic View

Algebraically, the dot product is the sum of the componentwise products:

\[\mathbf{a}\cdot\mathbf{b} = \mathbf{a}^T\mathbf{b} = \sum\limits_{i=1}^d\mathbf{a}_i\mathbf{b}_i\]

Proof:

Let \(\mathbf e_{i}\) be the orthonormal basis vector,

\[\begin{array}{rl}\mathbf{a}\cdot\mathbf{b} =& \mathbf {a} \cdot \sum \limits_{i=1}^d\mathbf {b}_{i}\mathbf {e} _{i}\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}(\mathbf {a} \cdot \mathbf {e} _{i})\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}(|\mathbf {a}| \cdot |\mathbf {e} _{i}|\cos\theta)\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}(|\mathbf {a}| \cos\theta)\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}\mathbf {a}_{i}\\=&\sum \limits_{i=1}^d\mathbf {a}_{i}\mathbf {b}_{i}\\\end{array}\]

Commutative

\[\mathbf{a}\cdot\mathbf{b}=\mathbf{b}\cdot\mathbf{a}\]

Proof: \[\mathbf{a}\cdot\mathbf{b}=|\mathbf{a}|\cdot|\mathbf{b}|\cos\theta=|\mathbf{b}|\cdot|\mathbf{a}|\cos\theta=\mathbf{b}\cdot\mathbf{a}\]

Distributive

\[\mathbf{a}\cdot(\mathbf{b}+\mathbf{c})=(\mathbf{a}\cdot\mathbf{b})+(\mathbf{a}\cdot\mathbf{c})\]

Proof:

\[\begin{array}{rl}\mathbf{a}\cdot(\mathbf{b}+\mathbf{c}) &= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot([\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d])\\&= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1+\mathbf{c}_1, \mathbf{b}_2+\mathbf{c}_2, ..., \mathbf{b}_d+\mathbf{c}_d]\\&= [\mathbf{a}_1(\mathbf{b}_1+\mathbf{c}_1), \mathbf{a}_2(\mathbf{b}_2+\mathbf{c}_2), ..., \mathbf{a}_d(\mathbf{b}_d+\mathbf{c}_d)]\\&= [\mathbf{a}_1\mathbf{b}_1+\mathbf{a}_1\mathbf{c}_1, \mathbf{a}_2\mathbf{b}_2+\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{b}_d+\mathbf{a}_d\mathbf{c}_d]\\&= [\mathbf{a}_1\mathbf{b}_1, \mathbf{a}_2\mathbf{b}_2, ..., \mathbf{a}_d\mathbf{b}_d]+[\mathbf{a}_1\mathbf{c}_1,\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{c}_d]\\&= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot(\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d]\\&=\mathbf{a}\cdot\mathbf{b}+\mathbf{a}\cdot\mathbf{c}\end{array}\]

Bilinear

\[\mathbf{a}\cdot(\alpha\mathbf{b}+\mathbf{c}) = \alpha(\mathbf{a}\cdot\mathbf{b})+(\mathbf{a}\cdot\mathbf{c})\]

Scalar Multiplication

\[(\alpha_1\mathbf{a})\cdot(\alpha_2\mathbf{b})=\alpha_1\alpha_2(\mathbf{a}\cdot\mathbf{b})\]

Dot product is homogenious under scaling \[\alpha(\mathbf{a}\cdot\mathbf{b}) = (\alpha\mathbf{a})\cdot\mathbf{b} = \mathbf{a}\cdot(\alpha\mathbf{b})\]

No cancellation

If \(\mathbf{a}\cdot\mathbf{b}=\mathbf{a}\cdot\mathbf{c}\) and \(\mathbf{a}\neq \mathbf{0}\), we can write \(\mathbf{a}\cdot(\mathbf{b}-\mathbf{c})=0\) by distributive law, which means that \(\mathbf{a}\perp(\mathbf{b}-\mathbf{c})\), which allows \((\mathbf{b}-\mathbf{c})\neq\mathbf{0}\) and therefore \(\mathbf{b}\neq\mathbf{c}\).

Applications

Vector Projection

Given two vectors \(\mathbf{a}\) and \(\mathbf{b}\) we want to determine the projection of \(\mathbf{a}\) onto \(\mathbf{b}\), which is the vector \(\mathbf{a}'\) of length \(a'\) parallel to the vector \(\mathbf{b}\).

The length \(a'\) is called the Scalar Projection and is determined by

\[a' = |\mathbf{a}|\cos\theta = |\mathbf{a}|\underbrace{\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{a}|\cdot|\mathbf{b}|}}_{\cos\theta}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|}=\mathbf{a}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\mathbf{a}\cdot\hat{\mathbf{b}}\]

Having the Scalar Projection, we can then calculate the Vector Projection:

\[\mathbf{a}' = a'\cdot\hat{\mathbf{b}} = (\mathbf{a}\cdot\hat{\mathbf{b}})\cdot\hat{\mathbf{b}}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|^2}\mathbf{b}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\mathbf{b}=:proj_{\mathbf{b}}(\mathbf{a})\]

Scalar Projection Percentage: The length of the distance \(a'\) to the total length of \(\mathbf{b}\) can be expressed in percent as follows:

\[\frac{a'}{|\mathbf{b}|} = \frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|\cdot |\mathbf{b}|} =\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|^2}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\]

Which is interesting. It says that the Vector Projection is just a scaling of vector \(\mathbf{b}\) by the proportion of \(\mathbf{a}'\) to \(\mathbf{b}\).

Direction Cosines

In three dimensional space, a vector \(\mathbf{a}\) will form the angle \(\alpha\) with the x-axis, \(\beta\) with the y-axis and \(\gamma\) with the z-axis. These angles are called direction angles and the cosine of these angles are called direction cosines.

PLOT

When we use the standard basis vectors for the dimension \(\mathbf{i}, \mathbf{j}\) and \(\mathbf{k}\), the direction cosines are:

\[\begin{array}{rl}\cos\alpha =& \frac{\mathbf{a}\cdot\mathbf{i}}{|\mathbf{a}|\cdot|\mathbf{i}|} = \frac{\mathbf{a}_1}{|\mathbf{a}|}\\\cos\beta =& \frac{\mathbf{a}\cdot\mathbf{j}}{|\mathbf{a}|\cdot|\mathbf{j}|} = \frac{\mathbf{a}_2}{|\mathbf{a}|}\\\cos\gamma =& \frac{\mathbf{a}\cdot\mathbf{k}}{|\mathbf{a}|\cdot|\mathbf{k}|} = \frac{\mathbf{a}_3}{|\mathbf{a}|}\\\end{array}\]

As such, the following properties hold:

  1. The vector \(\mathbf{b}=[\cos\alpha, \cos\beta, \cos\gamma]^T\) is a unit vector.
  2. \(\cos^2\alpha+ \cos^2\beta+ \cos^2\gamma=1\)
  3. \(\mathbf{a} = |\mathbf{a}|[\cos\alpha, \cos\beta, \cos\gamma]^T\)

Law of Cosines

Interestingly, the well known law of cosines can be derived using the dot-product:

\[{\begin{array}{rl}\mathbf {c} \cdot \mathbf {c} &=(\mathbf {a} -\mathbf {b} )\cdot (\mathbf {a} -\mathbf {b} )\\&=\mathbf {a} \cdot \mathbf {a} -\mathbf {a} \cdot \mathbf {b} -\mathbf {b} \cdot \mathbf {a} +\mathbf {b} \cdot \mathbf {b} \\&=a^{2}-\mathbf {a} \cdot \mathbf {b} -\mathbf {a} \cdot \mathbf {b} +b^{2}\\&=a^{2}-2(\mathbf {a} \cdot \mathbf {b}) +b^{2}\\c^{2}&=a^{2}+b^{2}-2ab\cos \theta \\\end{array}}\]