Robert Eisele
Engineer, Systems Architect and DBA

Dot Product

The dot product of two vectors \(\mathbf{a}\) and \(\mathbf{b}\) is a value expressing the angular relationship between two vectors and is defined to be the product of the vector lengths times the cosine of the angle \(\theta\) between the vectors:

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle = |\mathbf{a}|\cdot|\mathbf{b}|\cos\theta\]

If the vectors are orthogonal \(\left(\theta=90^°\right)\) it follows that

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle = 0\]

If \(\theta<90^°\) it follows that

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle > 0\]

If \(\theta>90^°\) it follows that

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle < 0\]

If the vectors are contradirectional \(\left(\theta=180^°\right)\) it follows that

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle = -|\mathbf{a}|\cdot|\mathbf{b}|\]

If the vectors are codirectional \(\left(\theta=0^°\right)\) it follows that

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle = |\mathbf{a}|\cdot|\mathbf{b}|\]

We can use this property to see that the dot product with itself is

\[\langle\mathbf{a}\cdot\mathbf{a}\rangle = |\mathbf{a}|\cdot|\mathbf{a}|=|\mathbf{a}|^2\]

It turns out the vector length can therefore be defined as

\[|\mathbf{a}|=\sqrt{\langle\mathbf{a}\cdot\mathbf{a}\rangle}\]

Algebraic View

Algebraically, the dot product is the sum of the componentwise products:

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle = \mathbf{a}^T\mathbf{b} = \sum\limits_{i=1}^d\mathbf{a}_i\mathbf{b}_i\]

Proof:

Let \(\mathbf {e} _{i}\) be the orthonormal basis vector,

\[\begin{array}{rl}\langle\mathbf{a}\cdot\mathbf{b}\rangle =& \langle\mathbf {a} \cdot \sum \limits_{i=1}^d\mathbf {b}_{i}\mathbf {e} _{i}\rangle\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}\langle\mathbf {a} \cdot \mathbf {e} _{i}\rangle\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}(|\mathbf {a}| \cdot |\mathbf {e} _{i}|\cos\theta)\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}(|\mathbf {a}| \cos\theta)\\=&\sum \limits_{i=1}^d\mathbf {b}_{i}\mathbf {a}_{i}\\=&\sum \limits_{i=1}^d\mathbf {a}_{i}\mathbf {b}_{i}\\\end{array}\]

Commutative

\[\langle\mathbf{a}\cdot\mathbf{b}\rangle=\langle\mathbf{b}\cdot\mathbf{a}\rangle\]

Proof: \[\langle\mathbf{a}\cdot\mathbf{b}\rangle=|\mathbf{a}|\cdot|\mathbf{b}|\cos\theta=|\mathbf{b}|\cdot|\mathbf{a}|\cos\theta=\langle\mathbf{b}\cdot\mathbf{a}\rangle\]

Distributive

\[\langle\mathbf{a}\cdot(\mathbf{b}+\mathbf{c})\rangle=\langle\mathbf{a}\cdot\mathbf{b}\rangle+\langle\mathbf{a}\cdot\mathbf{c}\rangle\]

Proof:

\[\begin{array}{rl}\langle\mathbf{a}\cdot(\mathbf{b}+\mathbf{c})\rangle &= \langle[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot([\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d])\rangle\\&= \langle[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1+\mathbf{c}_1, \mathbf{b}_2+\mathbf{c}_2, ..., \mathbf{b}_d+\mathbf{c}_d]\rangle\\&= [\mathbf{a}_1(\mathbf{b}_1+\mathbf{c}_1), \mathbf{a}_2(\mathbf{b}_2+\mathbf{c}_2), ..., \mathbf{a}_d(\mathbf{b}_d+\mathbf{c}_d)]\\&= [\mathbf{a}_1\mathbf{b}_1+\mathbf{a}_1\mathbf{c}_1, \mathbf{a}_2\mathbf{b}_2+\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{b}_d+\mathbf{a}_d\mathbf{c}_d]\\&= [\mathbf{a}_1\mathbf{b}_1, \mathbf{a}_2\mathbf{b}_2, ..., \mathbf{a}_d\mathbf{b}_d]+[\mathbf{a}_1\mathbf{c}_1,\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{c}_d]\\&= \langle[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]\rangle+\langle[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot(\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d]\rangle\\&=\langle\mathbf{a}\cdot\mathbf{b}\rangle+\langle\mathbf{a}\cdot\mathbf{c}\rangle\end{array}\]

Bilinear

\[\langle\mathbf{a}\cdot(\alpha\mathbf{b}+\mathbf{c})\rangle = \alpha\langle\mathbf{a}\cdot\mathbf{b}\rangle+\langle\mathbf{a}\cdot\mathbf{c}\rangle\]

Scalar Multiplication

\[\langle\alpha_1\mathbf{a}\cdot\alpha_2\mathbf{b}\rangle=\alpha_1\alpha_2\langle\mathbf{a}\cdot\mathbf{b}\rangle\]

Dot product is homogenious under scaling \[\alpha\langle\mathbf{a}\cdot\mathbf{b}\rangle = \langle\alpha\mathbf{a}\cdot\mathbf{b}\rangle = \langle\mathbf{a}\cdot\alpha\mathbf{b}\rangle\]

No cancellation

If \(\langle\mathbf{a}\cdot\mathbf{b}\rangle=\langle\mathbf{a}\cdot\mathbf{c}\rangle\) and \(\mathbf{a}\neq \mathbf{0}\), we can write \(\langle\mathbf{a}\cdot(\mathbf{b}-\mathbf{c})\rangle=0\) by distributive law, which means that \(\mathbf{a}\perp(\mathbf{b}-\mathbf{c})\), which allows \((\mathbf{b}-\mathbf{c})\neq\mathbf{0}\) and therefore \(\mathbf{b}\neq\mathbf{c}\).

Applications

Vector Projection

Given two vectors \(\mathbf{a}\) and \(\mathbf{b}\) we want to determine the projection of \(\mathbf{a}\) onto \(\mathbf{b}\), which is the vector \(\mathbf{a}'\) of length \(a'\) parallel to the vector \(\mathbf{b}\).

The length \(a'\) is called the Scalar Projection and is determined by

\[a' = |\mathbf{a}|\cos\theta = |\mathbf{a}|\underbrace{\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{|\mathbf{a}|\cdot|\mathbf{b}|}}_{\cos\theta}=\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{|\mathbf{b}|}=\left\langle\mathbf{a}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}\right\rangle=\langle\mathbf{a}\cdot\hat{\mathbf{b}}\rangle\]

Having the Scalar Projection, we can then calculate the Vector Projection:

\[\mathbf{a}' = a'\cdot\hat{\mathbf{b}} = \langle\mathbf{a}\cdot\hat{\mathbf{b}}\rangle\cdot\hat{\mathbf{b}}=\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{|\mathbf{b}|}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{|\mathbf{b}|^2}\mathbf{b}=\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{\langle\mathbf{b}\cdot\mathbf{b}\rangle}\mathbf{b}=:proj_{\mathbf{b}}(\mathbf{a})\]

Scalar Projection Percentage: The length of the distance \(a'\) to the total length of \(\mathbf{b}\) can be expressed in percent as follows:

\[\frac{a'}{|\mathbf{b}|} = \frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{|\mathbf{b}|\cdot |\mathbf{b}|} =\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{|\mathbf{b}|^2}=\frac{\langle\mathbf{a}\cdot\mathbf{b}\rangle}{\langle\mathbf{b}\cdot\mathbf{b}\rangle}\]

Which is interesting. It says that the Vector Projection is just a scaling of vector \(\mathbf{b}\) by the proportion of \(\mathbf{a}'\) to \(\mathbf{b}\).

Direction Cosines

In three dimensional space, a vector \(\mathbf{a}\) will form the angle \(\alpha\) with the x-axis, \(\beta\) with the y-axis and \(\gamma\) with the z-axis. These angles are called direction angles and the cosine of these angles are called direction cosines.

PLOT

When we use the standard basis vectors for the dimension \(\mathbf{i}, \mathbf{j}\) and \(\mathbf{k}\), the direction cosines are:

\[\begin{array}{rl}\cos\alpha =& \frac{\langle\mathbf{a}\cdot\mathbf{i}\rangle}{|\mathbf{a}|\cdot|\mathbf{i}|} = \frac{\mathbf{a}_1}{|\mathbf{a}|}\\\cos\beta =& \frac{\langle\mathbf{a}\cdot\mathbf{j}\rangle}{|\mathbf{a}|\cdot|\mathbf{j}|} = \frac{\mathbf{a}_2}{|\mathbf{a}|}\\\cos\gamma =& \frac{\langle\mathbf{a}\cdot\mathbf{k}\rangle}{|\mathbf{a}|\cdot|\mathbf{k}|} = \frac{\mathbf{a}_3}{|\mathbf{a}|}\\\end{array}\]

As such, the following properties hold:

  1. The vector \(\mathbf{b}=[\cos\alpha, \cos\beta, \cos\gamma]\) is a unit vector.
  2. \(\cos^2\alpha+ \cos^2\beta+ \cos^2\gamma=1\)
  3. \(\mathbf{a} = |\mathbf{a}|[\cos\alpha, \cos\beta, \cos\gamma]\)

Law of Cosines

\[{\begin{array}{rl}\langle\mathbf {c} \cdot \mathbf {c}\rangle &=\langle(\mathbf {a} -\mathbf {b} )\cdot (\mathbf {a} -\mathbf {b} )\rangle\\&=\langle\mathbf {a} \cdot \mathbf {a}\rangle -\langle\mathbf {a} \cdot \mathbf {b}\rangle -\langle\mathbf {b} \cdot \mathbf {a}\rangle +\langle\mathbf {b} \cdot \mathbf {b}\rangle \\&=a^{2}-\langle\mathbf {a} \cdot \mathbf {b}\rangle -\langle\mathbf {a} \cdot \mathbf {b}\rangle +b^{2}\\&=a^{2}-2\langle\mathbf {a} \cdot \mathbf {b}\rangle +b^{2}\\c^{2}&=a^{2}+b^{2}-2ab\cos \theta \\\end{array}}\]