Vectors, Matrices, and Operations: A Discussion for Professionals

1. Vectors: The Fundamental Objects

A vector is a fundamental object in linear algebra, often visualized as an arrow in space or as an ordered list of numbers.

A 3-dimensional vector v is commonly represented as:

\[ v = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix} \]

This vector represents a point in three-dimensional space, denoted as \(\mathbb{R}^3\), and can be visualized as an arrow originating from the origin \((0,0,0)\) and pointing toward \((v_1, v_2, v_3)\).

Vector Operations

Vectors can be manipulated through two basic operations: addition and scalar multiplication.

• Addition: Vectors are added component-wise:

\[ \mathbf{u} + \mathbf{v} = \begin{bmatrix} u_1 \\ u_2 \\ u_3 \end{bmatrix} + \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix} = \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \\ u_3 + v_3 \end{bmatrix} \]

• Scalar Multiplication: A vector is multiplied by a scalar \(\alpha\), by multiplying each of its components:

\[ \alpha \mathbf{v} = \alpha \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix} = \begin{bmatrix} \alpha v_1 \\ \alpha v_2 \\ \alpha v_3 \end{bmatrix} \]

Linear Combinations

A linear combination of vectors is formed by multiplying each vector by a scalar and adding the results. For example, for vectors \(\mathbf{u}\) and \(\mathbf{v}\), the linear combination is:

\[ 2 \begin{bmatrix} 3 \\ 4 \\ 5 \end{bmatrix} - 3 \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} = \begin{bmatrix} 6 - 3 \\ 8 - 6 \\ 10 - 9 \end{bmatrix} = \begin{bmatrix} 3 \\ 2 \\ 1 \end{bmatrix} \]

Linear combinations are essential because they can represent any point on a line, plane, or higher-dimensional space spanned by the vectors.

2. Dot Product: Measuring Relationships Between Vectors

The dot product of two vectors \(\mathbf{u}\) and \(\mathbf{v}\), denoted as \(\mathbf{u} \cdot \mathbf{v}\), is calculated by multiplying corresponding components and summing the results:

\[ \mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + u_3 v_3 \]

This product also has a geometric interpretation:

\[ \mathbf{u} \cdot \mathbf{v} = ||\mathbf{u}|| ||\mathbf{v}|| \cos \theta \]

Where \(||\mathbf{u}||\) and \(||\mathbf{v}||\) are the lengths (or magnitudes) of the vectors, and \(\theta\) is the angle between them.

Applications of Dot Product

• Length of a Vector: The length of a vector \(\mathbf{v}\) is \(||\mathbf{v}|| = \sqrt{\mathbf{v} \cdot \mathbf{v}}\).
• Orthogonality: Two vectors are orthogonal if their dot product is zero: \(\mathbf{u} \cdot \mathbf{v} = 0\).
• Angles Between Vectors: The angle between two vectors can be found from the dot product:

\[ \cos \theta = \frac{\mathbf{u} \cdot \mathbf{v}}{||\mathbf{u}|| \, ||\mathbf{v}||} \]

Schwarz Inequality

The Schwarz inequality is a fundamental result that arises from the dot product:

\[ |\mathbf{u} \cdot \mathbf{v}| \leq ||\mathbf{u}|| \, ||\mathbf{v}|| \]

This inequality bounds the absolute value of the dot product by the product of the vector lengths.

3. Matrices: Representing Linear Transformations

A matrix is a rectangular array of numbers. A matrix with \(m\) rows and \(n\) columns is called an \(m \times n\) matrix.

Matrix-Vector Multiplication

Matrices operate on vectors through multiplication. If \(A\) is an \(m \times n\) matrix, and \(\mathbf{x}\) is a vector in \(\mathbb{R}^n\), then the product \(A \mathbf{x}\) is an \(m \times 1\) vector.

\[ A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}, \quad \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} \]

The result is the linear combination of the columns of \(A\), scaled by the components of \(\mathbf{x}\).

Column Space

The column space of a matrix \(A\), denoted as \(C(A)\), is the set of all possible vectors that can be formed by matrix-vector multiplication. It captures all possible outputs of the linear transformation represented by \(A\).

Problems to Solve

Problem 1: Vector Addition and Scalar Multiplication

Given two vectors \(\mathbf{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}\) and \(\mathbf{v} = \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}\), calculate \(2\mathbf{u} + 3\mathbf{v}\).

Hint: Perform scalar multiplication and vector addition.

Problem 2: Dot Product

Find the dot product of the vectors \(\mathbf{u} = \begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix}\) and \(\mathbf{v} = \begin{bmatrix} 1 \\ 4 \\ 2 \end{bmatrix}\).

Hint: Use \(\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + u_3 v_3\).

Problem 3: Vector Length and Orthogonality

Determine whether the vectors \(\mathbf{u} = \begin{bmatrix} 1 \\ 2 \\ 2 \end{bmatrix}\) and \(\mathbf{v} = \begin{bmatrix} 2 \\ -4 \\ 0 \end{bmatrix}\) are orthogonal.

Hint: Compute the dot product and check if it’s zero.

Problem 4: Matrix-Vector Multiplication

Given \(A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}\) and \(\mathbf{x} = \begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix}\), calculate \(A \mathbf{x}\).

Hint: Multiply each row of the matrix by the vector.

Problem 5: Column Space

Determine whether \(\mathbf{b} = \begin{bmatrix} 5 \\ 11 \end{bmatrix}\) is in the column space of \(A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}\).

Hint: Solve \(A \mathbf{x} = \mathbf{b}\) for \(\mathbf{x}\).

Takeaway

In this section, we explored the foundational concepts of vectors, matrices, and their operations. These concepts serve as the building blocks of linear algebra, enabling deeper exploration of topics like linear transformations, eigenvectors, and matrix factorizations. Mastery of these topics is essential for any professional working in data science, machine learning, or related fields.