Skip to content

Vectors

Definition / Introduction

  • In Linear Algebra, a vector is an ordered list (or array) of numbers (scalars). It can be thought of as representing a point in space or a quantity having both magnitude and direction.
  • Vectors are fundamental objects used to represent data points, features, parameters in models, embeddings, and directions in multi-dimensional spaces.
  • They are typically represented as column vectors (most common in ML/DL contexts) or row vectors.

Key Concepts

1. Representation

  • Notation: Usually denoted by lowercase bold letters (e.g., \(\mathbf{v}, \mathbf{x}, \mathbf{w}\)) or lowercase letters with an arrow (e.g., \(\vec{v}\)).
  • Components/Elements: The individual numbers (scalars) within the vector. \(v_i\) is the \(i\)-th component.
  • Dimension: The number of components (\(n\)) in the vector determines its dimension. A vector \(\mathbf{v} \in \mathbb{R}^n\) is an n-dimensional real vector.
  • Column Vector: Written vertically: $$ \mathbf{v} = \begin{bmatrix} v_1 \ v_2 \ \vdots \ v_n \end{bmatrix} \in \mathbb{R}^n $$
  • Row Vector: Written horizontally: $$ \mathbf{v}^T = \begin{bmatrix} v_1 & v_2 & \dots & v_n \end{bmatrix} \in \mathbb{R}^{1 \times n} $$ Often represented as the transpose (denoted by \(T\)) of a column vector.

2. Geometric Interpretation

  • A vector \(\mathbf{v}\) in \(\mathbb{R}^n\) can be visualized (for \(n=2, 3\)) as:
    • An arrow starting from the origin \(\mathbf{0} = [0, ..., 0]^T\) and ending at the point specified by its components \((v_1, v_2, ..., v_n)\).
    • The position vector of the point \((v_1, v_2, ..., v_n)\) relative to the origin.
  • The magnitude (or length or norm, often \(||\mathbf{v}||\)) of the vector corresponds to the length of the arrow.
  • The direction corresponds to the orientation of the arrow in space.

3. Algebraic Interpretation

  • An ordered list of numbers representing attributes or coordinates.

4. Examples in Data Science / AI

  • Data Point: A single observation (e.g., a user, an image) can be represented as a vector where each component is a feature value (e.g., \([\text{age}, \text{income}, \text{pages\_visited}]^T\)).
  • Feature Vector: The collection of features for one data point.
  • Parameters/Weights: The weights connecting neurons in a layer, or the weights of a linear model, are often organized into vectors (or matrices).
  • Word Embeddings (NLP): Words are represented as dense vectors in a high-dimensional space, where similar words have vectors close to each other (e.g., Word2Vec, GloVe).
  • Gradients: The gradient \(\nabla f\) of a scalar function \(f\) with respect to multiple variables is a vector pointing in the direction of the steepest ascent. Used in optimization.

Connections to Other Topics

Summary

  • A vector (\(\mathbf{v}\)) is an ordered list (array) of numbers (scalars).
  • Represents magnitude and direction, or a point/location in space \(\mathbb{R}^n\).
  • Typically written as column vectors \(\begin{bmatrix} ... \end{bmatrix}\) in ML contexts. Dimension = number of components.
  • Fundamental for representing data points, features, parameters, embeddings, and gradients.

Sources