Notation

Throughout this book, we adhere to the following notational conventions. Note that some of these symbols are placeholders, while others refer to specific objects. As a general rule of thumb, the indefinite article “a” often indicates that the symbol is a placeholder and that similarly formatted symbols can denote other objects of the same type. For example, “ $x$ : a scalar” means that lowercased letters generally represent scalar values, but “ $Z$ : the set of integers” refers specifically to the symbol $Z$ .

Numerical Objects

$x$ : a scalar
$x$ : a vector
$X$ : a matrix
$X$ : a general tensor
$I$ : the identity matrix (of some given dimension), i.e., a square matrix with $1$ on all diagonal entries and $0$ on all off-diagonals
$x_{i}$ , $[x]_{i}$ : the $i^{th}$ element of vector $x$
$x_{i j}$ , $x_{i, j}$ , $[X]_{i j}$ , $[X]_{i, j}$ : the element of matrix $X$ at row $i$ and column $j$ .

Set Theory

$X$ : a set
$Z$ : the set of integers
$Z^{+}$ : the set of positive integers
$R$ : the set of real numbers
$R^{n}$ : the set of $n$ -dimensional vectors of real numbers
$R^{a \times b}$ : The set of matrices of real numbers with $a$ rows and $b$ columns
$$ \mathcal{X} $: c a r d i n a l i t y (n u m b e r o f e l e m e n t s) o f s e t$ \mathcal{X}$$
$A \cup B$ : union of sets $A$ and $B$
$A \cap B$ : intersection of sets $A$ and $B$
$A ∖ B$ : set subtraction of $B$ from $A$ (contains only those elements of $A$ that do not belong to $B$ )

Functions and Operators

$f (\cdot)$ : a function
$\log (\cdot)$ : the natural logarithm (base $e$ )
$\log_{2} (\cdot)$ : logarithm to base $2$
$\exp (\cdot)$ : the exponential function
$1 (\cdot)$ : the indicator function; evaluates to $1$ if the boolean argument is true, and $0$ otherwise
$1_{X} (z)$ : the set-membership indicator function; evaluates to $1$ if the element $z$ belongs to the set $X$ and $0$ otherwise
${(\cdot)}^{⊤}$ : transpose of a vector or a matrix
$X^{- 1}$ : inverse of matrix $X$
$⊙$ : Hadamard (elementwise) product
$[\cdot, \cdot]$ : concatenation
$‖ \cdot ‖_{p}$ : $ℓ_{p}$ norm
$‖ \cdot ‖$ : $ℓ_{2}$ norm
$⟨ x, y ⟩$ : inner (dot) product of vectors $x$ and $y$
$\sum$ : summation over a collection of elements
$\prod$ : product over a collection of elements
$\overset{def}{=}$ : an equality asserted as a definition of the symbol on the left-hand side

Calculus

$\frac{d y}{d x}$ : derivative of $y$ with respect to $x$
$\frac{\partial y}{\partial x}$ : partial derivative of $y$ with respect to $x$
$\nabla_{x} y$ : gradient of $y$ with respect to $x$
$\int_{a}^{b} f (x) d x$ : definite integral of $f$ from $a$ to $b$ with respect to $x$
$\int f (x) d x$ : indefinite integral of $f$ with respect to $x$

Probability and Information Theory

$X$ : a random variable
$P$ : a probability distribution
$X \sim P$ : the random variable $X$ follows distribution $P$
$P (X = x)$ : the probability assigned to the event where random variable $X$ takes value $x$
$P (X ∣ Y)$ : the conditional probability distribution of $X$ given $Y$
$p (\cdot)$ : a probability density function (PDF) associated with distribution $P$
$E [X]$ : expectation of a random variable $X$
$X ⊥ Y$ : random variables $X$ and $Y$ are independent
$X ⊥ Y ∣ Z$ : random variables $X$ and $Y$ are conditionally independent given $Z$
$σ_{X}$ : standard deviation of random variable $X$
$Var (X)$ : variance of random variable $X$ , equal to $σ_{X}^{2}$
$Cov (X, Y)$ : covariance of random variables $X$ and $Y$
$ρ (X, Y)$ : the Pearson correlation coefficient between $X$ and $Y$ , equals $\frac{Cov (X, Y)}{σ_{X} σ_{Y}}$
$H (X)$ : entropy of random variable $X$
$D_{KL} (P ‖ Q)$ : the KL-divergence (or relative entropy) from distribution $Q$ to distribution $P$

Next - Preliminaries

Previous - Installation

d2l-pdl