Random variable: A random variable \(X\) is a function that maps the sample space \(\Omega\) to the set of real numbers \(\mathbb{R}\). It assigns to each element \(\omega \in \Omega\) one and only one value \(X(\omega) = x\).
Let \(X\) be a random variable. Then its cumulative distribution function (cdf) is defined by \[F_X(x) = P(X\le x)\]
A random variable \(X\) is a continuous random variable if its cumulative distribution function \(F_X(x)\) is a continuous function \(\forall\) \(x\in\mathbb{R}\).
The probability density function (pdf) of a random variable \(X\) is a non-negative function \(f_X(x)\) such that \[F_X(x) = \int_{-\infty}^x f_X(x) \ \mathrm dx, \quad \forall \ x \in \mathbb{R}\]
A cdf \(F_X(x)\) always has the following properties:
\({\displaystyle \lim_{x\to -\infty} F_X(x) = 0}\)
\({\displaystyle \lim_{x\to \infty} F_X(x) = 1}\)
\(F_X(x)\) is a non-decreasing function.
If \(f_X(x)\) is the pdf of a random variable \(X\), then at the points where \(f_X(x)\) is continuous, \[\dfrac{\mathrm d F_X(x)}{\mathrm d x} = f_X(x)\]
A pdf \(f_X(x)\) always has the following properties:
\(f_X(x)\ge 0\ \forall\ x\in\mathbb{R}\)
\(\int_{\mathbb{R}} f_X(x)\ \mathrm d x = 1\), where \(\int_A\) represents integration over the set \(A\).
If \(A\subset \mathbb{R}\), then \(P(A)=\int_A f_X(x)\ \mathrm dx\). In other words, \[P(a\le X \le b ) = \int_a^b f_X(x) \ \mathrm dx\]
The support of a random variable \(X\) is \[\textrm{support } X = \{ x \mid f_X(x) > 0 \}\]
For a random variable \(X\), its expectation or expected value is \[E[X] = \int_{-\infty}^\infty x f_X(x) \ \mathrm dx\] often denoted as \(\mu_X\).
If \(X\) be a random variable and \(g(X)\) be some function of the random variable, then \[E[g(X)] = \int_{-\infty} ^\infty g(x) f_X(x) \ \mathrm dx\]
Expectation is a linear operation:
\(E[aX+b] = a E[x] + b\) for \(a, b \in \mathbb{R}\)
\(E[X_1+X_2+\ldots + X_n] = E[X_1] + E[X_2] + \ldots + E[X_n]\)
For a random variable \(X\), its variance, denoted as \(\sigma_X^2\), is \[\sigma_X^2 = \mathrm{Var}(X) = E[(X-\mu_X)^2] = E[X^2] - (E[X])^2\] so that \[\mathrm{Var}(X) = \int_{-\infty}^\infty (x-\mu_X)^2 f_X(x)\ \mathrm dx\]
For a random variable \(X\) and \(a, b \in \mathbb{R}\): \[\mathrm{Var}(aX+b) = a^2 \mathrm{Var}(X)\]
If \(X\) is a continuous random variable, and \(A\) is the event that \(a<X<b\) (where possibly \(b \to \infty\) or \(a \to -\infty\)), then the conditional pdf of \(X\) given the event \(A\) is \[f_{X\mid A} (x) = \left\{ \begin{matrix} \dfrac{f_X(x)}{P(A)} & a\le x < b \\[2ex] 0 & \textrm{ otherwise} \end{matrix}\right.\]
The conditional expectation of \(X\) given \(A\) is \[E[X|A] = \int_{-\infty}^\infty x f_{X\mid A}(x)\ \mathrm{d} x\]
The conditional expectation of \(g(X)\) given \(A\) is \[E[g(X)|A] = \int_{-\infty}^\infty g(x) f_{X\mid A}(x)\ \mathrm{d} x\]
The conditional variance of \(X\) given \(A\) is \[\mathrm{Var}(X|A) = E[X^2|A] - \left( E[X|A]\right)^2\]
For a random variable \(X\) and any event \(A\), the total expectation of \(X\) is \[E[X] = P(A) E[X|A] + P(A^c) E[X|A^c]\]
If \(Y=g(X)\), where \(g:\mathbb{R}\to \mathbb{R}\) is a strictly monotonous differentiable function so that \(g\) has a unique inverse and let \(h = g^{-1}\). Then, the pdf of the random variable \(Y\) is given by \[f_Y(y) = f_X(h(y)) \vert h'(y)\vert\]
If \(Y=g(X)\), where \(g:\mathbb{R}\to \mathbb{R}\) is differentiable but \(g\) is not strictly monotonous. However, there is a partition of \(\mathbb{R}\) into disjoint intervals, say \(I_1\), \(I_2\), …in each of which \(g\) is strictly monotonous and let \(g_k(x) = g(x)\) for \(x\in I_k\). Then, the pdf of the random variable \(Y\) is given by \[F_Y(y) = \sum_{k} f_X\left(g_k^{-1}(y)\right) \left\vert \dfrac{\mathrm d}{\mathrm d y} g_k^{-1}(y)\right\vert\]
Two random variables \(X\) and \(Y\) are jointly continuous if there exists a non-negative function \(f_{X,Y} : \mathbb{R}^2 \to \mathbb{R}\), such that, for any set \(A\subset \mathbb{R}^2\): \[P\left( (X, Y) \in A \right) = \iint_A f_{X,Y}(x, y)\ \mathrm{d}x \,\mathrm{d} y\] The function \(f_{X,Y}\) is called the joint probability density function of \(X\) and \(Y\).
The joint pdf of \(X\) and \(Y\) must satisfy \[\int_{-\infty}^\infty \int_{-\infty}^\infty f_{X,Y} (x, y) \ \mathrm{d}x \,\mathrm{d} y = 1\]
The marginal pdf of \(X\) can be obtained from the joint pdf by integrating over all values of \(y\): \[f_X(x) = \int_{-\infty}^\infty f_{X,Y}(x, y) \ \mathrm dy, \quad \forall\, x\]
The marginal pdf of \(Y\) can be obtained from the joint pdf by integrating over all values of \(x\): \[f_Y(x) = \int_{-\infty}^\infty f_{X,Y}(x, y) \ \mathrm dx, \quad \forall\, y\]
If \(X\) and \(Y\) are jointly continuous, the conditional pdf of \(X\) given \(Y\) is \[f_{X\mid Y} (x\mid y) = \dfrac{f_{X,Y}(x,y)}{f_Y(y)}\]
For two jointly continuous random variables \(X\) and \(Y\):
The expected value of \(X\) given \(Y=y\) is \[E[X|Y=y] = \int_{-\infty}^\infty x f_{X\mid Y}(x\mid y)\ \mathrm{d} x\]
For a function \(g(X)\) of \(X\) \[E[g(X)|Y=y] = \int_{-\infty}^\infty g(x) f_{X\mid Y}(x\mid y)\ \mathrm{d} x\]
The conditional variance of \(X\) given \(A\) is \[\mathrm{Var}(X|Y=y) = E[X^2|Y=y] - \left( E[X|Y=y]\right)^2\]
If two jointly continuous random variables are independent if and only if \[f_{X\mid Y}(x|y) = f_X(x)\] Equivalently, two jointly continuous random variables are independent if and only if \[f_{X, Y}(x,y) = f_X(x) f_Y(y)\]
For independent random variables \(X\) and \(Y\) \[E[XY] = E[X]E[Y]\]
If \(X\) and \(Y\) be independent random variables, then \(\forall\) functions \(g\) and \(h\) \[E[g(X) h(Y)] = E[g(X)] E[h(X)]\]
Special distributions of independent random variables: Let \(X\) and \(Y\) be two independent random variables, with pdf \(f_X(x)\), \(f_Y(x)\) and cdf \(F_X(x)\), \(F_Y(x)\), respectively.
Sum: The pdf of \(Z=X+Y\) is obtained using the convolution integral: \[f_Z(z) = \int_{-\infty}^\infty f_X(x) f_Y(z-x)\ \mathrm{d}x\]
Maximum: The cdf of the function \(Z = \max \{X, Y\}\) is \[F_Z(z) = F_X(z) F_Y(z)\]
Minimum: The cdf of the function \(Z = \min \{X, Y\}\) is \[F_Z(z) = 1-\left(1-F_X(z)\right)\left(1- F_Y(z)\right)\]
If \(X\) and \(Y\) are random variables with expectations \(\mu_X\) and \(\mu_Y\) respectively, then their covariance is defined to be \[\mathrm{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)] = E[XY]-\mu_X\mu_Y\]
The correlation coefficient of random variables \(X\) and \(Y\) is defined to be \[\rho = \dfrac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)\mathrm{Var}(Y)}}\]
If the random variables \(X\) and \(Y\) are independent, then \[\mathrm{Cov}(X,Y) = 0\] (Note: the converse need not be true.)
Transformed random variables: Suppose the random variables \(X_1\), \(X_2\) are transformed to random variables \(Y_1\), \(Y_2\) by some one to one mapping: \[\begin{matrix} Y_1 = u_1(X_1, X_2),\\ Y_2 = u_2(X_1, X_2) \end{matrix}\] which can be inverted to \[\begin{matrix} X_1 = w_1(Y_1, Y_2),\\ X_2 = w_2(Y_1, Y_2) \end{matrix}\] Let \(J\) be the Jacobian of this transformation: \[J = \det \begin{bmatrix} \dfrac{\partial w_1}{\partial y_1} & \dfrac{\partial w_1}{\partial y_2} \\[3ex] \dfrac{\partial w_2}{\partial y_1} & \dfrac{\partial w_2}{\partial y_2} \end{bmatrix}\] Then the joint pdf of \(Y_1\), \(Y_2\) is \[f_{Y_1, Y_2} (y_1, y_2) = f_{X_1, X_2} \left( w_1(y_1, y_2), w_2(y_1, y_2)\right) |J|\] where \(|\cdot|\) represents the absolute value.