# Gradient

A **gradient **is a vector that stores the partial derivatives of multi variable functions, often denoted by $\nabla$. It helps us calculate the **slope **at a specific point on a curve for functions with multiple independent variables.

**Find a Gradient**

**Find a Gradient**

Consider a function with two variables (x and y): $f(x,y) = x^2 + y^3$

**
**1) Find **partial derivative **with respect to **x** (Treat y as a constant like a random number 12)

2)** **Find **partial derivative **with respect to** y **(Treat x as a constant)

3) Store partial derivatives in a **gradient**

### Properties of Gradients

There are two additional properties of gradients that are especially useful in deep learning. A gradient:

Always points in the direction of greatest increase of a function (explained here)

Is zero at a local maximum or local minimum

## Directional Derivative

The **directional derivative** $\nabla_{\vec v} f$ is the rate at which the function $f(x,y)$ changes at a point $(x_1,y_1)$ in the direction ${\vec v}$.

**Directional derivative** is computed by taking the **dot product** of the gradient of $f$ and a unit vector ${\vec v}$

Note: Directional derivative of a function is a **scalar **while gradient is a **vector**.

### Find Directional Derivative

Consider a function with two variables (x and y): $f(x,y) = x^2 + y^3$

As described above, we take the **dot product** of the gradient and the directional vector:

We can rewrite the dot product as:

Hence, the directional derivative $\nabla_{\vec v} f$at co-ordinates $(5, 4)$ is: $\nabla_{\vec v} f = 4x+9y^2 = 4(5)+9(4)^2 = 164$

Link: - http://wiki.fast.ai/index.php/Calculus_for_Deep_Learning

Last updated