INTERACTIVE READING NOTES / OPTIMIZATION
Understanding gradient descent, one step at a time
Imagine standing on a curved hillside. How could you reach lower ground using only the slope beneath your feet? A simple quadratic function makes the idea visible.
01Find the downhill direction
The gradient points in the direction of steepest increase. To reduce the loss, move in the opposite direction of the gradient. You do not need to know where the valley is; you only need to check the slope at your new position after each step.
02Choose the step size
Choosing the right direction is only the beginning. The learning rate η sets the step size: too small is slow, while too large may overshoot. Move the slider to see the next step from x = 4.
Next step: 4 − 0.25 × 8 = 2.00
03Keep understanding in place
An explanation belongs to a document when you can find it again. After sending a question, a discussion marker appears beside the passage. Select it to return to that sentence and its conversation.
∎