4. Deep Learning

Optimization Deep — Quiz

Test your understanding of optimization deep with 5 practice questions.

Practice Questions

Question 1

Which of the following optimizers is a variant of Stochastic Gradient Descent (SGD) that incorporates an exponentially decaying average of past gradients?

Question 2

When applying batch normalization, which of the following statistical measures are typically used to normalize the activations within a mini-batch?

Question 3

A learning rate schedule that reduces the learning rate by a fixed factor every few epochs is known as a:

Question 4

If the loss function for a model is given by $L(w) = 4w^3 - 2w^2 + 5w - 1$, what is the gradient of the loss function with respect to $w$?

Question 5

Which of the following techniques is primarily designed to prevent the issue of 'exploding gradients' in deep neural networks?