Which of the following scenarios would most likely lead to the 'exploding gradient' problem during neural network training?
Question 2
Consider a deep neural network with multiple hidden layers. During training, if the gradients become extremely small as they propagate backward through the layers, what is the most direct consequence for the early layers of the network?
Question 3
When implementing a feedforward neural network for a multi-class classification problem with $K$ classes, which activation function is typically used in the output layer, and why?
Question 4
A machine learning practitioner is training a very deep neural network and observes that the model's performance is not improving, and the weights in the initial layers are barely changing. Which of the following is the most probable cause?
Question 5
In the context of training deep neural networks, what is the primary purpose of using a learning rate scheduler?
