Skip to content

Conversation

MohamedMostafa259
Copy link

In the original code, the RegularizedDense wrapper applies he_normal initialization to all layers, including the output layer. However, he_normal is optimized for ReLU activations and is not ideal for the final softmax layer according to Table 11-1 (Initialization parameters for each type of activation function) on page 360.

This PR updates the last layer to use glorot_normal initialization, which is better suited for layers with softmax activation. The change is small but improves clarity and correctness by aligning the initialization strategy with the activation function used.

Summary of changes

Before:

RegularizedDense(10, activation="softmax")

After:

RegularizedDense(10, activation="softmax", kernel_initializer="glorot_normal")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant