Fix: Override output layer initializer to glorot_normal for softmax compatibility #203
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the original code, the
RegularizedDense
wrapper applieshe_normal
initialization to all layers, including the output layer. However,he_normal
is optimized for ReLU activations and is not ideal for the finalsoftmax
layer according to Table 11-1 (Initialization parameters for each type of activation function) on page 360.This PR updates the last layer to use
glorot_normal
initialization, which is better suited for layers with softmax activation. The change is small but improves clarity and correctness by aligning the initialization strategy with the activation function used.Summary of changes
Before:
After: