Skip to content

Commit 3d99ea7

Browse files
author
liuzi
committed
correction of math foramts and logic
1 parent b8d2180 commit 3d99ea7

File tree

1 file changed

+5
-3
lines changed

1 file changed

+5
-3
lines changed

_posts/2025-02-15-generative-models.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -249,16 +249,18 @@ $$
249249
p(y=1|x;\phi, \mu_0, \mu_1, \Sigma) = \frac{1}{1+\exp(-\theta^T x)}
250250
$$
251251

252-
This reveals a profound connection between logistic regression and GDA: the parameters $$\theta$$ in logistic regression can be directly expressed as functions of the GDA parameters $$\phi, \mu_0, \mu_1, \Sigma$$. Remarkably, the resulting form is mathematically identical to the hypothesis function used in logistic regression, which models the conditional probability $$p(y=1|x)$$ discriminatively. This equivalence demonstrates how these seemingly different approaches can converge to similar mathematical formulations despite their distinct underlying philosophies.
252+
This reveals a profound connection between logistic regression and GDA: the parameters $$\theta$$ in logistic regression can be directly expressed as functions of the GDA parameters $$\phi, \mu_0, \mu_1, \Sigma$$. Remarkably, the resulting form is mathematically identical to the hypothesis function used in logistic regression, which models the conditional probability
253+
$$p(y|x)$$ discriminatively. This equivalence demonstrates how these seemingly different approaches can converge to similar mathematical formulations despite their distinct underlying philosophies.
253254

254-
Generally, generative models and discriminative models generate different decision boundaries when trained on the same dataset. The following points show the differences between the generative GDA model and the discriminative logistic regression model:
255+
The GDA model and the logistic regression model produce distinct decision boundaries when trained on identical datasets, despite their mathematical connections. These fundamental differences between the generative GDA model and the discriminative logistic regression model can be understood through the following key points:
255256

256257
- In the GDA model, when
257258
$$p(x|y)$$ follows a multivariate Gaussian distribution with a shared covariance matrix, $$p(y|x)$$ will necessarily take the form of a sigmoid function. However, the converse doesn't hold: defining $$p(y|x)$$ as a sigmoid function doesn't guarantee that $$p(x|y)$$ follows a multivariate Gaussian distribution. This demonstrates that GDA imposes stronger modeling assumptions than logistic regression.
258259

259260
- Because of these stronger assumptions, GDA excels when these assumptions accurately reflect the underlying data distribution. In contrast, logistic regression, with its more flexible and weaker assumptions, demonstrates greater robustness across diverse data distributions, provided there's sufficient training data available.
260261

261-
To be more general, this comparison can be extended to all discriminative and generative models. Generative models learn the joint probability distribution p(x,y) and make stronger assumptions about the data, which is beneficial when these assumptions hold true and training data is limited. Discriminative models, on the other hand, directly learn the conditional probability p(y|x) without modeling the input distribution, making them more robust to misspecification of the data distribution but typically requiring larger datasets to achieve optimal performance.
262+
To be more general, this comparison can be extended to all discriminative and generative models. Generative models learn the joint probability distribution p(x,y) and make stronger assumptions about the data, which is beneficial when these assumptions hold true and training data is limited. Discriminative models, on the other hand, directly learn the conditional probability
263+
p(y|x) without modeling the input distribution, making them more robust to misspecification of the data distribution but typically requiring larger datasets to achieve optimal performance.
262264

263265

264266

0 commit comments

Comments
 (0)