CD & PCA-whitened

1) Why in CD learning if the weights get bigger we need more iterations of CD to get unbiased samples from the equilibrium distribution?

2) What is the PCA-whitening that is applied to the data as preprocessing?


1 Response to “CD & PCA-whitened”

  1. 1 Pierre Luc Carrier April 11, 2013 at 12:24

    Q1 : Briefly, bigger weights => sigmoids units tend to saturate more => Markkov Chains tend to converge more => mixing between modes becomes harder.

    Q2 : PCA projects the data in a space where the attributes are uncorrelated. PCA-Whitening does PCA + ensure that the attributes in the PCA space have unit variance.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: