Q1: How is the wake-sleep algorithm for training belief nets different from the two-phase algorithm (CD) used in RBMs?

Q2: Can you think of any ways in which it is similar?


1 Response to “Wake-Sleep”

  1. 1 Pierre Luc Carrier March 28, 2013 at 14:58

    R1 – Mainly, with wake-sleep, you have 2 sets of weights for each layer (recognition and generative weights). For an RBM trained with CD, there is only one set of weights for a layer.

    R2 – It turns out using Wake-Sleep on a model with only 1 hidden layer with the constraint that the encoding weights be equal to the generative weights is equivalent to using CD1 to train that same model.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: