RBMs and Half-CD

Quiz question :

Adam and Bob are arguing about RBMs. Adam proposes to use his new algorithm called Half-CD to train an RBM. In Half-CD, just like CD1, you start sampling from a training point x1. However, instead of doing like CD1 in which you do the following steps :

  1. Sample h1, x2 and then h2
  2. Compute the positive phase gradient on (x1,h1) and the negative phase gradient on (x2,h2)

in Half-CD he proposes to do :

  1. Sample h1 and then x2
  2. Compute the positive phase gradient on (x1,h1) and the negative phase gradient on (x2,h1)

Adam argues that , since in practice CD-1 is often good enough to allow learning and that Half-CD performs only one ‘half-step’  of Gibbs sampling less than CD-1, Half-CD should also be good enough to allow learning. Moreover, he says, doing one less ‘half-step’ should also save time and make learning faster.

Bob feels that this is wrong but doesn’t really know why. Who’s right and why?

Advertisements

1 Response to “RBMs and Half-CD”


  1. 1 Gabriel Bernier-Colborne March 19, 2013 at 10:57

    Adam’s idea is not as crazy as it seems, but the estimation will be less exact than with CD1. For more information on different variations of contrastive divergence, see Bengio & Delalleau (2009): http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/232.


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s





%d bloggers like this: