Boltzman machine

Q1: The input of Boltzman machine is a binary vector, how we process the real number array input? By example the MNIST.

Q2:I didn’t understand very well the training algorithm for boltzman machine introduce in the second video. And if we can train the fully connected Boltzmann machine, what’s the drawback of it?

Q2: If the RBM is deep enough can it have same capacity like normal Boltzman machine?


1 Response to “Boltzman machine”

  1. 1 Sina Honari March 25, 2013 at 01:12

    Q1) For MNIST, the gray level of the pixel is considered as the probability that the bit corresponding to that pixel is one or zero and then a sampling is done based on that probability to get a binary value. This approach works for MNIST but may not work for other datasets.

    Q2) Boltmann Machines are very slow at convergence. We don’t have an efficient way of training the Boltmann Machines. The main draw-back of Boltmann Machines is in it’s Gibbs Sampling, each sampling only changes one variable while keeping all other variables fixed. This causes the convergence of the model very slow and makes it very unlikely to explore the modes since the possibility of change in these types of sampling is very low. However, in RBM since there is Block Gibbs Sampling, the visibles or hidden units are sampled simultaneously given the other layer and this causes the mixing and exploration of the modes more likely which is due to its higher chance of deviation from its previous value. This, in turn, makes the RBM models converge faster.

    Q3) We don’t know the exact answer to this question. Some families of distributions can be better represented by different models like RBM, DBM , etc. There are more representation power in deeper models like DBM compared to less deep models like RBM.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: