Q1) Consider that you have a dataset of 2’s and 5’s digits from the MNIST dataset. How would you discriminate 2’s and 5’s using only one autoencoder without any labels? (Use your imagination-there is very simple way to do it)
Q2) Given that the autoencoders can learn the implicit dimensionality of the data (lower dimensional manifold), can we use it as a compression algorithm? How and what would be the necessary conditions to minimize the information loss/reconstruction error for the compression(e.g.: Using Tied weights, type of nonlinearity, sparsity, training objective….etc.). Would disentangling factors of variations help for compression as well?
Q3) Sigmoid belief networks have an explaining away effect, which models we have seen so far have the same problem? What is their common characteristic?
Representation Learning 