Posts Tagged 'semantic hashing'

Noise and Saturation

When training a deep auto-encoder for the purposes of semantic hashing, Hinton adds noise to the input during the fine-tuning stage, because this forces the activities of the sigmoid units in the code to become bimodal (saturate). Why does adding noise have this effect?