When training a deep auto-encoder for the purposes of semantic hashing, Hinton adds noise to the input during the fine-tuning stage, because this forces the activities of the sigmoid units in the code to become bimodal (saturate). Why does adding noise have this effect?
Posts Tagged 'semantic hashing'
Noise and Saturation
Published April 18, 2013 apr18 , hinton-lectures , quiz 1 CommentTags: auto-encoders, Gabriel, semantic hashing