50 things I learned at NIPS 2016

Current machine learning techniques work best when training data and real data come from the same distribution. When it’s likely that an algorithm will be applied in a setting that is different from the training setting, it can be good to have the test set come from a different distribution than the training set, hopefully mirroring how the real application data will again come from a different distribution. This way, you get a better sense for how the algorithm does under distribution shift

Why does deep learning work now, but not 20 years ago, even though many of the core ideas were there? In one sentence: We have more data, more compute, better software engineering, and a few algorithmic innovations

For some products at Baidu, the main purpose is to acquire data from users, not revenue

For speech recognition, ML algorithms are now benchmarked against teams of humans, not individuals