How neural networks build up their understanding of images. Feature visualization allows us to see how GoogLeNet, trained on the ImageNet dataset, builds up its understanding of images over many layers. Visualizations of all channel are available in the appendix.
We often think of Momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. But it has other interesting behavior. It allows a larger range of step-sizes to be used, and creates its own oscillations. What is going on?
Every map projection introduces distortion, and each has its own set of problems. One of the most common criticisms of the Mercator map is that it exaggerates the size of countries nearer the poles (US, Russia, Europe), while downplaying the size of those near the equator (the African Continent).
Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading. By exploring how it behaves in simple cases, we can learn to use it more effectively.