Why are kernel methods with RBFs effective for handwritten digits (letters) classification?

Why are kernel methods with RBFs effective for handwritten digits (letters) classification?

• The question emerged while reading Ch. 3 of http://www.gaussianprocess.org/gpml/ . In the end of this chapter, the authors gave results for the problem of handwritten digits classification (16x16 greyscale pictures); features are 256 pixel intensities + bias. I was surprised that in such a high-dimensional problem, 'metric' methods, like Gaussian processes with squared exponential kernel, or SVM with the same kernel, behave quite nice without any dimension reduction preceeded. Also, I heard sometimes that SVM is good for [essentially bag-of-word] text classification. Why aren't they suffering from the curse of dimensionality?

• Answer:

  To put this interesting discussion on the ground, one can modify the toy script from http://scikit-learn.org/stable/auto_examples/plot_digits_classification.html to get some simple statistics. For example, let us look at the last test vector, and how far it is from 519 support vectors (the space has 8*6=42 dimensions). Distances are big enough, laying in the interval [26.7394839142 64.88451279]. However, gamma is small (0.001), i.e. variance is big so "effective distance" is small (and that is the key, I think!). As a consequence, scalar product (i.e. exp(-gamma*|test-support|^2) with all 519 support vectors lays in the interval [0.937175688934 0.973614850818]