Why isn't a gaussian kernel subject to the curse of dimensionality?

by Amy   Last Updated July 12, 2019 10:19 AM - source

This has been bugging me for a while now. I understand from this answer why gaussian kernels are effective. But I can't wrap my head around the intuition of why the infinite dimensional feature map 𝜙(𝑥) in the gaussian kernel is not subject to problems arising in this infinite dimensional space. I know that the "feature mapped" vector is never directly computed, but the inner product for example is evaluated and used as a measure of similarity between vectors.

But let's assume we are working with data sampled from a hypersphere, if in higher dimensions, the data becomes subject to problems like the concentration of measure phenomenon where everything shrinks to near zero volume and sampled points becomes randomly distributed/orthogonal on average, how would a gaussian kernel benefit in this setting since it would be taking the input data and mapping it to infinite dimensional space where in theory, all points should be uniformly close together around the center of the sphere and orthogonal on average, thus providing no insight for classification/clustering?

Or does the gaussian kernel map these points to a different geometrical structure in infinite dimensional Hilbert space? If so, why is this not then subject to other problems that arise due to the curse of dimensionality?



Related Questions



Is a kernel space necessarily high dimensional?

Updated April 02, 2016 08:08 AM


Understanding Kernel PCA

Updated August 02, 2018 11:19 AM

A Gaussian Kernel

Updated March 03, 2018 18:19 PM