Dimensionality Reduction From Several Angles

I will present five past and current projects that attack the problem of dimensionality reduction (DR) from quite different methodological angles. Two projects nicely fit into the usual mold of technique-driven work on algorithms for DR. Glimmer is a multilevel multidimensional scaling (MDS) algorithm that exploits the GPU. Glint is a new MDS framework that achieves high performance on costly distance functions. In contrast, the DimStiller project is a foray into systems rather than algorithms, built around the idea of "DR for the rest of us". It is a toolkit for DR that provides local and global guidance to users who may not be experts in the mathematics of high-dimensional data analysis. A third kind of project combines evaluation and the creation of taxonomies. Our recent taxonomy of visual cluster separation factors arose from the systematic qualitative examination of over 800 scatterplots of dimensionally reduced data, and includes an analysis of the reasons for failure of previous cluster separation metrics. I will also discuss the current work of a task taxonomy that is grounded in a two-year qualitative study of high-dimensional data analysts in many domains, to discover how the use of DR "in the wild" does and does not match up with the assumptions that underlie previous algorithmic work.