OpenSource Internship opportunity by OpenGenus for programmers. Apply now.
Table of Contents:
 Introduction to Projection Pursuit
 Purpose
 Projection Index
 Method
 Implementation in python
 Advantages
 limitations
 Application in real world
Introduction to Projection Pursuit
Projection Pursuit is a statistical technique to find the most "interesting" possible projections of multidimensional data. Generally, projections with more deviation from normal distribution are preferred and considered as more interesting.
William Henry Kruskal was the first who proposed the concept of Projection Pursuit techniques. Jerome H. Friedman and John Tukey implemented it successfully.
Years later, Friedman and Stuetzle extended the idea behind projection pursuit and added projection pursuit regression (PPR), projection pursuit classification (PPC), and projection pursuit density estimation (PPDE).
purpose
Projection pursuit is used for dimensionality reduction that is used to find nonlinear projections with lesser dimension of highdimensional data. Main purpose is searching for nonlinear patterns in the data and find projections that highlight these patterns and reveal hidden structures and relationships in the data.
Projection Index
Projection Index measures the degree of nonlinearity in the projection. It defines quality of the projections. It helps to choose desired projection with maximum nonlinearity. Value of the projection index of the optimum desired projection should be higher or lower depends on the goal. For maximizing variance, separability of classes, or cluster compactness higher value of the projection index is consider to be better. In the same way, to minimize withincluster variance or projection error higher value of the projection index is more preferred. In practical scenarios, a balanced criteria of maximization and minimization of various properties. The projection index is typically a nonlinear, nonconvex function.
Method
Let us consider that we have a two dimensional data set with two features, x1, and x2. f is the projection of the data on to the onedimensional space.
Step1: Projection_Index = sum of the 2nd order partial derivatives of the projection with respect to x1 and x2
= âˆ‚Â²f/âˆ‚x1Â² + âˆ‚Â²f/âˆ‚x1âˆ‚x2 + âˆ‚Â²f/âˆ‚x2âˆ‚x1 + âˆ‚Â²f/âˆ‚x2Â²
Gradient descent is the one of the optimization algorithms to find the projection with maximum projection index. If we want to implement it, first we have to compute gradient of the projection index with respect to the projection f.
Step2: âˆ‡f projection_index = âˆ‚/âˆ‚f (Projection_Index)
= [âˆ‚/âˆ‚f (âˆ‚Â²f/âˆ‚x1Â² + âˆ‚Â²f/âˆ‚x1âˆ‚x2 + âˆ‚Â²f/âˆ‚x2âˆ‚x1 + âˆ‚Â²f/âˆ‚x2Â²)]
if we plot projection index in z axis x1 in x axis and x2 in y axis, gradient of the projection index with respect to the projection is a vector which have the direction towards the maximum slope and the value of slop which is maximum in that directions.
Now we will update the projection by moving it towards the direction of the gradient with a small step, n in each iteration.
Step3: f = f  Î· * âˆ‡f projection_index
This steps will continue in every iteration until we find the maximum projection index or a predefined convergence criteria is met.
Implementation in python
Projection pursuit can be implemented using the skpp(ScikitPP) library in python. It have a Projection Pursuit Regressor to perform projection pursuit regression.
Command to install sklearn and skpp modules from the repository Python Package Index (PyPI).
pip install projectionpursuit
Code to perform projection pursuit regression on datasets
from skpp import ProjectionPursuitRegressor
from sklearn.datasets import make_regression
#Generate a simple regression data set
X, y = make_regression(n_samples=100,
n_features=10,
random_state=0)
#Create a Projection Pursuit Regressor
#model and fit it to the data
model = ProjectionPursuitRegressor()
model.fit(X, y)
#Use the fitted model to make predictions
predictions = model.predict(X)
This code will give a data set as output and fit a projection pursuit regression model to it. Predictions can be made using the same data set.
Advantages
Projection pursuit have some advantages over other dimensionality reduction techniques
 It is a flexible approach to customize the projection to fulfill different criteria, according to the goal of analysis.
 It can capture nonlinear projections of data. There is no assumptions of linearity.
 It is more useful for multimodal distributions than other dimension reduction techniques.
limitations
There are also some limitations of this technique. The limitations are 

Computation of this technique is complex as optimization techniques like gradient descent is required.

Sometimes the result of the projection pursuit depends on the initialization of the projection, as the optimization algorithm may or may not converge. If it converges it may converge to a local maximum of the projection index instead of the global maximum. This means that different initializations of the projection may lead to different results and that the optimization algorithms will not always find the best projection of the data.

As user can specify the projection index there are enough flexibility in this technique. But this can lead to overfitting of data. In that case, interpreting the data will be difficult.

Projection pursuit is not effective for data with vary high dimensions as computation of the optimization of the projection index becomes infeasible.
Applications in real world
Projection Pursuit is used for various purposes. it is more suitable technique for Exploratory analysis, Feature selection. It is also used for image compression, Feature extraction, gene expression analysis, protein structure prediction, economic forecasting, market segmentation, environmental monitoring. Overall, projection pursuit is a useful tool in statistics and data analysis.