ParsimonY contains the following features:

• parsimony provides structured and sparse penalties in machine learning. It currently contains:
  • Loss functions:
    • Linear regression
    • Logistic regression
  • Penalties:
    • L1 (Lasso)
    • L2 (Ridge)
    • Total variation (TV)
    • Overlapping Group LASSO (GL)
    • Any combination of the above
  • Algorithms:
    • I terative S hrinkage-T hresholding A lgorithm (fista)
    • F ast I terative S hrinkage-T hresholding A lgorithm (fista)
    • CO ntinuation of NEST erov’s smoothing A lgorithm (conesta)
    • Excessive gap method
  • Estimators
    • LinearRegression
    • Lasso
    • ElasticNet
    • LinearRegressionL1L2TV
    • LinearRegressionL1L2GL
    • LogisticRegressionL1L2TL
    • LogisticRegressionL1L2GL
    • LinearRegressionL2SmoothedL1TV

The reference environment for pylearn-parsimony is Ubuntu 12.04 LTS with Python 2.7.3, Numpy 1.6.1 and Scipy 0.9.0. More recent versions likely work, but have not been tested thoroughly.

Unless you already have Numpy and Scipy installed, you need to install them:

$ sudo apt-get install python-numpy python-scipy

In order to run the tests, you may also need to install Nose:

$ sudo apt-get install python-nose

In order to show plots, you may need to install Matplotlib:

$ sudo apt-get install python-matplotlib

To install: Download the release of pylearn-parsimony from https://github.com/neurospin/pylearn-parsimony/releases. Unpack the file, go to the pylearn-parsimony directory and type:

$ sudo python setup.py install

You are now ready to use your fresh installation of pylearn-parsimony!

A simple example: We first build a simulated dataset X and y.

import numpy as np
np.random.seed(42)
shape = (1, 4, 4)  # Three-dimension matrix
num_samples = 10  # The number of samples
num_ft = shape[0] * shape[1] * shape[2] # The number of features per sample
# Randomly generate X
X = np.random.rand(num_samples, num_ft)
beta = np.random.rand(num_ft, 1) # Define beta
# Add noise to y
y = np.dot(X, beta) + 0.001 * np.random.rand(num_samples, 1)
X_train = X[0:6, :]
y_train = y[0:6]
X_test = X[6:10, :]
y_test = y[6:10]

We build a simple estimator using the OLS (ordinary least squares) loss function and minimize using Gradient descent.

import parsimony.estimators as estimators
import parsimony.algorithms as algorithms
ols = estimators.LinearRegression(algorithm_params=dict(max_iter=1000))

Then we fit the model, estimate beta, and predict on test set.

res = ols.fit(X_train, y_train)
print "Estimated beta error = ", np.linalg.norm(ols.beta - beta)
print "Prediction error = ", np.linalg.norm(ols.predict(X_test) - y_test)

Now we build an estimator with the OLS loss function and a Total Variation penalty and minimize using FISTA.

import parsimony.estimators as estimators
import parsimony.algorithms as algorithms
import parsimony.functions.nesterov.tv as tv
k = 0.0  # l2 ridge regression coefficient
l = 0.0  # l1 lasso coefficient
g = 1.0  # tv coefficient
A = tv.linear_operator_from_shape(shape)  # Memory allocation for TV
olstv = estimators.LinearRegressionL1L2TV(k, l, g, A, mu=0.0001,
                                         algorithm=algorithms.proximal.FISTA(),
                                         algorithm_params=dict(max_iter=1000))

We fit the model, estimate beta, and predict on the test set.

res = olstv.fit(X_train, y_train)
print "Estimated beta error = ", np.linalg.norm(olstv.beta - beta)
print "Prediction error = ", np.linalg.norm(olstv.predict(X_test) - y_test)

• Tutorials

• Documentation