#The Information Sieve

Recent work establishes results that allow the efficient construction of representations that are maximally informative about the data. A recent extension allows us to build maximally informative representations in an incremental way. The other advantage of this approach is that it keeps track of remainder information, so that we can enable lossy and lossless compression.

This is described here:
The Information Sieve
Greg Ver Steeg and Aram Galstyan, ICML 2016,
http://arxiv.org/abs/1507.02284

A much more practical version of the information sieve for continuous variables is introduced here:
Sifting Common Information from Many Variables
Greg Ver Steeg, Shuyang Gao, Kyle Reing, and Aram Galstyan
The code for this version is in this repository: https://github.com/gregversteeg/LinearSieve

Papers describing previous work:
Discovering Structure in High-Dimensional Data Through Correlation Explanation
Greg Ver Steeg and Aram Galstyan, NIPS 2014, http://arxiv.org/abs/1406.1222

Maximally Informative Hierarchical Representions of High-Dimensional Data
Greg Ver Steeg and Aram Galstyan, AISTATS 2015, http://arxiv.org/abs/1410.7404

###Dependencies

Sieve requires numpy and scipy. If you use OS X, I recommend installing the Scipy Superpack:
http://fonnesbeck.github.io/ScipySuperpack/

###Install

To install, download using the link on the right or clone the project by executing this command in your target directory:

git clone git://github.com/gregversteeg/discrete_sieve.git

Use git pull to get updates. The code is under heavy development. Please feel free to raise issues or request features using the github interface.

import corex as ce

X = np.array([[0,0,0,0,0], # A matrix with rows as samples and columns as variables.
              [0,0,0,1,1],
              [1,1,1,0,0],
              [1,1,1,1,1]], dtype=int)

s = sieve.Sieve(max_layers=2)  # Initialize. Can limit number of layers (default is automatic) and pass kwargs to CorEx
s.fit(X)

s.labels  # Each variable/column is associated with one Y_j
s.tcs  # TC(X; Y) for each latent factor
s.layers[0].corex  # Access the CorEx object 

More examples are in the paper and in the tests and paper_figures folder.

See https://github.com/gregversteeg/CorEx for more info about the CorEx module (the one included here is a simplified version.)