RLScore - regularized least-squares machine learning algorithms package.
- Authors
- Homepage
- Version
0.6
- License
- Date
February 16. 2016
RLScore is a machine learning software package for regularized kernel methods, focusing especially on Regularized Least-Squares (RLS) based methods. The main advantage of the RLS family of methods is that they admit a closed form solution, expressed as a system of linear equations. This allows deriving highly efficient algorithms for RLS methods, based on matrix algebraic optimization. Classical results include computational short-cuts for multi-target learning, regularization parameter selection and leave-one-out cross-validation. RLScore takes these results further by implementing a wide variety of additional computational shortcuts for different types of cross-validation strategies, single- and multi-target feature selection, multi-task and zero-shot learning with Kronecker kernels, ranking, stochastic hill climbing based clustering etc. The majority of the implemented methods are such that are not available in any other software package.
- Regression and classification
- Regularized least-squares (RLS)
- multi-target learning
- selection of regularization parameter
- leave-one-out cross-validation
- leave-pair-out cross-validation
- k-fold cross-validation
- Feature selection for regression and classification
- Greedy regularized least-squares (Greedy RLS)
- greedy forward selection; selects features based on leave-one-out error
- joint feature selection for multi-target problems
- Ranking
- Regularized least-squares ranking (GlobalRankRLS)
- minimizes magnitude preserving ranking error
- multi-target learning
- selection of regularization parameter
- leave-pair-out cross-validation
- k-fold cross-validation
- Regularized least-squares ranking for query-structured data (QueryRankRLS)
- minimizes magnitude preserving ranking error, computed for each query separately
- multi-target learning
- selection of regularization parameter
- leave-query-out cross-validation
- Pairwise data and zero-shot learning
- Kronecker RLS
RLScore is written in Python and thus requires a working installation of Python 2.7.x. The package is also dependent on the NumPy package for matrix operations, and SciPy package for sparse matrix implementations.
- Major overhaul of learner interface, leaners now trained directly when initialized
- TwoStep-learning method, better Kronecker learners
- Cythonization of leave-pair-out cross-validation
- Automated regularization parameter selection via cross-validation for RLS and RankRLS added
- Old documentation removed as out-of-date, new documentation and tutorials in preparation
Version 0.5.1 (2014.07.31) ------------------------- This is a work in progress version maintained in a github repository. - The command line functionality is dropped and the main focus is shifted towards the library interface. - The interface has been considerably simplified to ease the use of the library. - Learning with tensor (Kronecker) product kernels considerably extended. - Many learners now implemented with cython to improve speed. - Support for a new type of interactive classification usable for image segmentation and various other tasks. - Numerous internal changes in the software.
- CGRLS and CGRankRLS learners for conjugate gradient -based training of RLS/RankRLS on large and high-dimensional, but sparse data.
- CGRankRLS supports learning from pairwise preferences between data points in addition to learning from utility values.
- Library interface for Python. Code examples for almost all included learning algorithms.
- Support for learning with Kronecker kernels.
- Numerous internal changes in the software.
- A linear time greedy forward feature selection with leave-one-out criterion for RLS (greedy RLS) included.
- Example data and codes for basic use cases included in the distribution.
- Fixed a bug causing problems when reading/writing binary files in Windows.
- Modifications to the configuration file format.
- All command line interfaces other than rls_core.py removed.
- Major restructuring of the code to make the software more modular.
- Configuration files introduced for more flexible use of software.
- Evolutionary maximum-margin clustering included.
- Model file format changed.
- Fixed a bug causing one of the features to get ignored.
- Major overhaul of the file formats.
- RLScore now supports learning multiple tasks simultaneously.
- Reduced set approximation included for large scale learning.
- Fixed a bug causing a memory leak after training with sparse data and linear kernel.
- First public release.
- Former Contributors
Evgeni Tsivtsivadze -participated in designing the version 0.1 and co-authored some of the articles in which the implemented methods were proposed.