Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

regreg

regreg is a Python framework for regularized regression and, more generally, for convex problems of the form

minimizeβℓ(Xβ)+∑jhj(Djβ),\text{minimize}_{\beta} \quad \ell(X\beta) + \sum_j h_j(D_j \beta),

where ℓ\ell is a smooth loss (squared error, a GLM negative log-likelihood, the Cox partial likelihood, a Huber loss, …) and each hjh_j is a nonsmooth penalty or constraint (the ℓ1\ell_1 norm, group norms, SLOPE, the nuclear norm, nonnegativity, linear constraints, …), possibly composed with a linear map DjD_j. Problems are solved by proximal gradient methods (FISTA), through the dual when there are several nonsmooth terms, or by ADMM and TFOCS-style smoothing.

The emphasis is on building problems out of pieces rather than on any one model:

On top of these, regreg.paths computes regularization paths with strong rules and cross-validation, and regreg.sklearn wraps models as scikit-learn estimators.

Installation

These pages describe the development version, which installs from the repository (this builds the Cython extensions with meson-python):

pip install git+https://github.com/regreg/regreg

or pip install . from a clone. The release on PyPI (0.1.3) predates the current API.

The optional extras are regreg[sklearn] (scikit-learn, for the regreg.sklearn estimators and cross-validation splitters) and regreg[test].

A first example

The LASSO, 12∥Y−Xβ∥22+λ∥β∥1\frac{1}{2}\|Y - X\beta\|_2^2 + \lambda \|\beta\|_1, is a loss and an atom:

array([ 2.67 , -1.811, 0.978, 0. , 0. , -0. , 0. , -0. , -0. , 0. , -0. , -0. , -0. , -0. , -0. , -0. , 0. , -0. , 0. , -0. ])

Swapping the atom changes the model: rr.group_lasso(groups, lagrange=...) gives the group LASSO, rr.l1norm(p, bound=...) the constrained form, and rr.nonnegative(p) nonnegative least squares. The Getting started page goes through this example in detail.