regreg is a Python framework for regularized regression and, more
generally, for convex problems of the form
where is a smooth loss (squared error, a GLM negative log-likelihood, the Cox partial likelihood, a Huber loss, …) and each is a nonsmooth penalty or constraint (the norm, group norms, SLOPE, the nuclear norm, nonnegativity, linear constraints, …), possibly composed with a linear map . Problems are solved by proximal gradient methods (FISTA), through the dual when there are several nonsmooth terms, or by ADMM and TFOCS-style smoothing.
The emphasis is on building problems out of pieces rather than on any one model:
Atoms are the nonsmooth pieces. Each is a seminorm or a cone constraint, in Lagrange form () or bound form (), with a proximal map and a conjugate, so that duality comes for free. See Atoms, duality and proximal maps.
Smooth losses know their value and gradient, and compose with a design matrix or any linear operator. See Losses and GLMs.
Affine transforms are matrices, sparse matrices or matrix-free operators, and atoms can be composed with them: the fused lasso, trend filtering and isotonic regression are an atom of a difference operator. See Constraints and linear transforms.
Problems pair them up and are solved by Solvers.
On top of these, regreg.paths computes regularization paths
with strong rules and cross-validation, and regreg.sklearn wraps models as
scikit-learn estimators.
Installation¶
These pages describe the development version, which installs from the repository (this builds the Cython extensions with meson-python):
pip install git+https://github.com/regreg/regregor pip install . from a clone. The release on PyPI (0.1.3) predates the
current API.
The optional extras are regreg[sklearn] (scikit-learn, for the
regreg.sklearn estimators and cross-validation splitters) and
regreg[test].
A first example¶
The LASSO, , is a loss and an atom:
import numpy as np
import regreg.api as rr
rng = np.random.default_rng(0)
n, p = 100, 20
X = rng.standard_normal((n, p))
beta = np.zeros(p)
beta[:3] = [3, -2, 1.5]
Y = X @ beta + rng.standard_normal(n)
loss = rr.squared_error(X, Y)
penalty = rr.l1norm(p, lagrange=20.)
problem = rr.simple_problem(loss, penalty)
coef = problem.solve(tol=1e-10)
np.round(coef, 3)array([ 2.67 , -1.811, 0.978, 0. , 0. , -0. , 0. , -0. ,
-0. , 0. , -0. , -0. , -0. , -0. , -0. , -0. ,
0. , -0. , 0. , -0. ])Swapping the atom changes the model: rr.group_lasso(groups, lagrange=...)
gives the group LASSO, rr.l1norm(p, bound=...) the constrained form,
and rr.nonnegative(p) nonnegative least squares. The
Getting started page goes through this example in detail.
Links¶
Source code and issues: https://
github .com /regreg /regreg