Atoms#
regreg.atoms#
- class regreg.atoms.atom(shape, offset=None, quadratic=None, initial=None)#
Bases:
nonsmoothBase class for atoms: simple nonsmooth functions with a known prox.
An atom is a support function (or, more generally, a simple convex function) with an inexpensive proximal map. Subclasses implement nonsmooth_objective, proximal and get_conjugate.
- property conjugate#
The conjugate of an atom.
- property dual#
The dual of an atom.
- property linear_transform#
The linear transform applied before a penalty is computed.
Defaults to regreg.affine.identity.
Examples
>>> from regreg.api import l1norm >>> penalty = l1norm(30, lagrange=3.4) >>> type(penalty.linear_transform) <class 'regreg.affine.identity'>
- nonsmooth_objective(arg, check_feasibility=False)#
The nonsmooth part of the objective; here just the quadratic.
- Parameters:
x (ndarray) – Point at which to evaluate.
check_feasibility (bool, optional) – Check whether x is feasible (unused here).
- Returns:
value (float) – Value of the nonsmooth part.
- smoothed(smoothing_quadratic)#
Return a smoothed version of the atom.
The smoothing is achieved by adding smoothing_quadratic to the conjugate of the atom and taking the conjugate again (Nesterov smoothing).
- Parameters:
smoothing_quadratic (regreg.identity_quadratic.identity_quadratic) – Quadratic term added to the conjugate. Its coefficient must be nonzero.
- Returns:
smoothed_atom (regreg.problems.composite.smooth_conjugate) – A smooth approximation of the atom.
- class regreg.atoms.affine_atom(atom_obj, atransform)#
Bases:
objectAn atom composed with an affine transform.
Given a seminorm on \(\mathbb{R}^p\), i.e. \(\beta \mapsto h_K(\beta)\), this class creates a new seminorm that evaluates \(h_K(D\beta+\alpha)\).
This class does not have a prox, but its dual does. The prox of the dual is
\[\text{minimize} \frac{1}{2} \|y-x\|^2_2 + x^T\alpha \ \text{s.t.} \ x \in \lambda K\]- Parameters:
atom_obj (atom) – The atom to compose with the affine transform. A copy is stored.
atransform (regreg.affine.affine_transform) – The affine transform. Any affine offset is moved into the offset of the atom, so that the stored transform is linear.
- latexify(var=None, idx='')#
Return a LaTeX representation of an object.
- Parameters:
var (str, optional) – Argument of the function.
idx (str, optional) – Optional subscript index.
- Returns:
L (str) – A LaTeX representation of the atom.
Examples
>>> from regreg.api import l1norm >>> penalty = l1norm(10, lagrange=0.9) >>> penalty.latexify(var=r'\gamma') '\\lambda_{} \\|\\gamma\\|_1'
- property dual#
a pair of the linear transform and the conjugate of the underlying atom.
- Type:
The dual of the affine atom
- nonsmooth_objective(arg, check_feasibility=False)#
Evaluate the atom at the linear transform of arg.
Returns
self.atom.nonsmooth_objective(self.linear_transform.linear_map(arg)).- Parameters:
arg (np.ndarray) – Argument of the objective.
check_feasibility (bool, optional) – Passed to the underlying atom.
- smoothed(smoothing_quadratic)#
Return a smoothed version of the affine atom.
- Parameters:
smoothing_quadratic (regreg.identity_quadratic.identity_quadratic) – Quadratic term added to the conjugate of the atom. Its coefficient must be nonzero.
- Returns:
smoothed (regreg.smooth.affine_smooth) – The smoothed atom composed with the linear transform.
regreg.atoms.seminorms#
- class regreg.atoms.seminorms.seminorm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
atomA seminorm atom that can be in Lagrange or bound form.
In Lagrange form the atom is \(\lambda h(\beta)\), in bound form it is the constraint \(h(\beta) \leq \delta\).
- Parameters:
shape (int or tuple) – Shape of the argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Exactly one of lagrange and bound must be specified.
bound (float, optional) – Bound parameter. Exactly one of lagrange and bound must be specified.
offset (np.ndarray, optional) – Offset subtracted from the argument before evaluating the seminorm.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- latexify(var=None, idx='')#
Return a LaTeX representation of an object.
- Parameters:
var (str, optional) – Argument of the function.
idx (str, optional) – Optional subscript index.
- Returns:
L (str) – A LaTeX representation of the atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(10, lagrange=0.9) >>> penalty.latexify(var=r'\gamma') '\\lambda_{} \\|\\gamma\\|_1'
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.get_conjugate() supnorm(..., bound=3.4...)
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.l1norm((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1norm((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1norm((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.l1norm((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \|\beta\|\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- nonsmooth_objective(arg, check_feasibility=False)#
The nonsmooth objective function of the atom.
Includes self.quadratic.objective(arg).
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
check_feasibility (bool, optional) – If True, then return np.inf if appropriate.
- Returns:
value (float) – The seminorm of arg.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> print(penalty.nonsmooth_objective([3, 4, 5, 9])) 42.0 >>> 2 * sum([3, 4, 5, 9]) 42
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=2.3) >>> penalty l1norm(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, supnorm(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)#
Create an atom composed with an affine transform.
This is the same as the linear class method but with offset as a positional argument.
- Parameters:
linear_operator (np.ndarray or regreg.affine.linear_transform) – Linear part of the affine transform.
offset (np.ndarray) – Offset of the affine transform, i.e. the seminorm is evaluated at
linear_operator(x) + offset.lagrange (float, optional) – Lagrange parameter.
diag (bool, optional) – If True, linear_operator is interpreted as the diagonal of a diagonal matrix.
bound (float, optional) – Bound parameter.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
- Returns:
atom (regreg.atoms.affine_atom) – The seminorm composed with the affine transform.
- classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)#
Create an atom composed with a linear transform.
- Parameters:
linear_operator (np.ndarray or regreg.affine.linear_transform) – The linear transform.
lagrange (float, optional) – Lagrange parameter.
diag (bool, optional) – If True, linear_operator is interpreted as the diagonal of a diagonal matrix.
bound (float, optional) – Bound parameter.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
offset (np.ndarray, optional) – Offset, i.e. the seminorm is evaluated at
linear_operator(x) + offset.
- Returns:
atom (regreg.atoms.affine_atom) – The seminorm composed with the linear transform.
- classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)#
Create an atom evaluated at a shifted argument.
The resulting seminorm is evaluated at
x + offset.- Parameters:
offset (np.ndarray) – The shift.
lagrange (float, optional) – Lagrange parameter.
diag (bool, optional) – Ignored.
bound (float, optional) – Bound parameter.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
- Returns:
atom (seminorm) – The shifted seminorm.
- static check_subgradient(atom, prox_center)#
Verify the KKT conditions of the proximal problem for a seminorm.
The proximal problem is
\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(u)\]where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.
If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.
- Parameters:
atom (seminorm) – The seminorm whose proximal map is checked.
prox_center (np.ndarray) – Center for the proximal map.
- Returns:
v1, v2 (float) – Two values that should be equal if the proximal map is correct.
- class regreg.atoms.seminorms.l1norm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe \(\ell_1\) norm.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|\beta\|_1\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|\beta\|_1 \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_1\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_1 \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \|\beta\|_1\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1norm((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1norm((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.l1norm((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.l1norm((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.get_conjugate() supnorm(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=2.3) >>> penalty l1norm(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, supnorm(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.seminorms.supnorm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe \(\ell_{\infty}\) norm.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|\beta\|_{\infty}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|\beta\|_{\infty} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_{\infty}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_{\infty} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \|\beta\|_{\infty}\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.supnorm((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.supnorm((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.supnorm((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.supnorm((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.supnorm((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.supnorm((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.supnorm((30,), lagrange=3.4) >>> penalty.get_conjugate() l1norm(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.supnorm((30,), lagrange=2.3) >>> penalty supnorm(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l1norm(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.seminorms.l2norm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe \(\ell_2\) norm.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|\beta\|_2\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|\beta\|_2 \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_2\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_2 \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \|\beta\|_2\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l2norm((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l2norm((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.l2norm((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2norm((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2norm((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.l2norm((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2norm((30,), lagrange=3.4) >>> penalty.get_conjugate() l2norm(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.l2norm((30,), lagrange=2.3) >>> penalty l2norm(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l2norm(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- regreg.atoms.seminorms.positive_part_lagrange(shape, lagrange, offset=None, quadratic=None, initial=None)#
The positive_part atom in Lagrange form, represented as an l1norm.
The positive_part atom in Lagrange form can be represented by an l1norm atom with the addition of a linear term and half the Lagrange parameter. This reflects the fact that \([0,1]^p = [-1/2,1/2]^p + 1/2 \pmb{1}\).
- Parameters:
shape (int or tuple) – Shape of the argument.
lagrange (float) – Lagrange parameter of the positive_part atom.
offset (np.ndarray, optional) – Offset of the atom.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- Returns:
atom (l1norm) – An l1norm atom equivalent to the positive_part atom.
- class regreg.atoms.seminorms.positive_part(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe positive part seminorm.
This is the support function of \([0,1]^p\), i.e. \(\beta \mapsto \sum_i \max(\beta_i, 0)\).
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \left(\sum_{i=1}^{p} (\beta)_i^+\right)\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\left(\sum_{i=1}^{p} (\beta)_i^+\right) \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \left(\sum_{i=1}^{p} (\beta)_i^+\right)\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \left(\sum_{i=1}^{p} (\beta)_i^+\right) \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \left(\sum_{i=1}^{p} (\beta)_i^+\right)\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.positive_part((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.positive_part((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.positive_part((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.positive_part((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.positive_part((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.positive_part((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.positive_part((30,), lagrange=3.4) >>> penalty.get_conjugate() constrained_max(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.positive_part((30,), lagrange=2.3) >>> penalty positive_part(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, constrained_max(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.seminorms.constrained_max(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe seminorm \(\max_i \beta_i\) subject to \(\beta \geq 0\).
This is the dual seminorm of positive_part.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right)\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right)\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right)\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.constrained_max((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.constrained_max((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.constrained_max((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.constrained_max((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.constrained_max((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.constrained_max((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.constrained_max((30,), lagrange=3.4) >>> penalty.get_conjugate() positive_part(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.constrained_max((30,), lagrange=2.3) >>> penalty constrained_max(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, positive_part(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.seminorms.constrained_positive_part(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe support function of \([-\infty,1]^p\).
This is \(\sum_i \beta_i\) subject to \(\beta \geq 0\).
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i)\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i) \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i)\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i) \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i)\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.constrained_positive_part((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.constrained_positive_part((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.constrained_positive_part((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.constrained_positive_part((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.constrained_positive_part((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.constrained_positive_part((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.constrained_positive_part((30,), lagrange=3.4) >>> penalty.get_conjugate() max_positive_part(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.constrained_positive_part((30,), lagrange=2.3) >>> penalty constrained_positive_part(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, max_positive_part(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.seminorms.max_positive_part(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe support function of the standard simplex.
This is \(\max_i \max(\beta_i, 0)\).
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|\beta^+\|_{\infty}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|\beta^+\|_{\infty} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta^+\|_{\infty} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta^+\|_{\infty}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \|\beta^+\|_{\infty}\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.max_positive_part((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.max_positive_part((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.max_positive_part((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.max_positive_part((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.max_positive_part((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.max_positive_part((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.max_positive_part((30,), lagrange=3.4) >>> penalty.get_conjugate() constrained_positive_part(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.max_positive_part((30,), lagrange=2.3) >>> penalty max_positive_part(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, constrained_positive_part(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- regreg.atoms.seminorms.get_lagrange(atom, lagrange=None)#
Return the appropriate lagrange parameter.
Returns lagrange if not None, else atom.lagrange. Raises ValueError if both are None.
- regreg.atoms.seminorms.n1#
alias of
constrained_positive_part
- regreg.atoms.seminorms.n2#
alias of
max_positive_part
- regreg.atoms.seminorms.get_bound(atom, bound=None)#
Return the appropriate bound parameter.
Returns bound if not None, else atom.bound. Raises ValueError if both are None.
regreg.atoms.weighted_atoms#
Weighted versions of the \(\ell_1\) and \(\ell_{\infty}\) norms.
- class regreg.atoms.weighted_atoms.seminorm(weights, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormBase class for weighted seminorms.
- Parameters:
weights (np.ndarray) – Nonnegative weights, one per coordinate. The shape of the atom is the shape of weights.
lagrange (float, optional) – Lagrange parameter (exactly one of lagrange and bound must be specified).
bound (float, optional) – Bound parameter.
offset (np.ndarray, optional) – Offset subtracted from the argument before the seminorm is evaluated.
quadratic (identity_quadratic, optional) – Quadratic term added to the seminorm.
initial (np.ndarray, optional) – Initial value of the coefficients.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm((30,), lagrange=3.4) >>> penalty.get_conjugate() supnorm(..., bound=3.4...)
- form_transform(subsample=False)#
Form the linear transform applied before the seminorm.
By subsampling we can get rid of some variables that have 0 weights.
- Parameters:
subsample (bool, optional) – If True and some weights are 0, the transform is a selector keeping only the coordinates with nonzero weights. Otherwise it is the identity.
- Returns:
transform (affine_transform) – The (cached) linear transform.
- property linear_transform#
Form the linear transform applied before the seminorm.
By subsampling we can get rid of some variables that have 0 weights.
- Parameters:
subsample (bool, optional) – If True and some weights are 0, the transform is a selector keeping only the coordinates with nonzero weights. Otherwise it is the identity.
- Returns:
transform (affine_transform) – The (cached) linear transform.
- class regreg.atoms.weighted_atoms.l1norm(weights, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe weighted \(\ell_1\) norm.
This is \(\|W\beta\|_1 = \sum_i w_i |\beta_i|\) where \(W\) is the diagonal matrix of weights. Coordinates with weight 0 are unpenalized; coordinates with infinite weight are constrained to be 0.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|W\beta\|_1\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|W\beta\|_1 \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|W\beta\|_1\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|W\beta\|_1 \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- terms(arg)#
Return the terms that are summed in computing the seminorm.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
- Returns:
terms (np.ndarray) – The values
np.fabs(arg) * self.weights.
- class regreg.atoms.weighted_atoms.supnorm(weights, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe weighted \(\ell_{\infty}\) norm.
This is \(\|W\beta\|_{\infty} = \max_i w_i |\beta_i|\) where \(W\) is the diagonal matrix of weights. It is the dual of the weighted l1norm with weights \(1/w_i\).
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|W\beta\|_{\infty}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(arg, bound=None)#
Verify \(\|W\beta\|_{\infty} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|W\beta\|_{\infty}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|W\beta\|_{\infty} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- terms(arg, check_feasibility=True)#
Return the terms that are maximized in computing the seminorm.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
check_feasibility (bool, optional) – If True, entries with weight 0 are set to
np.infwhere arg is nonzero (and 0 otherwise).
- Returns:
terms (np.ndarray) – The values
np.fabs(arg) / self.weights.
regreg.atoms.cones#
- class regreg.atoms.cones.cone(shape, offset=None, quadratic=None, initial=None)#
Bases:
atomBase class for cone constraints.
A cone constraint is the convex indicator \(I^{\infty}_K\) of a closed convex cone \(K\): it is 0 on \(K\) and \(+\infty\) elsewhere. Its conjugate is the indicator of the polar cone \(K^{\circ}\). Subclasses implement
constraintandcone_prox(the Euclidean projection onto \(K\)).- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonnegative((30,)) >>> penalty.get_conjugate() nonpositive((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonnegative((30,)) >>> penalty nonnegative((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nonpositive((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- property linear_transform#
The linear transform applied before the constraint is evaluated. Defaults to regreg.affine.identity.
- constraint(x)#
Evaluate the constraint
\[\|\beta\|\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- nonsmooth_objective(x, check_feasibility=False)#
The nonsmooth objective function of the atom.
For a cone constraint this is 0 (or
np.infif check_feasibility is True and x minus the offset is not in the cone), plusself.quadratic.objective(x, 'func').- Parameters:
x (np.ndarray) – Argument of the objective.
check_feasibility (bool, optional) – If True, return
np.infwhen x violates the constraint.
- Returns:
value (float) – The value of the nonsmooth objective at x.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative(4) >>> print(cone.nonsmooth_objective([3, 4, 5, 9])) 0.0
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = \|\beta\|\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + \|\beta\|\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- classmethod linear(linear_operator, diag=False, offset=None, quadratic=None)#
Compose a cone constraint with a linear transform.
- Parameters:
linear_operator (np.ndarray or linear_transform) – The linear operator \(D\). If not already a linear_transform, it is wrapped in one.
diag (bool, optional) – Passed to linear_transform: if True, linear_operator is interpreted as the diagonal of a diagonal matrix.
offset (np.ndarray, optional) – If not None, the constraint is imposed on \(D\beta + \text{offset}\).
quadratic (identity_quadratic, optional) – Quadratic term added to the cone constraint.
- Returns:
atom (affine_cone) – The composition of the cone constraint with \(D\).
- classmethod affine(linear_operator, offset, diag=False, quadratic=None)#
Compose a cone constraint with an affine transform.
This is the same as the linear class method but with offset as a positional argument.
- Parameters:
linear_operator (np.ndarray or linear_transform) – The linear operator \(D\). If not already a linear_transform, it is wrapped in one.
offset (np.ndarray or None) – If not None, the constraint is imposed on \(D\beta + \text{offset}\).
diag (bool, optional) – Passed to linear_transform: if True, linear_operator is interpreted as the diagonal of a diagonal matrix.
quadratic (identity_quadratic, optional) – Quadratic term added to the cone constraint.
- Returns:
atom (affine_cone) – The composition of the cone constraint with the affine map.
- static check_subgradient(atom, prox_center)#
Verify the KKT conditions for the proximal problem of a cone.
The proximal problem is
\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(u)\]where \(z\) is prox_center and \(h\) is atom.
For a cone, the minimizer and the residual are orthogonal, so this should return two values that are 0: the first is the inner product of the minimizer and the residual, the second is just 0.
- Parameters:
atom (cone) – A cone instance with a proximal method.
prox_center (np.ndarray) – Center for the proximal map.
- Returns:
v1, v2 (float) – Two values that should be equal (to 0) if the proximal map is correct.
- class regreg.atoms.cones.affine_cone(atom_obj, atransform)#
Bases:
affine_atomComposition of a cone constraint with a linear transform.
- class regreg.atoms.cones.nonnegative(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe non-negative cone constraint (which is the support function of the non-positive cone constraint).
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\beta \succeq 0)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\beta \succeq 0)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\beta \succeq 0)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonnegative((30,)) >>> penalty.get_conjugate() nonpositive((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonnegative((30,)) >>> penalty nonnegative((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nonpositive((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.nonpositive(shape, offset=None, quadratic=None, initial=None)#
Bases:
nonnegativeThe non-positive cone constraint (which is the support function of the non-negative cone constraint).
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\beta \preceq 0)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\beta \preceq 0)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\beta \preceq 0)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonpositive((30,)) >>> penalty.get_conjugate() nonnegative((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonpositive((30,)) >>> penalty nonpositive((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nonnegative((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.zero(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe zero seminorm, i.e. the support function of \(\{0\}\).
- constraint(x)#
Evaluate the constraint
\[{\cal Z}(\beta)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + {\cal Z}(\beta)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = {\cal Z}(\beta)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.zero((30,)) >>> penalty.get_conjugate() zero_constraint((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.zero((30,)) >>> penalty zero((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, zero_constraint((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.zero_constraint(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe zero constraint, i.e. the support function of \(\mathbb{R}^p\).
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\beta = 0)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\beta = 0)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\beta = 0)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.zero_constraint((30,)) >>> penalty.get_conjugate() zero((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.zero_constraint((30,)) >>> penalty zero_constraint((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, zero((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.l2_epigraph(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe \(\ell_2\) epigraph constraint.
The constraint is \(\|\beta[:-1]\|_2 \leq \beta[-1]\), i.e. the last coordinate holds the norm and the others the coefficients.
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\|\beta[:-1]\|_2 \leq \beta[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\|\beta[:-1]\|_2 \leq \beta[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\|\beta[:-1]\|_2 \leq \beta[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2_epigraph((30,)) >>> penalty.get_conjugate() l2_epigraph_polar((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2_epigraph((30,)) >>> penalty l2_epigraph((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l2_epigraph_polar((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.l2_epigraph_polar(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe polar of the l2_epigraph constraint, which is the negative of the l2 epigraph.
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\|\beta[:-1]\|_2 \leq -\beta[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(arg)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\|\beta[:-1]\|_2 \leq -\beta[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\|\beta[:-1]\|_2 \leq -\beta[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2_epigraph_polar((30,)) >>> penalty.get_conjugate() l2_epigraph((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.l2_epigraph_polar((30,)) >>> penalty l2_epigraph_polar((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l2_epigraph((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.l1_epigraph(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe \(\ell_1\) epigraph constraint.
The constraint is \(\|\beta[:-1]\|_1 \leq \beta[-1]\), i.e. the last coordinate holds the norm and the others the coefficients.
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\|\beta[:-1]\|_1 \leq \beta[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\|\beta[:-1]\|_1 \leq \beta[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\|\beta[:-1]\|_1 \leq \beta[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_epigraph((30,)) >>> penalty.get_conjugate() l1_epigraph_polar((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_epigraph((30,)) >>> penalty l1_epigraph((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l1_epigraph_polar((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.l1_epigraph_polar(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe polar of the l1_epigraph constraint, which is the negative of the linf epigraph.
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\|\beta[:-1]\|_{\infty} \leq - \beta[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(arg)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\|\beta[:-1]\|_{\infty} \leq - \beta[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\|\beta[:-1]\|_{\infty} \leq - \beta[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_epigraph_polar((30,)) >>> penalty.get_conjugate() l1_epigraph((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_epigraph_polar((30,)) >>> penalty l1_epigraph_polar((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l1_epigraph((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.linf_epigraph(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe \(\ell_{\infty}\) epigraph constraint.
The constraint is \(\|\beta[:-1]\|_{\infty} \leq \beta[-1]\), i.e. the last coordinate holds the norm and the others the coefficients.
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\|\beta[:-1]\|_{\infty} \leq \beta[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(arg)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\|\beta[:-1]\|_{\infty} \leq \beta[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\|\beta[:-1]\|_{\infty} \leq \beta[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_epigraph((30,)) >>> penalty.get_conjugate() linf_epigraph_polar((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_epigraph((30,)) >>> penalty linf_epigraph((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, linf_epigraph_polar((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.cones.linf_epigraph_polar(shape, offset=None, quadratic=None, initial=None)#
Bases:
coneThe polar of the linf_epigraph constraint, which is just the negative of the l1_epigraph.
- constraint(x)#
Evaluate the constraint
\[I^{\infty}(\|\beta[:-1]\|_1 \leq -\beta[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(arg)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}(\|\beta[:-1]\|_1 \leq -\beta[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}(\|\beta[:-1]\|_1 \leq -\beta[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_epigraph_polar((30,)) >>> penalty.get_conjugate() linf_epigraph((30,), offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_epigraph_polar((30,)) >>> penalty linf_epigraph_polar((30,), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, linf_epigraph((30,), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- regreg.atoms.cones.n1#
alias of
linf_epigraph
- regreg.atoms.cones.n2#
alias of
linf_epigraph_polar
regreg.atoms.group_lasso#
- class regreg.atoms.group_lasso.group_lasso(groups, weights={}, offset=None, lagrange=None, bound=None, quadratic=None, initial=None)#
Bases:
seminormThe group LASSO seminorm.
\[\beta \mapsto \sum_g w_g \|\beta[g]\|_2\]- Parameters:
groups (array-like) – Group labels for each coordinate of the argument.
weights (dict, optional) – Weights \(w_g\) keyed by group label. Groups without a weight get a default weight of the square root of the group size.
offset (np.ndarray, optional) – Offset subtracted from the argument before evaluating the seminorm.
lagrange (float, optional) – Lagrange parameter. Exactly one of lagrange and bound must be specified.
bound (float, optional) – Bound parameter. Exactly one of lagrange and bound must be specified.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- set_weight(group, weight)#
Set a group’s weight after initialization.
- Parameters:
group (hashable) – Label of the group.
weight (float) – New weight for the group.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso([1, 1, 2, 2, 2], lagrange=3.4) >>> penalty.get_conjugate() group_lasso_dual(..., bound=3.4...)
- terms(arg)#
Return the terms that are summed in computing the seminorm.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
- Returns:
terms (np.ndarray) – Weighted Euclidean norms of each group of arg, ordered by sorted group label.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> groups = [1,1,2,2,2] >>> penalty = rr.group_lasso(groups, lagrange=1.) >>> arg = [2,4,5,3,4] >>> [float(t) for t in penalty.terms(arg)] [6.3245..., 12.2474...] >>> penalty.seminorm(arg) 18.5720... >>> float(np.sqrt((2**2 + 4**2)*2)), float(np.sqrt((5**2 + 3**2 + 4**2) * 3.)) (6.3245..., 12.2474...) >>> float(np.sqrt((2**2 + 4**2)*2) + np.sqrt((5**2 + 3**2 + 4**2) * 3.)) 18.5720...
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \sum_g w_g \|\beta[g]\|_2\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \sum_g w_g \|\beta[g]\|_2\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \sum_g w_g \|\beta[g]\|_2 \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- constraint(arg, bound=None)#
Verify \(\sum_g w_g \|\beta[g]\|_2 \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \sum_g w_g \|\beta[g]\|_2\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.group_lasso([1, 1, 2, 2, 2], bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.group_lasso([1, 1, 2, 2, 2], bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.group_lasso([1, 1, 2, 2, 2], lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso([1, 1, 2, 2, 2], lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso([1, 1, 2, 2, 2], lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.group_lasso([1, 1, 2, 2, 2], bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.group_lasso([1, 1, 2, 2, 2], lagrange=2.3) >>> penalty group_lasso(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, group_lasso_dual(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.group_lasso.group_lasso_dual(groups, weights={}, offset=None, lagrange=None, bound=None, quadratic=None, initial=None)#
Bases:
group_lassoThe dual of the group LASSO seminorm.
\[\beta \mapsto \max_g \|\beta[g]\|_2 / w_g\]Takes the same arguments as group_lasso.
- terms(arg, check_feasibility=True)#
Return the terms that are maximized in computing the seminorm.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
check_feasibility (bool, optional) – If True, return np.inf for groups with a weight of 0 whose norms are not 0. Otherwise, return 0 for these entries.
- Returns:
terms (np.ndarray) – Euclidean norms of each group of arg divided by the group weights, ordered by sorted group label.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> groups = [1,1,2,2,2] >>> penalty = rr.group_lasso_dual(groups, lagrange=1.) >>> arg = [2,4,5,3,4] >>> [float(t) for t in penalty.terms(arg)] [3.1622..., 4.0824...] >>> float(np.sqrt((2**2 + 4**2)/2)), float(np.sqrt((5**2 + 3**2 + 4**2) / 3.)) (3.1622..., 4.0824...) >>> penalty.seminorm(arg) 4.0824...
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \max_g \|\beta[g]\|_2 / w_g\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \max_g \|\beta[g]\|_2 / w_g \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \max_g \|\beta[g]\|_2 / w_g\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(\beta) = \max_g \|\beta[g]\|_2 / w_g\]If the atom is in bound mode, then this has the form
\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.group_lasso_dual((30,), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.group_lasso_dual((30,), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.group_lasso_dual((30,), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual((30,), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual((30,), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.group_lasso_dual((30,), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual((30,), lagrange=3.4) >>> penalty.get_conjugate() group_lasso(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.group_lasso_dual((30,), lagrange=2.3) >>> penalty group_lasso_dual(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, group_lasso(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.group_lasso.group_lasso_cone(groups, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
coneA superclass for the group LASSO and group LASSO dual epigraph cones.
The argument of the cone has one more coordinate than groups: the last coordinate is the epigraph variable.
- Parameters:
groups (array-like) – Group labels for each coordinate (excluding the last one) of the argument.
weights (dict, optional) – Weights keyed by group label. See group_lasso.
offset (np.ndarray, optional) – Offset subtracted from the argument.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- seminorm_class#
alias of
group_lasso
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nonnegative((30,)) >>> penalty.get_conjugate() nonpositive((30,), offset=None)
- property weights#
Weights of the groups, keyed by group label.
- property groups#
Group labels of the coordinates (excluding the epigraph variable).
- constraint(arg)#
Evaluate the constraint
\[\|\beta\|\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- class regreg.atoms.group_lasso.group_lasso_epigraph(groups, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
group_lasso_coneThe epigraph of the group LASSO seminorm.
- seminorm_class#
alias of
group_lasso
- cone_prox(arg, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}\left(\sum_g \|\beta[g]\|_2 \leq \beta[-1]\right)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}\left(\sum_g \|\beta[g]\|_2 \leq \beta[-1]\right)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_epigraph([1,1,2,2,2]) >>> penalty.get_conjugate() group_lasso_epigraph_polar([1,1,2,2,2], offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_epigraph([1,1,2,2,2]) >>> penalty group_lasso_epigraph([1,1,2,2,2], offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, group_lasso_epigraph_polar([1,1,2,2,2], offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.group_lasso.group_lasso_epigraph_polar(groups, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
group_lasso_coneThe polar of the epigraph of the group LASSO seminorm.
- seminorm_class#
alias of
group_lasso_dual
- cone_prox(arg, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}\left(\max_g \|\beta[g]\|_2 \leq -\beta[-1]\right)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- constraint(arg)#
Evaluate the constraint
\[I^{\infty}\left(\max_g \|\beta[g]\|_2 \leq -\beta[-1]\right)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}\left(\max_g \|\beta[g]\|_2 \leq -\beta[-1]\right)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_epigraph_polar([1,1,2,2,2]) >>> penalty.get_conjugate() group_lasso_epigraph([1,1,2,2,2], offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_epigraph_polar([1,1,2,2,2]) >>> penalty group_lasso_epigraph_polar([1,1,2,2,2], offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, group_lasso_epigraph([1,1,2,2,2], offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.group_lasso.group_lasso_dual_epigraph(groups, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
group_lasso_coneThe group LASSO conjugate epigraph constraint.
- seminorm_class#
alias of
group_lasso_dual
- cone_prox(arg, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}\left(\max_g \|\beta[g]\|_2 \leq \beta[-1]\right)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- constraint(arg)#
Evaluate the constraint
\[I^{\infty}\left(\max_g \|\beta[g]\|_2 \leq \beta[-1]\right)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}\left(\max_g \|\beta[g]\|_2 \leq \beta[-1]\right)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual_epigraph([1,1,2,2,2]) >>> penalty.get_conjugate() group_lasso_dual_epigraph_polar([1,1,2,2,2], offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual_epigraph([1,1,2,2,2]) >>> penalty group_lasso_dual_epigraph([1,1,2,2,2], offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, group_lasso_dual_epigraph_polar([1,1,2,2,2], offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.group_lasso.group_lasso_dual_epigraph_polar(groups, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
group_lasso_coneThe polar of the group LASSO dual epigraph constraint.
- seminorm_class#
alias of
group_lasso
- cone_prox(arg, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + I^{\infty}\left(\sum_g \|\beta[g]\|_2 \leq -\beta[-1]\right)\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- constraint(arg)#
Evaluate the constraint
\[I^{\infty}\left(\sum_g \|\beta[g]\|_2 \leq -\beta[-1]\right)\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = I^{\infty}\left(\sum_g \|\beta[g]\|_2 \leq -\beta[-1]\right)\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual_epigraph_polar([1,1,2,2,2]) >>> penalty.get_conjugate() group_lasso_dual_epigraph([1,1,2,2,2], offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.group_lasso_dual_epigraph_polar([1,1,2,2,2]) >>> penalty group_lasso_dual_epigraph_polar([1,1,2,2,2], offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, group_lasso_dual_epigraph([1,1,2,2,2], offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- regreg.atoms.group_lasso.strong_set(glasso, lagrange_cur, lagrange_new, grad, slope_estimate=1)#
Guess the active groups at lagrange_new based on the gradient at lagrange_cur.
This is the strong rule for discarding inactive groups when computing a path of solutions.
- Parameters:
glasso (group_lasso) – The group LASSO penalty.
lagrange_cur (float) – Current value of the Lagrange parameter.
lagrange_new (float) – Next value of the Lagrange parameter.
grad (np.ndarray) – Gradient of the smooth loss at the solution for lagrange_cur.
slope_estimate (float, optional) – Estimate of the slope of the gradient as a function of the Lagrange parameter.
- Returns:
groups (list) – Labels of the candidate active groups.
- regreg.atoms.group_lasso.check_KKT(glasso, grad, solution, lagrange, tol=0.01)#
Check whether (grad, solution) satisfy the KKT conditions.
Assumes glasso is a group_lasso in Lagrange form, so that glasso.lagrange is not None.
- Parameters:
glasso (group_lasso) – The group LASSO penalty.
grad (np.ndarray) – Gradient of the smooth loss at solution.
solution (np.ndarray) – Candidate solution.
lagrange (float) – Lagrange parameter (currently unused; glasso.lagrange is used).
tol (float, optional) – Tolerance for the KKT conditions.
- Returns:
failing (bool) – True if the KKT conditions fail at tolerance tol.
regreg.atoms.sparse_group_lasso#
Implementation of the sparse group LASSO atom.
This penalty is defined by groups and individual feature weights and is
where \(\alpha_j\) are the feature (LASSO) weights and \(w_g\) are the group weights.
- class regreg.atoms.sparse_group_lasso.sparse_group_lasso(groups, lasso_weights, weights={}, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
group_lasso,seminormThe sparse group LASSO seminorm.
\[\beta \mapsto \sum_j \alpha_j |\beta_j| + \sum_g w_g \|\beta[g]\|_2\]- Parameters:
groups (array-like) – Group labels for each coordinate of the argument.
lasso_weights (array-like) – Feature weights \(\alpha_j\). If not of the same shape as groups, it is broadcast to that shape.
weights (dict, optional) – Group weights \(w_g\) keyed by group label. Groups without a weight get a default weight of the square root of the group size.
lagrange (float, optional) – Lagrange parameter. Exactly one of lagrange and bound must be specified.
bound (float, optional) – Bound parameter. Exactly one of lagrange and bound must be specified.
offset (np.ndarray, optional) – Offset subtracted from the argument before evaluating the seminorm.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \left[ \sum_j \alpha_j |\beta_j| + \sum_g \lambda_g \|\beta[g]\|_2 \right]\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(x, bound=None)#
Verify \(\left[ \sum_j \alpha_j |\beta_j| + \sum_g \lambda_g \|\beta[g]\|_2 \right] \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(x, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \left[ \sum_j \alpha_j |\beta_j| + \sum_g \lambda_g \|\beta[g]\|_2 \right]\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(x, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \left[ \sum_j \alpha_j |\beta_j| + \sum_g \lambda_g \|\beta[g]\|_2 \right] \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- terms(arg)#
Return the terms that are summed in computing the seminorm.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
- Returns:
terms (np.ndarray) – For each group (ordered by sorted group label), the weighted Euclidean norm plus the weighted \(\ell_1\) norm of that group of arg.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> groups = [1,1,2,2,2] >>> l1weights = [1,1,1,1,1] >>> penalty = rr.sparse_group_lasso(groups, l1weights, lagrange=1.) >>> arg = [2,4,5,3,4] >>> [float(t) for t in penalty.terms(arg)] [12.3245..., 24.2474...] >>> float(penalty.seminorm(arg)) 36.5720... >>> float(np.sqrt((2**2 + 4**2)*2) + 6), float(np.sqrt((5**2 + 3**2 + 4**2) * 3.) + 12) (12.3245..., 24.2474...) >>> float(np.sqrt((2**2 + 4**2)*2) + np.sqrt((5**2 + 3**2 + 4**2) * 3.) + np.sum(np.fabs(arg))) 36.5720...
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.sparse_group_lasso([1, 1, 2, 2, 2], [0, 0.5, 0.2, 0.2, 0.2], lagrange=3.4) >>> penalty.get_conjugate() sparse_group_lasso_dual(..., bound=3.4...)
- class regreg.atoms.sparse_group_lasso.sparse_group_lasso_dual(groups, lasso_weights, weights={}, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
sparse_group_lassoThe dual of the sparse group LASSO seminorm.
Takes the same arguments as sparse_group_lasso. The seminorm is computed by a binary search for the gauge of the dual ball.
- property base_conjugate#
The primal atom in Lagrange form matching this dual atom.
For a sparse group LASSO dual in bound form, this is a sparse_group_lasso atom in Lagrange form whose Lagrange parameter equals the bound of this atom.
Notes
This conjugate does not take into account the atom’s quadratic and offset terms. Its Lagrange parameter is 1: every use scales it explicitly (the gauge is homogeneous), so it also works for an atom in Lagrange form. It is cached, and rebuilt if the weights change.
- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot {\cal D}^{group sparse}(\beta)\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(x, bound=None)#
Verify \({\cal D}^{group sparse}(\beta) \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(x, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda {\cal D}^{group sparse}(\beta)\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(x, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ {\cal D}^{group sparse}(\beta) \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- terms(arg)#
Return the terms that are maximized in computing the seminorm.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
- Returns:
terms (np.ndarray) – The dual norm of each group of arg (ordered by sorted group label).
Examples
>>> import regreg.api as rr >>> groups = [1,1,2,2,2] >>> l1weights = [1,1,1,1,1] >>> penalty = rr.sparse_group_lasso_dual(groups, l1weights, lagrange=1.) >>> arg = [2,4,5,3,4] >>> [float(t) for t in penalty.terms(arg)] [1.666..., 2.0833...]
- regreg.atoms.sparse_group_lasso.inside_set(atom, point)#
Check whether point is in the dual ball.
If the atom is the primal (necessarily in Lagrange form), we check whether point is in the dual ball (i.e. the one determining the norm).
If the atom is a dual (necessarily in bound form), we just check whether projecting point returns the same point.
- Parameters:
atom (sparse_group_lasso or sparse_group_lasso_dual) – The atom determining the dual ball.
point (np.ndarray) – The point to check.
- Returns:
inside (bool) – Whether point is in the dual ball.
- regreg.atoms.sparse_group_lasso.gauge_function_dual(atom, point, tol=1e-10, max_iter=50)#
Compute the dual norm of the sparse group LASSO by binary search.
- Parameters:
atom (sparse_group_lasso or sparse_group_lasso_dual) – The atom determining the dual ball.
point (np.ndarray) – Point at which to evaluate the dual norm.
tol (float, optional) – Relative tolerance for the binary search.
max_iter (int, optional) – Maximum number of halvings / doublings used to bracket the dual norm.
- Returns:
value (float) – The dual norm of point.
Notes
This will have problems if the atom has infinite feature weights.
- regreg.atoms.sparse_group_lasso.n1#
alias of
sparse_group_lasso
- regreg.atoms.sparse_group_lasso.n2#
alias of
sparse_group_lasso_dual
regreg.atoms.mixed_lasso#
Mixed LASSO penalties.
A mixed LASSO penalty combines, on different coordinates of the same vector, a group LASSO penalty, an \(\ell_1\) penalty, a positive part penalty, a nonnegativity constraint or no penalty at all. The treatment of each coordinate is specified by a penalty_structure array of labels: the reserved labels UNPENALIZED, L1_PENALTY, POSITIVE_PART and NONNEGATIVE (all negative integers) select the corresponding treatment, and any other label is the name of a group.
- class regreg.atoms.mixed_lasso.mixed_lasso(penalty_structure, lagrange, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
atomMixed LASSO penalty in Lagrange form.
The penalty is
\[\lambda \left(\sum_g w_g \|\beta_g\|_2 + \sum_{j \in L} |\beta_j| + \sum_{j \in P} \max(\beta_j, 0) \right)\]where \(g\) ranges over the groups, \(L\) is the set of coordinates labelled L1_PENALTY and \(P\) the set of coordinates labelled POSITIVE_PART. Coordinates labelled UNPENALIZED are not penalized and coordinates labelled NONNEGATIVE are constrained to be nonnegative. The proximal map also sets negative POSITIVE_PART coordinates to zero.
- Parameters:
penalty_structure (np.ndarray) – One label per coordinate. Labels in reserved select the special treatments described above; every other label defines a group.
lagrange (float) – Lagrange parameter \(\lambda\).
weights (dict, optional) – Maps group labels to weights \(w_g\). Groups not in weights get weight \(\sqrt{|g|}\).
offset (np.ndarray, optional) – Offset subtracted from the argument before the penalty is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- property conjugate#
The conjugate of an atom.
- property dual#
The dual of an atom.
- property linear_transform#
The linear transform applied before a penalty is computed.
Defaults to regreg.affine.identity.
Examples
>>> from regreg.api import l1norm >>> penalty = l1norm(30, lagrange=3.4) >>> type(penalty.linear_transform) <class 'regreg.affine.identity'>
- nonsmooth_objective(x, check_feasibility=False)#
The nonsmooth objective of the atom, including its quadratic term.
- Parameters:
x (np.ndarray) – Argument of the objective.
check_feasibility (bool, optional) – If True, return np.inf when x violates the nonnegativity constraints.
- Returns:
value (float) – Value of the penalty plus self.quadratic at x.
- seminorm(x, check_feasibility=False)#
The mixed LASSO penalty, multiplied by self.lagrange.
- Parameters:
x (np.ndarray) – Argument of the penalty. The offset is not applied here: nonsmooth_objective subtracts it before calling this method.
check_feasibility (bool, optional) – If True, return np.inf when the NONNEGATIVE or POSITIVE_PART coordinates are not (strictly) positive.
- Returns:
value (float) – Value of the penalty.
- proximal(proxq, prox_control=None)#
The proximal operator.
The atom is always in Lagrange mode, so the proximal map is
\[v^{\lambda}(x) = \text{argmin}_{v \in \mathbb{R}^p} \frac{L}{2} \|x-v\|^2_2 + \lambda h(v-\alpha) + \langle v, \eta \rangle\]where \(h\) is the mixed LASSO penalty, \(\alpha\) is self.offset, and \(L\), \(x\) and \(\eta\) are the coefficient, center and linear term of the sum of proxq and self.quadratic.
- Parameters:
proxq (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – Ignored.
- Returns:
Z (np.ndarray) – The proximal map.
- class regreg.atoms.mixed_lasso.mixed_lasso_dual(penalty_structure, bound, weights={}, offset=None, quadratic=None, initial=None)#
Bases:
mixed_lassoConjugate of the mixed LASSO penalty, in bound form only.
This is the indicator of the set where the dual seminorm
\[h^*(u) = \max\left(\max_g \frac{\|u_g\|_2}{w_g}, \max_{j \in L} |u_j|, \max_{j \in P} \max(u_j, 0)\right)\]is at most bound (see mixed_lasso for notation).
- Parameters:
penalty_structure (np.ndarray) – One label per coordinate, as in mixed_lasso.
bound (float) – Bound on the dual seminorm.
weights (dict, optional) – Maps group labels to weights \(w_g\). Groups not in weights get weight \(\sqrt{|g|}\).
offset (np.ndarray, optional) – Offset subtracted from the argument before the constraint is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- property conjugate#
The conjugate of an atom.
- property dual#
The dual of an atom.
- property linear_transform#
The linear transform applied before a penalty is computed.
Defaults to regreg.affine.identity.
Examples
>>> from regreg.api import l1norm >>> penalty = l1norm(30, lagrange=3.4) >>> type(penalty.linear_transform) <class 'regreg.affine.identity'>
- constraint(x, bound=None)#
Check whether the seminorm of x is at most bound.
As with seminorm, the offset is not applied here: nonsmooth_objective subtracts it before calling this method.
- Parameters:
x (np.ndarray) – Argument of the constraint.
bound (float) – Bound on the seminorm. Must be supplied; a ValueError is raised if it is None.
- Returns:
value (float) – 0 if the seminorm of x is at most bound, np.inf otherwise.
- nonsmooth_objective(x, check_feasibility=False)#
The nonsmooth objective of the atom, including its quadratic term.
- Parameters:
x (np.ndarray) – Argument of the objective.
check_feasibility (bool, optional) – If True, include the result of self.constraint (0 if feasible, np.inf otherwise); otherwise the constraint term is taken to be 0.
- Returns:
value (float) – Value of the objective at x.
- seminorm(x, lagrange=1, check_feasibility=False)#
The dual seminorm of the mixed LASSO penalty.
- Parameters:
x (np.ndarray) – Argument of the dual seminorm. The offset is not applied here: nonsmooth_objective subtracts it before calling constraint.
lagrange (float, optional) – Ignored.
check_feasibility (bool, optional) – If True, return np.inf when x is outside the dual set: a nonzero unpenalized coordinate or group of weight 0, a negative positive-part coordinate, or a positive nonnegative one.
- Returns:
value (float) – Value of the dual seminorm.
- proximal(proxq, prox_control=None)#
The proximal operator.
The atom is always in bound mode, so the proximal map is
\[v^{\delta}(x) = \text{argmin}_{v \in \mathbb{R}^p} \frac{L}{2} \|x-v\|^2_2 + \langle v, \eta \rangle \ \text{s.t.} \ h^*(v-\alpha) \leq \delta\]where \(h^*\) is the dual seminorm, \(\delta\) is self.bound, \(\alpha\) is self.offset, and \(L\), \(x\) and \(\eta\) are the coefficient, center and linear term of the sum of proxq and self.quadratic.
- Parameters:
proxq (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – Ignored.
- Returns:
Z (np.ndarray) – The proximal map.
- regreg.atoms.mixed_lasso.strong_set(glasso, lagrange_cur, lagrange_new, grad, slope_estimate=1)#
Guess the active variables at a new value of the Lagrange parameter.
Uses the strong rules, based on the gradient at the current value of the Lagrange parameter, to guess which variables are active at lagrange_new.
- Parameters:
glasso (mixed_lasso) – The penalty.
lagrange_cur (float) – Current value of the Lagrange parameter.
lagrange_new (float) – New value of the Lagrange parameter.
grad (np.ndarray) – Gradient of the smooth part of the objective at the solution for lagrange_cur.
slope_estimate (float, optional) – Estimate of the slope used by the strong rules.
- Returns:
value (np.ndarray) – Boolean array, True for the variables guessed to be active.
selector (regreg.affine.selector) – Selector of the guessed active variables.
- regreg.atoms.mixed_lasso.check_KKT(glasso, grad, solution, lagrange, tol=0.01, kkt_tol=None)#
Check whether (grad, solution) satisfy the KKT conditions.
- Parameters:
glasso (mixed_lasso) – The penalty.
grad (np.ndarray) – Gradient of the smooth part of the objective at solution.
solution (np.ndarray) – Candidate solution.
lagrange (float) – Lagrange parameter.
tol (float, optional) – Relative tolerance for the checks.
kkt_tol (float, optional) – Tighter relative tolerance for the dual feasibility of the \(\ell_1\), positive part and group coordinates at zero, which decides whether they enter the model. If None, only tol is used.
- Returns:
failing (np.ndarray) – Boolean array, True for the coordinates where the KKT conditions fail.
- regreg.atoms.mixed_lasso.n1#
alias of
mixed_lasso
- regreg.atoms.mixed_lasso.n2#
alias of
mixed_lasso_dual
regreg.atoms.slope#
Implementation of the SLOPE penalty and its proximal operator.
See Bogdan, van den Berg, Sabatti, Su and Candes, “SLOPE – adaptive variable selection via convex optimization”, https://statweb.stanford.edu/~candes/papers/SLOPE.pdf
- class regreg.atoms.slope.slope(weights, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormThe SLOPE penalty.
The sorted \(\ell_1\) norm \(\sum_j w_j |\beta|_{(j)}\), where \(|\beta|_{(1)} \geq |\beta|_{(2)} \geq \dots\) are the sorted absolute values of \(\beta\).
- Parameters:
weights (np.ndarray) – Positive, non-increasing weights \(w_j\). The shape of the atom is the shape of weights.
lagrange (float, optional) – Lagrange parameter (exactly one of lagrange and bound must be specified).
bound (float, optional) – Bound parameter.
offset (np.ndarray, optional) – Offset subtracted from the argument before the penalty is evaluated.
quadratic (identity_quadratic, optional) – Quadratic term added to the penalty.
initial (np.ndarray, optional) – Initial value of the coefficients.
Notes
Only the Lagrange form has a proximal map implemented: bound_prox raises
NotImplementedError.- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \sum_j \lambda_j |\beta_{(j)}|\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(x, bound=None)#
Verify \(\sum_j \lambda_j |\beta_{(j)}| \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(x, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \sum_j \lambda_j |\beta_{(j)}|\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(x, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \sum_j \lambda_j |\beta_{(j)}| \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.slope([3., 2., 1.], lagrange=3.4) >>> penalty.get_conjugate() slope_conjugate(..., bound=3.4...)
- class regreg.atoms.slope.slope_conjugate(weights, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
slopeThe dual norm of the SLOPE penalty.
For weights \(w\), this is \(\max_k \left(\sum_{j \leq k} |\beta|_{(j)}\right) / \left(\sum_{j \leq k} w_j\right)\).
Notes
Only the bound form has a proximal map implemented, computed from the SLOPE proximal map via the Moreau decomposition: lagrange_prox raises
NotImplementedError.- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot {\cal D}^{SLOPE}(\beta)\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(x, bound=None)#
Verify \({\cal D}^{SLOPE}(\beta) \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(x, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda {\cal D}^{SLOPE}(\beta)\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(x, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ {\cal D}^{SLOPE}(\beta) \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- regreg.atoms.slope.n2#
alias of
slope_conjugate
regreg.atoms.block_norms#
Block norms of matrices.
This module contains the implementation of block norms, i.e. \(\ell_1/\|\cdot\|\) and \(\ell_{\infty}/\|\cdot\|\) norms that apply a norm to each row of a matrix and then combine the row norms with an \(\ell_1\) (sum) or \(\ell_{\infty}\) (max) norm. These are used in multiresponse LASSOs.
- class regreg.atoms.block_norms.block_sum(atom_cls, shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormSum of the norms of the rows of a matrix.
For a matrix \(B\) of shape \((n, p)\) and a seminorm \(\|\cdot\|\) on \(\mathbb{R}^p\) this is
\[\|B\|_{1,\|\cdot\|} = \sum_{i=1}^n \|B[i]\|\]where \(B[i]\) is the \(i\)-th row of \(B\).
- Parameters:
atom_cls (type) – Class of the seminorm applied to each row, e.g. regreg.atoms.seminorms.l2norm.
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- seminorms(x, lagrange=None, check_feasibility=False)#
Compute the seminorm of each row of the block norm’s argument.
- Parameters:
x (np.ndarray) – Matrix whose rows are passed to the row seminorm.
lagrange (float, optional) – Lagrange parameter passed to the row seminorm.
check_feasibility (bool, optional) – Ignored: feasibility is never checked for the rows.
- Returns:
value (np.ndarray) – Array of shape
(self.shape[0],)with the seminorm of each row.
- seminorm(x, check_feasibility=False, lagrange=None)#
Return \(\lambda \cdot \|B\|_{1,\|\cdot\|}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(x)#
Verify \(\|B\|_{1,\|\cdot\|} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(x, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{1,\|\cdot\|}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(x, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{1,\|\cdot\|} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=3.4) >>> penalty.get_conjugate() block_max(..., bound=3.4...)
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{1,\|\cdot\|}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.block_sum(rr.l1norm, (5, 4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.block_sum(rr.l1norm, (5, 4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.block_sum(rr.l1norm, (5, 4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=2.3) >>> penalty block_sum(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, block_max(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.block_norms.block_max(atom_cls, shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
block_sumMaximum of the norms of the rows of a matrix.
For a matrix \(B\) of shape \((n, p)\) and a seminorm \(\|\cdot\|\) on \(\mathbb{R}^p\) this is
\[\|B\|_{\infty,\|\cdot\|} = \max_{1 \leq i \leq n} \|B[i]\|\]where \(B[i]\) is the \(i\)-th row of \(B\). It is the dual of block_sum with the dual row seminorm.
- Parameters:
atom_cls (type) – Class of the seminorm applied to each row, e.g. regreg.atoms.seminorms.l2norm.
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{\infty,\| \cdot\|}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(x, bound=None)#
Verify \(\|B\|_{\infty,\| \cdot\|} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(x, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\infty,\| \cdot\|}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(x, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\infty,\| \cdot\|} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{\infty,\| \cdot\|}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.block_max(rr.l1norm, (5, 4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.block_max(rr.l1norm, (5, 4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.block_max(rr.l1norm, (5, 4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=2.3) >>> penalty block_max(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, block_sum(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=3.4) >>> penalty.get_conjugate() block_sum(..., bound=3.4...)
- class regreg.atoms.block_norms.linf_l2(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
block_maxMaximum of the \(\ell_2\) norms of the rows of a matrix.
\[\|B\|_{\infty,2} = \max_{1 \leq i \leq n} \|B[i]\|_2\]This is the dual norm of l1_l2.
- Parameters:
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- constraint(arg)#
Verify \(\|B\|_{\infty,2} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{\infty,2}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\infty,2} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\infty,2}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_l2((5,4), lagrange=3.4) >>> penalty.get_conjugate() l1_l2(..., bound=3.4...)
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{\infty,2}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.linf_l2((5,4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.linf_l2((5,4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.linf_l2((5,4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_l2((5,4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_l2((5,4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.linf_l2((5,4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.linf_l2((5,4), lagrange=2.3) >>> penalty linf_l2(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l1_l2(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.block_norms.linf_linf(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
linf_l2Maximum absolute value of the entries of a matrix.
\[\|B\|_{\infty,\infty} = \max_{i,j} |B_{ij}|\]This is the dual norm of l1_l1.
- Parameters:
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- constraint(arg)#
Verify \(\|B\|_{\infty,\infty} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{\infty,\infty}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\infty,\infty} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\infty,\infty}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{\infty,\infty}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.linf_linf((5,4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.linf_linf((5,4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.linf_linf((5,4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_linf((5,4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_linf((5,4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.linf_linf((5,4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.linf_linf((5,4), lagrange=2.3) >>> penalty linf_linf(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, l1_l1(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.linf_linf((5,4), lagrange=3.4) >>> penalty.get_conjugate() l1_l1(..., bound=3.4...)
- class regreg.atoms.block_norms.l1_l2(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
block_sumSum of the \(\ell_2\) norms of the rows of a matrix.
\[\|B\|_{1,2} = \sum_{i=1}^n \|B[i]\|_2\]This is a group LASSO penalty with the rows of \(B\) as groups, and is the dual norm of linf_l2.
- Parameters:
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{1,2}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{1,2} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- constraint(x)#
Verify \(\|B\|_{1,2} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{1,2}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{1,2}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1_l2((5,4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1_l2((5,4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.l1_l2((5,4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_l2((5,4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_l2((5,4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.l1_l2((5,4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.l1_l2((5,4), lagrange=2.3) >>> penalty l1_l2(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, linf_l2(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_l2((5,4), lagrange=3.4) >>> penalty.get_conjugate() linf_l2(..., bound=3.4...)
- class regreg.atoms.block_norms.l1_l1(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
l1_l2Sum of the absolute values of the entries of a matrix.
\[\|B\|_{1,1} = \sum_{i,j} |B_{ij}|\]This is the dual norm of linf_linf.
- Parameters:
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{1,1}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- constraint(arg)#
Verify \(\|B\|_{1,1} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- seminorm(arg, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{1,1}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{1,1} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{1,1}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1_l1((5,4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.l1_l1((5,4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.l1_l1((5,4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_l1((5,4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_l1((5,4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.l1_l1((5,4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.l1_l1((5,4), lagrange=2.3) >>> penalty l1_l1(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, linf_linf(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1_l1((5,4), lagrange=3.4) >>> penalty.get_conjugate() linf_linf(..., bound=3.4...)
regreg.atoms.sparse_group_block#
Implementation of the sparse group block norm and its dual.
The sparse group block norm of a matrix \(B\) is
where \(B[i]\) is the \(i\)-th row of \(B\). It is a sparse group LASSO in which the groups are the rows of \(B\), used in multiresponse problems.
- class regreg.atoms.sparse_group_block.sparse_group_block(shape, l1_weight, l2_weight, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
l1_l2The sparse group block norm.
\[B \mapsto w_1 \sum_{i,j} |B_{ij}| + w_2 \sum_i \|B[i]\|_2\]- Parameters:
shape (tuple) – Shape of the matrix argument.
l1_weight (float) – Weight \(w_1\) of the elementwise \(\ell_1\) norm.
l2_weight (float) – Weight \(w_2\) of the sum of the \(\ell_2\) norms of the rows.
lagrange (float, optional) – Lagrange parameter. Exactly one of lagrange and bound must be specified.
bound (float, optional) – Bound parameter. Exactly one of lagrange and bound must be specified.
offset (np.ndarray, optional) – Offset subtracted from the argument before evaluating the seminorm.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda w_1\|B\|_{1,1} + w_2\|B\|_{1,2}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ w_1\|B\|_{1,1} + w_2\|B\|_{1,2} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- constraint(x)#
Verify \(w_1\|B\|_{1,1} + w_2\|B\|_{1,2} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot w_1\|B\|_{1,1} + w_2\|B\|_{1,2}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.sparse_group_block((5, 4), 1, 2, lagrange=3.4) >>> penalty.get_conjugate() sparse_group_block_dual(..., bound=3.4...)
- terms(arg)#
Return the terms that are summed in computing the seminorm.
- Parameters:
arg (np.ndarray) – Matrix argument of the seminorm.
- Returns:
terms (np.ndarray) – The contribution of each row of arg to the seminorm.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.sparse_group_block((2, 3), 1, 2, lagrange=1.) >>> arg = np.array([[1., 2, 2], [0, 3, 4]]) >>> penalty.terms(arg) array([11., 17.]) >>> print(penalty.seminorm(arg)) 28.0
- class regreg.atoms.sparse_group_block.sparse_group_block_dual(shape, l1_weight, l2_weight, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
sparse_group_blockThe dual of the sparse group block norm.
Takes the same arguments as sparse_group_block. The dual norm is computed by a binary search.
- lagrange_prox(arg, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{w_1,w_2,\text{block}}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(arg, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{w_1,w_2,\text{block}} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- constraint(x)#
Verify \(\|B\|_{w_1,w_2,\text{block}} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- seminorm(x, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{w_1,w_2,\text{block}}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.sparse_group_block_dual((5, 4), 1, 2, lagrange=3.4) >>> penalty.get_conjugate() sparse_group_block(..., bound=3.4...)
- terms(arg)#
Return the terms that are maximized in computing the seminorm.
- Parameters:
arg (np.ndarray) – Matrix argument of the seminorm.
- Returns:
terms (np.ndarray) – The dual norm of each row of arg.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.sparse_group_block_dual((2, 3), 1, 1, bound=1.) >>> arg = np.array([[1., 2, 2], [0, 3, 4]]) >>> np.round(penalty.terms(arg), 3) array([1.172, 2.101]) >>> print(np.round(penalty.seminorm(arg, lagrange=1), 3)) 2.101
- regreg.atoms.sparse_group_block.n1#
alias of
sparse_group_block
- regreg.atoms.sparse_group_block.n2#
alias of
sparse_group_block_dual
regreg.atoms.svd_norms#
Norms and cones defined in terms of singular values.
This module contains the implementation of the operator and nuclear norms, as well as their epigraphs and the polars of these epigraphs. These are used in matrix completion problems and other low-rank factorization problems.
- class regreg.atoms.svd_norms.svd_atom(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
seminormBase class for seminorms of a matrix defined through its singular values.
- class regreg.atoms.svd_norms.nuclear_norm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
svd_atomThe nuclear norm.
The nuclear (trace) norm of a matrix \(B\) is the sum of its singular values
\[\|B\|_* = \sum_i \sigma_i(B).\]Its dual norm is operator_norm.
- Parameters:
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- seminorm(X, check_feasibility=False, lagrange=None)#
Return \(\lambda \cdot \|B\|_*\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(X, bound=None)#
Verify \(\|B\|_* \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(X, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_*\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(X, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_* \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_*\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.nuclear_norm((5,4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.nuclear_norm((5,4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.nuclear_norm((5,4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm((5,4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm((5,4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.nuclear_norm((5,4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm((5,4), lagrange=3.4) >>> penalty.get_conjugate() operator_norm(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.nuclear_norm((5,4), lagrange=2.3) >>> penalty nuclear_norm(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, operator_norm(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.svd_norms.operator_norm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)#
Bases:
svd_atomThe operator norm.
The operator (spectral) norm of a matrix \(B\) is its largest singular value
\[\|B\|_{\text{op}} = \max_i \sigma_i(B).\]Its dual norm is nuclear_norm.
- Parameters:
shape (tuple) – Shape \((n, p)\) of the matrix argument.
lagrange (float, optional) – Lagrange parameter (penalty form). Exactly one of lagrange and bound should be specified.
bound (float, optional) – Bound parameter (constraint form).
offset (np.ndarray, optional) – Offset \(\alpha\) subtracted from the argument before the norm is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- seminorm(X, lagrange=None, check_feasibility=False)#
Return \(\lambda \cdot \|B\|_{\text{op}}\).
Here \(\lambda\) is lagrange. If check_feasibility is True, and the seminorm is unbounded, will return np.inf if appropriate.
The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
lagrange (float, optional) – Lagrange parameter. Defaults to self.lagrange.
check_feasibility (bool, optional) – If True, return np.inf if arg is infeasible.
- Returns:
value (float) – The value of the seminorm.
- constraint(X, bound=None)#
Verify \(\|B\|_{\text{op}} \leq \delta\).
Here \(\delta\) is bound. If the constraint is satisfied, returns 0, else returns np.inf.
The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the seminorm.
bound (float, optional) – Bound parameter. Defaults to self.bound.
- Returns:
value (float) – 0 if the constraint is satisfied, np.inf otherwise.
- lagrange_prox(X, lipschitz=1, lagrange=None)#
Return the unique minimizer of the Lagrange form proximal problem.
\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\text{op}}\]Above, \(\lambda\) is the Lagrange parameter, \(L\) is the Lipschitz parameter and \(\theta\) is arg.
If the argument lagrange is None and the atom is in Lagrange mode,
self.lagrangeis used as the Lagrange parameter, else an exception is raised.The base class seminorm just returns the appropriate Lagrange parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
lipschitz (float, optional) – Coefficient in front of the quadratic.
lagrange (float, optional) – Lagrange factor in front of the seminorm. Defaults to self.lagrange.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- bound_prox(X, bound=None)#
Return the unique minimizer of the bound form proximal problem.
\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\text{op}} \leq \delta\]where \(\delta\) is the bound parameter and \(\theta\) is arg.
If the argument bound is None and the atom is in bound mode,
self.boundis used as the bound parameter, else an exception is raised.The base class seminorm just returns the appropriate bound parameter for use by the subclasses.
- Parameters:
arg (np.ndarray) – Argument of the proximal map.
bound (float, optional) – Bound for the constraint on the seminorm. Defaults to self.bound.
- Returns:
Z (np.ndarray) – The proximal map of arg.
- proximal(quadratic, prox_control=None)#
The proximal operator.
If the atom is in Lagrange mode, this has the form
\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and
\[h(B) = \|B\|_{\text{op}}\]If the atom is in bound mode, then this has the form
\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (dict, optional) – This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> penalty = rr.l1norm(4, lagrange=2) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> penalty.proximal(Q) array([ 1.6666..., -2.6666..., -0. , 0. ])
- get_bound()#
Get method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.operator_norm((5,4), bound=2.3) >>> constraint.bound 2.3
- set_bound(bound)#
Set method of the bound property.
Examples
>>> import regreg.api as rr >>> constraint = rr.operator_norm((5,4), bound=3.4) >>> constraint.bound 3.4 >>> constraint.bound = 2.3 >>> constraint.bound 2.3 >>> penalty = rr.operator_norm((5,4), lagrange=2.3) >>> penalty.bound = 3.4 Traceback (most recent call last): ... AttributeError: atom is in lagrange mode
- get_lagrange()#
Get method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm((5,4), lagrange=3.4) >>> penalty.lagrange 3.4
- set_lagrange(lagrange)#
Set method of the lagrange property.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm((5,4), lagrange=3.4) >>> penalty.lagrange 3.4 >>> penalty.lagrange = 2.3 >>> penalty.lagrange 2.3 >>> constraint = rr.operator_norm((5,4), bound=3.4) >>> constraint.lagrange = 3.4 Traceback (most recent call last): ... AttributeError: atom is in bound mode
- get_conjugate()#
Return the conjugate of a given atom.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm((5,4), lagrange=3.4) >>> penalty.get_conjugate() nuclear_norm(..., bound=3.4...)
- get_dual()#
Return the dual of an atom.
This dual is formed by introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (regreg.affine.linear_transform) – The linear transform of the atom.
conjugate (atom) – The conjugate of the atom.
Examples
>>> import numpy as np >>> import regreg.api as rr >>> penalty = rr.operator_norm((5,4), lagrange=2.3) >>> penalty operator_norm(..., lagrange=2.3...) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nuclear_norm(..., bound=2.3...))
If there is a linear part to the penalty, the linear_transform may not be the identity. For example, the 1D fused LASSO penalty:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.l1norm.linear(D, lagrange=2.3) >>> linear_atom affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
- class regreg.atoms.svd_norms.svd_cone(input_shape, offset=None, quadratic=None, initial=None)#
Bases:
coneBase class for cones defined through the singular values of a matrix.
The argument of such a cone is a vector of length \(np+1\): the first \(np\) entries are a flattened \(n \times p\) matrix and the last entry is a scalar.
- Parameters:
input_shape (tuple) – Shape \((n, p)\) of the matrix part of the argument.
offset (np.ndarray, optional) – Offset subtracted from the argument before the constraint is evaluated.
quadratic (regreg.identity_quadratic.identity_quadratic, optional) – Quadratic term added to the atom.
initial (np.ndarray, optional) – Initial value for the coefficients.
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm_epigraph((5,4)) >>> penalty.get_conjugate() nuclear_norm_epigraph_polar((5,4), offset=None)
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[B^{*}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle + h(B - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(B) = \|B\|\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm_epigraph((5,4)) >>> penalty nuclear_norm_epigraph((5,4), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nuclear_norm_epigraph_polar((5,4), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.svd_norms.nuclear_norm_epigraph(input_shape, offset=None, quadratic=None, initial=None)#
Bases:
svd_coneThe epigraph of the nuclear norm.
\[\{(B, t) : \|B\|_* \leq t\}\]Its polar cone is nuclear_norm_epigraph_polar.
See svd_cone for the constructor arguments.
- constraint(normX)#
Evaluate the constraint
\[I^{\infty}(\|B[:-1]\|_* \leq B[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(normX, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[B^{*}(u) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{1}{2} \|B-u\|^2_2 + I^{\infty}(\|B[:-1]\|_* \leq B[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[B^{*}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle + h(B - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(B) = I^{\infty}(\|B[:-1]\|_* \leq B[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm_epigraph((5,4)) >>> penalty nuclear_norm_epigraph((5,4), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nuclear_norm_epigraph_polar((5,4), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm_epigraph((5,4)) >>> penalty.get_conjugate() nuclear_norm_epigraph_polar((5,4), offset=None)
- class regreg.atoms.svd_norms.nuclear_norm_epigraph_polar(input_shape, offset=None, quadratic=None, initial=None)#
Bases:
svd_coneThe polar of the epigraph of the nuclear norm.
\[\{(B, t) : \|B\|_{\text{op}} \leq -t\}\]Its polar cone is nuclear_norm_epigraph.
See svd_cone for the constructor arguments.
- constraint(normX)#
Evaluate the constraint
\[I^{\infty}(\|B[:-1]\|_{op} \leq -B[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(normX, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[B^{*}(u) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{1}{2} \|B-u\|^2_2 + I^{\infty}(\|B[:-1]\|_{op} \leq -B[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[B^{*}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle + h(B - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(B) = I^{\infty}(\|B[:-1]\|_{op} \leq -B[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm_epigraph_polar((5,4)) >>> penalty nuclear_norm_epigraph_polar((5,4), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, nuclear_norm_epigraph((5,4), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.nuclear_norm_epigraph_polar((5,4)) >>> penalty.get_conjugate() nuclear_norm_epigraph((5,4), offset=None)
- class regreg.atoms.svd_norms.operator_norm_epigraph(input_shape, offset=None, quadratic=None, initial=None)#
Bases:
svd_coneThe epigraph of the operator norm.
\[\{(B, t) : \|B\|_{\text{op}} \leq t\}\]Its polar cone is operator_norm_epigraph_polar.
See svd_cone for the constructor arguments.
- constraint(normX)#
Evaluate the constraint
\[I^{\infty}(\|B[:-1]\|_{op} \leq B[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(normX, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[B^{*}(u) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{1}{2} \|B-u\|^2_2 + I^{\infty}(\|B[:-1]\|_{op} \leq B[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[B^{*}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle + h(B - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(B) = I^{\infty}(\|B[:-1]\|_{op} \leq B[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm_epigraph((5,4)) >>> penalty operator_norm_epigraph((5,4), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, operator_norm_epigraph_polar((5,4), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm_epigraph((5,4)) >>> penalty.get_conjugate() operator_norm_epigraph_polar((5,4), offset=None)
- class regreg.atoms.svd_norms.operator_norm_epigraph_polar(input_shape, offset=None, quadratic=None, initial=None)#
Bases:
svd_coneThe polar of the epigraph of the operator norm.
\[\{(B, t) : \|B\|_* \leq -t\}\]Its polar cone is operator_norm_epigraph.
See svd_cone for the constructor arguments.
- constraint(normX)#
Evaluate the constraint
\[I^{\infty}(\|B[:-1]\|_{*} \leq -B[-1])\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(normX, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[B^{*}(u) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{1}{2} \|B-u\|^2_2 + I^{\infty}(\|B[:-1]\|_{*} \leq -B[-1])\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(quadratic, prox_control=None)#
The proximal operator.
This has the form
\[B^{*}(\theta) = \text{argmin}_{B \in \mathbb{R}^{{n \times p + 1}}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle + h(B - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(B) = I^{\infty}(\|B[:-1]\|_{*} \leq -B[-1])\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm_epigraph_polar((5,4)) >>> penalty operator_norm_epigraph_polar((5,4), offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, operator_norm_epigraph((5,4), offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.operator_norm_epigraph_polar((5,4)) >>> penalty.get_conjugate() operator_norm_epigraph((5,4), offset=None)
regreg.atoms.linear_constraints#
- class regreg.atoms.linear_constraints.linear_constraint(shape, basis, offset=None, initial=None, quadratic=None)#
Bases:
coneBase class for linear (subspace) constraints.
This class allows specification of linear constraints of the form \(x \in \text{row}(L)\) by specifying an orthonormal basis for the rowspace of \(L\).
If the constraint is of the form \(Ax=0\), then this linear constraint can also be created using the linear classmethod of the zero_constraint cone in regreg.atoms.cones.
- Parameters:
shape (int or tuple) – Shape of the argument of the constraint.
basis (np.ndarray) – Matrix whose rows are an orthonormal basis for the subspace.
offset (np.ndarray, optional) – Offset subtracted from the argument before the constraint is evaluated.
initial (np.ndarray, optional) – Initial value of the coefficients.
quadratic (identity_quadratic, optional) – Quadratic term added to the constraint.
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.projection((4,), [[1,0,0,0],[0,1,0,0]]) >>> penalty.get_conjugate() projection_complement((4,), [[1,0,0,0],[0,1,0,0]], offset=None)
- classmethod linear(linear_operator, basis, diag=False, offset=None, quadratic=None)#
Compose a cone constraint with a linear transform.
- Parameters:
linear_operator (np.ndarray or linear_transform) – The linear operator \(D\). If not already a linear_transform, it is wrapped in one.
diag (bool, optional) – Passed to linear_transform: if True, linear_operator is interpreted as the diagonal of a diagonal matrix.
offset (np.ndarray, optional) – If not None, the constraint is imposed on \(D\beta + \text{offset}\).
quadratic (identity_quadratic, optional) – Quadratic term added to the cone constraint.
- Returns:
atom (affine_cone) – The composition of the cone constraint with \(D\).
- class regreg.atoms.linear_constraints.projection(shape, basis, offset=None, initial=None, quadratic=None)#
Bases:
linear_constraintAn atom representing a linear constraint.
The constraint restricts the argument to the span of the rows of basis, a matrix that is assumed to be a set of row vectors spanning the space.
Notes
It is assumed (without checking) that the rows of basis are orthonormal, so that projecting x onto their span is simply
np.dot(basis.T, np.dot(basis, x)).- constraint(x)#
Evaluate the constraint
\[\|\beta\|\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + \|\beta\|\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(proxq, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = \|\beta\|\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.projection((4,), [[1,0,0,0],[0,1,0,0]]) >>> penalty.get_conjugate() projection_complement((4,), [[1,0,0,0],[0,1,0,0]], offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.projection((4,), [[1,0,0,0],[0,1,0,0]]) >>> penalty projection((4,), [[1,0,0,0],[0,1,0,0]], offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, projection_complement((4,), [[1,0,0,0],[0,1,0,0]], offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- class regreg.atoms.linear_constraints.projection_complement(shape, basis, offset=None, initial=None, quadratic=None)#
Bases:
linear_constraintAn atom representing a linear constraint.
The orthogonal complement of projection: the argument is constrained to the orthogonal complement of the span of the rows of basis, which are assumed to be orthonormal.
- constraint(x)#
Evaluate the constraint
\[\|\beta\|\]- Parameters:
x (np.ndarray) – Argument of the constraint.
- Returns:
value (float) – 0 if x satisfies the constraint (up to the tolerance
self.tol), otherwisenp.inf.
- cone_prox(x, lipschitz=1)#
Euclidean projection onto the cone.
Returns the unique minimizer
\[\beta^{*}(u) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\beta-u\|^2_2 + \|\beta\|\]where \(u\) is x.
- Parameters:
x (np.ndarray) – Point to project.
- Returns:
projection (np.ndarray) – The projection of x onto the cone.
- proximal(proxq, prox_control=None)#
The proximal operator.
This has the form
\[\beta^{*}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle + h(\beta - \alpha)\]where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center, \(L\) is quadratic.coef and
\[h(\beta) = \|\beta\|\]- Parameters:
quadratic (regreg.identity_quadratic.identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – This argument is ignored for cones, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of quadratic.
Examples
>>> import regreg.api as rr >>> cone = rr.nonnegative((4,)) >>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0) >>> np.allclose(cone.proximal(Q), [3, 0, 0, 1]) True
- get_conjugate()#
Return the conjugate of the cone constraint.
The conjugate of the indicator of a cone is the indicator of its polar cone. If the atom has a nonzero quadratic term, a smooth conjugate is returned instead.
Examples
>>> import regreg.api as rr >>> penalty = rr.projection_complement((4,), [[1,0,0,0],[0,1,0,0]]) >>> penalty.get_conjugate() projection((4,), [[1,0,0,0],[0,1,0,0]], offset=None)
- get_dual()#
Return the dual of an atom.
This dual is formed by making the substitution \(v=Ax\) where \(A\) is self.linear_transform.
- Returns:
linear_transform (linear_transform) – The linear transform \(A\).
conjugate (cone) – The conjugate of the cone constraint.
Examples
>>> import regreg.api as rr >>> penalty = rr.projection_complement((4,), [[1,0,0,0],[0,1,0,0]]) >>> penalty projection_complement((4,), [[1,0,0,0],[0,1,0,0]], offset=None) >>> penalty.dual (<regreg.affine.identity object at 0x...>, projection((4,), [[1,0,0,0],[0,1,0,0]], offset=None))
If there is a linear part to the penalty, the linear_transform may not be the identity:
>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1] >>> D array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]]) >>> linear_atom = rr.nonnegative.linear(D) >>> linear_atom affine_cone(nonnegative((3,), offset=None), array([[ 1., -1., 0., 0.], [ 0., 1., -1., 0.], [ 0., 0., 1., -1.]])) >>> linear_atom.dual (<regreg.affine.linear_transform object at 0x...>, nonpositive((3,), offset=None))
- regreg.atoms.linear_constraints.n1#
alias of
projection
- regreg.atoms.linear_constraints.n2#
alias of
projection_complement
regreg.atoms.simplex#
The simplex constraint and its conjugate, the maximum function.
- class regreg.atoms.simplex.simplex_constraint(shape, offset=None, quadratic=None, initial=None)#
Bases:
atomThe simplex constraint.
The indicator of the probability simplex \(\{\beta : \beta_i \geq 0, \sum_i \beta_i = 1\}\). Its conjugate is the maximum function.
- proximal(proxq, prox_control=None)#
Projection onto the simplex.
Computes the proximal map of the simplex constraint, i.e. the Euclidean projection onto the simplex of the center implied by proxq (combined with self.quadratic and self.offset).
- Parameters:
proxq (identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – Ignored.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of proxq.
- nonsmooth_objective(arg, check_feasibility=False)#
Value of the simplex constraint.
Always 0 (plus the value of self.quadratic) unless check_feasibility is True.
- Parameters:
arg (np.ndarray) – Argument of the constraint.
check_feasibility (bool, optional) – If True, then return
np.infif arg is not in the simplex.
- Returns:
value (float) – The value of the constraint at arg.
- class regreg.atoms.simplex.maximum(shape, offset=None, quadratic=None, initial=None)#
Bases:
atomThe conjugate of the simplex constraint:
x.max().- proximal(proxq, prox_control=None)#
Proximal map of the maximum function.
By the Moreau decomposition, this is the residual after projecting the implied center of proxq onto the simplex scaled by
1 / L, where \(L\) is the coefficient of the combined quadratic.- Parameters:
proxq (identity_quadratic) – A quadratic added to the atom before minimizing.
prox_control (None or dict, optional) – Ignored.
- Returns:
Z (np.ndarray) – The proximal map of the implied center of proxq.
- nonsmooth_objective(arg, check_feasibility=False)#
Value of the maximum function.
Returns the maximum entry of arg (after applying the offset) plus the value of self.quadratic.
- Parameters:
arg (np.ndarray) – Argument of the maximum function.
check_feasibility (bool, optional) – Ignored: the maximum function is finite everywhere.
- Returns:
value (float) – The value of the maximum function at arg.
- regreg.atoms.simplex.n1#
alias of
simplex_constraint