atoms.seminorms

Module: atoms.seminorms

Inheritance diagram for regreg.atoms.seminorms:

digraph inheritance6a1e1e2540 { rankdir=LR; size="8.0, 12.0"; "atoms.seminorms.constrained_max" [URL="#regreg.atoms.seminorms.constrained_max",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The seminorm x.max() s.t. x geq 0."]; "atoms.seminorms.seminorm" -> "atoms.seminorms.constrained_max" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.constrained_positive_part" [URL="#regreg.atoms.seminorms.constrained_positive_part",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Support function of $[-\infty,1]^p$"]; "atoms.seminorms.seminorm" -> "atoms.seminorms.constrained_positive_part" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.l1norm" [URL="#regreg.atoms.seminorms.l1norm",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The l1 norm"]; "atoms.seminorms.seminorm" -> "atoms.seminorms.l1norm" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.l2norm" [URL="#regreg.atoms.seminorms.l2norm",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The l2 norm"]; "atoms.seminorms.seminorm" -> "atoms.seminorms.l2norm" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.max_positive_part" [URL="#regreg.atoms.seminorms.max_positive_part",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="support function of the standard simplex"]; "atoms.seminorms.seminorm" -> "atoms.seminorms.max_positive_part" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.positive_part" [URL="#regreg.atoms.seminorms.positive_part",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The positive_part seminorm (which is the support"]; "atoms.seminorms.seminorm" -> "atoms.seminorms.positive_part" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.seminorm" [URL="#regreg.atoms.seminorms.seminorm",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="An atom that can be in lagrange or bound form."]; "regreg.atoms.atom" -> "atoms.seminorms.seminorm" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.supnorm" [URL="#regreg.atoms.seminorms.supnorm",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The :math:`\ell_{\infty}` norm"]; "atoms.seminorms.seminorm" -> "atoms.seminorms.supnorm" [arrowsize=0.5,style="setlinewidth(0.5)"]; "problems.composite.composite" [URL="regreg.problems.composite.html#regreg.problems.composite.composite",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A generic way to specify a problem in composite form."]; "problems.composite.nonsmooth" [URL="regreg.problems.composite.html#regreg.problems.composite.nonsmooth",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A composite subclass that explicitly returns 0"]; "problems.composite.composite" -> "problems.composite.nonsmooth" [arrowsize=0.5,style="setlinewidth(0.5)"]; "regreg.atoms.atom" [URL="regreg.atoms.html#regreg.atoms.atom",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A class that defines the API for support functions."]; "problems.composite.nonsmooth" -> "regreg.atoms.atom" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Classes

constrained_max

class regreg.atoms.seminorms.constrained_max(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

The seminorm x.max() s.t. x geq 0.

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.constrained_max((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.constrained_max((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
positive_part(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.constrained_max((30,), lagrange=2.3)
>>> penalty 
constrained_max(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, positive_part(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.constrained_max((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\left\\|%(var)s\\right\\|_{\\infty} + I^{\\infty}\\left(\\min(%(var)s) \\in [0,+\\infty)\\right) '
objective_vars = {'dualnormklass': 'positive_part', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'constrained_max', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \left\|\beta\right\|_{\infty} + I^{\infty}\left(\min(\beta) \in [0,+\infty)\right) \)lambda` is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.constrained_max((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.constrained_max((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.constrained_max((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.constrained_max((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

constrained_positive_part

class regreg.atoms.seminorms.constrained_positive_part(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

Support function of \([-\infty,1]^p\)

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i) \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i) \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.constrained_positive_part((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.constrained_positive_part((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
max_positive_part(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.constrained_positive_part((30,), lagrange=2.3)
>>> penalty 
constrained_positive_part(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, max_positive_part(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.constrained_positive_part((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i) \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{1} + \\sum_{i=1}^{%(shape)s} \\delta_{[0,+\\infty]}(%(var)s_i)'
objective_vars = {'dualnormklass': 'max_positive_part', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'constrained_positive_part', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i)\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|\beta\|_{1} + \sum_{i=1}^{p} \delta_{[0,+\infty]}(\beta_i)\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.constrained_positive_part((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.constrained_positive_part((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.constrained_positive_part((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.constrained_positive_part((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

l1norm

class regreg.atoms.seminorms.l1norm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

The l1 norm

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_1 \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\|\beta\|_1 \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1norm((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
supnorm(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=2.3)
>>> penalty 
l1norm(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, supnorm(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_1 \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_1'
objective_vars = {'dualnormklass': 'supnorm', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'l1norm', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
prox_tol = 1e-10
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \|\beta\|_1\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|\beta\|_1\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1norm((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.l1norm((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.l1norm((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

l2norm

class regreg.atoms.seminorms.l2norm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

The l2 norm

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_2 \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\|\beta\|_2 \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l2norm((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.l2norm((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
l2norm(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.l2norm((30,), lagrange=2.3)
>>> penalty 
l2norm(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, l2norm(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l2norm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_2 \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_2'
objective_vars = {'dualnormklass': 'l2norm', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'l2norm', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \|\beta\|_2\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|\beta\|_2\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l2norm((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.l2norm((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l2norm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.l2norm((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

max_positive_part

class regreg.atoms.seminorms.max_positive_part(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

support function of the standard simplex

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta^+\|_{\infty} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\|\beta^+\|_{\infty} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.max_positive_part((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.max_positive_part((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
constrained_positive_part(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.max_positive_part((30,), lagrange=2.3)
>>> penalty 
max_positive_part(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, constrained_positive_part(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.max_positive_part((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta^+\|_{\infty} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s^+\\|_{\\infty}'
objective_vars = {'dualnormklass': 'constrained_positive_part', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'max_positive_part', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \|\beta^+\|_{\infty}\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|\beta^+\|_{\infty}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.max_positive_part((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.max_positive_part((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.max_positive_part((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.max_positive_part((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

positive_part

class regreg.atoms.seminorms.positive_part(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

The positive_part seminorm (which is the support function of [0,l]^p).

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \left(\sum_{i=1}^{p} (\beta)_i^+\right) \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\left(\sum_{i=1}^{p} (\beta)_i^+\right) \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.positive_part((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.positive_part((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
constrained_max(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.positive_part((30,), lagrange=2.3)
>>> penalty 
positive_part(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, constrained_max(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.positive_part((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \left(\sum_{i=1}^{p} (\beta)_i^+\right) \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\left(\\sum_{i=1}^{%(shape)s} (%(var)s)_i^+\\right)'
objective_vars = {'dualnormklass': 'constrained_max', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'positive_part', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \left(\sum_{i=1}^{p} (\beta)_i^+\right)\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \left(\sum_{i=1}^{p} (\beta)_i^+\right)\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.positive_part((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.positive_part((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.positive_part((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.positive_part((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

seminorm

class regreg.atoms.seminorms.seminorm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.atom

An atom that can be in lagrange or bound form.

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)
static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)
property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1norm((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
supnorm(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=2.3)
>>> penalty 
l1norm(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, supnorm(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)
latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|'
objective_vars = {'dualnormklass': 'supnorm', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'l1norm', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \|\beta\|\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)
set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1norm((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.l1norm((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1norm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.l1norm((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

supnorm

class regreg.atoms.seminorms.supnorm(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

The \(\ell_{\infty}\) norm

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{\beta}^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{1}{2} \|\theta-\beta\|^2_2 \ \text{s.t.} \ \|\beta\|_{\infty} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg, bound=None)

Verify \(\|\beta\|_{\infty} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.supnorm((30,), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.supnorm((30,), lagrange=3.4)
>>> penalty.get_conjugate() 
l1norm(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.supnorm((30,), lagrange=2.3)
>>> penalty 
supnorm(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, l1norm(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.supnorm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{\beta}^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda \|\beta\|_{\infty} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{\\infty}'
objective_vars = {'dualnormklass': 'l1norm', 'initargs': '(30,)', 'linear': 'D', 'normklass': 'supnorm', 'offset': '\\alpha', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[\beta^{\lambda}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \lambda h(\beta-\alpha) + \langle \beta, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(\beta) = \|\beta\|_{\infty}\]

If the atom is in bound mode, then this has the form

\[\beta^{\delta}(\theta) = \text{argmin}_{\beta \in \mathbb{R}^{p}} \frac{L}{2} \|\theta-\beta\|^2_2 + \langle \beta, \eta \rangle \ \text{s.t.} \ h(\beta - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|\beta\|_{\infty}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.supnorm((30,), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.supnorm((30,), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.supnorm((30,), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.supnorm((30,), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

Functions

regreg.atoms.seminorms.get_bound(atom, bound=None)

Return appropriate bound parameter.

regreg.atoms.seminorms.get_lagrange(atom, lagrange=None)

Return appropriate lagrange parameter.

regreg.atoms.seminorms.positive_part_lagrange(shape, lagrange, offset=None, quadratic=None, initial=None)

The positive_part atom in lagrange form can be represented by an l1norm atom with the addition of a linear term and half the lagrange parameter. This reflects the fact that \([0,1]^p = [-1/2,1/2]^p + 1/2 \pmb{1}\).