atoms.block_norms

Module: atoms.block_norms

Inheritance diagram for regreg.atoms.block_norms:

digraph inheritance029ba6e895 { rankdir=LR; size="8.0, 12.0"; "atoms.block_norms.block_max" [URL="#regreg.atoms.block_norms.block_max",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "atoms.block_norms.block_sum" -> "atoms.block_norms.block_max" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.block_norms.block_sum" [URL="#regreg.atoms.block_norms.block_sum",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "atoms.seminorms.seminorm" -> "atoms.block_norms.block_sum" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.block_norms.l1_l1" [URL="#regreg.atoms.block_norms.l1_l1",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "atoms.block_norms.l1_l2" -> "atoms.block_norms.l1_l1" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.block_norms.l1_l2" [URL="#regreg.atoms.block_norms.l1_l2",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "atoms.block_norms.block_sum" -> "atoms.block_norms.l1_l2" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.block_norms.linf_l2" [URL="#regreg.atoms.block_norms.linf_l2",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "atoms.block_norms.block_max" -> "atoms.block_norms.linf_l2" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.block_norms.linf_linf" [URL="#regreg.atoms.block_norms.linf_linf",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "atoms.block_norms.linf_l2" -> "atoms.block_norms.linf_linf" [arrowsize=0.5,style="setlinewidth(0.5)"]; "atoms.seminorms.seminorm" [URL="regreg.atoms.seminorms.html#regreg.atoms.seminorms.seminorm",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="An atom that can be in lagrange or bound form."]; "regreg.atoms.atom" -> "atoms.seminorms.seminorm" [arrowsize=0.5,style="setlinewidth(0.5)"]; "problems.composite.composite" [URL="regreg.problems.composite.html#regreg.problems.composite.composite",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A generic way to specify a problem in composite form."]; "problems.composite.nonsmooth" [URL="regreg.problems.composite.html#regreg.problems.composite.nonsmooth",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A composite subclass that explicitly returns 0"]; "problems.composite.composite" -> "problems.composite.nonsmooth" [arrowsize=0.5,style="setlinewidth(0.5)"]; "regreg.atoms.atom" [URL="regreg.atoms.html#regreg.atoms.atom",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A class that defines the API for support functions."]; "problems.composite.nonsmooth" -> "regreg.atoms.atom" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

This module contains the implementation of block norms, i.e. l1/l*, linf/l* norms. These are used in multiresponse LASSOs.

Classes

block_max

class regreg.atoms.block_norms.block_max(atom_cls, shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.block_norms.block_sum

__init__(atom_cls, shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(x, bound=None)

Return unique minimizer

\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\infty,\| \cdot\|} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(x, bound=None)

Verify \(\|B\|_{\infty,\| \cdot\|} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.block_max(rr.l1norm, (5, 4), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=3.4)
>>> penalty.get_conjugate() 
block_sum(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=2.3)
>>> penalty 
block_max(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, block_sum(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(x, lipschitz=1, lagrange=None)

Return unique minimizer

\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\infty,\| \cdot\|} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{\\infty,\\| \\cdot\\|}'
objective_vars = {'dualnormklass': 'block_sum', 'initargs': 'rr.l1norm, (5, 4)', 'linear': 'D', 'normklass': 'block_max', 'offset': '\\alpha', 'shape': 'n \\times p', 'var': 'B'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(B) = \|B\|_{\infty,\| \cdot\|}\]

If the atom is in bound mode, then this has the form

\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(x, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|B\|_{\infty,\| \cdot\|}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

seminorms(x, lagrange=None, check_feasibility=False)

Compute all seminorms in the block norm.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.block_max(rr.l1norm, (5, 4), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.block_max(rr.l1norm, (5, 4), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.block_max(rr.l1norm, (5, 4), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

block_sum

class regreg.atoms.block_norms.block_sum(atom_cls, shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.seminorms.seminorm

__init__(atom_cls, shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(x, bound=None)

Return unique minimizer

\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{1,\|\cdot\|} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(x)

Verify \(\|B\|_{1,\|\cdot\|} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.block_sum(rr.l1norm, (5, 4), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=3.4)
>>> penalty.get_conjugate() 
block_max(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=2.3)
>>> penalty 
block_sum(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, block_max(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(x, lipschitz=1, lagrange=None)

Return unique minimizer

\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{1,\|\cdot\|} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{1,\\|\\cdot\\|}'
objective_vars = {'dualnormklass': 'block_max', 'initargs': 'rr.l1norm, (5, 4)', 'linear': 'D', 'normklass': 'block_sum', 'offset': '\\alpha', 'shape': 'n \\times p', 'var': 'B'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(B) = \|B\|_{1,\|\cdot\|}\]

If the atom is in bound mode, then this has the form

\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(x, check_feasibility=False, lagrange=None)

Return \(\lambda \cdot \|B\|_{1,\|\cdot\|}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

seminorms(x, lagrange=None, check_feasibility=False)

Compute all seminorms in the block norm.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.block_sum(rr.l1norm, (5, 4), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.block_sum(rr.l1norm, (5, 4), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.block_sum(rr.l1norm, (5, 4), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

l1_l1

class regreg.atoms.block_norms.l1_l1(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.block_norms.l1_l2

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{1,1} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg)

Verify \(\|B\|_{1,1} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1_l1((5,4), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.l1_l1((5,4), lagrange=3.4)
>>> penalty.get_conjugate() 
linf_linf(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.l1_l1((5,4), lagrange=2.3)
>>> penalty 
l1_l1(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, linf_linf(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1_l1((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{1,1} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{1,1}'
objective_vars = {'dualnormklass': 'linf_linf', 'initargs': '(5,4)', 'linear': 'D', 'normklass': 'l1_l1', 'offset': '\\alpha', 'shape': 'n \\times p', 'var': 'B'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(B) = \|B\|_{1,1}\]

If the atom is in bound mode, then this has the form

\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|B\|_{1,1}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

seminorms(x, lagrange=None, check_feasibility=False)

Compute all seminorms in the block norm.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1_l1((5,4), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.l1_l1((5,4), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1_l1((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.l1_l1((5,4), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

l1_l2

class regreg.atoms.block_norms.l1_l2(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.block_norms.block_sum

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{1,2} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(x)

Verify \(\|B\|_{1,2} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1_l2((5,4), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.l1_l2((5,4), lagrange=3.4)
>>> penalty.get_conjugate() 
linf_l2(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.l1_l2((5,4), lagrange=2.3)
>>> penalty 
l1_l2(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, linf_l2(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1_l2((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{1,2} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{1,2}'
objective_vars = {'dualnormklass': 'linf_l2', 'initargs': '(5,4)', 'linear': 'D', 'normklass': 'l1_l2', 'offset': '\\alpha', 'shape': 'n \\times p', 'var': 'B'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(B) = \|B\|_{1,2}\]

If the atom is in bound mode, then this has the form

\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(x, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|B\|_{1,2}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

seminorms(x, lagrange=None, check_feasibility=False)

Compute all seminorms in the block norm.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.l1_l2((5,4), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.l1_l2((5,4), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.l1_l2((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.l1_l2((5,4), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

linf_l2

class regreg.atoms.block_norms.linf_l2(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.block_norms.block_max

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\infty,2} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg)

Verify \(\|B\|_{\infty,2} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.linf_l2((5,4), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.linf_l2((5,4), lagrange=3.4)
>>> penalty.get_conjugate() 
l1_l2(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.linf_l2((5,4), lagrange=2.3)
>>> penalty 
linf_l2(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, l1_l2(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.linf_l2((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\infty,2} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{\\infty,2}'
objective_vars = {'dualnormklass': 'l1_l2', 'initargs': '(5,4)', 'linear': 'D', 'normklass': 'linf_l2', 'offset': '\\alpha', 'shape': 'n \\times p', 'var': 'B'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(B) = \|B\|_{\infty,2}\]

If the atom is in bound mode, then this has the form

\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|B\|_{\infty,2}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

seminorms(x, lagrange=None, check_feasibility=False)

Compute all seminorms in the block norm.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.linf_l2((5,4), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.linf_l2((5,4), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.linf_l2((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.linf_l2((5,4), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05

linf_linf

class regreg.atoms.block_norms.linf_linf(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Bases: regreg.atoms.block_norms.linf_l2

__init__(shape, lagrange=None, bound=None, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

classmethod affine(linear_operator, offset, lagrange=None, diag=False, bound=None, quadratic=None)

This is the same as the linear class method but with offset as a positional argument

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property bound
bound_prox(arg, bound=None)

Return unique minimizer

\[{B}^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{1}{2} \|\theta-B\|^2_2 \ \text{s.t.} \ \|B\|_{\infty,\infty} \leq \delta\]

where \(\delta\) is the bound parameter and \(\theta\) is arg.

If the argument bound is None and the atom is in bound mode, self.bound is used as the bound parameter, else an exception is raised.

The class atom’s bound_prox just returns the appropriate bound parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

bound : float (optional)

Bound for the constraint on the seminorm. Defaults to self.bound.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

static check_subgradient(atom, prox_center)

For a given seminorm, verify the KKT condition for the problem for the proximal problem

\[\text{minimize}_u \frac{1}{2} \|u-z\|^2_2 + h(z)\]

where \(z\) is the prox_center and \(h\) is atom which may be in Lagrange or bound form.

If the atom is in Lagrange form, this function should return two values equal to the seminorm of the minimizer. If it is bound form it should return two values equal to the dual seminorm of the residual, i.e. the prox_center minus the minimizer.

Parameters

atom : seminorm

prox_center : np.ndarray(np.float)

Center for the proximal map.

Returns

v1, v2 : float

Two values that should be equal if the proximal map is correct.

property conjugate
constraint(arg)

Verify \(\|B\|_{\infty,\infty} \leq \delta\), where \(\delta\) is bound.

If the result is True, returns 0, else returns np.inf.

The class seminorm’s constraint just returns the appropriate bound parameter for use by the subclasses.

property dual
get_bound()

Get method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.linf_linf((5,4), bound=2.3) 
>>> constraint.bound
2.3
get_conjugate()

Return the conjugate of an given atom.

>>> import regreg.api as rr
>>> penalty = rr.linf_linf((5,4), lagrange=3.4)
>>> penalty.get_conjugate() 
l1_l1(..., bound=3.4...)
get_dual()

Return the dual of an atom. This dual is formed by making introducing new variables \(v=Ax\) where \(A\) is self.linear_transform.

>>> import regreg.api as rr
>>> penalty = rr.linf_linf((5,4), lagrange=2.3)
>>> penalty 
linf_linf(..., lagrange=2.3...)
>>> penalty.dual 
(<regreg.affine.identity object at 0x...>, l1_l1(..., bound=2.3...))

If there is a linear part to the penalty, the linear_transform may not be identity. For example, the 1D fused LASSO penalty:

>>> D = (np.identity(4) + np.diag(-np.ones(3),1))[:-1]
>>> D
array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]])
>>> linear_atom = rr.l1norm.linear(D, lagrange=2.3)
>>> linear_atom 
affine_atom(l1norm((3,), lagrange=2.3...), array([[ 1., -1.,  0.,  0.],
       [ 0.,  1., -1.,  0.],
       [ 0.,  0.,  1., -1.]]))
>>> linear_atom.dual 
(<regreg.affine.linear_transform object at 0x...>, supnorm((3,), bound=2.3...))
get_lagrange()

Get method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.linf_linf((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
get_offset()
get_quadratic()

Get the quadratic part of the composite.

property lagrange
lagrange_prox(arg, lipschitz=1, lagrange=None)

Return unique minimizer

\[{B}^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda \|B\|_{\infty,\infty} \]

Above, \(\lambda\) is the Lagrange parameter and \(L\) is the Lipschitz parameter and \(\theta\) is arg.

If the argument lagrange is None and the atom is in lagrange mode, self.lagrange is used as the lagrange parameter, else an exception is raised.

The class atom’s lagrange_prox just returns the appropriate lagrange parameter for use by the subclasses.

Parameters

arg : np.ndarray(np.float)

Argument of the proximal map.

lipschitz : float

Coefficient in front of the quadratic.

lagrange : float (optional)

Lagrange factor in front of the seminorm. Defaults to self.lagrange.

Returns

Z : np.ndarray(np.float)

The proximal map of arg.

latexify(var=None, idx='')

Return a LaTeX representation of an object.

>>> import regreg.api as rr
>>> penalty = rr.l1norm(10, lagrange=0.9)
>>> penalty.latexify(var=r'\gamma') 
'\\lambda_{} \\|\\gamma\\|_1'
Parameters

var : string

Argument of the functions

idx : string

Optional subscript index.

Returns

L : string

A LaTeX representation of the atom.

classmethod linear(linear_operator, lagrange=None, diag=False, bound=None, quadratic=None, offset=None)
property linear_transform

The linear transform applied before a penalty is computed. Defaults to regreg.affine.identity

>>> from regreg.api import l1norm
>>> penalty = l1norm(30, lagrange=3.4)
>>> type(penalty.linear_transform)
<class 'regreg.affine.identity'>
nonsmooth_objective(arg, check_feasibility=False)

The nonsmooth objective function of the atom. Includes self.quadratic.objective(arg).

>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> penalty.nonsmooth_objective([3, 4, 5, 9])
42.0
>>> 2 * sum([3, 4, 5, 9])
42
Parameters

arg : np.ndarray(np.float)

Argument of the seminorm.

check_feasibility : bool

If True, then return np.inf if appropriate.

Returns

value : np.float

The seminorm of arg.

objective(x, check_feasibility=False)
objective_template = '\\|%(var)s\\|_{\\infty,\\infty}'
objective_vars = {'dualnormklass': 'l1_l1', 'initargs': '(5,4)', 'linear': 'D', 'normklass': 'linf_linf', 'offset': '\\alpha', 'shape': 'n \\times p', 'var': 'B'}
property offset
proximal(quadratic, prox_control=None)

The proximal operator. If the atom is in Lagrange mode, this has the form

\[B^{\lambda}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \lambda h(B-\alpha) + \langle B, \eta \rangle\]

where \(\alpha\) is self.offset, \(\eta\) is quadratic.linear_term, \(\theta\) is quadratic.center and

\[h(B) = \|B\|_{\infty,\infty}\]

If the atom is in bound mode, then this has the form

\[B^{\delta}(\theta) = \text{argmin}_{B \in \mathbb{R}^{n \times p}} \frac{L}{2} \|\theta-B\|^2_2 + \langle B, \eta \rangle \ \text{s.t.} \ h(B - \alpha) \leq \delta\]
>>> import regreg.api as rr
>>> penalty = rr.l1norm(4, lagrange=2)
>>> Q = rr.identity_quadratic(1.5, [3, -4, -1, 1], 0, 0)
>>> penalty.proximal(Q) 
array([ 1.6666..., -2.6666..., -0.        ,  0.        ])
Parameters

quadratic : regreg.identity_quadratic.identity_quadratic

A quadratic added to the atom before minimizing.

prox_control : [None, dict]

This argument is ignored for seminorms, but otherwise is passed to regreg.algorithms.FISTA if the atom needs to be solved iteratively.

Returns

Z : np.ndarray(np.float)

The proximal map of the implied center of quadratic.

proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

seminorm(arg, lagrange=None, check_feasibility=False)

Return \(\lambda \cdot \|B\|_{\infty,\infty}\), where \(\lambda\) is lagrange. If check_feasibility is True, and seminorm is unbounded, will return np.inf if appropriate.

The class seminorm’s seminorm just returns the appropriate lagrange parameter for use by the subclasses.

seminorms(x, lagrange=None, check_feasibility=False)

Compute all seminorms in the block norm.

set_bound(bound)

Set method of the bound property.

>>> import regreg.api as rr
>>> constraint = rr.linf_linf((5,4), bound=3.4) 
>>> constraint.bound
3.4
>>> constraint.bound = 2.3
>>> constraint.bound
2.3
>>> penalty = rr.linf_linf((5,4), lagrange=2.3)
>>> penalty.bound = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in lagrange mode
set_lagrange(lagrange)

Set method of the lagrange property.

>>> import regreg.api as rr
>>> penalty = rr.linf_linf((5,4), lagrange=3.4)
>>> penalty.lagrange
3.4
>>> penalty.lagrange = 2.3
>>> penalty.lagrange
2.3
>>> constraint = rr.linf_linf((5,4), bound=3.4)
>>> constraint.lagrange = 3.4 
Traceback (most recent call last):
...
AttributeError: atom is in bound mode
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, lagrange=None, diag=False, bound=None, quadratic=None)
smooth_objective(x, mode='both', check_feasibility=False)

The zero function.

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)
tol = 1e-05