scikit-learn estimators

regreg.sklearn.base

Mixin classes for building sklearn estimators from regreg problems.

If sklearn is importable, this module defines mixin classes that pair regression or classification losses with an atom so that, for example, cross-validation error can be evaluated.

class regreg.sklearn.base.base_mixin(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: object

Base mixin pairing a loss with an atom used as a regularizer.

fit(X, y)

Fit a regularized regression estimator.

Parameters:

• X (ndarray) – Feature matrix of shape (n, p).

• y (ndarray) – Response, of shape (n,), or a 2-d array that also holds case weights and/or offset columns (see the Notes of __init__).

Returns:

self

predict(X)

Predict new responses in the regression setting.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

yhat (ndarray) – Linear predictor \(X\hat{\beta}\) of shape (n,).

score(X, y, sample_weight=None)

Score the fitted model on data X, y.

The score is computed according to score_method. Deviance-type scores are negated so that larger is better, as sklearn’s model selection tools maximize scores.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Test samples.

• y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – True values for X, possibly with case weight and offset columns as in fit.

• sample_weight (array-like, shape (n_samples,), optional) – Sample weights, multiplied into the case weights if y includes them, and used as the case weights otherwise.

Returns:

score (float)

class regreg.sklearn.base.lagrange_mixin(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: base_mixin

A regularized estimator where the atom is in Lagrange form.

Allows for an elastic net type ridge term. The atom constructor can use the gradient of the loss at 0 (or at the null solution) to construct the appropriate atom – commonly the Lagrange parameter is taken to be some multiple in [0, 1] of the dual norm of this vector.

fit(X, y)

Fit a regularized regression estimator.

Parameters:

• X (ndarray) – Feature matrix of shape (n, p).

• y (ndarray) – Response, of shape (n,), or a 2-d array that also holds case weights and/or offset columns (see the Notes of __init__).

Returns:

self

class regreg.sklearn.base.classifier_mixin(atom_constructor, atom_params, case_weights=False, coef=1.0, quadratic=None, score_method='accuracy', solve_args={}, initial=None)

Bases: base_mixin

Mixin for classifiers.

Predictions are labels rather than linear predictors.

predict(X)

Predict new labels in the classification setting.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

labels (ndarray) – Boolean labels, True where the linear predictor is positive.

predict_proba(X)

Predict new probabilities in the classification setting (abstract).

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

probs (ndarray) – Predicted probabilities, of shape (n,).

class regreg.sklearn.base.classifier_lagrange_mixin(atom_constructor, atom_params, case_weights=False, enet_alpha=0.0, coef=1.0, score_method='accuracy', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: classifier_mixin, lagrange_mixin

Classifier mixin with the atom in Lagrange form.

class regreg.sklearn.base.survival_mixin(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: base_mixin

Mixin for survival models.

class regreg.sklearn.base.survival_mixin_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: survival_mixin, lagrange_mixin

Survival mixin with the atom in Lagrange form.

class regreg.sklearn.base.sklearn_regression_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: lagrange_mixin, BaseEstimator, RegressorMixin

sklearn regressor with a regularizer in Lagrange form.

class regreg.sklearn.base.sklearn_regression(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: base_mixin, BaseEstimator, RegressorMixin

sklearn regressor with an arbitrary atom as regularizer.

class regreg.sklearn.base.sklearn_classifier(atom_constructor, atom_params, case_weights=False, coef=1.0, quadratic=None, score_method='accuracy', solve_args={}, initial=None)

Bases: classifier_mixin, BaseEstimator, ClassifierMixin

sklearn classifier with an arbitrary atom as regularizer.

class regreg.sklearn.base.sklearn_classifier_lagrange(atom_constructor, atom_params, case_weights=False, enet_alpha=0.0, coef=1.0, score_method='accuracy', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: classifier_lagrange_mixin, BaseEstimator, ClassifierMixin

sklearn classifier with a regularizer in Lagrange form.

class regreg.sklearn.base.sklearn_survival(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: survival_mixin, BaseEstimator, RegressorMixin

sklearn survival estimator with an arbitrary atom as regularizer.

class regreg.sklearn.base.sklearn_survival_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: survival_mixin_lagrange, BaseEstimator, RegressorMixin

sklearn survival estimator with a regularizer in Lagrange form.

regreg.sklearn.gaussian

sklearn estimators for Gaussian (least squares) regression.

class regreg.sklearn.gaussian.sklearn_gaussian(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: sklearn_regression

Regularized Gaussian (least squares) regression for sklearn.

Any atom can be used as a regularizer.

class regreg.sklearn.gaussian.sklearn_gaussian_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_gaussian

Lagrange-form Gaussian regression for sklearn.

regreg.sklearn.logistic

sklearn estimators for logistic regression and classification.

class regreg.sklearn.logistic.sklearn_logistic(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: sklearn_regression

Regularized logistic regression for sklearn.

Any atom can be used as a regularizer.

The response passed to fit is either a vector of binary successes or a 2-d array with columns (successes, trials).

class regreg.sklearn.logistic.sklearn_logistic_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_logistic

Lagrange-form logistic regression for sklearn.

class regreg.sklearn.logistic.sklearn_logistic_classifier(atom_constructor, atom_params, case_weights=False, coef=1.0, quadratic=None, score_method='accuracy', solve_args={}, initial=None)

Bases: sklearn_classifier

Regularized logistic classifier for sklearn.

Any atom can be used as a regularizer.

predict_proba(X)

Predict success probabilities in the classification setting.

The probabilities are the inverse logit applied to the linear predictor.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

probs (ndarray) – Predicted probabilities, of shape (n,).

class regreg.sklearn.logistic.sklearn_logistic_classifier_lagrange(atom_constructor, atom_params, case_weights=False, enet_alpha=0.0, coef=1.0, score_method='accuracy', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_classifier_lagrange, sklearn_logistic_classifier

Lagrange-form logistic classifier for sklearn.

regreg.sklearn.probit

sklearn estimators for probit regression and classification.

class regreg.sklearn.probit.sklearn_probit(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: sklearn_regression

Regularized probit regression for sklearn.

Any atom can be used as a regularizer.

The response passed to fit is either a vector of binary successes or a 2-d array with columns (successes, trials).

class regreg.sklearn.probit.sklearn_probit_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_probit

Lagrange-form probit regression for sklearn.

class regreg.sklearn.probit.sklearn_probit_classifier(atom_constructor, atom_params, case_weights=False, coef=1.0, quadratic=None, score_method='accuracy', solve_args={}, initial=None)

Bases: sklearn_classifier

Regularized probit classifier for sklearn.

Any atom can be used as a regularizer.

predict_proba(X)

Predict success probabilities in the classification setting.

The probabilities are the standard normal CDF applied to the linear predictor.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

probs (ndarray) – Predicted probabilities, of shape (n,).

class regreg.sklearn.probit.sklearn_probit_classifier_lagrange(atom_constructor, atom_params, case_weights=False, enet_alpha=0.0, coef=1.0, score_method='accuracy', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_classifier_lagrange, sklearn_probit_classifier

Lagrange-form probit classifier for sklearn.

regreg.sklearn.cloglog

sklearn estimators for complementary log-log regression models.

class regreg.sklearn.cloglog.sklearn_cloglog(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: sklearn_regression

Regularized complementary log-log regression for sklearn.

Any atom can be used as a regularizer.

The response passed to fit is either a vector of binary successes or a 2-d array with columns (successes, trials).

class regreg.sklearn.cloglog.sklearn_cloglog_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_cloglog

Lagrange-form complementary log-log regression for sklearn.

class regreg.sklearn.cloglog.sklearn_cloglog_classifier(atom_constructor, atom_params, case_weights=False, coef=1.0, quadratic=None, score_method='accuracy', solve_args={}, initial=None)

Bases: sklearn_classifier

Regularized complementary log-log classifier for sklearn.

Any atom can be used as a regularizer.

predict(X)

Predict new labels in the classification setting.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

labels (ndarray) – Boolean labels, True where the fitted probability exceeds 1/2, i.e. where the linear predictor exceeds \(\log(\log 2)\).

predict_proba(X)

Predict success probabilities in the classification setting.

The probabilities are \(1 - \exp(-e^{\eta})\) applied to the linear predictor.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

probs (ndarray) – Predicted probabilities, of shape (n,).

class regreg.sklearn.cloglog.sklearn_cloglog_classifier_lagrange(atom_constructor, atom_params, case_weights=False, enet_alpha=0.0, coef=1.0, score_method='accuracy', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_classifier_lagrange, sklearn_cloglog_classifier

Lagrange-form cloglog classifier for sklearn.

regreg.sklearn.multinomial

sklearn estimators for multinomial regression and classification.

class regreg.sklearn.multinomial.sklearn_multinomial(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: sklearn_regression

Regularized multinomial regression for sklearn.

Any atom can be used as a regularizer.

The response passed to fit is an (n, J) array of counts for each of the J categories.

class regreg.sklearn.multinomial.sklearn_multinomial_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_multinomial

Lagrange-form multinomial regression for sklearn.

class regreg.sklearn.multinomial.sklearn_multinomial_classifier(atom_constructor, atom_params, case_weights=False, coef=1.0, quadratic=None, score_method='accuracy', solve_args={}, initial=None)

Bases: sklearn_classifier

Regularized multinomial classifier for sklearn.

Any atom can be used as a regularizer.

predict(X)

Predict labels in the classification setting.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

labels (ndarray) – Index of the category with the largest linear predictor, of shape (n,).

predict_proba(X)

Predict category probabilities in the classification setting.

Parameters:

X (ndarray) – Feature matrix of shape (n, p).

Returns:

probs (ndarray) – Predicted probabilities (softmax of the linear predictor), of shape (n, J).

class regreg.sklearn.multinomial.sklearn_multinomial_classifier_lagrange(atom_constructor, atom_params, case_weights=False, enet_alpha=0.0, coef=1.0, score_method='accuracy', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_classifier_lagrange, sklearn_multinomial_classifier

Lagrange-form multinomial classifier for sklearn.

regreg.sklearn.cox

sklearn estimators for Cox proportional hazards regression.

class regreg.sklearn.cox.sklearn_cox(atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, quadratic=None, score_method='deviance', solve_args={}, initial=None)

Bases: sklearn_regression

Regularized Cox proportional hazards regression for sklearn.

Any atom can be used as a regularizer.

The response passed to fit has columns (event_times, censoring).

class regreg.sklearn.cox.sklearn_cox_lagrange(atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_cox

Lagrange-form Cox proportional hazards regression for sklearn.

regreg.sklearn.huber

sklearn estimators for regression with the Huber loss.

class regreg.sklearn.huber.sklearn_huber(smoothing_parameter, atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, score_method='deviance', quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression

Regularized Huber loss regression for sklearn.

Any atom can be used as a regularizer.

class regreg.sklearn.huber.sklearn_huber_lagrange(smoothing_parameter, atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_huber

Lagrange-form Huber loss regression for sklearn.

regreg.sklearn.huber_svm

sklearn estimators for binary responses with the Huberized SVM loss.

class regreg.sklearn.huber_svm.sklearn_huber_svm(smoothing_parameter, atom_constructor, atom_params, case_weights=False, offset=False, coef=1.0, score_method='deviance', quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression

Regularized Huberized SVM for sklearn.

Any atom can be used as a regularizer.

class regreg.sklearn.huber_svm.sklearn_huber_svm_lagrange(smoothing_parameter, atom_constructor, atom_params, case_weights=False, offset=False, enet_alpha=0.0, coef=1.0, score_method='deviance', unpenalized=False, quadratic=None, solve_args={}, initial=None)

Bases: sklearn_regression_lagrange, sklearn_huber_svm

Lagrange-form Huberized SVM for sklearn.