scikits.learn.linear_model.sparse.SGDRegressor

class scikits.learn.linear_model.sparse.SGDRegressor(loss='squared_loss', penalty='l2', alpha=0.0001, rho=0.84999999999999998, fit_intercept=True, n_iter=5, shuffle=False, verbose=0, p=0.10000000000000001, seed=0, learning_rate='invscaling', eta0=0.01, power_t=0.25)

Linear model fitted by minimizing a regularized empirical loss with SGD

SGD stands for Stochastic Gradient Descent: the gradient of the loss is estimated each sample at a time and the model is updated along the way with a decreasing strength schedule (aka learning rate).

The regularizer is a penalty added to the loss function that shrinks model parameters towards the zero vector using either the squared euclidean norm L2 or the absolute norm L1 or a combination of both (Elastic Net). If the parameter update crosses the 0.0 value because of the regularizer, the update is truncated to 0.0 to allow for learning sparse models and achieve online feature selection.

This implementation works with data represented as dense numpy arrays of floating point values for the features.

Parameters :

loss : str, ‘squared_loss’ or ‘huber’

The loss function to be used. Defaults to ‘squared_loss’ which refers to the ordinary least squares fit. ‘huber’ is an epsilon insensitive loss function for robust regression.

penalty : str, ‘l2’ or ‘l1’ or ‘elasticnet’

The penalty (aka regularization term) to be used. Defaults to ‘l2’ which is the standard regularizer for linear SVM models. ‘l1’ and ‘elasticnet’ migh bring sparsity to the model (feature selection) not achievable with ‘l2’.

alpha : float

Constant that multiplies the regularization term. Defaults to 0.0001

rho : float

The Elastic Net mixing parameter, with 0 < rho <= 1. Defaults to 0.85.

fit_intercept: bool :

Whether the intercept should be estimated or not. If False, the data is assumed to be already centered. Defaults to True.

n_iter: int :

The number of passes over the training data (aka epochs). Defaults to 5.

shuffle: bool :

Whether or not the training data should be shuffled after each epoch. Defaults to False.

seed: int, optional :

The seed of the pseudo random number generator to use when shuffling the data.

verbose: integer, optional :

The verbosity level

p : float

Epsilon in the epsilon insensitive huber loss function; only if loss==’huber’.

learning_rate : string, optional

The learning rate: constant: eta = eta0 optimal: eta = 1.0/(t+t0) invscaling: eta = eta0 / pow(t, power_t) [default]

eta0 : double, optional

The initial learning rate [default 0.01].

power_t : double, optional

The exponent for inverse scaling learning rate [default 0.25].

See also

RidgeRegression, ElasticNet, Lasso, SVR

Examples

>>> import numpy as np
>>> from scikits.learn import linear_model
>>> n_samples, n_features = 10, 5
>>> np.random.seed(0)
>>> y = np.random.randn(n_samples)
>>> X = np.random.randn(n_samples, n_features)
>>> clf = linear_model.sparse.SGDRegressor()
>>> clf.fit(X, y)
SGDRegressor(loss='squared_loss', power_t=0.25, shuffle=False, verbose=0,
       n_iter=5, learning_rate='invscaling', fit_intercept=True,
       penalty='l2', p=0.1, seed=0, eta0=0.01, rho=1.0, alpha=0.0001)

Attributes

coef_	array, shape = [n_features]	Weights asigned to the features.
intercept_	array, shape = [1]	The intercept term.

Methods

__init__(loss='squared_loss', penalty='l2', alpha=0.0001, rho=0.84999999999999998, fit_intercept=True, n_iter=5, shuffle=False, verbose=0, p=0.10000000000000001, seed=0, learning_rate='invscaling', eta0=0.01, power_t=0.25)

fit(X, y, coef_init=None, intercept_init=None, sample_weight=None, **params)

Fit linear model with Stochastic Gradient Descent.

Parameters :

X : numpy array of shape [n_samples,n_features]

Training data

y : numpy array of shape [n_samples]

Target values

coef_init : array, shape = [n_features]

The initial coeffients to warm-start the optimization.

intercept_init : array, shape = [1]

The initial intercept to warm-start the optimization.

sample_weight : array-like, shape = [n_samples], optional

Weights applied to individual samples (1. for unweighted).

Returns :

self : returns an instance of self.

predict(X)

Predict using the linear model

Parameters :

X : array or scipy.sparse matrix of shape [n_samples, n_features]

Whether the numpy.array or scipy.sparse matrix is accepted dependes on the actual implementation

Returns :

array, shape = [n_samples] :

Array containing the predicted class labels.

score(X, y)

Returns the coefficient of determination of the prediction

Parameters :

X : array-like, shape = [n_samples, n_features]

Training set.

y : array-like, shape = [n_samples]

Returns :

z : float