8.9.1. sklearn.gaussian_process.GaussianProcess

class sklearn.gaussian_process.GaussianProcess(regr='constant', corr='squared_exponential', beta0=None, storage_mode='full', verbose=False, theta0=0.1, thetaL=None, thetaU=None, optimizer='fmin_cobyla', random_start=1, normalize=True, nugget=2.2204460492503131e-15)

The Gaussian Process model class.

Parameters :

regr : string or callable, optional

A regression function returning an array of outputs of the linear regression functional basis. The number of observations n_samples should be greater than the size p of this basis. Default assumes a simple constant regression trend. Available built-in regression models are:

'constant', 'linear', 'quadratic'

corr : string or callable, optional

A stationary autocorrelation function returning the autocorrelation between two points x and x’. Default assumes a squared-exponential autocorrelation model. Built-in correlation models are:

'absolute_exponential', 'squared_exponential',
'generalized_exponential', 'cubic', 'linear'

beta0 : double array_like, optional

The regression weight vector to perform Ordinary Kriging (OK). Default assumes Universal Kriging (UK) so that the vector beta of regression weights is estimated using the maximum likelihood principle.

storage_mode : string, optional

A string specifying whether the Cholesky decomposition of the correlation matrix should be stored in the class (storage_mode = ‘full’) or not (storage_mode = ‘light’). Default assumes storage_mode = ‘full’, so that the Cholesky decomposition of the correlation matrix is stored. This might be a useful parameter when one is not interested in the MSE and only plan to estimate the BLUP, for which the correlation matrix is not required.

verbose : boolean, optional

A boolean specifying the verbose level. Default is verbose = False.

theta0 : double array_like, optional

An array with shape (n_features, ) or (1, ). The parameters in the autocorrelation model. If thetaL and thetaU are also specified, theta0 is considered as the starting point for the maximum likelihood rstimation of the best set of parameters. Default assumes isotropic autocorrelation model with theta0 = 1e-1.

thetaL : double array_like, optional

An array with shape matching theta0’s. Lower bound on the autocorrelation parameters for maximum likelihood estimation. Default is None, so that it skips maximum likelihood estimation and it uses theta0.

thetaU : double array_like, optional

An array with shape matching theta0’s. Upper bound on the autocorrelation parameters for maximum likelihood estimation. Default is None, so that it skips maximum likelihood estimation and it uses theta0.

normalize : boolean, optional

Input X and observations y are centered and reduced wrt means and standard deviations estimated from the n_samples observations provided. Default is normalize = True so that data is normalized to ease maximum likelihood estimation.

nugget : double or ndarray, optional

Introduce a nugget effect to allow smooth predictions from noisy data. If nugget is an ndarray, it must be the same length as the number of data points used for the fit. The nugget is added to the diagonal of the assumed training covariance; in this way it acts as a Tikhonov regularization in the problem. In the special case of the squared exponential correlation function, the nugget mathematically represents the variance of the input values. Default assumes a nugget close to machine precision for the sake of robustness (nugget = 10. * MACHINE_EPSILON).

optimizer : string, optional

A string specifying the optimization algorithm to be used. Default uses ‘fmin_cobyla’ algorithm from scipy.optimize. Available optimizers are:

'fmin_cobyla', 'Welch'

‘Welch’ optimizer is dued to Welch et al., see reference [WBSWM1992]. It consists in iterating over several one-dimensional optimizations instead of running one single multi-dimensional optimization.

random_start : int, optional

The number of times the Maximum Likelihood Estimation should be performed from a random starting point. The first MLE always uses the specified starting point (theta0), the next starting points are picked at random according to an exponential distribution (log-uniform on [thetaL, thetaU]). Default does not use random starting point (random_start = 1).

Notes

The presentation implementation is based on a translation of the DACE Matlab toolbox, see reference [NLNS2002].

References

[NLNS2002]

(1, 2) H.B. Nielsen, S.N. Lophaven, H. B. Nielsen and J. Sondergaard. DACE - A MATLAB Kriging Toolbox. (2002) http://www2.imm.dtu.dk/~hbn/dace/dace.pdf

[WBSWM1992]

(1, 2) W.J. Welch, R.J. Buck, J. Sacks, H.P. Wynn, T.J. Mitchell, and M.D. Morris (1992). Screening, predicting, and computer experiments. Technometrics, 34(1) 15–25. http://www.jstor.org/pss/1269548

Examples

>>> import numpy as np
>>> from sklearn.gaussian_process import GaussianProcess
>>> X = np.array([[1., 3., 5., 6., 7., 8.]]).T
>>> y = (X * np.sin(X)).ravel()
>>> gp = GaussianProcess(theta0=0.1, thetaL=.001, thetaU=1.)
>>> gp.fit(X, y) 
GaussianProcess(beta0=None, corr=...,
        normalize=..., nugget=...,
        ...

Methods

arg_max_reduced_likelihood_function()	This function estimates the autocorrelation parameters theta as the maximizer of the reduced likelihood function.
fit(X, y)	The Gaussian Process model fitting method.
get_params([deep])	Get parameters for the estimator
predict(X[, eval_MSE, batch_size])	This function evaluates the Gaussian Process model at x.
reduced_likelihood_function([theta])	This function determines the BLUP parameters and evaluates the reduced
score(X, y)	Returns the coefficient of determination R^2 of the prediction.
set_params(**params)	Set the parameters of the estimator.

__init__(regr='constant', corr='squared_exponential', beta0=None, storage_mode='full', verbose=False, theta0=0.1, thetaL=None, thetaU=None, optimizer='fmin_cobyla', random_start=1, normalize=True, nugget=2.2204460492503131e-15)

arg_max_reduced_likelihood_function()

This function estimates the autocorrelation parameters theta as the maximizer of the reduced likelihood function. (Minimization of the opposite reduced likelihood function is used for convenience)

Parameters :

self : All parameters are stored in the Gaussian Process model object.

Returns :

optimal_theta : array_like

The best set of autocorrelation parameters (the sought maximizer of the reduced likelihood function).

optimal_reduced_likelihood_function_value : double

The optimal reduced likelihood function value.

optimal_par : dict

The BLUP parameters associated to thetaOpt.

fit(X, y)

The Gaussian Process model fitting method.

Parameters :

X : double array_like

An array with shape (n_samples, n_features) with the input at which observations were made.

y : double array_like

An array with shape (n_features, ) with the observations of the scalar output to be predicted.

Returns :

gp : self

A fitted Gaussian Process model object awaiting data to perform predictions.

get_params(deep=True)

Get parameters for the estimator

Parameters :

deep: boolean, optional :

If True, will return the parameters for this estimator and contained subobjects that are estimators.

predict(X, eval_MSE=False, batch_size=None)

This function evaluates the Gaussian Process model at x.

Parameters :

X : array_like

An array with shape (n_eval, n_features) giving the point(s) at which the prediction(s) should be made.

eval_MSE : boolean, optional

A boolean specifying whether the Mean Squared Error should be evaluated or not. Default assumes evalMSE = False and evaluates only the BLUP (mean prediction).

batch_size : integer, optional

An integer giving the maximum number of points that can be evaluated simulatneously (depending on the available memory). Default is None so that all given points are evaluated at the same time.

Returns :

y : array_like

An array with shape (n_eval, ) with the Best Linear Unbiased Prediction at x.

MSE : array_like, optional (if eval_MSE == True)

An array with shape (n_eval, ) with the Mean Squared Error at x.

reduced_likelihood_function(theta=None)

This function determines the BLUP parameters and evaluates the reduced likelihood function for the given autocorrelation parameters theta.

Maximizing this function wrt the autocorrelation parameters theta is equivalent to maximizing the likelihood of the assumed joint Gaussian distribution of the observations y evaluated onto the design of experiments X.

Parameters :

theta : array_like, optional

An array containing the autocorrelation parameters at which the Gaussian Process model parameters should be determined. Default uses the built-in autocorrelation parameters (ie theta = self.theta).

Returns :

reduced_likelihood_function_value : double

The value of the reduced likelihood function associated to the given autocorrelation parameters theta.

par : dict

A dictionary containing the requested Gaussian Process model parameters:

sigma2

Gaussian Process variance.

beta

Generalized least-squares regression weights for Universal Kriging or given beta0 for Ordinary Kriging.

gamma

Gaussian Process weights.

C

Cholesky decomposition of the correlation matrix [R].

Ft

Solution of the linear equation system : [R] x Ft = F

G

QR decomposition of the matrix Ft.

score(X, y)

Returns the coefficient of determination R^2 of the prediction.

The coefficient R^2 is defined as (1 - u/v), where u is the regression sum of squares ((y - y_pred) ** 2).sum() and v is the residual sum of squares ((y_true - y_true.mean()) ** 2).sum(). Best possible score is 1.0, lower values are worse.

Parameters :

X : array-like, shape = [n_samples, n_features]

Training set.

y : array-like, shape = [n_samples]

Returns :

z : float

set_params(**params)

Set the parameters of the estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The former have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns :self :