User guide: contents

• 1. Installing scikit-learn
  • 1.1. Installing an official release
    • 1.1.1. Installing from source
      • 1.1.1.1. Easy install
      • 1.1.1.2. From source package
    • 1.1.2. Windows installer
    • 1.1.3. Building on windows
  • 1.2. Third party distributions of scikit-learn
    • 1.2.1. Debian and derivatives (Ubuntu)
    • 1.2.2. Python(x, y)
    • 1.2.3. Enthought python distribution
    • 1.2.4. Macports
    • 1.2.5. NetBSD
  • 1.3. Bleeding Edge
  • 1.4. Testing
• 2. Getting started: an introduction to machine learning with scikit-learn
  • 2.1. Machine learning: the problem setting
  • 2.2. Loading an example dataset
  • 2.3. Learning and Predicting
  • 2.4. Model persistence
• 3. Supervised learning
  • 3.1. Generalized Linear Models
    • 3.1.1. Ordinary Least Squares
      • 3.1.1.1. Ordinary Least Squares Complexity
    • 3.1.2. Ridge Regression
      • 3.1.2.1. Ridge Complexity
      • 3.1.2.2. Setting alpha: generalized Cross-Validation
    • 3.1.3. Lasso
      • 3.1.3.1. Setting alpha
        • 3.1.3.1.1. Using cross-validation
        • 3.1.3.1.2. Information-criteria based model selection
    • 3.1.4. Elastic Net
    • 3.1.5. Least Angle Regression
    • 3.1.6. LARS Lasso
      • 3.1.6.1. Mathematical formulation
    • 3.1.7. Orthogonal Matching Pursuit (OMP)
    • 3.1.8. Bayesian Regression
      • 3.1.8.1. Bayesian Ridge Regression
    • 3.1.9. Automatic Relevance Determination - ARD
      • 3.1.9.1. Mathematical formulation
    • 3.1.10. Logisitic regression
    • 3.1.11. Stochastic Gradient Descent - SGD
  • 3.2. Support Vector Machines
    • 3.2.1. Classification
      • 3.2.1.1. Multi-class classification
      • 3.2.1.2. Unbalanced problems
    • 3.2.2. Regression
    • 3.2.3. Density estimation, outliers detection
    • 3.2.4. Support Vector machines for sparse data
    • 3.2.5. Complexity
    • 3.2.6. Tips on Practical Use
    • 3.2.7. Kernel functions
      • 3.2.7.1. Custom Kernels
        • 3.2.7.1.1. Using python functions as kernels
        • 3.2.7.1.2. Using the Gram matrix
    • 3.2.8. Mathematical formulation
      • 3.2.8.1. SVC
      • 3.2.8.2. NuSVC
    • 3.2.9. Implementation details
  • 3.3. Stochastic Gradient Descent
    • 3.3.1. Classification
    • 3.3.2. Regression
    • 3.3.3. Stochastic Gradient Descent for sparse data
    • 3.3.4. Complexity
    • 3.3.5. Tips on Practical Use
    • 3.3.6. Mathematical formulation
      • 3.3.6.1. SGD
    • 3.3.7. Implementation details
  • 3.4. Nearest Neighbors
    • 3.4.1. Unsupervised Nearest Neighbors
    • 3.4.2. Nearest Neighbors Classification
    • 3.4.3. Nearest Neighbors Regression
    • 3.4.4. Nearest Neighbor Algorithms
      • 3.4.4.1. Brute Force
      • 3.4.4.2. K-D Tree
      • 3.4.4.3. Ball Tree
      • 3.4.4.4. Choice of Nearest Neighbors Algorithm
      • 3.4.4.5. Effect of leaf_size
  • 3.5. Gaussian Processes
    • 3.5.1. An introductory regression example
    • 3.5.2. Mathematical formulation
      • 3.5.2.1. The initial assumption
      • 3.5.2.2. The best linear unbiased prediction (BLUP)
      • 3.5.2.3. The empirical best linear unbiased predictor (EBLUP)
    • 3.5.3. Correlation Models
    • 3.5.4. Regression Models
    • 3.5.5. Implementation details
  • 3.6. Partial Least Squares
  • 3.7. Naive Bayes
    • 3.7.1. Gaussian Naive Bayes
    • 3.7.2. Multinomial Naive Bayes
    • 3.7.3. Bernoulli Naive Bayes
  • 3.8. Multiclass algorithms
    • 3.8.1. One-Vs-The-Rest
    • 3.8.2. One-Vs-One
    • 3.8.3. Error-Correcting Output-Codes
  • 3.9. Feature selection
    • 3.9.1. Univariate feature selection
      • 3.9.1.1. Feature scoring functions
        • 3.9.1.1.1. For classification
        • 3.9.1.1.2. For regression
    • 3.9.2. Recursive feature elimination
    • 3.9.3. L1-based feature selection
• 4. Unsupervised learning
  • 4.1. Gaussian mixture models
    • 4.1.1. Expectation-maximization
    • 4.1.2. Variational inference
    • 4.1.3. The Dirichlet Process
      • 4.1.3.1. GMM classifier
      • 4.1.3.2. Variational Gaussian mixtures: VBGMM classifier
      • 4.1.3.3. Infinite Gaussian mixtures: DPGMM classifier
  • 4.2. Manifold learning
    • 4.2.1. Introduction
    • 4.2.2. Isomap
      • 4.2.2.1. Complexity
    • 4.2.3. Locally Linear Embedding
      • 4.2.3.1. Complexity
    • 4.2.4. Modified Locally Linear Embedding
      • 4.2.4.1. Complexity
    • 4.2.5. Hessian Eigenmapping
      • 4.2.5.1. Complexity
    • 4.2.6. Local Tangent Space Alignment
      • 4.2.6.1. Complexity
    • 4.2.7. Tips on practical use
  • 4.3. Clustering
    • 4.3.1. K-means
      • 4.3.1.1. Mini Batch K-Means
    • 4.3.2. Affinity propagation
    • 4.3.3. Mean Shift
    • 4.3.4. Spectral clustering
    • 4.3.5. Hierarchical clustering
      • 4.3.5.1. Adding connectivity constraints
    • 4.3.6. DBSCAN
    • 4.3.7. Clustering performance evaluation
      • 4.3.7.1. Inertia
        • 4.3.7.1.1. Presentation and usage
        • 4.3.7.1.2. Advantages
        • 4.3.7.1.3. Drawbacks
      • 4.3.7.2. Ajusted Rand index
        • 4.3.7.2.1. Presentation and usage
        • 4.3.7.2.2. Advantages
        • 4.3.7.2.3. Drawbacks
        • 4.3.7.2.4. Mathematical formulation
      • 4.3.7.3. Homogeneity, completeness and V-measure
        • 4.3.7.3.1. Presentation and usage
        • 4.3.7.3.2. Advantages
        • 4.3.7.3.3. Drawbacks
        • 4.3.7.3.4. Mathematical formulation
  • 4.4. Decomposing signals in components (matrix factorization problems)
    • 4.4.1. Principal component analysis (PCA)
      • 4.4.1.1. Exact PCA and probabilistic interpretation
      • 4.4.1.2. Approximate PCA
      • 4.4.1.3. Kernel PCA
      • 4.4.1.4. Sparse Principal Components Analysis (SparsePCA and MiniBatchSparsePCA)
    • 4.4.2. Dictionary Learning
      • 4.4.2.1. Generic dictionary learning
      • 4.4.2.2. Mini-batch dictionary learning
    • 4.4.3. Independent component analysis (ICA)
    • 4.4.4. Non-negative matrix factorization (NMF or NNMF)
  • 4.5. Covariance estimation
    • 4.5.1. Empirical covariance
    • 4.5.2. Shrunk Covariance
      • 4.5.2.1. Basic shrinkage
      • 4.5.2.2. Ledoit-Wolf shrinkage
      • 4.5.2.3. Oracle Approximating Shrinkage
  • 4.6. Hidden Markov Models
• 5. Model Selection
  • 5.1. Cross-Validation
    • 5.1.1. Examples
    • 5.1.2. Cross validation iterators
      • 5.1.2.1. Leave-One-Out - LOO
      • 5.1.2.2. Leave-P-Out - LPO
      • 5.1.2.3. K-fold
      • 5.1.2.4. Stratified K-Fold
      • 5.1.2.5. Leave-One-Label-Out - LOLO
      • 5.1.2.6. Leave-P-Label-Out
      • 5.1.2.7. Random permutations cross-validation a.k.a. Shuffle & Split
      • 5.1.2.8. Bootstrapping cross-validation
    • 5.1.3. Computing cross-validated metrics
  • 5.2. Grid Search
    • 5.2.1. Examples
    • 5.2.2. Alternatives to brute force grid search
      • 5.2.2.1. Model specific cross-validation
        • 9.2.3. sklearn.linear_model.RidgeCV
        • 5.2.2.1.2. sklearn.linear_model.RidgeClassifierCV
        • 9.2.10. sklearn.linear_model.LarsCV
        • 9.2.11. sklearn.linear_model.LassoLarsCV
        • 9.2.5. sklearn.linear_model.LassoCV
        • 9.2.7. sklearn.linear_model.ElasticNetCV
      • 5.2.2.2. Information Criterion
        • 9.2.12. sklearn.linear_model.LassoLarsIC
• 6. Dataset loading utilities
  • 6.1. Toy datasets
  • 6.2. Sample images
  • 6.3. Sample generators
  • 6.4. Datasets in svmlight / libsvm format
  • 6.5. The Olivetti faces dataset
  • 6.6. The 20 newsgroups text dataset
    • 6.6.1. Usage
    • 6.6.2. Examples
  • 6.7. Downloading datasets from the mldata.org repository
  • 6.8. The Labeled Faces in the Wild face recognition dataset
    • 6.8.1. Usage
    • 6.8.2. Examples
• 7. Preprocessing data
  • 7.1. Standardization or Mean Removal and Variance Scaling
  • 7.2. Normalization
  • 7.3. Binarization
    • 7.3.1. Feature binarization
• 8. Feature extraction
  • 8.1. Text feature extraction
  • 8.2. Image feature extraction
    • 8.2.1. Patch extraction
• 9. Class Reference
  • 9.1. Support Vector Machines
    • 9.1.1. sklearn.svm.SVC
    • 9.1.2. sklearn.svm.LinearSVC
    • 9.1.3. sklearn.svm.NuSVC
    • 9.1.4. sklearn.svm.SVR
    • 9.1.5. sklearn.svm.NuSVR
    • 9.1.6. sklearn.svm.OneClassSVM
    • 9.1.7. sklearn.svm.l1_min_c
    • 9.1.8. For sparse data
      • 9.1.8.1. sklearn.svm.sparse.SVC
      • 9.1.8.2. sklearn.svm.sparse.NuSVC
      • 9.1.8.3. sklearn.svm.sparse.SVR
      • 9.1.8.4. sklearn.svm.sparse.NuSVR
      • 9.1.8.5. sklearn.svm.sparse.OneClassSVM
      • 9.1.8.6. sklearn.svm.sparse.LinearSVC
    • 9.1.9. Low-level methods
      • 9.1.9.1. sklearn.svm.libsvm.fit
      • 9.1.9.2. sklearn.svm.libsvm.decision_function
      • 9.1.9.3. sklearn.svm.libsvm.predict
      • 9.1.9.4. sklearn.svm.libsvm.predict_proba
      • 9.1.9.5. sklearn.svm.libsvm.cross_validation
  • 9.2. Generalized Linear Models
    • 9.2.1. sklearn.linear_model.LinearRegression
    • 9.2.2. sklearn.linear_model.Ridge
    • 9.2.3. sklearn.linear_model.RidgeCV
    • 9.2.4. sklearn.linear_model.Lasso
    • 9.2.5. sklearn.linear_model.LassoCV
    • 9.2.6. sklearn.linear_model.ElasticNet
    • 9.2.7. sklearn.linear_model.ElasticNetCV
    • 9.2.8. sklearn.linear_model.Lars
    • 9.2.9. sklearn.linear_model.LassoLars
    • 9.2.10. sklearn.linear_model.LarsCV
    • 9.2.11. sklearn.linear_model.LassoLarsCV
    • 9.2.12. sklearn.linear_model.LassoLarsIC
    • 9.2.13. sklearn.linear_model.LogisticRegression
    • 9.2.14. sklearn.linear_model.OrthogonalMatchingPursuit
    • 9.2.15. sklearn.linear_model.SGDClassifier
    • 9.2.16. sklearn.linear_model.SGDRegressor
    • 9.2.17. sklearn.linear_model.BayesianRidge
    • 9.2.18. sklearn.linear_model.ARDRegression
    • 9.2.19. sklearn.linear_model.lasso_path
    • 9.2.20. sklearn.linear_model.lars_path
    • 9.2.21. sklearn.linear_model.orthogonal_mp
    • 9.2.22. sklearn.linear_model.orthogonal_mp_gram
    • 9.2.23. For sparse data
      • 9.2.23.1. sklearn.linear_model.sparse.Lasso
      • 9.2.23.2. sklearn.linear_model.sparse.ElasticNet
      • 9.2.23.3. sklearn.linear_model.sparse.SGDClassifier
      • 9.2.23.4. sklearn.linear_model.sparse.SGDRegressor
      • 9.2.23.5. sklearn.linear_model.sparse.LogisticRegression
  • 9.3. Naive Bayes
    • 9.3.1. Naive Bayes models
    • 9.3.2. sklearn.naive_bayes.GaussianNB
    • 9.3.3. sklearn.naive_bayes.MultinomialNB
    • 9.3.4. sklearn.naive_bayes.BernoulliNB
  • 9.4. Nearest Neighbors
    • 9.4.1. sklearn.neighbors.NearestNeighbors
    • 9.4.2. sklearn.neighbors.KNeighborsClassifier
    • 9.4.3. sklearn.neighbors.RadiusNeighborsClassifier
    • 9.4.4. sklearn.neighbors.NeighborsClassifier
    • 9.4.5. sklearn.neighbors.KNeighborsRegressor
    • 9.4.6. sklearn.neighbors.RadiusNeighborsRegressor
    • 9.4.7. sklearn.neighbors.NeighborsRegressor
    • 9.4.8. sklearn.neighbors.BallTree
    • 9.4.9. sklearn.neighbors.kneighbors_graph
    • 9.4.10. sklearn.neighbors.radius_neighbors_graph
  • 9.5. Gaussian Mixture Models
    • 9.5.1. sklearn.mixture.GMM
    • 9.5.2. sklearn.mixture.DPGMM
    • 9.5.3. sklearn.mixture.VBGMM
  • 9.6. Hidden Markov Models
    • 9.6.1. sklearn.hmm.GaussianHMM
    • 9.6.2. sklearn.hmm.MultinomialHMM
    • 9.6.3. sklearn.hmm.GMMHMM
  • 9.7. Clustering
    • 9.7.1. sklearn.cluster.KMeans
    • 9.7.2. sklearn.cluster.MiniBatchKMeans
    • 9.7.3. sklearn.cluster.MeanShift
    • 9.7.4. sklearn.cluster.SpectralClustering
    • 9.7.5. sklearn.cluster.AffinityPropagation
    • 9.7.6. sklearn.cluster.Ward
  • 9.8. Metrics
    • 9.8.1. Classification metrics
      • 9.8.1.1. sklearn.metrics.confusion_matrix
      • 9.8.1.2. sklearn.metrics.roc_curve
      • 9.8.1.3. sklearn.metrics.auc
      • 9.8.1.4. sklearn.metrics.precision_score
      • 9.8.1.5. sklearn.metrics.recall_score
      • 9.8.1.6. sklearn.metrics.fbeta_score
      • 9.8.1.7. sklearn.metrics.f1_score
      • 9.8.1.8. sklearn.metrics.precision_recall_fscore_support
      • 9.8.1.9. sklearn.metrics.classification_report
      • 9.8.1.10. sklearn.metrics.precision_recall_curve
      • 9.8.1.11. sklearn.metrics.zero_one_score
      • 9.8.1.12. sklearn.metrics.zero_one
      • 9.8.1.13. sklearn.metrics.hinge_loss
    • 9.8.2. Regression metrics
      • 9.8.2.1. sklearn.metrics.r2_score
      • 9.8.2.2. sklearn.metrics.mean_square_error
    • 9.8.3. Clustering metrics
      • 9.8.3.1. sklearn.metrics.adjusted_rand_score
      • 9.8.3.2. sklearn.metrics.homogeneity_completeness_v_measure
      • 9.8.3.3. sklearn.metrics.homogeneity_score
      • 9.8.3.4. sklearn.metrics.completeness_score
      • 9.8.3.5. sklearn.metrics.v_measure_score
    • 9.8.4. Pairwise metrics
      • 9.8.4.1. sklearn.metrics.pairwise.euclidean_distances
      • 9.8.4.2. sklearn.metrics.pairwise.linear_kernel
      • 9.8.4.3. sklearn.metrics.pairwise.polynomial_kernel
      • 9.8.4.4. sklearn.metrics.pairwise.rbf_kernel
      • 9.8.4.5. sklearn.metrics.pairwise.distance_metrics
      • 9.8.4.6. sklearn.metrics.pairwise.pairwise_distances
      • 9.8.4.7. sklearn.metrics.pairwise.kernel_metrics
      • 9.8.4.8. sklearn.metrics.pairwise.pairwise_kernels
  • 9.9. Covariance Estimators
    • 9.9.1. Covariance estimators
    • 9.9.2. sklearn.covariance.EmpiricalCovariance
    • 9.9.3. sklearn.covariance.ShrunkCovariance
    • 9.9.4. sklearn.covariance.LedoitWolf
    • 9.9.5. sklearn.covariance.OAS
    • 9.9.6. sklearn.covariance.empirical_covariance
    • 9.9.7. sklearn.covariance.ledoit_wolf
    • 9.9.8. sklearn.covariance.shrunk_covariance
    • 9.9.9. sklearn.covariance.oas
  • 9.10. Signal Decomposition
    • 9.10.1. sklearn.decomposition.PCA
    • 9.10.2. sklearn.decomposition.ProbabilisticPCA
    • 9.10.3. sklearn.decomposition.ProjectedGradientNMF
    • 9.10.4. sklearn.decomposition.RandomizedPCA
    • 9.10.5. sklearn.decomposition.KernelPCA
    • 9.10.6. sklearn.decomposition.FastICA
    • 9.10.7. sklearn.decomposition.NMF
    • 9.10.8. sklearn.decomposition.SparsePCA
    • 9.10.9. sklearn.decomposition.MiniBatchSparsePCA
    • 9.10.10. sklearn.decomposition.DictionaryLearning
    • 9.10.11. sklearn.decomposition.MiniBatchDictionaryLearning
    • 9.10.12. sklearn.decomposition.fastica
    • 9.10.13. sklearn.decomposition.dict_learning
    • 9.10.14. sklearn.decomposition.dict_learning_online
    • 9.10.15. sklearn.decomposition.sparse_encode
    • 9.10.16. sklearn.decomposition.sparse_encode_parallel
  • 9.11. Linear Discriminant Analysis
    • 9.11.1. sklearn.lda.LDA
  • 9.12. Partial Least Squares
    • 9.12.1. sklearn.pls.PLSRegression
    • 9.12.2. sklearn.pls.PLSCanonical
    • 9.12.3. sklearn.pls.CCA
    • 9.12.4. sklearn.pls.PLSSVD
  • 9.13. Cross Validation
    • 9.13.1. sklearn.cross_validation.LeaveOneOut
    • 9.13.2. sklearn.cross_validation.LeavePOut
    • 9.13.3. sklearn.cross_validation.KFold
    • 9.13.4. sklearn.cross_validation.StratifiedKFold
    • 9.13.5. sklearn.cross_validation.LeaveOneLabelOut
    • 9.13.6. sklearn.cross_validation.LeavePLabelOut
    • 9.13.7. sklearn.cross_validation.Bootstrap
    • 9.13.8. sklearn.cross_validation.ShuffleSplit
  • 9.14. Grid Search
    • 9.14.1. sklearn.grid_search.GridSearchCV
  • 9.15. Feature Selection
    • 9.15.1. sklearn.feature_selection.rfe.RFE
    • 9.15.2. sklearn.feature_selection.rfe.RFECV
    • 9.15.3. sklearn.feature_selection.univariate_selection.chi2
  • 9.16. Feature Extraction
    • 9.16.1. From images
      • 9.16.1.1. sklearn.feature_extraction.image.img_to_graph
      • 9.16.1.2. sklearn.feature_extraction.image.grid_to_graph
      • 9.16.1.3. sklearn.feature_extraction.image.extract_patches_2d
      • 9.16.1.4. sklearn.feature_extraction.image.reconstruct_from_patches_2d
      • 9.16.1.5. sklearn.feature_extraction.image.PatchExtractor
    • 9.16.2. From text
      • 9.16.2.1. sklearn.feature_extraction.text.RomanPreprocessor
      • 9.16.2.2. sklearn.feature_extraction.text.WordNGramAnalyzer
      • 9.16.2.3. sklearn.feature_extraction.text.CharNGramAnalyzer
      • 9.16.2.4. sklearn.feature_extraction.text.CountVectorizer
      • 9.16.2.5. sklearn.feature_extraction.text.TfidfTransformer
      • 9.16.2.6. sklearn.feature_extraction.text.Vectorizer
  • 9.17. Preprocessing and normalization
    • 9.17.1. sklearn.preprocessing.Scaler
    • 9.17.2. sklearn.preprocessing.Normalizer
    • 9.17.3. sklearn.preprocessing.Binarizer
    • 9.17.4. sklearn.preprocessing.LabelBinarizer
    • 9.17.5. sklearn.preprocessing.KernelCenterer
    • 9.17.6. sklearn.preprocessing.scale
    • 9.17.7. sklearn.preprocessing.normalize
    • 9.17.8. sklearn.preprocessing.binarize
  • 9.18. Manifold learning
    • 9.18.1. sklearn.manifold.LocallyLinearEmbedding
    • 9.18.2. sklearn.manifold.Isomap
    • 9.18.3. sklearn.manifold.locally_linear_embedding
  • 9.19. Datasets
    • 9.19.1. Loaders
      • 9.19.1.1. sklearn.datasets.load_files
      • 9.19.1.2. sklearn.datasets.load_diabetes
      • 9.19.1.3. sklearn.datasets.load_digits
      • 9.19.1.4. sklearn.datasets.load_iris
      • 9.19.1.5. sklearn.datasets.load_linnerud
      • 9.19.1.6. sklearn.datasets.load_lfw_pairs
      • 9.19.1.7. sklearn.datasets.fetch_lfw_pairs
      • 9.19.1.8. sklearn.datasets.load_lfw_people
      • 9.19.1.9. sklearn.datasets.fetch_lfw_people
      • 9.19.1.10. sklearn.datasets.load_20newsgroups
      • 9.19.1.11. sklearn.datasets.fetch_20newsgroups
    • 9.19.2. Samples generator
      • 9.19.2.1. sklearn.datasets.make_classification
      • 9.19.2.2. sklearn.datasets.make_regression
      • 9.19.2.3. sklearn.datasets.make_blobs
      • 9.19.2.4. sklearn.datasets.make_friedman1
      • 9.19.2.5. sklearn.datasets.make_friedman2
      • 9.19.2.6. sklearn.datasets.make_friedman3
      • 9.19.2.7. sklearn.datasets.make_low_rank_matrix
      • 9.19.2.8. sklearn.datasets.make_sparse_coded_signal
      • 9.19.2.9. sklearn.datasets.make_sparse_uncorrelated
      • 9.19.2.10. sklearn.datasets.make_spd_matrix
      • 9.19.2.11. sklearn.datasets.make_swiss_roll
      • 9.19.2.12. sklearn.datasets.make_s_curve
  • 9.20. Pipeline
    • 9.20.1. sklearn.pipeline.Pipeline
  • 9.21. Utilities
    • 9.21.1. sklearn.utils.check_random_state
    • 9.21.2. sklearn.utils.resample
    • 9.21.3. sklearn.utils.shuffle