3. Supervised learning

• 3.1. Generalized Linear Models
  • 3.1.1. Ordinary Least Squares
    • 3.1.1.1. Ordinary Least Squares Complexity
  • 3.1.2. Ridge Regression
    • 3.1.2.1. Ridge Complexity
    • 3.1.2.2. Setting alpha: generalized Cross-Validation
  • 3.1.3. Lasso
    • 3.1.3.1. Setting alpha
      • 3.1.3.1.1. Using cross-validation
      • 3.1.3.1.2. Information-criteria based model selection
  • 3.1.4. Elastic Net
  • 3.1.5. Least Angle Regression
  • 3.1.6. LARS Lasso
    • 3.1.6.1. Mathematical formulation
  • 3.1.7. Orthogonal Matching Pursuit (OMP)
  • 3.1.8. Bayesian Regression
    • 3.1.8.1. Bayesian Ridge Regression
  • 3.1.9. Automatic Relevance Determination - ARD
    • 3.1.9.1. Mathematical formulation
  • 3.1.10. Logisitic regression
  • 3.1.11. Stochastic Gradient Descent - SGD
• 3.2. Support Vector Machines
  • 3.2.1. Classification
    • 3.2.1.1. Multi-class classification
    • 3.2.1.2. Unbalanced problems
  • 3.2.2. Regression
  • 3.2.3. Density estimation, outliers detection
  • 3.2.4. Support Vector machines for sparse data
  • 3.2.5. Complexity
  • 3.2.6. Tips on Practical Use
  • 3.2.7. Kernel functions
    • 3.2.7.1. Custom Kernels
      • 3.2.7.1.1. Using python functions as kernels
      • 3.2.7.1.2. Using the Gram matrix
  • 3.2.8. Mathematical formulation
    • 3.2.8.1. SVC
    • 3.2.8.2. NuSVC
  • 3.2.9. Implementation details
• 3.3. Stochastic Gradient Descent
  • 3.3.1. Classification
  • 3.3.2. Regression
  • 3.3.3. Stochastic Gradient Descent for sparse data
  • 3.3.4. Complexity
  • 3.3.5. Tips on Practical Use
  • 3.3.6. Mathematical formulation
    • 3.3.6.1. SGD
  • 3.3.7. Implementation details
• 3.4. Nearest Neighbors
  • 3.4.1. Unsupervised Nearest Neighbors
  • 3.4.2. Nearest Neighbors Classification
  • 3.4.3. Nearest Neighbors Regression
  • 3.4.4. Nearest Neighbor Algorithms
    • 3.4.4.1. Brute Force
    • 3.4.4.2. K-D Tree
    • 3.4.4.3. Ball Tree
    • 3.4.4.4. Choice of Nearest Neighbors Algorithm
    • 3.4.4.5. Effect of leaf_size
• 3.5. Gaussian Processes
  • 3.5.1. An introductory regression example
  • 3.5.2. Mathematical formulation
    • 3.5.2.1. The initial assumption
    • 3.5.2.2. The best linear unbiased prediction (BLUP)
    • 3.5.2.3. The empirical best linear unbiased predictor (EBLUP)
  • 3.5.3. Correlation Models
  • 3.5.4. Regression Models
  • 3.5.5. Implementation details
• 3.6. Partial Least Squares
• 3.7. Naive Bayes
  • 3.7.1. Gaussian Naive Bayes
  • 3.7.2. Multinomial Naive Bayes
  • 3.7.3. Bernoulli Naive Bayes
• 3.8. Multiclass algorithms
  • 3.8.1. One-Vs-The-Rest
  • 3.8.2. One-Vs-One
  • 3.8.3. Error-Correcting Output-Codes
• 3.9. Feature selection
  • 3.9.1. Univariate feature selection
    • 3.9.1.1. Feature scoring functions
      • 3.9.1.1.1. For classification
      • 3.9.1.1.2. For regression
  • 3.9.2. Recursive feature elimination
  • 3.9.3. L1-based feature selection