3. Supervised learning

• 3.1. Generalized Linear Models
  • 3.1.1. Ordinary Least Squares (OLS)
    • 3.1.1.1. OLS Complexity
  • 3.1.2. Ridge Regression
    • 3.1.2.1. Ridge Complexity
    • 3.1.2.2. Generalized Cross-Validation
  • 3.1.3. Lasso
  • 3.1.4. Elastic Net
  • 3.1.5. Least Angle Regression
  • 3.1.6. LARS Lasso
    • 3.1.6.1. Mathematical formulation
  • 3.1.7. Bayesian Regression
    • 3.1.7.1. Bayesian Ridge Regression
  • 3.1.8. Automatic Relevance Determination - ARD
    • 3.1.8.1. Mathematical formulation
  • 3.1.9. Logisitic regression
  • 3.1.10. Stochastic Gradient Descent - SGD
• 3.2. Support Vector Machines
  • 3.2.1. Classification
    • 3.2.1.1. Multi-class classification
    • 3.2.1.2. Unbalanced problems
  • 3.2.2. Regression
  • 3.2.3. Density estimation, outliers detection
  • 3.2.4. Support Vector machines for sparse data
  • 3.2.5. Complexity
  • 3.2.6. Tips on Practical Use
  • 3.2.7. Kernel functions
    • 3.2.7.1. Custom Kernels
      • 3.2.7.1.1. Using python functions as kernels
      • 3.2.7.1.2. Using the Gram matrix
  • 3.2.8. Mathematical formulation
    • 3.2.8.1. SVC
    • 3.2.8.2. NuSVC
  • 3.2.9. Implementation details
• 3.3. Stochastic Gradient Descent
  • 3.3.1. Classification
  • 3.3.2. Regression
  • 3.3.3. Stochastic Gradient Descent for sparse data
  • 3.3.4. Complexity
  • 3.3.5. Tips on Practical Use
  • 3.3.6. Mathematical formulation
    • 3.3.6.1. SGD
  • 3.3.7. Implementation details
• 3.4. Nearest Neighbors
  • 3.4.1. Classification
  • 3.4.2. Regression
  • 3.4.3. Efficient implementation: the ball tree
• 3.5. Feature selection
  • 3.5.1. Univariate feature selection
    • 3.5.1.1. Feature scoring functions
      • 3.5.1.1.1. For classification
      • 3.5.1.1.2. For regression
• 3.6. Gaussian Processes
  • 3.6.1. An introductory regression example
  • 3.6.2. Mathematical formulation
    • 3.6.2.1. The initial assumption
    • 3.6.2.2. The best linear unbiased prediction (BLUP)
    • 3.6.2.3. The empirical best linear unbiased predictor (EBLUP)
  • 3.6.3. Correlation Models
  • 3.6.4. Regression Models
  • 3.6.5. Implementation details
• 3.7. Partial Least Squares
• 3.8. Naive Bayes
  • 3.8.1. Gaussian Naive Bayes