Quality Control in Machine Learning Models

Quality control in machine learning models is essential to ensure the reliability and accuracy of the models. Two prominent methods used for quality control are the Jackknife Quality Control Index Values and the Hodges-Lehmann Null Distribution. These methods help in estimating the bias and variance of statistical estimates, thereby improving the robustness of models like Self-Organizing Maps (SOM) and One-Class Support Vector Machines (SVM).

The implementation is based on research paper: Self-Organizing Map Quality Control Index[1].

Jackknife Quality Control Index Values

The Jackknife method, also known as jackknife cross-validation, is a resampling technique used to estimate the bias and standard error of a statistic. This method involves systematically recomputing the statistic estimate by leaving out one or more observations at a time from the sample set. The basic steps involved in the Jackknife method are:

1. Resampling: For a dataset with n observations, the Jackknife method creates n subsamples, each omitting a different single observation.

2. Estimation: The statistic of interest is calculated for each of these subsamples.

3. Aggregation: The results from these subsamples are aggregated to estimate the bias and variance of the statistic.

The Jackknife method is particularly useful for estimating the variance and bias of complex statistics and is widely used in various machine learning models, including SOM and SVM.

Hodges-Lehmann Null Distribution

The Hodges-Lehmann estimator is a robust statistical method used to estimate the median of a population. It is particularly useful for symmetric distributions and is known for its consistency and median-unbiased properties. The Hodges-Lehmann estimator is defined as the median of all pairwise averages of the sample data.

[1]

Sila Kittiwachana, Diana L. S. Ferreira, Louise A. Fido, Duncan R. Thompson, Richard E. A. Escott, and Richard G. Brereton. Self-organizing map quality control index. Analytical Chemistry, 82(14):5972–5982, 2010. PMID: 20557073. URL: https://doi.org/10.1021/ac100383g, arXiv:https://doi.org/10.1021/ac100383g, doi:10.1021/ac100383g.