How Spectra Lab calculates the numbers

What real peaks look like

Thirty-six intact mango fruit, measured 684 to 990 nm. Every thin line is one fruit; the heavy line is their mean. The labelled features are found in the data, not drawn in.

The formulas, exactly as the calculators use them

Let r_i be the laboratory (reference) value and p_i the model prediction for sample i, for n samples the model did not learn from. d_i = p_i − r_i.

Two conventions differ between software packages, so state them whenever you compare numbers: SEP here divides by n − 1 (the bias is removed first), and RMSEP divides by n. RMSEP² = SEP² (n − 1)/n + bias², so the two agree only when the bias is small and n is large. The slope is the fit of predicted on reference; some packages fit the other way round, which gives a different number.

Bias is tested with t = bias / (SEP / √n) against Student's t at n − 1 degrees of freedom. The calculators accept 3 or more pairs; a result from a handful of samples is a poor estimate, whatever the arithmetic says.

Eight samples, every number reproducible

Illustrative values chosen to show the arithmetic, not measured data. Paste them into the free check and you get the same results.

These results are computed by the same JavaScript file the page ships (nir_stats.js) when this page is built, not typed in by hand.

Compared with numpy and scipy on five test sets

The browser engine and a reference implementation (numpy and scipy.stats.linregress, with the n − 1 convention) were run on five synthetic sets of 12, 30, 80, 200 and 500 pairs from a fixed random seed. Every statistic was compared: bias, SEP, RMSEP, slope, intercept, R², SD of reference and RPD.

• Largest relative difference over all statistics and all five sets: 8 × 10⁻¹⁴ (floating-point rounding).
• The check can fail: computing SEP with n instead of n − 1 changes SEP by 4.3 % on the 12-pair set, far above that tolerance, so a convention error would be caught.
• The test sets are synthetic and show that the arithmetic is right. They do not show that any particular NIR model is good.

What these numbers cannot tell you

• Use samples the model never saw. Statistics on the calibration set describe how well the model memorised, not how it will perform.
• RPD bands are rules of thumb. Suggested cut-offs differ between fields and authors. Read RPD together with the range of your samples, the laboratory error (SEL) and what accuracy your decision needs.
• A validation holds for the population it covers. Samples from a new season, instrument, supplier or range need their own check.
• The reference method has error too. A SEP close to SEL is near the best any model can show.
• The free check needs only pairs. It cannot see outliers in the spectra, wavelength choices or overfitting inside the model. The Lab report covers those on real data.

Standards these statistics relate to

• ASTM E1655-17(2024), Standard Practices for Infrared Multivariate Quantitative Analysis
  Covers NIR (about 780 to 2500 nm) and mid-infrared calibration practice, definitions and validation criteria.
• ASTM E2617-17(2024), Standard Practice for Validation of Empirically Derived Multivariate Calibrations
  Validation of a calibration against an accepted reference method, with the caution not to extrapolate beyond the validated population.
• ISO 12099:2017, Animal feeding stuffs, cereals and milled cereal products: Guidelines for the application of near infrared spectrometry
  NIR guidelines for moisture, fat, protein, starch, crude fibre and digestibility in feed and cereals.

Listed from the publishers' catalogue pages. The standards are paid documents and we have not reproduced their text; consult them for their exact procedures and limits. Where a standard and this page differ, follow the standard.

Run the check on your own model

Check my calibration See a real validation report