Measurement system variation

Since it is a process in itself, the act of measuring is subject to variability like all processes. It is extremely important to understand measurement variation, as many decisions can be made based on the measurement results. Some basic questions that we will try to answer are:

1. What are the basic sources of variation?

2. Is the system statistically stable over time?

3. How close to the “truth” are the measured results? How is this quantified?

4. What are some means of quantifying or characterizing variation in a measurement system?

Types of variability

Variability in measurement, of course, involves special and common causes. Variability (or errors) can be divided into three categories: human error, systematic error, and random error.

Human errors are the most elusive type to try to control. They occur randomly, intermittently, and can be large or small. Incorrect reading of instruments or equipment, transposing numbers, entering incorrect values ​​into a computer or calculator, and measuring the wrong sample are examples. Most are impossible to control and correct, as carelessness is often the main cause.

Systematic errors or assignable errors are always of the same sign, whether positive or negative. They are constant regardless of the number of measurements performed. These are errors due to bias, as defined in the following paragraphs. As such, they can generally be identified. After identification, they can be removed or negated by correction factors. Elimination is always preferred to correction as a control method.

Random errors represent the common cause variability of the measurement system. They are both positive and negative in effect and occur by chance. Some examples are the slight variations that may exist in sample injection techniques for a gas chromatograph or the lower temperature of a drying oven or the sensitivity limitations of a pH electrode.

While they cannot be completely eliminated, they can be reduced. They can be statistically estimated and used to validate measurement results.

Our objective should be to control, monitor and estimate the variability in the measurement results and eliminate the effects of systematic errors.

Measurement terminology

There are some terms that are in widespread use when it comes to measurements. Before continuing, these need to be discussed.

Stability

Stability refers to the total variation in measurement obtained with the same equipment at the same standard over an extended period of time. The statistical stability of a measurement system implies that the test is predictable over time. Without this, any analysis of measurement variability is only applicable to the time period of the study. Statistical stability allows the results to be used to characterize future performance. Unless there is objective evidence of statistical stability of measurement systems, do not use the results of a measurement variability study to predict future test / equipment performance.

The means of demonstrating statistical stability is the control chart. Graphs of standards in average and range or individual and moving range graphs not only represent the stability of the measurements, but also serve as indicators that calibration is required. Calibration while the system is still indicating a control condition will generally only serve to increase variation in measurement systems.

Statistical stability, or statistical control, does not mean that the measurement process has been optimized. Several different organizations may use similar measurement methods with each in statistical control, but their performance may differ markedly.

Accuracy, bias and precision

Precision is the closeness of agreement between a test result and the “true” or accepted reference value. In other words, how close we are to the “truth”. To better define precision, two additional terms are used.

Bias refers to a systematic error that contributes to the difference between the population mean of measurements or test results and an accepted reference or true value.

Precision is the closeness of agreement between randomly selected individual measurements or test results obtained under prescribed conditions. An exact method is one capable of producing accurate and unbiased results. With measurements, we assess inaccuracy; we tried to quantify bias and imprecision.

The accepted reference value is a value that serves as an agreed reference for comparison and is derived as:

A theoretical or established value based on scientific principles,

An assigned value based on experimental work such as NIST or

A consensus value, based on collaborative experimental work (such as the ASTM Interlaboratory Cross-Verification Sample Exchange Program).

ASTM D6299 provides an accepted methodology for statistically determining an accepted reference value.

Standard deviation is a mathematically calculated quantity that measures the precision or “noise” of a process,

Σ, commonly known as ‘sigma’

Estimated from current and historical data using statistical techniques.

A measure of variation

The standard deviation of the measurement error can be used as a measure of precision or actually “imprecision”.

Calibration or recalibration can improve the accuracy of a measurement by reducing error or bias. However, the calibration does not necessarily have any effect on the precision of the measurements.

Measurement system variability

The accuracy, bias and precision of a measurement system can be divided into a part attributable to the equipment or apparatus and that associated with different people or laboratories performing the test. The special terms for these precision components are as follows:

Repeatability

The repeatability of a measurement process implies that the test variation is consistent. It is a measure of the degree of agreement between independent test results obtained in a short time interval with the same test method in the same laboratory by the same operator using the same equipment and the same samples. By keeping so many factors the same, repeatability represents the inherent variability in the test equipment or apparatus.

Reproducibility

Reproducibility is a measure of the degree of agreement between test results obtained in different laboratories with the same test method using the same samples. Includes differences such as operators, equipment, and supervision that will exist between laboratories. As a result, it can never be less than the repeatability of a test. ASTM uses this definition and repeatability to characterize the performance of the test method for any laboratory.

There are differences in terminology because AIAG does not use the ASTM definitions. While its definition of repeatability is essentially the same, the AIAG methodology uses reproducibility to refer to the variability associated with operators. Its ASTM equivalent of reproducibility is called R&R, or the combination of equipment and operator variability.

You must know the terminology that your customers use.

Sources of variability

The systematic and random errors that can influence measurement results can come from a multitude of sources. These can generally be summarized into the following categories:

Team

The equipment, be it a sophisticated automated electronic analyzer or glassware, has been manufactured to certain tolerances. Inherent variation in equipment specifications will be reflected in the test results. Component wear, failure, or improper maintenance will increase the variance in test results. Any inconsistency in the verification and / or recalibration of the calibration will also affect the consistency of the results obtained from the equipment.

People

People almost always contribute to variation simply because none of us are exactly the same. We differ in dexterity, reaction times, color sensitivity, and other ways. Even the same operators can perform differently at different times due to degrees of mental and physical alertness. Some degree of differences between operators are practically unavoidable. Of course, some tests are more sensitive to the effects of differences between operators. Incomplete or non-explicit test methods open the door to another difference in operators, the “interpretation” of the requirements.

Laboratory environment

Some samples and equipment may be susceptible to temperature, humidity, atmospheric pressure, and other environmental factors. Because these cannot be perfectly controlled within or between laboratories, they contribute to some extent to variation in test results.

Samples

Any non-uniformity of the sample can increase the variation in test results. When conducting studies to determine test variability, special effort should be made to obtain test samples that are as uniform or similar as possible.

Weather

All of the sources of variation mentioned above can change themselves over time. In measurement studies, efforts are generally made to keep the time span as short as possible.

Measurement systems analysis

Several different techniques are useful for analyzing the variability of the measurement system. These include measurement variability studies (both short and long), control charts, designed experiments, and analysis of variance. Donald Wheeler’s book, “Evaluating the Measurement Process,” does an excellent job of introducing the control chart approach. Check this out for a detailed discussion of the topic. The AIAG, MSA Handbook, Fourth Edition is the ‘bible’ for the automotive industry. To comply with the IATF 16949: 2016 standard, all MSA studies must comply with the methodology outlined in the MSA manual.

Interlaboratory versus intralaboratory studies

Establishing the repeatability of a method is accomplished almost as well in one laboratory as in another. Differences in results between laboratories are generally not due so much to differences in precision, but to systematic errors or biases.

Interlaboratory (interlaboratory) studies can establish the relative magnitudes of biases and precision. They do not offer much help in discovering the assignable causes of the biases.

To obtain the information necessary to identify assignable causes and eliminate their effects, measurement system studies should be conducted in a single laboratory. (Intralaboratory) These studies may involve independent verification of results from a laboratory.

Independent verification activities:

Blind sample programs

Cross-checks between laboratories

Audits

Review questions

1. What are some sources of measurement variation?

2. Explain the differences between bias and precision.

3. How do you judge whether a measurement system is statistically stable?

4. List a few different techniques for analyzing measurement variability.

5. Explain what repea