Open access peer-reviewed chapter

The Data Evaluation of Interlaboratory Comparisons for Calibration Laboratories

Written By

Oleh Velychko and Tetyana Gordiyenko

Submitted: 24 June 2021 Reviewed: 16 July 2021 Published: 21 August 2021

DOI: 10.5772/intechopen.99489

From the Edited Volume

Applied Aspects of Modern Metrology

Edited by Oleh Velychko

Chapter metrics overview

365 Chapter Downloads

View Full Metrics

Abstract

National accreditation agencies in different countries have set quite strict requirements for accreditation of testing and calibration laboratories. Interlaboratory comparisons (ILCs) are a form of experimental verification of laboratory activities to determine technical competence in a particular activity. Successful results of conducting ILCs for the laboratory are a confirmation of competence in carrying out certain types of measurements by a specific specialist on specific equipment. To obtain reliable results of ILC accredited laboratories, it is necessary to improve the methods of processing these results. These methods are based on various data processing algorithms. Therefore, it is necessary to choose the most optimal method of processing the obtained data, which would allow to obtain reliable results. In addition, it is necessary to take into account the peculiarities of the calibration laboratories (CLs) when evaluating the results of ILС. Such features are related to the need to provide calibration of measuring instruments for testing laboratories. The evaluation results for ILCs for CLs are presented. The results for all participants of ILCs were evaluated using the En and z indexes. The obtained results showed that for the such ILCs it is also necessary to evaluate the data using the z index also.

Keywords

  • interlaboratory comparison
  • data evaluation
  • referent laboratory
  • calibration laboratory
  • calibration
  • measurement uncertainty

1. Introduction

Participants in the International Laboratory Accreditation Cooperation (ILAC) Mutual Recognition Agreement (MRA) recognize the calibration or test results obtained by each other’s accredited calibration and testing laboratories [1, 2, 3, 4]. ILAC Policy and Procedural publications are for the operation of the ILAC MRA. ILAC has a special policy for participation in proficiency testing activities, on metrological traceability of measurement results, for measurement uncertainty in calibration [5, 6, 7]. The policy for measurement uncertainty to base on the Guide to Uncertainty in Measurement (GUM) [8, 9, 10, 11] and retains the common understanding of the term calibration and measurement capabilities (CMCs) from the joint declaration issued by the International Bureau of Weights and Measures (BIPM) and ILAC [12]. ILAC has a special guideline for measurement uncertainty in testing [13]. This document provides guidance for the evaluation and reporting of measurement uncertainty in testing accordance with the requirements of the International Standard ISO/IEC 17025 [14].

National accreditation agencies in different countries have set quite strict requirements for accreditation of testing and calibration laboratories. Laboratory accreditation criteria in most accreditation systems include three main groups: laboratory technical equipment, personnel competence, and the effectiveness of the quality system. Interlaboratory comparisons (ILCs) are a form of experimental verification of laboratory activities to determine technical competence in a particular activity. Successful results of conducting ILCs for the laboratory are a confirmation of competence in carrying out certain types of measurements by a specific specialist on specific equipment.

To obtain reliable results of ILC accredited laboratories, it is necessary to improve the methods of processing these results. These methods are based on various data processing algorithms as required by international and regional guidelines and standards. To conduct ILC for CLs, it is necessary to take into account the relevant requirements of the international standards ISO/IEC 17025 [14] and ISO/IEC 17043 [15]. Therefore, it is necessary to choose the most optimal method of processing the obtained data, which would have a minimum number of restrictions on the application and allow to obtain reliable results. In addition, it is necessary to take into account the peculiarities of the calibration laboratories (CLs) when evaluating the results of ILС. Such features are related to the need to provide calibration of measuring instruments for testing laboratories.

ILCs for CLs are held nationally in different countries. Such ILCs are carried out to establish the competence of the CLs in calibrating various measuring instruments and working standards for various measured quantities [16, 17, 18, 19, 20, 21, 22, 23, 24]. For their implementation, various calibration objects are used. To evaluate the ILC data, various methods of their data processing are used [25, 26, 27, 28, 29, 30], and to estimate the measurement uncertainty, the regional guidance EA-04/02 М [31] is additionally used, in addition to the ILAC documents [8, 13]. However, in addition to the method of data evaluation, it is necessary to take into account other influencing factors on the CL result of ILC. In particular, unsatisfactory ILC results for all participating CLs may be associated with a large time drift of the calibrated measuring instrument.

The growing practical need of ILCs for CLs to ensure recognition of the obtained results at both national and international levels underscores the relevance of this research.

Advertisement

2. The national interlaboratory comparisons for calibration laboratories

The main purpose of accredited CLs is to calibrate working standards and measuring instruments for accredited testing laboratories. Significantly more testing laboratories are accredited by national accreditation bodies than CLs. For example, at the middle of 2021, 837 testing and 35 calibration laboratories were accredited in Ukraine. This represents only 4% of accredited CLs of the total number of all accredited laboratories. Therefore, the number of ILCs for testing laboratories is objectively much larger than for CLs.

The State Enterprise “Ukrmetrteststandard” (Ukraine) as a referent laboratory (RL) organized and carry out seven ILCs for accredited CLs from 2016 to 2019 [32, 33, 34, 35, etc]. The list of these ILCs is shown in Table 1. The calibration objects for these ILCs were working standards and measuring instruments for electrical quantities, and time and frequency. When carrying out comparisons, CLs calibrated objects in accordance with the requirements of the international standard ISO/IEC 17025 [14]. The total number of calibration object parameters ranged from 3 to 12. The total number of CLs with RL that took part in these comparisons ranged from 5 to 10.

ILCCalibration objectNumber of parametersNumber of participantsPeriod of carrying out
ILC1Precision measuring thermocoupleAC voltage at 5 frequencies5 labs2016–2018
ILC2Measures of electrical resistance (1th round)3 nominations of resistance8 labs2016
ILC3Measures of electrical resistance (2th round)3 nominations of resistance5 labs2018–2019
ILC4Precision measure of electric power6 power factors at 2 frequencies8 labs2016–2018
ILC5Low frequency signal generatorAC voltage at one frequency, total harmonic factor at 4 frequencies, 5 frequencies4 labs2016
ILC6Electronic stopwatch3 time intervals9 labs2016
ILC7High-frequency signal generator3 frequencies10 labs2018

Table 1.

The list of national ILCs for CLs.

In all presented ILCs, the assigned value (AV) with its uncertainty was taken as the value with its uncertainty of the RL. This was done because the RL had the best measurement capabilities among all CLs that took part in the comparisons. For many years RL has taken part in international comparisons of national measurement standards of electrical quantities within the framework of Regional Metrological Organizations (COOMET, EURAMET, and GULFMET) and had positive results. RL also had published CMCs for some electrical quantities in the BIPM Key Comparison Database [36].

A program for all ILCs was implemented in accordance with the requirements of ISO/IEC 17043 [15]. CLs that participated in the ILCs performed calibration of the measuring instruments (calibration object) provided to the RL in accordance with their own methods according to the radial scheme [4]. RL sent the calibration object to the participating laboratory, and this laboratory returned this object back to RL. In this case, the RL constantly monitored the stability of the calibration object [35, 37]. The RL determined the characteristics of the instability of the calibration object before and after its research in the CLs participating in the ILC.

In accordance with the adopted ILC programs, RL analyzed the calibration data provided by the CLs [38], in particular, analyzed the declared measurement uncertainty. The data obtained from CLs were necessarily checked by RL for their consistency. Indicators for assessing of consistency were En and z indexes set and defined in [4, 15]. The general algorithm of processing of the received primary data of ILCs given in [34] was used. In case of inconsistent data, RL reported this appropriate CL and analyzed the responses received from this laboratory. RL prepared a report on the comparisons, evaluating the data of all CLs. In the event that the laboratory or laboratories received inconsistent results of comparisons, RL suggested that they take the necessary corrective action.

Advertisement

3. The traditional data evaluation of interlaboratory comparisons

The traditional assessment of ILC data for CLs is carried out in accordance with the requirements of ISO/IEC 17043 [15]. During of the evaluation of primary data from the participating CLs, the interlaboratory deviation of the measurement results or degree of equivalences (DoE) was calculated based on the ILCs results.

The DoE for j-th CLs participant of ILC is calculated using Equation [4, 34, 38].

Dlabj=xlabjXAV,E1

where xlabj is measured value for i-th CL; XAV is AV for ILC.

Expanded uncertainty of the result of each participant Uxlabj and expanded uncertainty of AV UXAV were used to check the consistency of the primary ILC data and to calculate En index (En number) using equation

En=Dlabj/U2xlabj+U2XAV.E2

|En| ≤ 1.0 indicates satisfactory performance and |En| > 1.0 indicates unsatisfactory performance [4, 15].

On Figures 13 show the traditional graphical interpretation of the results of three ILCs at one of the calibration points (ILC 2–1, Figure 1, ILC 4–2, Figure 2 and ILC 6–1, Figure 3 respectively). The evaluation of primary data of all ILCs is carried out by means of the specially developed software “Interlaboratory comparisons” (Ukraine) which implements the algorithm presented in [34]. To prepare reports on ILCs, RL used specified software that allowed calculating the En and z indexes and constructing a graphical display of the results. The figures show the DoE with expanded uncertainty for all participating CLs in ILCs. The green dashed line shows the measurement uncertainty limits of the AV of ILC.

Figure 1.

DoE of CLs for ILC 2–1.

Figure 2.

DoE of CLs for ILC 4–2.

Figure 3.

DoE of CLs for ILC 6–1.

Only two laboratories (lab 4 and lab 6 for ILC 2–1) have an unsatisfactory result for two ILCs using the En index. En index more characterizes the reliability of measurement results of laboratories participating in the ILC, but is not always sufficient to determine the accuracy of measurement results.

Advertisement

4. The additional data evaluation of interlaboratory comparisons

The consistency evaluation of data using En and z indicators is important not only to confirm the technical competence of laboratories participating in the ILC. This will also help to increase the accuracy of calibration by the laboratory participating in the ILC with a corresponding reduction in measurement uncertainty.

The z index compares the measurement results of all laboratories with each other and gives better information about the accuracy of measurements in laboratory. The measurement accuracy is an important characteristic for CL, therefore this index is more suitable for evaluating ILC data for CLs.

z index (z score) is calculated by the equation

z=Dlabj/σ,E3

where σ is the standard deviation for qualification assessment (ILC).

|z| ≤ 2.0 indicates a satisfactory performance characteristic and does not require adjustment or response measures, 2.0 < |z| < 3.0 indicates a dubious performance characteristic and requires precautionary measures, and |z| ≥ 3.0 indicates an unsatisfactory performance characteristic and requires adjustment or response measures.

In Tables 28 shows the calculated results of En and z indexes at all points of the calibration for all ILCs. En and z indexes are zero for RL. Cells with unsatisfactory results are highlighted in grey in the tables. An unsatisfactory result is the excess for En index of the value 1, and for the z index of 2 (does not require adjustment or response measures) or 3 (requires adjustment or response measures) [4, 15].

ILC dataIndexLab 1Lab 2Lab 3Lab 4
ILC1–1En−0.8540.4440.438−0.312
z−0.5840.1140.224−2.462
ILC1–2En−0.4510.8181.522−0.064
z−1.1460.2660.605−2.090
ILC1–3En−0.6450.8820.9870.022
z−1.1670.4830.7811.895
ILC1–4En−0.1470.2380.1290.452
z−0.0250.0190.0182.503
ILC1–5En0.7530.382
z2.2651.939

Table 2.

Results of ILCs for calibration of precision measuring thermocouple.

ILC dataIndexLab 1Lab 2Lab 3Lab 4Lab 5Lab 6Lab 7
ILC2–1En0.249−0.245−0.008−4.631−0.1002.352−0.021
z2.677−0.027−0.996−0.208−0.0050.188−0.169
ILC2–2En−1.9556.066−0.018−0.8140.256−0.646−0.032
z−2.4040.880−1.460−0.1970.056−0.253−1.460
ILC2–3En−0.8994.179−0.0860.171−0.4700.975−0.058
z−2.2511.318−0.9690.051−0.1270.367−0.969

Table 3.

Results of ILCs for calibration of measures of electrical resistance (1-th round).

ILC dataIndexLab 1Lab 2Lab 3Lab 4
ILC3–1En0.3010.1330.3670.322
z0.1172.6580.1520.961
ILC3–2En0.1940.0650.0420.051
z0.0122.5790.0040.457
ILC3–3En0.3010.1330.3670.322
z0.1172.6580.1520.961

Table 4.

Results of ILCs for calibration of measures of electrical resistance (2-th round).

ILC dataIndexLab 1Lab 2Lab 3Lab 4Lab 5Lab 6
ILC4–1En0.0000.0040.001−0.0130.0000.000
z0.0020.2150.029−2.8040.0180.018
ILC4–2En0.0000.0020.0020.0340.0000.000
z0.0010.0460.0462.8750.0040.006
ILC4–3En0.0000.0020.0010.0000.0000.000
z0.1572.6830.679−0.192−0.1050.157
ILC4–4En0.0000.0020.0010.0000.0000.000
z0.2072.6571.055−0.170−0.0750.207
ILC4–5En0.0000.002−0.0010.1190.0000.000
z0.0010.013−0.0092.8590.0000.001
ILC4–6En0.0000.0020.0020.1370.0000.000
z0.0000.0120.0122.8620.0000.001
ILC4–7En0.0000.0030.0030.0000.0000.000
z−0.0432.2601.919−0.0430.000−0.043
ILC4–8En0.0000.0010.0010.0000.0000.000
z0.1762.4392.1870.1760.5530.176
ILC4–9En0.0010.0020.0000.0000.0000.001
z0.5322.7870.622−0.009−0.0990.532
ILC4–10En0.0000.0020.0000.000−0.0010.000
z0.1082.5700.569−0.277−0.3540.108
ILC4–11En0.0010.0020.0020.0000.0000.001
z0.3312.4622.0680.0950.2530.331
ILC4–12En0.0010.0020.0010.0000.0000.001
z0.4982.9891.3290.4060.4060.498

Table 5.

Results of ILCs for calibration of precision measure of electric power.

ILC dataIndexLab 1Lab 2Lab 3
ILC5–1En−0.235−0.014−0.305
z−2.479−0.077−1.044
ILC5–2En−0.050−0.0560.036
z−1.095−1.4611.095
ILC5–3En0.0400.0000.035
z1.2060.0002.412
ILC5–4En0.0740.0330.068
z1.4141.4142.828
ILC5–5En0.7370.3970.139
z2.3992.1810.727
ILC5–6En−0.087−0.055−0.016
z−2.557−1.627−0.465
ILC5–7En0.2890.2780.122
z2.4482.2921.011
ILC5–8En−0.0860.016−3.258
z−0.0630.012−2.326
ILC5–9En−0.284−0.0610.029
z−2.315−0.4890.233
ILC5–10En−0.947−1.0240.692
z−1.335−1.4420.975

Table 6.

Results of ILCs for calibration of low frequency signal generator.

ILC dataIndexLab 1Lab 2Lab 3Lab 4Lab 5Lab 6Lab 7Lab 8Lab 9
ILC6–1En0.713−0.2120.9920.3790.8940.3280.897−0.2900.982
z1.811−0.0750.5030.3020.5030.3653.245−0.1760.415
ILC6–2En0.998−0.9430.7330.2180.6760.1610.192−0.3790.914
z3.160−0.3340.4860.7290.7900.5011.686−0.2731.413
ILC6–3En0.711−0.5870.4590.1570.2960.0650.804−0.2730.754
z2.008−0.1930.4680.9080.4680.2893.012−0.1651.582

Table 7.

Results of ILCs for calibration of electronic stopwatch.

ILC dataIndexLab 1Lab 2Lab 3Lab 4Lab 5Lab 6Lab 7Lab 8Lab 9Lab 10
ILC7–1En0.0500.662−0.072−0.0250.643−0.164−0.384−0.039−0.012−0.124
z1.2432.666−0.687−0.6340.857−0.124−0.588−0.631−0.118−0.433
ILC7–2En0.0670.912−0.076−0.0340.775−0.253−0.475−0.0680.005−0.052
z1.2612.533−0.695−0.6290.761−0.182−0.725−1.0400.034−0.145
ILC7–3En0.0621.134−0.072−0.0530.164−0.160−0.648−0.042−0.014−0.104
z1.0452.847−0.746−0.8340.164−0.131−0.574−0.427−0.09−0.276

Table 8.

Results of ILCs for calibration of high-frequency signal generator.

Оn Figure 4 shows the graphical interpretation of the results of estimation of En (a) and z (b) indexes for ILC 2–1, on Figure 5 – for ILC 4–2, and in Figure 6 – for ILC 6–1, respectively.

Figure 4.

Values of En and z indexes for ILC 2–1: a is En index, b is z index.

Figure 5.

Values of En and z indexes for ILC 4–2: a is En index, b is z index.

Figure 6.

Values of En and z indexes for ILC 6–1: a is En index, b is z index.

Advertisement

5. The summarized results of interlaboratory comparisons

The summarized results of estimation of En and z indexes for all ILCs are shown in Table 9 and Figure 7. The percentage of discrepancies two assessments for ILCs 1, 3, 4 and 6 estimates are 100, for ILCs 2, 5 and 7 estimates are from 95 to 97. This suggests that the conclusions that can be drawn about the technical competence of the laboratories participating in these ILCs are completely inconsistent.

ILCNumber of participants*Number of parametersEn > 1 for labz > 3 for labz > 2 for labPercentage of discrepancies in evaluation
ILC14 labs20 points1 point (5%)0 point (0%)4 points (20%)100%
ILC27 labs21 points5 points (24%)0 point (0%)3 points (14%)95%
ILC34 labs12 points0 point (0%)0 point (0%)3 points (25%)100%
ILC46 labs72 points0 point (0%)0 point (0%)14 points (19%)100%
ILC53 labs30 points2 points (7%)0 point (0%)10 points (33%)97%
ILC69 labs27 points0 point (0%)3 points (11%)1 point (4%)100%
ILC710 labs30 points1 point (3%)0 point (0%)3 points (10%)97%
Total:212 points8 points (4%)3 points (1,4%)38 points (18%)

Table 9.

The summarized results of estimation of En and z indexes for all ILCs.

Without RL.


Figure 7.

The summarized results of estimation of En and z indexes for all ILCs: a is absolute value, b is percentage value (%).

Only one result of ILC1 according to En index have inconsistency (lab 3). At the same time, 4 results of ILC1 according to z indexes have inconsistencies (lab 2 and lab 4).

5 results of ILC2 according to En indexes have inconsistencies (for labs 1, 2, 4 and 6). At the same time, 3 results of ILC2 according to z indexes have inconsistencies (only for lab 1).

ILC3, ILC4, and ILC6 according to En index have no inconsistencies. At the same time, 3 results of ILC3 according to z indexes have inconsistencies (only for lab 2), 14 results of ILC4 according to z indexes have inconsistencies (for labs 2, 3 and 4), and 4 results of the ILC6 according to z indexes have inconsistencies (for labs 2, 3, and 4), including 3 from 4 are very large (z > 3.0).

Only one ILC7 result according to En index have inconsistency (for lab 2). At the same time, 3 results of the ILC7 according to z indexes have inconsistencies (for lab 2 also).

The results of the data consistency analysis show that all ILCs, taking into account both indexes, have measurement points with unsatisfactory results. Analysis of the data taking into account the En index shows that only three ILCs (ILC3, ILC4, and ILC6) have satisfactory results. At the same time, analysis of the data taking into account the z index (z > 2) shows that all ILCs have measurement points with unsatisfactory results. ILС6 has measurement points with significantly unsatisfactory results, taking into account the z index (z > 3).

If we return to the analysis of Figures 13, it can be seen that lab 4 for ILCs 2–1 and lab 1 and lab 7 for ILC6–1 have very large declared measurement uncertainties with large DoEs. This led to unsatisfactory results, taking into account the z index. The main reason for the unsatisfactory result of lab 3 for ILCs 1–2, taking into account En index, is, on the contrary, a very small declared measurement uncertainty.

The general recommendation for lab 3 and lab 4 for ILC1–2, as well as for lab 4 for ILC4–2, and lab 1 and lab 7 for ILC6–1 is to revise the estimate of the measurement uncertainty, taking into account guides [8, 31]. This measurement uncertainty can be influenced by both the calibration results of the laboratory working standards and the level of competence of the laboratory personnel. Taking these recommendations into account can improve the results of that laboratories participation in other rounds of ILCs or new ILCs.

Advertisement

6. The influence of travelling standards instability

The travelling standards instability can affect the results of ILCs for CLs. Some works are devoted to assessing its influence, in particular compensation for its instability. The repeatability of a good measuring instrument is below 10% of its maximum error as shown in [39]. The travelling standard with 0,2% shows variations of random errors below x˜±0.02% where x˜ is the average of the readings during calibration. This a small Type A uncertainty in relation to other components is show.

Typically, RL already takes into account the travelling standards instability in the ILC assigned value XAV and its expanded uncertainty [4].

UXAV=2u2xref+u2xinst,E4

where uxref is the standard measurement uncertainty obtained by calibrating of travelling standard with a RL; uxinst is the standard measurement uncertainty from the travelling standard instability of during ILC period

uxinst=ΔXmax/3,E5

ΔXmax is the maximum change in nominal value of travelling standard during ILC period.

The absence of a significant effect of the travelling standards instability on the evaluation of the CL result in the ILC can be at its maximum instability, which is determined by the expression [39].

xinstU2xlabj+U2XAV.E6

The value of the travelling standards instability can be obtained for several cases: measurements of the RL of the travelling standard in the process of carry out of ILC; from the technical specification for the travelling standard, measurements of the RL of the travelling standard for a long time, and etc. Results of calculating the En index for various options for accounting for travelling standards instability are shows in [39].

In any case, from expression (4) it follows that with an increase in the value of the measurement uncertainty associated with instability, the value of the En index only decreases. In this case, it can be stated that the use of a more unstable travelling standard can improve the consistency of the ILC data, which is not acceptable for CL. To carry out ILСs for CLs, it is more preferable to use working standards as calibration objects. Typically, a working standard has less instability than a measuring instrument. The use of a measuring instrument as a travelling standard can lead to somewhat distorted results of such ILCs.

An analysis RL of the travelling standard instability for all calibration points of the ILC7 is given in [33]. The drift of travelling standard for ILC7 at all frequencies is presented on Figure 8. The uncertainty of travelling standards instability for ILC7 is presented in Table 10. The contribution of the uncertainty from the long-term drift of the travelling standard to the standard uncertainty of AV for the entire duration of ILC7 is from 5.3 to 8.3% for all calibration points. Such a drift of the measuring instrument used as a calibration object is acceptable for the ILC. It does not distort the ILC results for the participating CLs.

Figure 8.

The drift of travelling standard for ILC7: a is frequency 130 MHz, b is frequency 168 MHz, c is frequency 223 MHz.

ILC7 pointFrequency (MHz)uXAV (Hz)uxinst (Hz)Drift contribution to uncertainty AV (%)
ILC7–11300.120.018.3
ILC7–21680.170.015.9
ILC7–32230.190.015.3

Table 10.

The uncertainty of travelling standards instability for ILC7.

The list of travelling standard for all ILCs and values of En and z indexes are shown in Table 11.

ILCCalibration objectWorking standardMeasuring instrumentEn indexz index
Unsatisfactory (%)
ILC1Precision measuring thermocoupleYesNo520
ILC2Measures of electrical resistance (1th round)YesNo2414
ILC3Measures of electrical resistance (2th round)YesNo025
ILC4Precision measure of electric powerYesNo019
ILC5Low frequency signal generatorNoYes733
ILC6Electronic stopwatchNoYes045
ILC7High-frequency signal generatorNoYes313

Table 11.

The list of travelling standard for ILCs and values of En and z indexes.

The use of a measuring instrument as a calibration object leads to a slight increase in the values of the z index and practically does not affect the En index, as can be seen from Table 11.

Advertisement

7. The improvement of the evaluation of interlaboratory comparison results

Statistical methods for use in proficiency testing by ILCs are presented in [26, 27]. The aim of creating alternative statistics in order to improve the analysis and evaluation of ILC measurement results is research work [40]. The improvement of statistical indicators is proposed by addressing two specific issues: robustness and reliability. The proposed methodology is not traditional for ILC, but it can be used as an additional methodology for checking the results of ILC.

The following conditions are provided for data evaluation of international comparison of national standards: the travelling measurement standard is stable, the measurement results presented by laboratories are reciprocally independent, and the Gaussian distribution is assigned to a measurand in each laboratory [41, 42, 43, 44]. The same conditions can be extended for data evaluation of ILCs for CLs. Frequently the measurement procedures for supplementary comparisons of national standards [44] are the calibration procedures of these laboratories. Such calibration procedures can also be extended to ILCs for CLs. In such a case, the calibration capabilities of the laboratory can be confirmed.

The application of z index for evaluation of CL results recommended instead of En index since this number is not applicable due to the difficulty in determining the AV [16]. Of course, for accredited test laboratories, it is preferable to use the services of a CL with the best calibration capabilities. Better calibration capabilities of laboratories are characterized by lower calibration uncertainties of working standards and measuring instruments. CLs with a satisfactory value of the En index in the ILC, but having large calibration uncertainties become uncompetitive. If we return to the analysis of Figures 13, it can be seen that lab 1, 3 and 7 for ILCs 1–2, lab 4 for ILC4–2, and lab 1 and lab 7 for ILC6–1 have very large declared measurement uncertainties.

The declared measurement uncertainties of CL for ILC are judged as confirmed if the following equation is satisfied [4, 43].

Dlabj<2uDlabj.E7

In case the declared uncertainties CL don’t confirmed during the ILC and for their confirmation it is necessary to participate in other similar ILCs.

Often, a national metrological institute or an accredited CL, which is an RL in ILC, performs high-precision calibration of working standards and measuring instruments for CLs participating in this ILC. In this case, a correlation of the obtained CL results is formed, which must be taken into account when evaluating the data of such an ILC. Covariance’s are estimated by careful analysis of the uncertainty budget of CLs by the RL

covxlabjXAV=u02,E8

where u02 is common input to the uncertainty budgets of both results [43].

In this case, the value of the En index is calculated by the formula:

En=Dlabj/U2xlabj+U2XAV2covxlabjXAV.E9

If the value of the En index meets the specified requirement (≤ 1.0), then the minimum standard measurement uncertainty, that can be claimed as calibration capability of CL participating in ILC, is:

uCCxlabj=uILCxlabj.E10

If the value of the En index not meets the specified requirement (> 1.0), then the minimum standard measurement uncertainty, that can be claimed as calibration capability of CL participating in ILC, is:

uCCxlabj=Dlabj24+u2XAV,E11

where uXAV is the standard measurement uncertainty of AV.

Correspondingly, the extended uncertainty is U0.95xlabj=2uILCxlabj .

The same requirements can be extended for compliance (≤ 2.0) or inconsistency (> 2.0) of the value of the z index with the established requirements. In this case, the minimum standard measurement uncertainty, that can be claimed as calibration capability of CL participating in ILC, will be determined by formulas (10) and (11), respectively.

If the standard uncertainty uXAV of the AV is too large in comparison with the standard deviation σ for ILC, then there is a risk that some laboratories will receive action and warning signals because of inaccuracy in the determination of the AV, not because of any cause within the laboratories. If

uXAV0.3σE12

then the uncertainty of the AV is negligible and need not be included in the interpretation of the ILC results. Further, all CLs participating in ILC shall carry out the same number of replicate measurements. This approach assumes that CLs have generally similar repeatability [26].

To evaluate the ILC data, can use z′ index also [26] which calculated by the equation

z=Dlabj/σ2+u2XAVE13

This equation may be used when the AV is not calculated using the results reported by CLs participating in ILC. z′ index shall be interpreted in the same way as z index and using the same critical values of 2.0 and 3.0.

Comparison of the equations for z and z′ indexes shows that z′ index for ILC will all be smaller than the corresponding z index by a constant factor of σ/σ2+u2XAV .

When the inequality established by expression (12) is satisfied, then this factor will fall in the range: 0.96σ/σ2+u2XAV1.00. In this case, z′ index will be nearly identical to z index, and it may be concluded that the uncertainty of the AV is negligible. When the inequality established by expression (12) is not satisfied, the difference in magnitude of the z′ and z indexes may be such that some z index exceeds the critical values of 2.0 or 3.0.

Advertisement

8. Conclusions

To perform an ILC for CLs, RL must provide a stable working standard or measuring instrument as a calibration object and monitor its drift throughout the ILC. The use of a measuring instrument as a calibration object leads to a slight increase in the values of the z index and practically does not affect the En index. The application of z index for evaluation of CL results recommended instead of En index.

The analysis of the results of the ILC for CLs for consistency should include not only the analysis of the values of the En index, but also the z index. If we restrict ourselves to only the En index, then it is possible to get unreliable results of the ILC and not identify problems in the CL-participants of the ILC. In 3 from 7 ILCs examined, the En index showed completely satisfactory results, while the z index in all of these 3 ILCs revealed problematic results from the participating laboratories.

The stable travelling standard, the independent measurement results of laboratories with Gaussian distribution are main conditions for data evaluation of ILC for CLs. To participate in the ILC when declaring its measurement uncertainty, CLs must conduct a thorough analysis of the components of this uncertainty. It is necessary to take into account the correlation of the laboratory data of the participants of the ILC when evaluating its results. Covariance is estimated by carefully analyzing the CL uncertainty budget using RL.

The minimum standard measurement uncertainty that can be claimed as the calibration capability of a CL participating in an ILC can be determined in different ways depending on the value of the obtained En index or z index. If the standard uncertainty of AV is too large compared to the standard deviation for the ILC, there is a risk of unreliable results for some CLs.

References

  1. 1. ILAC B7:10/2015. The ILAC Mutual Recognition Arrangement, ILAC, 2015; 8 p.
  2. 2. ILAC P4:05/2019. ILAC Mutual Recognition Arrangement: Policy and Management, ILAC, 2019, 11 p.
  3. 3. ILAC P5:05/2019 ILAC Mutual Recognition Arrangement: Scope and Obligations, ILAC; 2019, 9 p.
  4. 4. Velychko O., Gordiyenko T. Metrological Traceability at Different Measurement Levels. Standards, Methods and Solutions of Metrology, Published by IntechOpen, London, United Kingdom, 2019; Chapter 1, pp. 1–21. DOI: 10.5772/intechopen.84853.
  5. 5. ILAC P9:06/2014 ILAC Policy for Participation in Proficiency Testing Activities, ILAC; 2014, 8 p.
  6. 6. ILAC Р10:07/2020. ILAC Policy on Traceability of Measurement Results, ILAC; 2020, 11 p.
  7. 7. ILAC P14:09/2020 ILAC Policy for Measurement Uncertainty in Calibration, ILAC; 2020, 14 p.
  8. 8. JCGM 100:2008. Evaluation of measurement data – Guide to the expression of uncertainty in measurement (GUM), JCGM; 2008. 134 p.
  9. 9. JCGM 100:2008. Evaluation of measurement data — Supplement 1 to the “Guide to the expression of uncertainty in measurement” – Propagation of distributions using a Monte Carlo method, JCGM; 2008. 90 p.
  10. 10. JCGM 104:2009. Evaluation of measurement data – An introduction to the “Guide to the expression of uncertainty in measurement” and related documents, JCGM; 2009. 28 p.
  11. 11. JCGM GUM-6:2020. Guide to the expression of uncertainty in measurement – Part 6: Developing and using measurement models, JCGM; 2020. 103 p.
  12. 12. ILAC–BIPM Partnership [Internet]. Available from: https://ilac.org/about-ilac/partnerships/international-partners/bipm/ [Accessed: 2021-06-25].
  13. 13. ILAC G17:01/2021. ILAC Guidelines for Measurement Uncertainty in Testing, ILAC; 2021, 12 p.
  14. 14. ISO/IEC 17025:2017. General requirements for the competence of testing and calibration laboratories. Switzerland: ISO/IEC; 2017. 30 p.
  15. 15. ISO/IEC 17043:2010. Conformity assessment. General requirements for proficiency testing. Switzerland: ISO/IEC; 2010. 39 p.
  16. 16. Beckert S. F., Fischer G. E. Interlaboratory comparison of roughness measurement: Application of Algorithm A of ISO 13528:2015 in determining the designated value and the standard deviation. XXII World Congress of the International Measurement Confederation (IMEKO 2018). Journal of Physics: Conf. Series. IOP Publication. 2018;1065;082007:4. DOI: 10.1088/1742-6596/1065/8/082007.
  17. 17. Iacobescu F., Poenaru M. M., Anghel M.-A. Reactive Power Quality Assessment through Interlaboratories Comparison. 22th IMEKO TC 4 Symposium “Supporting World development through electrical and electronic measurements”, 2017; Iasi, Romania, 13–19.
  18. 18. Poenaru M. M., Iacobescu F., Anghel M.-A. Length сalibration Quality assessment through Interlaboratories Comparison. 22th IMEKO TC 4 Symposium “Supporting World development through electrical and electronic measurements”, 2017; Iasi, Romania, 20–26.
  19. 19. Poenaru M. M., Iacobescu F., Anghel M.-A. Pressure Calibration Quality Assessment through Interlaboratories Comparison. 22th IMEKO TC 4 Symposium “Supporting World development through electrical and electronic measurements”, 2017; Iasi, Romania, 27–32.
  20. 20. Poenaru M. M., Iacobescu F., Anghel A.-C., Salceanu A., Anghel M.-A. Active power quality assessment through interlaboratories comparison, 21st ІМЕКО TC4 International Symposium and 19th International Workshop on ADC Modelling and Testing Understanding the World through Electrical and Electronic Measurement, Budapest, Hungary, 2016, 224–228.
  21. 21. Furuichi N., Terao Y., Ogawa S., Cordova L., Shimada T. Inter-laboratory comparison of small water flow calibration facilities with extremely low uncertainty. Measurement, 2016;91:548–556.
  22. 22. Bermanec L. G., Zvizdic D. Interlaboratory comparison in the pressure range from 0 to 2 MPa for accredited calibration laboratories. Int. J. Metrol. Qual. Eng., 2015;6, 307. DOI: 10.1051/ijmqe/2015021.
  23. 23. Claudio J., Costa M. Brazilian energy interlaboratory program applicative. In: Proceedings of the XX IMEKO World Congress “Metrology for Green Growth”; 2012; Busan, Republic of Korea. IMEKO; 2012. 6 p.
  24. 24. Briggs P. Proficiency testing for calibration laboratories. In: Proceedings of the XX IMEKO World Congress “Metrology for Green Growth”; 2012; Busan, Republic of Korea. IMEKO; 2012. 5 р.
  25. 25. Velychko O., Gordiyenko T. The estimation of the measurement results with using statistical methods. Journal of Physics: Conf. Series. 2015; 588;012017:6. DOI: 10.1088/1742-6596/588/1/012017.
  26. 26. ISO 13528:2015. Statistical methods for use in proficiency testing by interlaboratory comparisons. Switzerland: ISO; 2015. 89 p.
  27. 27. Velychko O., Gordiyenko T. The implementation of general guides and standards on regional level in the field of metrology. Journal of Physics: Conf. Series. 2010; 238;012044:6. DOI: 10.1088/1742-6596/238/1/012044.
  28. 28. Chunovkina A., Zviagin N., Burmistrova N. Interlaboratory comparisons. Practical approach for data evaluation. In: Proceedings of the XX IMEKO World Congress “Metrology for Green Growth”; 2012; Busan, Republic of Korea. IMEKO; 2012. 5 р.
  29. 29. Jackson G. S., Muzikar P., Goehring B. A Bayesian approach to an interlaboratory comparison. Chemometrics and Intelligent Laboratory Systems, (2015) Vol. 141, 94–99
  30. 30. Acko B., Brezovnik S., Sluban B. Verification of Software Applications for Evaluating Interlaboratory Comparison Results. Procedia Engineering. 2014;69, 263–272.
  31. 31. EA-04/02 М. Evaluation of the Uncertainty of Measurement in Calibration. EA; 2013. 75 p.
  32. 32. Velychko O., Gordiyenko T., Boriak K. Linking the Results of Inter-laboratory Comparisons for DC Electrical Resistance Measures. Ukrainian Metrological Journal. 2020;1:4–11. DOI: 10.24027/2306-7039.1.2020. 204155.
  33. 33. Velychko O., Shevkun S., Mescheriak O., Gordiyenko T., Kursini S. Interlaboratory comparisons of the calibration results of signal generator. Eastern-European Journal of Enterprise Technologies. 2019;3/9(99):14–20. DOI: 10.15587/1729-4061.2019.166504.
  34. 34. Velychko O., Shevkun S., Gordiyenko T., Mescheriak O. Interlaboratory comparisons of the calibration results of time meters. Eastern-European Journal of Enterprise Technologies. 2018;1/9(91):4–11. DOI: 10.15587/1729-4061.2018.121089.
  35. 35. Velychko O., Isaiev V. Interlaboratory comparison in context of inappropriate results of voltage thermal converter calibration. Journal of Electrical Engineering and Information Technologies. 2018;3;1–2:5–12.
  36. 36. The BIPM key comparison database (KCDB) [Internet]. Available from: http://kcdb.bipm.org/ [Accessed: 2021-06-25].
  37. 37. Velychko O., Isaiev V. A comparative analysis of AC/DC transfer standards for comparison of national standards. Eastern-European Journal of Enterprise Technologies. Applied physics. 2018;6/5(96):14–24. DOI: 10.15587/1729-4061.2018.150459.
  38. 38. Velychko O., Gordiyenko T. Features of the processing of results and estimation of measurement uncertainty of inter-laboratory comparison for calibration laboratories. Information Processing Systems. 2018;4(155):77–83. DOI: 10.30748/soi.2018.155.10.
  39. 39. Sousa J. J. L., Leitão L. T. S., Costa M. M., Faria M. C. Considerations on the influence of travelling standards instability in an interlaboratory comparison program. In: Proceedings of the XX IMEKO World Congress “Metrology for Green Growth”; 2012; Busan, Republic of Korea. IMEKO; 2012. 4 p.
  40. 40. Berni R., Carobbi C. Alternative statistical analysis of interlaboratory comparison measurement results. In: Proceedings of the XXI IMEKO World Congress “Measurement in Research and Industry”; 2015; Prague, Czech Republic. IMEKO; 2015. 5 p.
  41. 41. Cox M. G. The evaluation of key comparison data: determining the largest consistent subset. Metrologia. 2007;44:187–200.
  42. 42. Cox M. G. The evaluation of key comparison data. Metrologia. 2002;39:589–595. DOI: 10.1088/0026-1394/39/6/10.
  43. 43. CООМЕТ R/GM/14:2016. Guidelines for data evaluation of COOMET key comparison, COOMET; 2016, 6 p.
  44. 44. CООМЕТ R/GM/19:2016. Guideline on COOMET supplementary comparison evaluation, COOMET; 2016, 5 p.

Written By

Oleh Velychko and Tetyana Gordiyenko

Submitted: 24 June 2021 Reviewed: 16 July 2021 Published: 21 August 2021