The list of national ILCs for CLs.
National accreditation agencies in different countries have set quite strict requirements for accreditation of testing and calibration laboratories. Interlaboratory comparisons (ILCs) are a form of experimental verification of laboratory activities to determine technical competence in a particular activity. Successful results of conducting ILCs for the laboratory are a confirmation of competence in carrying out certain types of measurements by a specific specialist on specific equipment. To obtain reliable results of ILC accredited laboratories, it is necessary to improve the methods of processing these results. These methods are based on various data processing algorithms. Therefore, it is necessary to choose the most optimal method of processing the obtained data, which would allow to obtain reliable results. In addition, it is necessary to take into account the peculiarities of the calibration laboratories (CLs) when evaluating the results of ILС. Such features are related to the need to provide calibration of measuring instruments for testing laboratories. The evaluation results for ILCs for CLs are presented. The results for all participants of ILCs were evaluated using the En and z indexes. The obtained results showed that for the such ILCs it is also necessary to evaluate the data using the z index also.
- interlaboratory comparison
- data evaluation
- referent laboratory
- calibration laboratory
- measurement uncertainty
Participants in the International Laboratory Accreditation Cooperation (ILAC) Mutual Recognition Agreement (MRA) recognize the calibration or test results obtained by each other’s accredited calibration and testing laboratories [1, 2, 3, 4]. ILAC Policy and Procedural publications are for the operation of the ILAC MRA. ILAC has a special policy for participation in proficiency testing activities, on metrological traceability of measurement results, for measurement uncertainty in calibration [5, 6, 7]. The policy for measurement uncertainty to base on the Guide to Uncertainty in Measurement (GUM) [8, 9, 10, 11] and retains the common understanding of the term calibration and measurement capabilities (CMCs) from the joint declaration issued by the International Bureau of Weights and Measures (BIPM) and ILAC . ILAC has a special guideline for measurement uncertainty in testing . This document provides guidance for the evaluation and reporting of measurement uncertainty in testing accordance with the requirements of the International Standard ISO/IEC 17025 .
National accreditation agencies in different countries have set quite strict requirements for accreditation of testing and calibration laboratories. Laboratory accreditation criteria in most accreditation systems include three main groups: laboratory technical equipment, personnel competence, and the effectiveness of the quality system. Interlaboratory comparisons (ILCs) are a form of experimental verification of laboratory activities to determine technical competence in a particular activity. Successful results of conducting ILCs for the laboratory are a confirmation of competence in carrying out certain types of measurements by a specific specialist on specific equipment.
To obtain reliable results of ILC accredited laboratories, it is necessary to improve the methods of processing these results. These methods are based on various data processing algorithms as required by international and regional guidelines and standards. To conduct ILC for CLs, it is necessary to take into account the relevant requirements of the international standards ISO/IEC 17025  and ISO/IEC 17043 . Therefore, it is necessary to choose the most optimal method of processing the obtained data, which would have a minimum number of restrictions on the application and allow to obtain reliable results. In addition, it is necessary to take into account the peculiarities of the calibration laboratories (CLs) when evaluating the results of ILС. Such features are related to the need to provide calibration of measuring instruments for testing laboratories.
ILCs for CLs are held nationally in different countries. Such ILCs are carried out to establish the competence of the CLs in calibrating various measuring instruments and working standards for various measured quantities [16, 17, 18, 19, 20, 21, 22, 23, 24]. For their implementation, various calibration objects are used. To evaluate the ILC data, various methods of their data processing are used [25, 26, 27, 28, 29, 30], and to estimate the measurement uncertainty, the regional guidance EA-04/02 М  is additionally used, in addition to the ILAC documents [8, 13]. However, in addition to the method of data evaluation, it is necessary to take into account other influencing factors on the CL result of ILC. In particular, unsatisfactory ILC results for all participating CLs may be associated with a large time drift of the calibrated measuring instrument.
The growing practical need of ILCs for CLs to ensure recognition of the obtained results at both national and international levels underscores the relevance of this research.
2. The national interlaboratory comparisons for calibration laboratories
The main purpose of accredited CLs is to calibrate working standards and measuring instruments for accredited testing laboratories. Significantly more testing laboratories are accredited by national accreditation bodies than CLs. For example, at the middle of 2021, 837 testing and 35 calibration laboratories were accredited in Ukraine. This represents only 4% of accredited CLs of the total number of all accredited laboratories. Therefore, the number of ILCs for testing laboratories is objectively much larger than for CLs.
The State Enterprise “Ukrmetrteststandard” (Ukraine) as a referent laboratory (RL) organized and carry out seven ILCs for accredited CLs from 2016 to 2019 [32, 33, 34, 35, etc]. The list of these ILCs is shown in Table 1. The calibration objects for these ILCs were working standards and measuring instruments for electrical quantities, and time and frequency. When carrying out comparisons, CLs calibrated objects in accordance with the requirements of the international standard ISO/IEC 17025 . The total number of calibration object parameters ranged from 3 to 12. The total number of CLs with RL that took part in these comparisons ranged from 5 to 10.
|ILC||Calibration object||Number of parameters||Number of participants||Period of carrying out|
|ILC1||Precision measuring thermocouple||AC voltage at 5 frequencies||5 labs||2016–2018|
|ILC2||Measures of electrical resistance (1th round)||3 nominations of resistance||8 labs||2016|
|ILC3||Measures of electrical resistance (2th round)||3 nominations of resistance||5 labs||2018–2019|
|ILC4||Precision measure of electric power||6 power factors at 2 frequencies||8 labs||2016–2018|
|ILC5||Low frequency signal generator||AC voltage at one frequency, total harmonic factor at 4 frequencies, 5 frequencies||4 labs||2016|
|ILC6||Electronic stopwatch||3 time intervals||9 labs||2016|
|ILC7||High-frequency signal generator||3 frequencies||10 labs||2018|
In all presented ILCs, the assigned value (AV) with its uncertainty was taken as the value with its uncertainty of the RL. This was done because the RL had the best measurement capabilities among all CLs that took part in the comparisons. For many years RL has taken part in international comparisons of national measurement standards of electrical quantities within the framework of Regional Metrological Organizations (COOMET, EURAMET, and GULFMET) and had positive results. RL also had published CMCs for some electrical quantities in the BIPM Key Comparison Database .
A program for all ILCs was implemented in accordance with the requirements of ISO/IEC 17043 . CLs that participated in the ILCs performed calibration of the measuring instruments (calibration object) provided to the RL in accordance with their own methods according to the radial scheme . RL sent the calibration object to the participating laboratory, and this laboratory returned this object back to RL. In this case, the RL constantly monitored the stability of the calibration object [35, 37]. The RL determined the characteristics of the instability of the calibration object before and after its research in the CLs participating in the ILC.
In accordance with the adopted ILC programs, RL analyzed the calibration data provided by the CLs , in particular, analyzed the declared measurement uncertainty. The data obtained from CLs were necessarily checked by RL for their consistency. Indicators for assessing of consistency were
3. The traditional data evaluation of interlaboratory comparisons
The traditional assessment of ILC data for CLs is carried out in accordance with the requirements of ISO/IEC 17043 . During of the evaluation of primary data from the participating CLs, the interlaboratory deviation of the measurement results or degree of equivalences (DoE) was calculated based on the ILCs results.
where is measured value for
Expanded uncertainty of the result of each participant and expanded uncertainty of AV were used to check the consistency of the primary ILC data and to calculate
On Figures 1–3 show the traditional graphical interpretation of the results of three ILCs at one of the calibration points (ILC 2–1, Figure 1, ILC 4–2, Figure 2 and ILC 6–1, Figure 3 respectively). The evaluation of primary data of all ILCs is carried out by means of the specially developed software “Interlaboratory comparisons” (Ukraine) which implements the algorithm presented in . To prepare reports on ILCs, RL used specified software that allowed calculating the
Only two laboratories (lab 4 and lab 6 for ILC 2–1) have an unsatisfactory result for two ILCs using the
4. The additional data evaluation of interlaboratory comparisons
The consistency evaluation of data using
where σ is the standard deviation for qualification assessment (ILC).
In Tables 2–8 shows the calculated results of
|ILC data||Index||Lab 1||Lab 2||Lab 3||Lab 4|
|ILC data||Index||Lab 1||Lab 2||Lab 3||Lab 4||Lab 5||Lab 6||Lab 7|
|ILC data||Index||Lab 1||Lab 2||Lab 3||Lab 4|
|ILC data||Index||Lab 1||Lab 2||Lab 3||Lab 4||Lab 5||Lab 6|
|ILC data||Index||Lab 1||Lab 2||Lab 3|
|ILC data||Index||Lab 1||Lab 2||Lab 3||Lab 4||Lab 5||Lab 6||Lab 7||Lab 8||Lab 9|
|ILC data||Index||Lab 1||Lab 2||Lab 3||Lab 4||Lab 5||Lab 6||Lab 7||Lab 8||Lab 9||Lab 10|
5. The summarized results of interlaboratory comparisons
The summarized results of estimation of
|ILC||Number of participants*||Number of parameters||Percentage of discrepancies in evaluation|
|ILC1||4 labs||20 points||1 point (5%)||0 point (0%)||4 points (20%)||100%|
|ILC2||7 labs||21 points||5 points (24%)||0 point (0%)||3 points (14%)||95%|
|ILC3||4 labs||12 points||0 point (0%)||0 point (0%)||3 points (25%)||100%|
|ILC4||6 labs||72 points||0 point (0%)||0 point (0%)||14 points (19%)||100%|
|ILC5||3 labs||30 points||2 points (7%)||0 point (0%)||10 points (33%)||97%|
|ILC6||9 labs||27 points||0 point (0%)||3 points (11%)||1 point (4%)||100%|
|ILC7||10 labs||30 points||1 point (3%)||0 point (0%)||3 points (10%)||97%|
Only one result of ILC1 according to
5 results of ILC2 according to
ILC3, ILC4, and ILC6 according to
Only one ILC7 result according to
The results of the data consistency analysis show that all ILCs, taking into account both indexes, have measurement points with unsatisfactory results. Analysis of the data taking into account the
If we return to the analysis of Figures 1–3, it can be seen that lab 4 for ILCs 2–1 and lab 1 and lab 7 for ILC6–1 have very large declared measurement uncertainties with large DoEs. This led to unsatisfactory results, taking into account the z index. The main reason for the unsatisfactory result of lab 3 for ILCs 1–2, taking into account
The general recommendation for lab 3 and lab 4 for ILC1–2, as well as for lab 4 for ILC4–2, and lab 1 and lab 7 for ILC6–1 is to revise the estimate of the measurement uncertainty, taking into account guides [8, 31]. This measurement uncertainty can be influenced by both the calibration results of the laboratory working standards and the level of competence of the laboratory personnel. Taking these recommendations into account can improve the results of that laboratories participation in other rounds of ILCs or new ILCs.
6. The influence of travelling standards instability
The travelling standards instability can affect the results of ILCs for CLs. Some works are devoted to assessing its influence, in particular compensation for its instability. The repeatability of a good measuring instrument is below 10% of its maximum error as shown in . The travelling standard with 0,2% shows variations of random errors below where is the average of the readings during calibration. This a small Type A uncertainty in relation to other components is show.
Typically, RL already takes into account the travelling standards instability in the ILC assigned value and its expanded uncertainty .
where is the standard measurement uncertainty obtained by calibrating of travelling standard with a RL; is the standard measurement uncertainty from the travelling standard instability of during ILC period
is the maximum change in nominal value of travelling standard during ILC period.
The absence of a significant effect of the travelling standards instability on the evaluation of the CL result in the ILC can be at its maximum instability, which is determined by the expression .
The value of the travelling standards instability can be obtained for several cases: measurements of the RL of the travelling standard in the process of carry out of ILC; from the technical specification for the travelling standard, measurements of the RL of the travelling standard for a long time, and etc. Results of calculating the
In any case, from expression (4) it follows that with an increase in the value of the measurement uncertainty associated with instability, the value of the
An analysis RL of the travelling standard instability for all calibration points of the ILC7 is given in . The drift of travelling standard for ILC7 at all frequencies is presented on Figure 8. The uncertainty of travelling standards instability for ILC7 is presented in Table 10. The contribution of the uncertainty from the long-term drift of the travelling standard to the standard uncertainty of AV for the entire duration of ILC7 is from 5.3 to 8.3% for all calibration points. Such a drift of the measuring instrument used as a calibration object is acceptable for the ILC. It does not distort the ILC results for the participating CLs.
|ILC7 point||Frequency (MHz)||(Hz)||(Hz)||Drift contribution to uncertainty AV (%)|
The list of travelling standard for all ILCs and values of
|ILC||Calibration object||Working standard||Measuring instrument|
|ILC1||Precision measuring thermocouple||Yes||No||5||20|
|ILC2||Measures of electrical resistance (1th round)||Yes||No||24||14|
|ILC3||Measures of electrical resistance (2th round)||Yes||No||0||25|
|ILC4||Precision measure of electric power||Yes||No||0||19|
|ILC5||Low frequency signal generator||No||Yes||7||33|
|ILC7||High-frequency signal generator||No||Yes||3||13|
The use of a measuring instrument as a calibration object leads to a slight increase in the values of the
7. The improvement of the evaluation of interlaboratory comparison results
Statistical methods for use in proficiency testing by ILCs are presented in [26, 27]. The aim of creating alternative statistics in order to improve the analysis and evaluation of ILC measurement results is research work . The improvement of statistical indicators is proposed by addressing two specific issues: robustness and reliability. The proposed methodology is not traditional for ILC, but it can be used as an additional methodology for checking the results of ILC.
The following conditions are provided for data evaluation of international comparison of national standards: the travelling measurement standard is stable, the measurement results presented by laboratories are reciprocally independent, and the Gaussian distribution is assigned to a measurand in each laboratory [41, 42, 43, 44]. The same conditions can be extended for data evaluation of ILCs for CLs. Frequently the measurement procedures for supplementary comparisons of national standards  are the calibration procedures of these laboratories. Such calibration procedures can also be extended to ILCs for CLs. In such a case, the calibration capabilities of the laboratory can be confirmed.
The application of
In case the declared uncertainties CL don’t confirmed during the ILC and for their confirmation it is necessary to participate in other similar ILCs.
Often, a national metrological institute or an accredited CL, which is an RL in ILC, performs high-precision calibration of working standards and measuring instruments for CLs participating in this ILC. In this case, a correlation of the obtained CL results is formed, which must be taken into account when evaluating the data of such an ILC. Covariance’s are estimated by careful analysis of the uncertainty budget of CLs by the RL
where is common input to the uncertainty budgets of both results .
In this case, the value of the
If the value of the
If the value of the
where is the standard measurement uncertainty of AV.
Correspondingly, the extended uncertainty is .
The same requirements can be extended for compliance (≤ 2.0) or inconsistency (> 2.0) of the value of the
If the standard uncertainty of the AV is too large in comparison with the standard deviation σ for ILC, then there is a risk that some laboratories will receive action and warning signals because of inaccuracy in the determination of the AV, not because of any cause within the laboratories. If
then the uncertainty of the AV is negligible and need not be included in the interpretation of the ILC results. Further, all CLs participating in ILC shall carry out the same number of replicate measurements. This approach assumes that CLs have generally similar repeatability .
To evaluate the ILC data, can use
This equation may be used when the AV is not calculated using the results reported by CLs participating in ILC.
Comparison of the equations for
When the inequality established by expression (12) is satisfied, then this factor will fall in the range: . In this case,
To perform an ILC for CLs, RL must provide a stable working standard or measuring instrument as a calibration object and monitor its drift throughout the ILC. The use of a measuring instrument as a calibration object leads to a slight increase in the values of the
The analysis of the results of the ILC for CLs for consistency should include not only the analysis of the values of the
The stable travelling standard, the independent measurement results of laboratories with Gaussian distribution are main conditions for data evaluation of ILC for CLs. To participate in the ILC when declaring its measurement uncertainty, CLs must conduct a thorough analysis of the components of this uncertainty. It is necessary to take into account the correlation of the laboratory data of the participants of the ILC when evaluating its results. Covariance is estimated by carefully analyzing the CL uncertainty budget using RL.
The minimum standard measurement uncertainty that can be claimed as the calibration capability of a CL participating in an ILC can be determined in different ways depending on the value of the obtained