^{a}Engineering Materials, Environment and Modeling Laboratory (LIMME). Faculty of Sciences Dhar El Mahraz BP 1796 Atlas Fez, Morocco, ^{b}Application Organic Chemistry Laboratory, Faculty of Sciences and Techniques, BP 2202 route d’immouzerFèsUniversity Sidi Mohamed Ben AbedallahMorocco
Email: fayssaljhilal@gmail.com
Received: 20 Dec 2015 Revised and Accepted: 27 Feb 2016
ABSTRACT
Objective: A new and powerful statistical approach known as the uncertainty profile concept has been suggested for both testing the validity and making easy and straightforward interpretation of results obtained during the validation of an analytical method. The main goal of this paper is to confirm the applicability of this new strategy for the validation of a commercial kit, microbiological method, for the enumeration of the Enterobacteriaceae in foods and the estimate of the measurement uncertainty by using the newly provided formula and without referring to any additional experiments.
Methods: An innovative formula to assess the uncertainty by using validation data and without recourse to other additional experiments was proposed. The uncertainty was evaluated through the twosided βcontent, γconfidence tolerance interval, which is computed with three manners: the Mee’s approach, the Generalized Pivotal Confidence, and the Modified Large Simple procedure
Results: After the use of the three chemometric method of calculation of tolerance intervals, the obtained results with uncertainty profile show without doubt that the enumeration method is valid over the range of target values given that the upper and the lower 66.7 %content, 90 %confidence tolerance limits have fallen within the two acceptance limits of±0.25 Log unit. If the β is stretched to 80 %content, 90 %confidence, the three computed tolerance intervals lead to different decisions.
Conclusion: we have demonstrated the ability of the uncertainty profile to be used for testing the validity of enumeration method which represents the first application of an uncertainty profile to food microbiological methods, and provides good estimations of the uncertainty measurements for each concentration level.
Keywords:Validation, Uncertainty profile, βcontentγconfidence tolerance interval, Uncertainty measurement, Microbiological method
© 2016 The Authors. Published by Innovare Academic Sciences Pvt Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)
INTRODUCTION
Actually, it is well known that the analytical, microbiological methods are very crucial on account of their role and efficacy to provide accurate quantification of the analyte in pharmaceutical and medical field. Concerning tests implemented, manufacturers must verify that their microbiological methods are applicable to analyze the pharmaceuticals or medical devices and then yields a confidence in the validity of analytical results that are provided to customers.
An analytical measurement is a tool to make a suitable decision. Therefore, the effort must be heading essentially to the metrological quality of the measurements, which requires introducing two important concepts: analytical validation and estimation of measurement uncertainty. Then, to ensure an analytical measurement, the analyst is called to master both these concepts which are passages mandatory for all laboratories that are trying to be accredited according to ISO 17025 [2].
Thus, the validation of a bioanalytical method is crucial since it is mandatory to prove that an alternative method is able actually to deliver results that can be interpreted in the same way as those obtained with the reference method.
Otherwise, several papers like standards and guidelines have been published in order to propose a procedure for validating analytical and bioanalytical methods [39]. In general, they recommend checking a validation set of criteria that must be compared to threshold values. Although these documents have largely contributed to the advancement of the implementation of analytical validation, however, they have weaknesses on the conclusions of tests performed and as to aid in decision making related to acceptance limits defined for the use of an analytical procedure. In addition, the guide ISO 16140 [10], defined the final decision rules which are principally based on a set of null hypothesis tests, aiming to demonstrate that the estimated parameters for the alternative method do not differ significantly from those obtained with the reference method. This strategy presents many drawbacks that were extensively described in recent papers [11, 12]. Consequently, the SFSTP published a guide entitled ‘Harmonization of strategies for the validation of quantitative analytical procedures [3, 4] where a new interpretation strategy has been developed, called accuracy profile, based on global acceptance criteria so that such confusing conclusions can be avoided. This strategy of accuracy profile as a decision tool based on the concept of total error and uses the βexpectation tolerance interval. It has been proved its efficacy to select the most appropriate response function, to determine the limits of quantification and to assess the concentration range. In spite of its applicability and flexibility, this approach has shown, however, gaps in the estimation of measurement uncertainty [1321]. Undoubtedly, measurement uncertainty is a crucial parameter which the estimation is considered as a major problem for every laboratory. So as to assist analysts, standards and guides have been published to assess the measurement uncertainty [2226] such as the International Organization for Standardization (ISO) approach [24] (commonly known as “bottomup” approach) and the Analytical Methods Committee [27] (commonly known as “topdown” approach). From ISO Guide, EURACHEM was conceived and adapted specifically for chemical measurements [28].
The implementation of the bottomup approach to measurement uncertainty proposed by the GUM is hardly applicable for complex analytical or bioanalytical methods because of the difficulty of constructing an uncertainty budget model [2931]. The propagation of the uncertainties in this model following the law of error propagation is another difficulty. According to the Eura chem guide [28], uncertainty can be evaluated from the validation data. Thus, research studies have been published and dedicated to the one and only main purpose of simplifying and evaluating the uncertainty of measurement from validation data [3240]. Consequently, all researchers’ concentration was mainly to adopt a global, harmonic and easy to practice strategy to test the validity of an analytical method and to estimate the measurement uncertainty.
In this paper, we try to show the applicability of a new analytical validation strategy and estimation of measurement uncertainty and expand its field of application by applying it to a biological matrix. This approach called the uncertainty profile was developed in our laboratory not simply to validate an analytical method but also to estimate the measurement uncertainty. This means that thanks to this approach, we can perform a full validation.
It is based on the use the two sided βcontent, γconfidence tolerance interval in order to build a powerful graphical tool that allows simply making a decision about the validity of a bioanalytical method [1421].
The present study aims to illustrate the validity of an alternative method for the quantitative enumeration of Enterobacteriaceae.
Three Chemometrics methodologies are uncovered to build the βcontent, γconfidence tolerance interval, namely: the Satterthwaite’s approximation, the GPQ method (generalized pivotal confidence) and the MLS procedure (modified large simple). Furthermore, we illustrate the applicability and flexibility of the uncertainty profile to assess the fitness of the purpose of analytical, microbiological methods [20].
METHODS AND EXPERIENCES
Case study
The 3M Petri film Enterobacteriaceae Count Plate is a sampleready culture medium system applicable to the enumeration of Enterobacteriaceae in food products and the industrial environment. The measure and is colonyforming unit per gram (CFU/g). Indeed, two major features are met in the original application: the difficulty encountered in defining the measure and for microbiological methods; and the use of experimental data collected in the framework of a collaborative study instead of by a single laboratory. Actually, tests for Enterobacteriaceae are mandatory steps in the process hygiene for many foodstuffs [1]. Therefore, enumeration of Enterobacteriaceae in foods now shows evidence of increasing interest.
Uncertainty assessment: new approach
The uncertainty is a parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measure and.
A simple and efficient procedure has recently made available to researchers to estimate the uncertainty of their analytical methods without restoring to other studies or experiments. Indeed, this approach is based only on the validation data and brings together two major objectives, the validation of analytical methods and estimation of measurement uncertainty. The estimation of the uncertainty is crucial for the interpretation of the results obtained and especially their comparisons with the statutorily above compliance limits. The declaration of an analytical result is incomplete without knowing the measurement uncertainty. Hence, ISO 17025 requires since 2001 for testing laboratories, to apply procedures for estimating uncertainty and to be able, if necessary, to be associated with the results returned [2]. Generally, the uncertainty is, as it is indicated in references [16, 17, 20], expressed by the following equations:
We conclude the expression of the uncertainty:
With:
Construction of uncertainty profile
The building strategy of uncertainty profile, which comes after computing the uncertainty through the equation (8), is based on the following expression:
Where
We can also build the uncertainty profile by the following expression:
Where:
And is the true value
To achieve the uncertainty profile, we have to respect the following steps [1421]
Tolerance intervals
Before computing the uncertainty, we have to estimate the tolerance interval. Generally, two kinds of tolerance intervals are largely used, which are the βexpectation tolerance interval and βcontent, γconfidence tolerance interval. In this section, our main interest lies in βcontent, γconfidence tolerance intervals which are also called guaranteed coverage tolerance intervals [20].
In the following, we will detail three approaches to the construction of a twosided βcontent, γconfidence tolerance interval using Mee's approach [41], Hoffman–Kringle approach [42] and Liao–Lin–Iyer approach [43, 44].
Mee's approach
In this approach, the estimation of Chisquare is based on the Satterthwaite’s approximation.
The computation of uncertainty profile is based on the βγcontent tolerance interval [41]. This later is calculated according to Mee’s approach and can be expressed as [16, 17, 20]:
Where:
is the estimation of tolerance factor which should for a bilateral βγcontent tolerance interval in a balanced oneway ANOVA.
And:
is the variance between laboratories.
is the variance within laboratories.
Liao–Lin–Iyer approach
Liao and al proposed an approach that focuses on the βcontent tolerance interval. Indeed, have used the generalized pivotal confidence interval and Monte Carlo simulation to estimate the βcontent tolerance intervals [43, 44].
We consider the case of oneway random model with balanced data; Note that, in order to derive a twosided tolerance interval, we have to obtain a margin of statistic error D, a function of and , so that:
Where Once D is obtained, the tolerance interval is given by:
We define
To calculate D, we must determine γ upper confidence limit for using the generalized confidence interval idea.
Once the data are obtained, compute the observed values, SS_{b} and SS_{e}. Let M denote the number of simulation runs. For i=1, 2, 3,…. M. Then, Generate independent random variables and .
Finally compute:
The γ quantile of the D_{1,i} values gives an estimate of the γ upper confidence limit.
The square root of this upper confidence limit, multiplied byprovides a margin of error statistic D needed for computing the twosided βcontent tolerance interval. In the above, , denotes the (1+β)/2 quantile of a standard normal distribution.
Hoffman–Kringle approach
It’s a strategy based on the Modified Large Simple (MLS) procedure to compute the twosided bcontent tolerance interval in balanced and unbalanced oneway random models [42]. For a balanced oneway random model, we define:
And
The MLS upper confidence limit foris given by:
Once the S is obtained, the βcontent, γconfidence tolerance interval is given by:
Evaluation of the enumeration method
In many classical guides, validation is habitually limited to calculate a set of performance criteria. But according to the definition of validation proposed by ISO, it must also contain a demonstration of the fitness of the method to a defined goal based on these performance criteria; therefore, uncertainty profile is a possible solution. It consists of a graphical decisionmaking tool to help the analyst decide whether an analytical procedure is valid. It is obtained by simultaneously combining βcontent, γconfidence tolerance interval and acceptability limits in graphical format.
In order To demonstrate the applicability of the uncertainty profile strategy to the microbiological field, we evaluate the performance of the quantitative enumeration procedure which is selected from the literature [1]. In this procedure, the dosage is direct and we haven’t to calibration data, and so we aren’t in need of response function or linearity study.
For validation standards, three concentration levels have involved in experiment design. Each validation sample was analyzed two times (n= 2) for fourteen different days (a= 14).
RESULTS AND DISCUSSION
At each concentration level, recovery, Bias, Standard deviation (SD) between and within laboratories and freedom degrees are calculated. All obtained results are summarized in table 1.
Table 1: Statistical results in Log_{10 }(CFU/g)
Statistic 
Concentration levels 

First 
Second 
Third 

Concentrations 
2.267 
3.23 
4.176 
Number of days (a) 
14 
14 
14 
Number of replicates (n) 
2 
2 
2 
Bias 
0.043 
0.0135 
0.047 
Recovery (%) 
98.11 
100.42 
101.12 
SD between laboratories 
0.0308 
0.0524 
0.0711 
SD within laboratories 
0.1221 
0.1100 
0.1044 
Freedom degree 
26.74 
25.75 
23.98 
Lower acceptability (λ) 
0.25 Log unit 
0.25 Log unit 
0.25 Log unit 
Upper acceptability (+λ) 
0.25 Log unit 
0.25 Log unit 
0.25 Log unit 
In order to calculate the tolerance intervals, we set a value of β = 66.7% of the content preferred to be compliant, the population of future measures included within the acceptance limits with confidence levels set at 90 % and 95 %. We expanded the value of β to 80 % and verified the validity of the method for the two risk values.
As known, the uncertainty is a key parameter in the strategy of the uncertainty profile. Its estimation is, therefore, fundamental to the interpretation of the results and is quite logically becomes an unavoidable obligation for laboratories in the various sectors where quantitative analytical procedures are applied to provide analytical results. Indeed, since 2001, the standard ISO 17025 requires testing laboratories to apply procedures for the estimation of the uncertainty of obtained results [45, 46]. In table 2 all results about uncertainty, expanded uncertainty and uncertainty limits are presented.
Table 2: Point estimates of the uncertainty measurement and uncertainty limits obtained for each concentration level using the 66.7 and 80 %content, 90 and 95 %confidence tolerance interval computed by Mee, Liao–Lin–Iyer (GPQ with N=100000) and Hoffman–Kringle (MLS) methods
βcontent 
γconfidence 
Approach 
Target values 
Uncertainty 
Expanded (%) 
Lower uncertainty 
Upper uncertainty 
(Log_{10 }(CFU/g)) 
Uncertainty 
limit 
limit 

0.667 
0.95 
Mee 
2.267 
0.0810 
7.1459 
2.0580 
2.3905 
3.23 
0.0794 
4.7853 
3.0802 
3.4069 

4.176 
0.0833 
3.9899 
4.0510 
4.3949 

GPG 
2.267 
0.0832 
7.3416 
2.0534 
2.3951 

3.23 
0.0809 
4.8735 
3.0772 
3.4099 

4.176 
0.0842 
4.0324 
4.0492 
4.3967 

MLS 
2.267 
0.0820 
7.2339 
2.0559 
2.3926 

3.23 
0.0799 
4.8103 
3.0793 
3.4078 

4.176 
0.0833 
3.9887 
4.0511 
4.3949 

0.667 
0.90 
Mee 
2.267 
0.0909 
8.0198 
2.0694 
2.3791 
3.23 
0.0889 
5.5046 
3.0919 
3.3952 

4.176 
0.093 
4.4529 
4.0639 
4.3820 

GPG 
2.267 
0.0940 
8.2929 
2.0641 
2.3844 

3.23 
0.0912 
5.4914 
3.0880 
3.3991 

4.176 
0.0947 
4.5339 
4.0610 
4.3849 

MLS 
2.267 
0.0919 
8.1042 
2.0677 
2.3808 

3.23 
0.0894 
5.3850 
3.0910 
3.3961 

4.176 
0.0931 
4.4608 
4.0636 
4.3823 

0.8 
0.95 
Mee 
2.267 
0.1072 
9.4583 
2.0042 
2.4443 
3.23 
0.1051 
6.3337 
3.0273 
3.4598 

4.176 
0.1103 
5.2808 
3.9954 
4.4505 

GPG 
2.267 
0.1099 
9.6937 
1.9987 
2.4498 

3.23 
0.1071 
6.4500 
3.0233 
3.4637 

4.176 
0.1112 
5.3240 
3.9935 
4.4524 

MLS 
2.267 
0.1085 
9.5763 
2.0014 
2.4471 

3.23 
0.1057 
6.3679 
3.0261 
3.4609 

4.176 
0.1103 
5.2802 
3.9954 
4.4505 

0.8 
0.90 
Mee 
2.267 
0.1203 
10.6150 
2.0192 
2.4293 
3.23 
0.1176 
7.0869 
3.0428 
3.4443 

4.176 
0.1231 
5.8937 
4.0124 
4.4335 

GPG 
2.267 
0.1242 
10.9606 
2.0126 
2.4359 

3.23 
0.1205 
7.2616 
3.0379 
3.4492 

4.176 
0.1255 
6.0082 
4.0083 
4.4376 

MLS 
2.267 
0.1216 
10.7284 
2.0170 
2.4315 

3.23 
0.1183 
7.1286 
3.0416 
3.4454 

4.176 
0.1233 
5.9052 
4.0120 
4.4339 
Table 3: Uncertainty limits with difference to target values for two different values of βcontent and γconfidence
βcontent 
γconfidence 
Target 
Tolerance interval with Difference to target values 

values 
Mee’s approach 
GPQ approach 
MLS approach 

0.667 
0.95 
2.267 
[0.2090;0.1235] 
[0.2136;0.1281] 
[0.2111;0.1256] 
3.230 
[0.1498;0.1769] 
[0.1528;0.1799] 
[0.1507;0.1778] 

4.176 
[0.1250;0.2189] 
[0.1268;0.2207] 
[0.1249;0.2189] 

0.90 
2.267 
[0.1976;0.1121] 
[0.2029;0.1174] 
[0.1993;0.1138] 

3.230 
[0.1381;0.1652] 
[0.1420;0.1691] 
[0.1390;0.1661] 

4.176 
[0.1121;0.2060] 
[0.1150;0.2039] 
[0.1124;0.2063] 

0.8 
0.95 
2.267 
[0.2628;0.1773] 
[0.2683;0.1828] 
[0.2656;0.1801] 
3.230 
[0.2027;0.2298] 
[0.2067;0.2337] 
[0.2309;0.2309] 

4.176 
[0.1806;0.2745] 
[0.1825;0.2764] 
[0.1806;0.2745] 

0.90 
2.267 
[0.2478;0.1623] 
[0.2544;0.1689] 
[0.2500;0.1645] 

3.230 
[0.1872;0.2143] 
[0.1921;0.2192] 
[0.1884;0.2154] 

4.176 
[0.1636;0.2575] 
[0.1677;0.2616] 
[0.1640;0.2579] 
The uncertainty profiles are constructed from the results in table 3 which contain the tolerance intervals with the difference to the target levels.
The upper and lower uncertainty limits expressed in absolute value are presented in fig. below as a function of the target values generated by the reference method. As can be seen from the results, the method was considered valid since the uncertainty intervals with the difference to target value are included in the±0.25 Log unit acceptance limits for all levels tested.
Legend
The continuous gray lines are the tolerance limits computed by MLS method, the dashed green lines represent the uncertainty limits computed by GPQ method, the blue dotted lines are the upper and lower βcontent, γconfidence tolerance limits calculated by Mee’s method and the red dashed lines are set at±0.25 Log unit.
In order to illustrate this methodology, it was applied to the validation of enumeration method of Enterobacteriaceae in foods. As indicated several times above, the uncertainty profile, directly reflecting the analytical procedure potential, makes possible to appreciate the adequacy of different practices and to make decisions.
Fig. 1: Uncertainty profile of the alternative method of enumeration of Enterobacteriaceae using Mee, GPQ and MLS methods (A): profile carried out with β=66.7 %, γ= 90 %, (B): profile carried out with β=66.7 %, γ= 95 %
Fig. 2: Uncertainty profile of the alternative method of enumeration of Enterobacteriaceae using Mee, GPQ and MLS methods (C): profile carried out with β=80 %, γ= 90 %, (D): profile carried out with β=80 %, γ= 95 %
All the steps listed below have been applied successfully to evaluate the performance of the alternative method of enumeration. On the other hand, the proposed strategy allows a good estimate of measurement uncertainty by using validation data [1] and without having recourse to other experiments by using the formula equation (3) for estimating uncertainty based on the βcontent, γconfidence tolerance interval and computed by three different methods: the Satterthwaite’s approximation (Mee’s method) [41], the GPQ method (generalized pivotal confidence) [43, 44] and the MLS procedure (modified large simple).
All for β=66.7 %, γ= 90% tolerance limit values are included within the acceptability limits, and it can be concluded that the alternative method is fully validated for all the three chemometric methods. Indeed, the tolerance intervals computed by Mee, GPQ and MLS methods are, as can be shown in fig.1. A, within the acceptance limits for all target levels. Even if we take a risk of 5 %, all tolerance limits, presented in fig.1. B, are always inside the acceptance limits.
Consider that over the validation domain; the tolerance interval is included in the acceptability limits; the method can be declared as valid over this range. This can be interpreted as follows: by definition, the β=66.7 %, γ= 90 & 95 % tolerance limits is supposed to contain, on average, 66.7 % of the predicted future measurements. Therefore, the analyst can guarantee that, on average, 66.7 % of the future results will fall within the limits of this prediction interval. As far as this percentage is included in the acceptability limits, the analyst can guarantee that future measurements will be equivalent to those obtained by the reference method, with an acceptance of±0.25 Log unit.
The other hand, we compared the results obtained from the uncertainty profile, with that calculated by the accuracy profile, each time led to the same decision even if we chose a value of±0.25 Log unit as acceptability limits. But, our preference is for the uncertainty interval, because this type of strategy allows a better estimate of the uncertainty measurement compared to the total error interval.
If the analyst wants more future measurements included in the limits of acceptability, there is the possibility to extend the β value to 80%. As a result, for an acceptance of±0.25 Log unit, the method is not valid on all range of target levels, fig. 2, for the tolerance intervals Mee, MLS, and GPG for both risks 5% and 10 %.
Otherwise, if we choose an acceptance of±0.3Log unit, the uncertainty limits are included in the acceptability limits; the method, as shown in fig. 3, can be declared as valid over this range.
Fig. 3: Uncertainty profile of the alternative method of enumeration of Enterobacteriaceae using Mee, GPQ and MLS methods (C): profile carried out with β=80 %, γ= 90 %, (D): profile carried out with β=80 %, γ= 95 %. Acceptance limits are set at±0.3 Log unit
CONCLUSION
The validation of any analytical method is the ultimate step before the routine use. In this work, we have demonstrated the ability of the uncertainty profile to be used for testing the validity of enumeration method which represents the first application of an uncertainty profile to food microbiological methods, and provides good estimations of the uncertainty measurements for each concentration level.
The uncertainty profile is built from the β,γtolerance interval. Three chemometric methods have been used to assess the tolerance interval: the Satterthwaite’s approximation (Mee), the GPQ method (generalized pivotal confidence) and the MLS procedure (modified large simple). In the light of the obtained results, we have shown that these three statistic methods can easily be applied to evaluate the performance of enumeration method as well as to assess the uncertainty of results and led to the same decision.
CONFLICT OF INTERESTS
Declared none
REFERENCES