Introducing impact and cost into conformity assessment risks

Risks and costs

In general, the impact of a wrong decision in conformity assessment is expressed as a risk Risk, defined as the probability p of the wrong decision occurring multiplied by the cost (utility) C of the consequences of the in-correct decision:

Risk = p.C

In this section, new expressions for decision-making risks including costs are presented, together with a novel tool – the operating cost characteristic curve – as an extension of traditional statistical tools, with the addition of an economic decision-theory approach. Complementarity with the optimised uncertainty methodology [Fearn et al 2002] is emphasised in the concluding remarks.

Cost model

An in-correct accept on inspection of a non-conforming object will lead to customer costs associated with out-of-tolerance product. Overall costs, consisting of a sum of testing costs, D,  and the costs, C, associated with customer risk can be calculated with the expression:

Overall costs - equation

Overall costs - equation

ym within rpvwhere RPV denotes the region of permissible entity values, η – where test costs, D, are modelled as varying inversely to the squared dispersion σ² . The expression can be applied to both specific and global conformity assessment [Pendrill 2007].

Plots of cost against (i) quantity value and (ii) uncertainty

Overall costs, according to the above equation, can be either plotted over:

  1. a range of quantity values of  η for a given test dispersion, σ, and ‘guard-band’ factor h – yielding an “operating cost characteristic” analogous to the traditional, probability-based operating characteristic
  2. a range of test dispersion, σ, for a given quantity value η, the so-called “optimised uncertainty curve
3D costs

3D costs

Figure Overall costs versus Uncertainty and Entity value in the vicinity of a specification limit USL (adapted from [Pendrill 2008])

It is possible to view the two tools – the operating cost characteristic [I] and optimised uncertainty [II] methodologies – as together providing a complete basis for risk-assessment in conformity assessment: Overall costs are plotted in three dimensions – shown in figure – where at each entity value on the operating characteristic curve, the corresponding optimized uncertainty curve would cross in the orthogonal direction. The familiar U-shaped optimized uncertainty curve – where the costs of testing are balance against the costs of incorrect decision-making – is clearly visible along the ‘uncertainty’ axis of the 3D-plot of figure . In this way, the optimum uncertainty required at specific conformity assessment points, such as those for customer and supplier risk, could be identified across the full range of entity values.

Criticism of approach

A common criticism is that it is “very difficult to quantify the total cost in all possible applicative scenarios. On the other hand, it is known that, by setting a suitable amount of guard-banding, the risk of accepting an out-of-tolerance item or rejecting an in-tolerance item can be kept under control with good accuracy” [Macii and Petri 2009]

Answers to such criticism include:





G Beges, J Drnovsek and L R Pendrill 2009 Optimising calibration and measurement capabilities in terms of economics in conformity assessment”, Accred Qual Assur, DOI

Hake R 2010 “The Cult of Statistical Significance”,

T. Fearn, S. Fisher, M. Thompson and S. Ellison 2002, “A decision-theory approach to fitness for purpose in analytical measurement,” Analyst, vol. 127, pp. 818 – 824

Macii D and Petri D 2009 “Guidelines to manage measurement uncertainty in conformance testing procedures”, IEEE Trans Instrum Meas 58, 33 – 40

H Källgren and L R Pendrill 2006, “Exhaust gas analysers and optimised sampling, uncertainties and costs”, Accreditation and Quality Assurance – Journal for Quality, Reliability and Comparability in Chemical Measurement. Vol 11, 496- 505  (2006)

Källgren H, Pendrill L R, Lindlov K (2006) ”Uncertainty in conformity assessment in legal metrology (related to the MID)”, OIML Bull XLVII(3):15–21

Pendrill L R 2007 “Optimised Measurement Uncertainty and Decision-Making in Conformity Assessment”, NCSLi Measure, Vol. 2, no. 2, pp 76 – 86 Balancing the costs of testing against the consequence costs of in-correct decision-making

Pendrill L R 2008, “Operating ‘cost’ characteristics in sampling by variable and attributeAccred. Qual. Assur., 13, 619 – 631, DOI: 10.1007/s00769-008-0438-y Extending classical statistical significance tools to include measures of impact

L.R. Pendrill and H. Källgren 2008, “Optimised measurement uncertainty and decision-making in the metering of energy, fuel and exhaust gases,” Izmerite’lnaya Technika (Measurement Techniques), Vol 51, No. 4, pp. 370 – 7, April, 2008 DOI:

L. R. Pendrill 2009, An Optimised Uncertainty Approach to Guard-banding in Global Conformity Assessment”, Advanced Mathematical and Computational Tools in Metrology VIII in Data Modeling for Metrology and Testing in Measurement Science Series: Modeling and Simulation in Science, Engineering and Technology, Birkhauswer (Boston), 2009, ISBN: 978-0-8176-4592-2

L R Pendrill 2010 “Optimised uncertainty and cost operating characteristics: new tools for conformity assessment. Application to geometrical product control in automobile industry”, Int. J. Metrol. Qual. Eng.1, 105 – 110, DOI:

Ziliak, S.T. & D. McCloskey. 2008. “The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives (Economics, Cognition, and Society).” University of Michigan Press; publisher’s information at <>. information at <>. Note the searchable “Look Inside” feature.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: