Compliance with specifications
- Hard faults: often measured quantitatively and tested for compliance with a specification limit (e.g. mean time to failure < 10000 hours) on an interval or ratio measurement scale
- Soft faults: often measured qualitatively (e.g. the customer says, “How do I turn the thing on?!”) on perhaps an ordinal scale
Measuring & testing embedded systems
Two distinct approaches to the testing and measurement of embedded systems can be identified:
- choice of a measurement system (instrument, method, etc) suitable for testing an embedded system against specifications
- characterisation of an embedded system as though it were a measurement system in order to increase understanding of how the embedded system functions and its principal limitations
Any measurement, be it quantitatively of a hard fault or a more subjective judgement of a soft fault, will not be perfect. Limited time and resources to make the measurement will lead to a number of measurement errors which are not evaluated or compensated for. The resulting measurement uncertainty needs to be evaluated.
Decision-making, risks & costs
When making a decision about compliance of an embedded system with a specification limit, any measurement uncertainty will lead to a certain risk that incorrect decisions of conformity will be made.
Finally, these decision risks need to be evaluated, not only in terms of % supplier respective consumer risk, but also in impact terms (cost, danger etc)