In order to describe a phenomenon which contains some kind of randomness, a probabilty distribution can be used, see Figure 1. This distribution gives the probability (frequency) of various events, for example different results from repeated measurements of a quantity. Often a summary of a distribution is required in terms of a small number of measures (characteristics) for example a position measure (mean value, median etc) and a measure of scatter (variance, standard deviation, variation width etc). Even relations between different variables can be expressed with characteristics (covariance, correlation coefficients).
Figure 1. Probability distribution for a continuous random variable
The area under the curve is 1 and the area over a certain interval (black in the figure) gives the probability of an observation in this interval.
Distributions and relations can be illustrated graphically with frequency functions, histograms, pie charts, xy-plots etc.
Normally it is not possible to investigate the whole population of potential outcomes of interest, and instead distributions and statistics have to be estimated from samples.The uncertainty in these estimated depend both on the sampling method as well as the sample size.