Introduction: The Bayesian View of Probability and Statistics

Probabilities are [44] often used to represent the chances of particular outcomes of a sampling of a random variable, when the outcome is so far indeterminate. For example, before a standard die is thrown, the score which will be obtained might take any of the integer values from $1$ to $6$ inclusive. What is already known about the score can then be represented by a probability distribution; typically, a probability $\frac{1}{6}$ is assigned to each of the possible outcomes. There is [44] a perspective on probability and statistics, known as frequentism, in which the description of random variables, whose outcomes are so far indeterminate, is seen as the only appropriate use of probability. For most of the twentieth century, frequentism was [56,44] the dominant view in the field of probability.

However, an alternative perspective exists [44]. It is common to use a probability to represent the degree to which one believes in the truth of an assertion. Contestants on the television quiz Who Wants to Be a Millionaire, who ``phone a friend'' to ask for advice, often ask the friend to estimate a percentage confidence which they have in their answer. They are using probabilities to represent degrees of belief, in the way that is meant here. The use of probabilities to represent degrees of belief is [12] technically feasible, so long as the degrees of belief satisfy three conditions, known as the Cox axioms. The viewpoint in which it is seen as epistemologically valid to use probabilities to represent degrees of belief, as well as to describe random variables, is [44] known as Bayesianism.

There is [34,71,44] a mathematical principle known as Bayes' theorem, which, for two events or propositions $E$ and $T$, relates the conditional probability $P(E\vert T)$ of $E$ being true, given that $T$ is true, and the conditional probability $P(T\vert E)$ of $T$ being true, given that $E$ is true, with the marginal probabilities $P(E)$ and $P(T)$ of $E$ and $T$, that is to say the overall probabilities which $E$ and $T$ have, each averaged over the probability distribution of truth or falsehood of the other proposition, as follows:

$$P(T\vert E)=\frac{P(E\vert T)P(T)}{P(E)}\textrm{.}$$ (7)

In a frequentist perspective, Bayes' theorem is merely an interesting generalization about the mutual correlations between random variables. In the Bayesian world-view, on the other hand, Bayes' theorem sits at the heart of epistemology, because it provides a rule for generating the posterior probability $P(T\vert E)$, representing the degree of belief in a theory $T$, in the light of observational or experimental evidence $E$, from the likelihood $P(E\vert T)$, which is the probability that $T$ assigns to the occurrence of $E$, the prior probability $P(T)$, representing the degree of belief in $T$ before the occurrence of $E$, and the marginal likelihood $P(E)$, which is the average $\sum_{T\in{}S}P(E\vert T)P(T)$, over the set $S$ of all available theories, of the likelihood; that is to say, Bayes' theorem is a rule for drawing an inference, from evidence $E$, about the truth of theory $T$.

It will be useful to define some terminology here. For the purposes of this paper, a postulate is a descriptive statement about some aspect of the behaviour of the universe, and a theory is a set of postulates simultaneously believed to be true. A hypothesis is a prediction, derived from a theory using rules of logic⁵, about the consequences of some act or omission. A value is a prescriptive statement about the desirability of some process or situation, and an ideology is a set of values simultaneously believed to be good. A policy is a decision, derived from an ideology using hypotheses, to undertake some act or omission⁶. In Bayesian analysis, hypotheses take the form of likelihood values.

Bayesian inference has been used to interpret empirical data in many natural and social science research projects, including:

• the determination of the gravitational field strength at several individual locations in East Africa, using the accumulated measurements from all the locations, to obtain more precise results than would have been available had the measurements at each location been treated separately, given that the instruments used in all cases were similar; a Bayesian method was also used for a similar problem in Europe, in which it was only possible to rely on the similarity of the measuring instruments in some cases [36],
• the comparison of a hypothesis, in which the probability of an individual, who is a member of a pair of twins, being convicted of a crime is independent of whether the twins are identical or non-identical, with another (more adjustable) hypothesis, in which the probability of conviction may depend on whether the twins are identical or non-identical [36],
• the comparison of a hypothesis, in which the probability of a word, which is in a list whose psycho-analytical gender associations were stated by Freud, having a particular grammatical gender is independent of its psycho-analytical gender, with another (more adjustable) hypothesis, in which the probability of the word having a particular grammatical gender may depend on its psycho-analytical gender [36],
• the determination of the crystallographic structure of two organic compounds, from X-ray diffraction measurements [5], and
• the determination of the magnetic properties of a multi-layer sample, from Brillouin light scattering measurements [45].

As is [73] often the case in epistemology, the Bayesian world-view represents an ideal of knowledge-seeking which may never be attainable in full. However, unlike many epistemologies, the Bayesian world-view, by virtue of being expressed in mathematical terms, lends [44,49] itself to approximation methods (appendix 5) which allow real research activity to approach the ideal. For example, Minka [49] compares the performance of several approximate Bayesian inference methods in a variety of problems, including inferring the probability of an individual suffering from heart or thyroid disease, given some observed data about the individual and data about whether members of a ``training set'' of other individuals, with given observed data, suffered from heart or thyroid disease.

The inference method described above requires a prior probability distribution: the initial prior probability distribution represents the degrees of belief in the theories in the absence of any evidence, except where the approximation in section 5.1 is in use. Subsequently, the posterior probability distribution, which represents the degrees of belief in the various theories, after any given observation, becomes [34] the prior probability distribution, for inference from the next observation. The final posterior probability distribution is, nevertheless, independent of the order in which various pieces of evidence are presented; repeated applications of Bayes' theorem (equation 7) show:

$$P(T\vert E_1,E_2) = \frac{P(E_2\vert E_1,T)P(T\vert E_1)}{P(E_2\vert E_1)}\protect$$

$$= \frac{P(E_2\vert E_1,T)P(E_1\vert T)P(T)}{P(E_2\vert E_1)P(E_1)}\protect$$

$$= \frac{P(E_1\vert E_2,T)P(E_2\vert T)P(E_1\vert T)P(T)}{P(E_1\vert T)P(E_2\vert E_1)P(E_1)}\protect$$

$$= \frac{P(E_1\vert E_2,T)P(E_2\vert T)P(T)}{P(E_1\vert E_2)P(E_2)}\protect$$

$$= \frac{P(E_1\vert E_2,T)P(T\vert E_2)}{P(E_1\vert E_2)}\protect$$

$$= P(T\vert E_2,E_1)\textrm{.}$$ (8)

Inference according to Bayes' theorem unites [36,24,34] inductive and deductive reasoning, explicitly linking the inductive act of assessing several theories $T_i$, in the light of a particular piece of evidence $E$, by calculating the posterior probabilities $P(T_i\vert E)$, with the deductive act of generating the likelihoods $P(E_j\vert T)$ of several pieces of evidence $E_j$, from a particular theory $T$.

The purpose of this paper is to arrive at some of the epistemological insights of recent decades, many of which have originated in the social sciences, from this Bayesian starting-point, and in particular to emphasize similarities between Bayesian statistics and the standpoint epistemology that has grown out of the new social movements of feminism, anti-racism, and environmentalism, especially feminism; to this end, The Science Question in Feminism [28], and the works cited therein as having contributed to the development of standpoint epistemology, will be taken to be describing the standpoint epistemology, as will Is Science Multicultural? [29], and ``Towards a Methodology for Feminist Research'' [48]⁷. Having given a mathematical foundation to these insights, it is hoped to bring some of them to bear on current directions in the physical sciences. The author's aim in all of this is to encourage more natural and social science researchers to use Bayesian statistics and standpoint epistemology, in their selection of experiments and interpretation of evidence.

As is clear from the above, Bayesian inference is [42,43] a quantitative method of determining degrees of belief. This may require some justification, because many in the standpoint epistemology movement are [60] suspicious of quantitative methods. The author believes that this suspicion arises from a definition of ``quantitative,'' used in the social sciences, into which Bayesian inference does not fall, and which is significantly narrower than that used in mathematics and in the physical sciences. In the social sciences, ``quantitative research'' means a method of investigating social phenomena in which the answer to a question, whether that question is answered by a researcher making observations, or by a member of the community being studied, to whom the question is posed by a researcher, is given by the selection of one or more of a countable, finite, and usually rather small, set of pre-defined categories. Mathematicians and physical scientists might prefer to term this ``discrete research.''

Standpoint epistemologists, and others in the new social movements, criticize discrete research on several grounds. Firstly, they argue that the pre-definition of the categories constrains the members of the community being studied to answers or descriptions, which may be influenced more by the preconceptions of the researcher who devised the categories, than by the true opinions or behaviours of the members of that community. Secondly, they argue [18] that the small number of the categories destroys any hope of obtaining a useful description of a system as complicated as those studied in the social sciences. Thirdly, they argue [6,48] that the brevity of the ``category-selection'' responses, and the interrogation-like nature of questions intended to be answered in this way, militate against the construction of a relationship of trust and shared aims between researcher and researched, which could motivate the researched to take the pains necessary to answer accurately, or indeed at all.

The author believes that the quantitative nature of the description of degrees of belief in Bayesian inference is immune to the first two criticisms. For the first criticism, as long as the degrees of belief obey the Cox axioms, no other constraint is [12] placed on them by their description by probabilities. For the second criticism, the number of possible degrees of belief in Bayesian inference, far from being small, is infinite and uncountable. The third criticism may require that, if one is trying to draw out the beliefs of others, some other description of the beliefs should be solicited either instead of, or in addition to, the Bayesian probabilities. Perhaps, in any such study, discussing inference processes, rather than merely asking about the degrees of belief, which those processes have as final results, would be useful. However, that is not the concern of this paper.