Elicitation is the process of extracting knowledge about the parameters in the statistical model. This information can then be used to provide input for the prior distribution needed for Bayesian analysis. Several methods of prior elicitation are used in practice including the use of experts. Whereas other fields increasingly appreciate the use of experts to inform the priors, in the social sciences use of expert knowledge in this way is hardly ever used. The main reason might be that the estimates based on the data are trusted more than the subjective opinions of experts. But these arguments only hold if the data is truly a random sample from the population, which is almost never the case with limited data. In these situations, the data may be quite unreliable and I argue balancing the two sub-optimal sources of information via Bayesian statistics leads to more reliable conclusions than relying on imperfect data alone. Also, by quantifying the expert-data conflict a new type of research question can be answered: which expert, or group of experts disagree most with the data? Answering such a question opens a new area of research which can be especially useful to identify biased experts or for policy makers who can then focus their campaign on specific misconceptions between groups of experts.
You must be logged in to post a comment.