Prior selection
From GusWiki
Prior selection is a necessary part of Bayesian reasoning: one must pick a -:prior before doing Bayesian updates. There are no widely-satisfactory solutions to this problem, i.e. there is no good, principled way of choosing priors.
Contents |
Criteria for Selecting Priors
Principle of Indifference
Invariance under Goodman's Grue transformations
One simple solution is to simply give a uniform distribution over all events that have been conceptualized. The problem with this is that if we rename the entities, the resulting prior changes.
According to Kevin T Kelly, this is an attempt to get a free lunch, and will always be incoherent.
The Solomonoff Prior
Solomonoff's idea was to make simpler priors more likely.
Problems:
- arbitrary distortion are necessary to make the prior a pdf, i.e. to make its integral converge.
- choice of -:Turing Machine is arbitrary (up to a constant, but still arbitrary).
Bayesian
One bad argument against Bayesianism in the Bayesianism vs Frequentism debate is that Bayesians have no principles for choosing priors. -:E.T. Jaynes, however, showed that frequentism suffers from the exact same problem in a less explicit form.
Empirical science of science
Prior selection methods can also be evaluated empirically.
See Also
Model selection Occam's Razor Bayesianism and rationality
Sources to Check
Peter Grünwald's class on MDL
To include
"Prior Selection" is sometimes cited as a philosophical problem with Bayesianism. Some reasoners (e.g. economists in the Austrian tradition) may refuse to choose a prior by claiming "radical ignorance", and they argue that picking any prior would be an ad-hoc decision.
E.T. Jaynes has shown that frequentism suffers from the same problem, under the guise of model selection.
Some ways of picking a prior:
- Empirical Prior, Meta-Scientific Experiments (best, but impractical)
- Solomonoff Prior (better but incomputable)
- Maximum Entropy Principle
- Schmidhuber's Speed Prior (computable approximation to Solomonoff's M?)
Some desirable criteria for induction methods:
Goodman's Grue and Language Invariance
references: Kevin Kelly J.B. Paris - The Uncertain Reasoners' Companion
