Prior selection
From GusWiki
Prior selection is a necessary part of Bayesian reasoning: one must pick a -:prior before doing Bayesian updates. There are no widely-satisfactory solutions to this problem, i.e. there is no good, principled way of choosing priors.
Contents |
[edit] Criteria for Selecting Priors
[edit] Principle of Indifference
[edit] Invariance under Goodman's Grue transformations
One simple solution is to simply give a uniform distribution over all events that have been conceptualized. The problem with this is that if we rename the entities, the resulting prior changes.
According to Kevin T Kelly, this is an attempt to get a free lunch, and will always be incoherent.
[edit] The Solomonoff Prior
Solomonoff's idea was to make simpler priors more likely.
Problems:
- arbitrary distortion are necessary to make the prior a pdf, i.e. to make its integral converge.
- choice of -:Turing Machine is arbitrary (up to a constant, but still arbitrary).
[edit] Bayesian
One bad argument against Bayesianism in the Bayesianism vs Frequentism debate is that Bayesians have no principles for choosing priors. -:E.T. Jaynes, however, showed that frequentism suffers from the exact same problem in a less explicit form.
[edit] Empirical science of science
Prior selection methods can also be evaluated empirically.
[edit] See Also
Model selection Occam's Razor Bayesianism and rationality
[edit] Sources to Check
Peter Grünwald's class on MDL
[edit] To include
"Prior Selection" is sometimes cited as a philosophical problem with Bayesianism. Some reasoners (e.g. economists in the Austrian tradition) may refuse to choose a prior by claiming "radical ignorance", and they argue that picking any prior would be an ad-hoc decision.
E.T. Jaynes has shown that frequentism suffers from the same problem, under the guise of model selection.
Some ways of picking a prior:
- Empirical Prior, Meta-Scientific Experiments (best, but impractical)
- Solomonoff Prior (better but incomputable)
- Maximum Entropy Principle
- Schmidhuber's Speed Prior (computable approximation to Solomonoff's M?)
Some desirable criteria for induction methods:
Goodman's Grue and Language Invariance
references: Kevin Kelly J.B. Paris - The Uncertain Reasoners' Companion
