Theoretical ideas
From GusWiki
Information geometry, invariance to reparametrizations
Information theory, Kolmogorov Complexity and MDL
Learning theory
Good estimators are those that lead us to the truth (a.k.a. consistency) efficiently (i.e. fast convergence). likelihood maximization is often a standard (and optimal) way of achieving this, but there are settings in which other types of estimators are more desirable, especially when the likelihood is computationally intractable. See: Alternatives to the likelihood.
Philosophical rants
Computationalist Platonism the role of axioms is to discover truths, not make truths. "Absolute" or axiom-independent truths are those about whether a Turing Machine halts. We can check that a mathematical statement is "objectively meaningful" by translating it into such a form (see computable analysis). "$T \vdash \phi$" is always an "objectively meaningful" statement (as long as $T$ is recursively axiomatized), because we've relativized the truth of $\phi$ to the theory $T$. But the point is that some statements don't need to be relativized, and deserve the labels of "axiom-independent" and "objectively meaningful".
See also, from Scott Aaronson's paper: << David Deutsch, of quantum computing fame, has argued that even mathematical statements are ultimately about physical reality—since mathematics is rooted in computation, and the laws of physics determine what is and isn’t computable [16].12 Whether or not you agree with this, it does suggest the following “physical process criterion” for mathematical truth: We should expect a mathematical question to have a definite answer, if and only if we can phrase the question in terms of a physical process we can imagine. >>
Physics
- is entropy subjective? about Maxwell's Demon
- paradox in Classical Mechanics? energy vs force "is rocket-launching associative?"
- "energy = k amplitude^2" follows from energy conservation
- no substance can be totally incompressible, a "digital physics" proof: if some work is applied towards compressing an object which is completely non-responsive to it, then the universe is not storing the energy information anywhere. This violates a basic informational principle.
