From GusWiki

See also: Publications, Tutorials

Contents 1 Current 1.1 UBC MSc - stochastic block models for ranked relational data 2 Archive 2.1 Estimating Gaussian subspaces from ICA 2.2 Discovering causal graphical models with -:ICA (with Peter Spirtes) 2.3 coupling Markov Chains (with Cosma Shalizi) 3 Class Projects at UBC 3.1 Predicting Energy Usage in Office Buildings 3.2 An Empirical Analysis of Algorithms for Bayesian Sparse Linear Regression 3.3 Experiments with Stochastic Gradient Descent: Condensations of the Real line 4 Older 4.1 2004 4.2 2003 4.3 2002 4.4 2001 4.5 2000 4.6 1990's 5 Project Proposals 6 Half-Baked Ideas 7 See also

Current

UBC MSc - stochastic block models for ranked relational data

Gustavo Lacerda, Jennifer Bryan - Recovering Block Structure from Ranked Relational Data: a simulation study of the likelihood function

in progress...

Archive

Estimating Gaussian subspaces from ICA

nothing new here

Discovering causal graphical models with -:ICA (with Peter Spirtes)

We had two papers at UAI2008:

• my paper on discovering cyclic causal models (see the video of my talk)
• Patrik Hoyer's paper on discovering causal models when error terms can be either Gaussian or non-Gaussian, which we presented as a poster.

Further directions:

• handle noisy dynamics

coupling Markov Chains (with Cosma Shalizi)

working on some mathematical questions about coupled Markov Chains, with the goal of being able to confidently say that two experimental conditions (or two pairs of people) are different with respect to the -:mutual information statistic. This could be useful for:

• discovering the structure of social networks
• testing hypotheses of the form: "the information-connectivity of these two brain regions is different under these two conditions"
• much more

Class Projects at UBC

Predicting Energy Usage in Office Buildings

paper [not publically available]^*

This data had an hourly resolution, and multiple periodicities (day, week, year) reflecting changes in weather and occupancy. I experimented with prediction methods such as interpolation using periodic kernels, and regularized autoregressions with shrinkage between neighboring hours-of-the-day.

---

^{* - Some of the information on this report is proprietary; I will censor it accordingly once I have a tool to do so.}

An Empirical Analysis of Algorithms for Bayesian Sparse Linear Regression

paper slides

I implemented an anytime algorithm, Dobra's (2009) Mode-Oriented Stochastic Search (MOSS), and measured its performance over time (measured in number of model evaluations). We compared it against a greedy stochastic search (FBMP), on instances coming from 2 instance classes. This required adapting MOSS to use the same prior as FBMP, and to use fast Bayesian updates.

Experiments with Stochastic Gradient Descent: Condensations of the Real line

paper

I experimented with Hinton's (2006) Contrastive Divergence method for training Restricted Boltzmann Machines to compress images of handwritten digits. I found that the Widrow-Hoff update step can be improved by performing an exponential (but not a polynomial) transformation of the parameter space, in the sense that this leads to lower reconstruction error when controlling for the number of iterations.

Older

2004

• "Modeling Cognitive Theories of Autism: Interpretations of the Executive Dysfunction Theory"

• "Automatic Bilingual Dictionary Building from a Parallel Corpus". The results were published at Adafre, van Hage, Kamps, Lacerda, de Rijke - "The University of Amsterdam at CLEF 2004"

• with Henrik Nordmark: "Identifying Structure in the Narratives of Portuguese-Speaking Children.": reports on experiments performed with Portuguese children at the Spaarndammer School.

2003

• with Samson de Jager: "Comparing Baltag-Moss-Solecki Update Logic with Burrows-Abadi-Needham Logic of Authentication"

• with Dr. Jacques Robin, I gave a lecture about logic in AI, including non-monotonic logic, abduction, referential opacity & modal epistemic logic, and different systems for belief revision (most of the material came from Russell&Norvig)

• independent: "Incenting Honest Reporting of Probability Estimates": a non-trivial problem with a very elegant solution which I didn't know about at the time: proper scoring rules. Available as a blog post.

2002

• independent: "An Introduction to Tuning Theory." About the small-integer-ratio theory of consonance, and the tradeoff between just intervals and transposibility: Defined utility measures. I suggest that the perception of consonance will be dependent on the instrument's harmonic series.

2001

• independent: "Stochastic Simulation of a Sexual Population with a Sickle-Cell-Anemia like disease", where the simulations show that the frequency of this sometimes-beneficial (heterozygous individuals) allele tends to reach an equilibrium.

• with Dr. Karl Voss: "Using Wavelets for Lossy Data Compression": discusses how to pick a basis, presents evaluation measures, demonstration.

2000

• with Paul Jakowski: "Non-Parametric Kernel Density Estimation in a Circular Domain": investigated the distribution of emails over the course of a day and treated the issue of smoothing.

• with Dr. Alexei Ashikhmin: "An efficient algorithm for error-correction in noisy channels": about my work at Bell Labs. On this talk, I explained algebraically why the simple Viterbi algorithm works.

1990's

• 1999: with Dr. David Farmer: "On the number of Roots of Polynomials over Finite Fields": simple algebraic number theory applied to a seemingly-unexplored problem, intuitions supported by experiments on the computer.

• 1997: high school Extended Essay: sphere falling in a viscous liquid, where I generalize Stokes Law to account for the effect of walls. An empirical law is observed and partially justified by theory.

Project Proposals

• A QED Project for Science: The Promises of Computer-Level Formalization (2006)

• Formalizing Science Intuitively (2004)

• Using weak definitions in formal mathematics

Half-Baked Ideas

My ideas at the half-bakery (old ones)

See also

• Notebooks, my attempt at a Wikipedia-like guide to my research interests. This includes:
  • basic concepts that could I teach in an introductory class (my notebooks reflect my particular way of thinking about them)
  • ideas of my own

Fun project ideas