Introduction In the 1920s, Lotka (1909) and Volterra (1926) developed a model of a very simple predator-prey ecosystem. Although simple, it turns out that the Canadian lynx and showshoe hare are well represented by such a model. Furthermore, the Hudson Bay Company kept records of how many pelts of each species were trapped for almost … Continue reading Ecology, Dynamical…
#probability
23 posts
26 Jun 2016
6 Dec 2015
Introduction Let be a (hidden) Markov process. By hidden, we mean that we are not able to observe it. And let be an observable Markov process such that That is the observations are conditionally independent given the state of the hidden process. As an example let us take the one given in Särkkä (2013) where … Continue reading Naive Particle…
12 Aug 2015
Introduction Suppose we wish to model a process described by a differential equation and initial condition But we wish to do this in the presence of noise. It’s not clear how do to this but maybe we can model the process discretely, add noise and somehow take limits. Let be a partition of then we … Continue reading Stochastic Integration
27 Apr 2015
Introduction Suppose you want to sample from the truncated normal distribution. One way to do this is to use rejection sampling. But if you do this naïvely then you will run into performance problems. The excellent Devroye (1986) who references Marsaglia (1964) gives an efficient rejection sampling scheme using the Rayleigh distribution. The random-fu package … Continue reading Rejection Sampling
25 Dec 2014
Let an be stopping times and let the filtration on which they are defined be right continuous. Then , , and are stopping times where . For the first we have and both the latter are in by the definition of a stopping time. Similarly for the second . For the fourth we have since … Continue reading Stopping Times
9 Sept 2014
Summary An extended Kalman filter in Haskell using type level literals and automatic differentiation to provide some guarantees of correctness. Population Growth Suppose we wish to model population growth of bees via the logistic equation We assume the growth rate is unknown and drawn from a normal distribution but the carrying capacity is known and … Continue reading Fun with…
26 Aug 2014
Introduction Suppose we have a vector of weights which sum to 1.0 and we wish to sample n samples randomly according to these weights. There is a well known trick in Matlab / Octave using sampling from a uniform distribution. num_particles = 2*10^7 likelihood = zeros(num_particles,1); likelihood(:,1) = 1/num_particles; [_,index] = histc(rand(num_particles,1),[0;cumsum(likelihood/sum(likelihood))]); s = sum(index); … Continue reading Haskell Vectors…
6 Aug 2014
Introduction Suppose we have particle moving in at constant velocity in 1 dimension, where the velocity is sampled from a distribution. We can observe the position of the particle at fixed intervals and we wish to estimate its initial velocity. For generality, let us assume that the positions and the velocities can be perturbed at … Continue reading Fun with…
19 Jul 2014
Suppose we wish to estimate the mean of a sample drawn from a normal distribution. In the Bayesian approach, we know the prior distribution for the mean (it could be a non-informative prior) and then we update this with our observations to create the posterior, the latter giving us improved information about the distribution of … Continue reading Fun with…
11 May 2014
Introduction I have seen Hölder’s inequality and Minkowski’s inequality proved in several ways but this seems the most perspicuous (to me at any rate). Young’s Inequality If and such that then A and satisfying the premise are known as conjugate indices. Proof Since is convex we have Substituting in appropriate values gives or Now take … Continue reading Hölder’s and…
7 Dec 2013
Introduction About a year ago there was a reddit post on the Ising Model in Haskell. The discussion seems to have fizzled out but Ising models looked like a perfect fit for Haskell using repa. In the end it turns out that they are not a good fit for repa, at least not using the … Continue reading Haskell, Ising,…
30 Apr 2013
Introduction Having shown how to use automated differentiation to estimate parameters in the case of linear regression let us now turn our attention to the problem of classification. For example, we might have some data about people’s social networking such as volume of twitter interactions and number of twitter followers together with a label which … Continue reading Logistic Regression…
26 Apr 2013
Introduction Automated differentiation was developed in the 1960’s but even now does not seem to be that widely used. Even experienced and knowledgeable practitioners often assume it is either a finite difference method or symbolic computation when it is neither. This article gives a very simple application of it in a machine learning / statistics … Continue reading Regression and…
4 Nov 2012
One way of visualizing a distribution is to take lots of samples for it and plot the resulting histogram. To do this we use the diagrams package. {-# LANGUAGE TupleSections #-} import Diagrams.Prelude import Diagrams.Backend.Cairo.CmdLine import Data.Colour (withOpacity) To generate the actual samples (in this case from the beta distribution) we use the random-fu and … Continue reading Plotting a…
15 Aug 2012
A Simple Example In Section 7.1 of Doing Bayesian Data Analysis, John Kruschke gives a simple example of the Metropolis algorithm, in which we generate samples from a distribution without knowing the distribution itself. Of course the example is contrived as we really do know the distribtion. In the particular example, for i = 1…n where n … Continue reading…
7 May 2012
A few weeks ago, two of my colleagues and I had a birthday in the same week. What are the odds of that? There’s about 25 people in the department in which I work. We consider every outcome where pairs of people share birthdays but no 3 or more people share birthdays. Then we can … Continue reading A Triple…
1 Apr 2012
Suppose we want to find the price of a European call option. Then we need to solve the Black-Scholes equation: Although this particular equation can be solved explicitly, under more realistic assumptions we have to rely on numerical methods. We can approximate the partial differential equation by a difference equation (the minus sign on the … Continue reading Solving a…
17 Mar 2012
Protter(Protter 2004) notes that if some assumptions are made on how fast the mesh (see below for a definition) converges then the calculation of the almost sure value of the quadratic variation of Brownian Motion can be done using Chebyshev’s Inequality and the Borel-Cantelli Lemma. Williams and Rogers(Rogers and Williams 2000) also set this as … Continue reading The Quadratic…
28 Jan 2012
This is fairly standard. Let Xi be a symmetric random walk and let Ei be the event that Xi = 0. Then and . The diagram shows the number of ways of getting to a particular node. The dashed arrows are meant to represent that the random walk then continues. In general, at time 2n, the number … Continue reading…
17 Jul 2011
Doob’s Inequality for Right (Left) Continuous Martingales In all the proofs I have seen, some of the details are elided; here is my proof with all the details. Proof: Let be an increasing set of finite sets whose union is dense in and note that by right (left) continuity, we have: Thus: By monotone convergence … Continue reading Doob’s Inequality…
3 Jul 2011
Doob’s Inequality Let be a (discrete) non-negative sub-martingale, bounded in i.e. with for some fixed . Define by , and . Then . Further, . Proof: Fix and define the stopping time . Then which is -measurable and thus by The Stopping Time Lemma since Also by Fubini Using both of these facts and Fubini … Continue reading Doob’s Inequality
13 Mar 2011
is a martingale. We need to show: or alternatively that But is independent of and so
27 Dec 2008
See 3.12 of Williams. Let where and . and . is right continuous. Then by the proof . Thus and is therefore an upper bound (if not then such that and but by monotonicity ). Therefore . On the other hand suppose then (if not then but then is a lower bound for all such … Continue reading Skorokhod Representation…