You might recall that before my immensely long series on ways we could make C# a probabilistic programming language, I did a short series on how we can automatically computed the exact derivative in any direction of a real-valued function … Continue reading →
#fixing random
26 posts
11 Dec 2019
8 Nov 2019
Source code for this episode is here. Welcome to this special bonus episode of Fixing Random, the immensely long blog series where I discuss ways to add probabilistic programming features into C#. I ran into an interesting problem at work … Continue reading →
27 Jul 2019
All right, let’s finish this thing off! First, I want to summarize, second I want to describe a whole lot of interesting stuff that I did not get to, and third, I want to give a selection of papers and … Continue reading →
15 Jul 2019
Let’s sum up the last few episodes: Suppose we have a distribution of doubles, p, and a function f from double to double. We often want to answer the question “what is the average value of f when it is given samples … Continue reading →
8 Jul 2019
Last time on FAIC we were attacking our final problem in computing the expected value of a function f applied to a set of samples from a distribution p. We discovered that we could sometimes do a “stretch and shift” of … Continue reading →
1 Jul 2019
Last time on FAIC we finally wrote a tiny handful of lines of code to correctly implement importance sampling; if we have a distribution p that we’re sampling from, and a function f that we’re running those samples through, we can compute … Continue reading →
24 Jun 2019
One more time! Suppose we have our nominal distribution p that possibly has “black swans” and our helper distribution q which has the same support, but no black swans. We wish to compute the expected value of f when applied to samples … Continue reading →
17 Jun 2019
Last time on FAIC we deduced the idea behind the “importance sampling” technique for determining the average value of a function from double to double — call it f — when it is applied to samples from a possibly-non-normalized weighted distribution of … Continue reading →
10 Jun 2019
Last time on FAIC we implemented a better technique for estimating the expected value of a function f applied to samples from a distribution p: Compute the total area (including negative areas) under the function x => f(x) * p.Weight(x) … Continue reading →
3 Jun 2019
Last time on FAIC I showed why our naïve implementation of computing the expected value can be fatally flawed: there could be a “black swan” region where the “profit” function f is different enough to make a big difference in … Continue reading →
28 May 2019
Last time on FAIC we reviewed the meaning of “expected value”: when you get a whole bunch of samples from a distribution, and a function on those samples, what is the average value of the function’s value as the number … Continue reading →
20 May 2019
Last time in this series we saw that we could compute a continuous posterior distribution when given a continuous prior and a discrete likelihood function; I hope it is clear how that is useful, but I’d like to switch gears … Continue reading →
13 May 2019
Last time on FAIC I posed and solved a problem in Bayesian reasoning involving only discrete distributions, and then proposed a variation on the problem whereby we change the prior distribution to a continuous distribution, while preserving that the likelihood … Continue reading →
10 May 2019
[It is] a spectacular vindication of the principle that each individual coin spun individually is as likely to come down heads as tails and therefore should cause no surprise that each individual time it does. Thus Guildenstern (or is it … Continue reading →
6 May 2019
Last time on FAIC we implemented a technique for sampling from a non-normalized target PDF: Find an everywhere-larger helper PDF that we can sample from. Sample from it. Accept or reject the sample via a coin flip with the ratio … Continue reading →
2 May 2019
Last time on FAIC we went through a loose, hand-wavy definition of what it means to have a “weighted” continuous distribution: our weights are now doubles, and given by a Probability Distribution Function; the probability of a sample coming from … Continue reading →
29 Apr 2019
We’ve been mostly looking at small, discrete distributions in this series, but we started this series by looking at continuous distributions. Now that we have some understanding of how to solve probability problems on simple discrete distributions and Markov processes, … Continue reading →
26 Apr 2019
Last time on FAIC we implemented the Markov process distribution, which is a distribution over state sequences, where the initial state and each subsequent state is random. There are lots of applications of Markov processes; a silly one that I’ve … Continue reading →
23 Apr 2019
[Code for this episode is here.] So far in this series we’ve very briefly looked at continuous distributions on doubles, and spent a lot of time looking at discrete distributions with small supports. Let’s take a look at a completely … Continue reading →
17 Apr 2019
So… I’ve got good news and bad news. The good news is: I’ve described an interface for discrete probability distributions and implemented several distributions. I’ve shown how projecting a distribution is logically equivalent to the LINQ Select operator. I’ve shown … Continue reading →
15 Apr 2019
[Code for this episode is here.] Last time in this series I left you with several challenges for improving our DSL for imperative probabilistic workflows. But first, a puzzle: Question One: You are walking down the street when you see … Continue reading →
11 Apr 2019
Last time in this series I proposed a stripped-down DSL for probabilistic workflows. Today, let’s see how we could “lower” it to ordinary C# 7 code. I’ll assume of course that we have all of the types and extension methods that … Continue reading →
8 Apr 2019
Without further ado, here’s my proposed stripped-down C# that could be a DSL for probabilistic workflows; as we’ll see, it is quite similar to both enumerator blocks from C# 2 and async/await from C# 5. (Code for this episode can … Continue reading →
4 Apr 2019
I’ve got no code for you this time; instead here are some musings about language design informed by our discussion so far. One of the most important questions to ask when designing a language feature is: what should the balance … Continue reading →
2 Apr 2019
Before that silly diversion I mentioned that we will be needing the empty distribution; today, we’ll implement it. It’s quite straightforward, as you’d expect. [Code for this episode is here.] public sealed class Empty<T> : IDiscreteDistribution<T> { public static readonly Empty<T> Distribution = new Empty<T>(); private Empty() { } public T Sample() => throw new Exception(“Cannot sample from empty distribution”);…
1 Apr 2019
I just thought of a really cute application of the stochastic workflow technology we’ve been working on; most of the series has already been written but it fits in here, so I’m going to insert this extra bonus episode. We’ll … Continue reading →