This post assumes familiarity with our primer on Kolmogorov complexity. We recommend the uninformed reader begin there. We will do our best to keep consistent notation across both posts. Kolmogorov Complexity as a Metric Over the past fifty years mathematicians have been piling up more and more theorems about Kolmogorov complexity, and for good reason. One of the main interpretations…
4 Dec 2012
Search is front-and-center in the new SoundCloud, key to the consumer experience. We’ve made the search box one of the first things you see…
One of the four value statements from the Agile Manifesto states: Individuals and interactions over processes and tools. Two principles supporting the Agile Manifesto are: Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development…
3 Dec 2012
You can get good traceability for your builds by using Go's custom pipeline labels effectively.
2 Dec 2012
Define the Ramsey number $ R(k,m)$ to be the minimum number $ n$ of vertices required of the complete graph $ K_n$ so that for any two-coloring (red, blue) of the edges of $ K_n$ one of two things will happen: There is a red $ k$-clique; that is, a complete subgraph of $ k$ vertices for which all edges…
27 Nov 2012
The structure of a programming language reflects the challenges and solutions the designers decided to address. Each designer coming with his/her own background decides to tackle some specific issues in a novel way and/or often decides to borrow existing paradigms from other languages. We can’t, then, fairly judge a language without understanding what problem the language designer was trying to…
Last time around, I claimed that taking time to set up infrastructure for builds and baby-sitting a CI server was a less than ideal use of your time. However, I will also say this: I deeply believe that the ability to reliably release your application, early & often, with a degree of quality you feel comfortable with, is an essential…
26 Nov 2012
A couple of days ago I wrote about how to build a simple and non intrusive AJAX status with CoffeeScript in RubyOnRails.
Our software has bugs. We fix most of our bugs. So far our team probably sounds a lot like yours. Where we may begin to differ is in how we go about deciding which bugs to fix and how we signal to ourselves that we need to focus on bugs above other forms of valuable work. Exposure: Another view on…
23 Nov 2012
My desktop
21 Nov 2012
20 Nov 2012
18 Nov 2012
Not sure how I managed to miss this, but I’m watching this Tumblr presentation and they talk about their projects named after Arrested Development topics: Gob, Parmesan, Buster, Jetpants, Oscar, George and Motherboy. Still, the best software project name is probably still Apple’s BHA.
17 Nov 2012
I spoke at PubCon 2012 last month in Las Vegas. A couple people have asked for my slides, so here they are: Several of the slides have links to additional information, in case you’re interested. We announced the disavow links tool during my session so that’s what a lot of the slides are about.
14 Nov 2012
If you are currently a software engineer you need to realize two things: Now is an awesome time to be a software engineer (probably the best time ever). Your job might not be well suited for you. I’ll show you why we are lucky bastards, why we aren’t so good at picking the right jobs and some hints on how…
#4 in the Continuous Delivery webinar series Automated testing is a cornerstone of Continuous Delivery. Automated functional testing ensures that your software is working across your value stream and that your organization has fast feedback when it's not. Maintaining automated acceptance tests, once a team has more than a few tests, isn't trivial however. This webinar covers best practices for…
13 Nov 2012
In this first of a six-part series we'll do a deep dive into doing Continuous Delivery with Go The official Go documentation explains some key concepts with the help of a nice diagram:
A Go Pipeline does not necessarily map one-to-one with what is referred to as the automated deployment pipeline in continuous delivery literature. The automated deployment pipeline is essentially the end-to-end CD value stream. This end to end value stream is often better modeled using multiple Go Pipelines. For very simple cases, it may be enough to model the entire value…
11 Nov 2012
Last time we investigated the (very unintuitive) concept of a topological space as a set of “points” endowed with a description of which subsets are open. Now in order to actually arrive at a discussion of interesting and useful topological spaces, we need to be able to take simple topological spaces and build them up into more complex ones. This…
10 Nov 2012
Problem: Prove there are infinitely many primes Solution: Denote by $ \pi(n)$ the number of primes less than or equal to $ n$. We will give a lower bound on $ \pi(n)$ which increases without bound as $ n \to \infty$. Note that every number $ n$ can be factored as the product of a square free number $ r$…
8 Nov 2012
Last week two other Twilio engineers and I went to the Columbia engineering career fair. We had a great time and talked to a lot of really smart people. However I was surprised at some of the naive mistakes students made when we were talking. We're there to try to hire students and students are […]
4 Nov 2012
In our last primer we looked at a number of interesting examples of metric spaces, that is, spaces in which we can compute distance in a reasonable way. Our goal for this post is to relax this assumption. That is, we want to study the geometric structure of space without the ability to define distance. That is not to say…
2 Nov 2012
During RubyConf 2012 in Denver, Colorado Matt Aimonetti gave a talk entitled Ruby Vs. The World. ##Description of the talk: Ruby is an awesome programming language, it’s so pleasing you probably haven’t seriously looked at other languages since you switched. The programming world is evolving fast, new languages are created daily, new trends are emerging. Let’s take some time to…
The web has come a long way. We have APIs that allow developers to create amazing applications quickly, and browser technologies have…
Where’d my tail go? - Using Continuous Delivery principles to shorten our release tail This is the story of how the Thoughtworks Studios Mingle team used continuous delivery principles to reduce the testing period at the end of our release cycle from 2-3 weeks to 1 day.
I am a fan of teams where roles and people are disconnected. A model where all roles are hats and people can wear any hat that they are capable of wearing. Basically, try getting an m:n relation between roles and people. However, one of the key to this model is, like I said, capability.
1 Nov 2012
For October 2012, I tried to practice the ukulele every day. I ended up doing more traveling than I expected, but I managed to play ukulele most of the days. I’m still a total beginner, but it was a lot of fun! My favorite song to play so far is M.T.A. by the Kingston Trio. […]
31 Oct 2012
My fellow Americans, I’d like to introduce my latest Halloween costume: Matt Romney! My five-point plan for the Mitt Romney Halloween costume went like this: Start with a suit and tie. Put a little silver in my sideburns. My wife used some silver and white eye shadow. Swoosh the hair up a little bit and […]
Is Continuous Delivery all about the nuts and bolts of CI? Infrastructure automation and the design of build pipelines? The biggest constraint to being able to incrementally roll out new features are oftentimes the problems in the design of the system itself. And this is often drowned in the CD conversation. To enable rapid, frequent release, it is essential to…
29 Oct 2012
Something that pops up pretty frequently is to implement time decay, especially where you have recursive chains of jobs. For instance, say you want to keep track of a popularity score. You calculate today’s output by reading yesterday’s output, discounting it by $$ exp(-lambda Delta T) $$ and then adding some hit count for today. Typically you choose $$ lambda…
How do move from releasing every few months, to weeks and then perhaps daily? Is “Fail fast, Learn fast” the way to get there? Enough of talk… I’d like to see how it really can be done? Please do join us for our Go User Group webinar (GUG) on Tue, Oct 30, 2012. This interactive (plenty of Q&A time) webinar…
25 Oct 2012
[Webinar] Why Continuous Delivery Needs Devops and Why Devops Needs Infrastructure-as-Code
Thoughtworks InsightsBoth continuous delivery and devops have become mainstream, at least in terms of mindshare. As a result, a lot of vendors have jumped onto the bandwagon. Most products that have anything to do with deployment now try to associate themselves with devops and continuous delivery. In this webinar, Sriram will try to clear the air in a product independent manner.…
23 Oct 2012
Harnessing the power of marketing for positive social change.
Ever since Facebook and GMail began misusing the grip icon as a navicon, countless UI frameworks and websites have started to mix the icon metaphors as well. Here are the differences, and why it’s important to be consistent… The difference between gripping then pulling and tapping then navigating
22 Oct 2012
This is the third in a series of interviews on continuous delivery, this time with Elisabeth Hendrickson. You can see the first one, with Jesse Robbins, on the Thoughtworks Studios Blog, and the second, with John Allspaw, here.
“it’s possible for good people, in perversely designed systems, to casually perpetrate acts of great harm on strangers, sometimes without ever realising it.” — Ben Goldacre, Bad Pharma, p. xi
21 Oct 2012
Every once in a while I read something along the lines of: “most developers just want to write new features, they don’t want to work with maintenance and bug-fixing”. If that’s true, then most developers are missing out on the fun … Continue reading →
I’m shamelessly promoting my first major open source project. Luigi is a Python module that helps you build complex pipelines of batch jobs, handle dependency resolution, and create visualizations to help manage multiple workflows. It also comes with Hadoop support built in (because that’s where really where its strength becomes clear). We use Luigi internally at Spotify to run thousands…
17 Oct 2012
The case of Apple sometimes comes up in discussions around continuous delivery and the lean startup. For example, Richard Durnall described Apple’s strategy to me on Twitter as follows: Brilliant and unwavering product vision from a few amazing folks going to market infrequently with huge ceremony.
16 Oct 2012
11 Oct 2012
My grandfather has been seriously ill this week, so I’m flying tonight to be with him in Tennessee. If you’re waiting on me for a reply about something, it will probably need to wait. I still hope to attend PubCon next week but I can’t promise that I’ll be able to make it. Added: My […]
Agile enthusiasts are quick to make fun of people who pick and choose among various agile concepts, philosophies, techniques, and tools, rather than working hard to be "pure." We have a word for people who do that: dilettantes. But when we use two words we call them "cafeteria agilists."
Value Chains and Value Streams: Strategic Maps Are Better For Project Inception
Thoughtworks InsightsI was working on some training materials this week, and I needed to whip up as-is and to-be business process maps. These would be used to help trainees visualize how the software they were going to build in an agile way during the training would transform the fictional business for which it was being written.
9 Oct 2012
During Aloha RubyConf 2012 in Honolulu, Hawaii Matt Aimonetti gave a talk entitled mmm..mruby or why yet another Ruby implementation. ##Description of the talk: mruby is Matz’ new Ruby implementation, it’s not cooler than node.js, it doesn’t natively support Hypstermedia, it looks just like the good old Ruby. So why should we, as a community care? Matt’s talk is divided…
8 Oct 2012
Last time we investigated the k-nearest-neighbors algorithm and the underlying idea that one can learn a classification rule by copying the known classification of nearby data points. This required that we view our data as sitting inside a metric space; that is, we imposed a kind of geometric structure on our data. One glaring problem is that there may be…
5 Oct 2012
During PulsoConf 2012 in Bogotá, Colombia Matt Aimonetti gave a talk entitled Tower of Babel: a tour of programming languages. ##Description of the talk: Programming languages affect the way one looks and solves problems. But comparing programming languages isn’t as simple as drawing a table comparing features. In his talk, Matt shows what he likes, dislikes, the philosophy and concrete…
I’ve worked with teams who have some fairly sophisticated card walls, in terms of columns (some even warranted!). For my personal card wall, I really just want an at-a-glance view for my day: just the work that I’m dealing with today, nothing more, nothing less.
“We need to change our culture” – how many times have I heard that when I am working with organizations trying to adopt an agile approach? Well, I hear it almost every time I give one of our workshops, and it has driven me to change the way I start each workshop by trying to discover the culture of this…
4 Oct 2012
On a recent vacation I visited the Mingei International Museum in San Diego. During a tour by the museum director we were looking at a mid-1930′s Santo Domingo Pueblo (New Mexico) necklace. “Interesting about this necklace,” the director said, “are the ‘nots’– not coral, not obsidian (but old melted phonograph records for the black element). During the depression the Native…
3 Oct 2012
Demystifying memory: RES is real, VIRT is virtual.
2 Oct 2012
Numberphile posted a video today describing a neat trick based on complete sequences: The mathematics here is pretty simple, but I noticed at the end of the video that Dr. Grime was constructing the cards by hand, when really this is a job for a computer program. I thought it would be a nice warmup exercise (and a treat to…
On Saturday, we hosted Story Hack Boston along with P2PU and Mashery at the MIT Media Lab. About 50 people from content and technical backgrounds joined us to create new story telling experiences. The crowd was pretty evenly split, which made for a lot of awesome collaboration.
1 Oct 2012
Last month I did a secret 30 day challenge: everyday I did something nice for my wife. It could be surprising her with flowers, doing some chore around the house without her asking, or just trying to be present and focused when she wanted to talk. This challenge turned out really well. At first, I […]
This is the second in a series of interviews on continuous delivery, this time with John Allspaw, one of my co-authors on the Devops Cookbook. These interviews will eventually be put together – along with some additional exclusive material – into a set of LiveLessons, with the royalties going to Black Girls Code.
29 Sept 2012
Bazaarvoice R&D had a great year of intensive university recruiting with 12 interns joining our teams last summer and working side-by-side with the developers on our products. We have further expanded the program this year to accommodate two co-op positions for students from the University of Waterloo. The influx of fresh ideas and additional energy […]
26 Sept 2012
Problem: Prove there are infinitely many prime numbers. Solution: First recall that an arithmetic progression with difference $ d$ is a sequence of integers $ a_n \subset \mathbb{Z}$ so that for every pair $ a_k, a_{k+1}$ the difference $ a_{k+1} – a_k = d$. We proceed be defining a topology on the set of integers by defining a basis $…
The New York Times has a big new feature up explaining why data centers at Google, Facebook etc. waste tons of electricity. Diego Doval, a former CTO at a popular startup, skewers the NYT with a 5,000 word critique of the factual inaccuracies in the post. Doval is right on with his critique, but there's […]
25 Sept 2012
I jokingly say - “If you do not have a flaky functional tests build, you are not doing anything real”. I’ve spent a good amount of my professional career writing a lot of functional tests. I have interacted with a lot of teams in Thoughtworks trying to understand their functional testing issues. The most common issue in all these teams…
23 Sept 2012
When I found out about the book “How Google Tests Software“, it didn’t take long until I had ordered a copy. I find it quite fascinating to read about how Google does things, whether it is about their development process, their … Continue reading →
In my experience as a designer on complex software projects, I have come to realize that all people involved on a product team (developers, designers, and stakeholders) need to share certain universal values which influence the design of a software product.
21 Sept 2012
20 Sept 2012
Scala is a very interesting programming language. It has for goal to provide both Object Oriented and Functional Programming paradigms. Now Scala isn’t the only recent programming language out there mixing the two paradigms. Ruby, JavaScript and Clojure are other examples of popular languages implementing both functional and OO programming patterns. Of course, they each have a different take on…
#2 in the Continuous Delivery webinar series In this talk, Jez discusses how devops can be implemented in large organizations. He will provide a selection of the common issues with devops adoption, such as managing risk, compliance with regulation and frameworks such as ITIL, and meeting auditing requirements.
18 Sept 2012
We’re making some changes to how we manage our App Gallery and wanted to take some time to explain them to you, our developer community. The…
How many times have you heard people say - “[Your favorite build server] is just a glorified cron job!”? I am sure if you think about this statement for a few minutes, you’ll find that there is something wrong with it. If it is true you could just start using a cron job. But you don’t! There must be more…
“All of us like to think that human affairs are essentially rational. … The wealth of experience that fails to support this notion never seems to faze us. … That human affairs usually work not rationally but paradoxically.”
In this short video Dan Gilmer from Ancestry.com talks about how Go enabled them to optimize their Continuous Delivery processes. They did so by failing fast and learning fast. "During one day we were able to get the code out to production 5 times. But it took about 40 "fast fails" to be able to get those 5 good production…
I’ve spent most of my software career, 12 of 13 years, on the delivery side - the bulk of it in roles where it was my butt if we didn’t deliver or expectations weren’t met (or reset). One of the toughest delivery impediments I had to constantly deal with was the ubiquitous silo. Assuming you have a functional* team, based…
In our Mingle Product development we use Go extensively for our testing activities. We find it very helpful and here’s why:
I was just reflecting a bit on why I like Go (from Thoughtworks Studios). I had previously been using Jenkins (which I love), and while I still love Jenkins and would happily use it again, here are some of the reasons why Go wins in my book for trying to do continuous delivery.
17 Sept 2012
In moving to CD, teams often experience some early success. Cutting lead time from 6 months to 3 months turns out to be easy. That’s just smart scope management and getting people to actually talk to each other. But going from months to weeks, or even to days? Most teams hit a wall. This proves to be a chasm many…
My succinct summary of the Continuous Delivery book is "fail fast, learn fast". Pavan (former Go and Twist developer) and I have been testing this four-word phrase with people all over the CD continuum.
16 Sept 2012
This post comes in preparation for a post on decision trees (a specific type of tree used for classification in machine learning). While most mathematicians and programmers are familiar with trees, we have yet to discuss them on this blog. For completeness, we’ll give a brief overview of the terminology and constructions associated with trees, and describe a few common…
14 Sept 2012
Here at Bazaarvoice, we’re constantly focused on improving the user experience for our products. From the initial email invitation, to the submission form, to the way in which reviews are presented, we want to make sure that our interfaces are as flexible and intuitive as possible. Part of my job on the mobile team at […]
13 Sept 2012
This blog post shows you how to build a HTML5 chat app for both web and mobile, that also includes a chat user buddy list using Presence.
This blog post shows you how to build a HTML5 chat app for both web and mobile, that also includes a chat user buddy list using Presence.
12 Sept 2012
10 Sept 2012
When starting a project, or a new phase of an existing project, it is common for teams to try and capture a decent breadth of stories and prioritize and estimate them to form a backlog. From that backlog the team can then start to organize the stories to form some sort of plan and an idea of the overall size.
8 Sept 2012
Unlock your creativity: A playful guide to entering the open mode.
7 Sept 2012
I’m planning to record some new webmaster questions next week. I made a Google Moderator page where you can submit topics and vote for different questions. Please ask your questions in on the Google Moderator page, not in the comments here. When the comments are in Moderator, people can vote them up and down.
Sonar is a popular platform to manage code quality. This write-up details how to set up Go to do sonar analysis and then view the report from Go. The write-up assumes that sonar maven plugin will be used to do analysis. Let us start by having a pipeline with two stages, “build” and “test”. Set up a new stage "quality".…
4 Sept 2012
PubNub Channel Presence meant we also needed to provide a way for you to debug and learn about the Presence Event Delivery Format.
PubNub Channel Presence meant we also needed to provide a way for you to debug and learn about the Presence Event Delivery Format.
"High school success: A reliable predictor of life achievement, but not revolutionary thinking."
Stripe (https://stripe.com/) held their second capture the flag event, this time the CTF was dedicated to web-based vulnerabilities and exploits. As a new Security Engineer here at BV the timing of this was perfect. It allowed me to use it as a vehicle for awareness and to ramp up curiosity, interest and even excitement for […]
31 Aug 2012
Task for today was making the NuGet repository of TeamCity available in our local network. Sounds easier as it was as our TeamCity instance is available from the Internet but you can only access the rontend with valid domain credentials (LDAP/Active Directory authentication). Enabling the guest account feature in TeamCity would […] The post TeamCity: Make the NuGet repository available…
Democracy at work: Is it time to vote for your boss?
#1 in the Continuous Delivery webinar series This talk will introduce the principles and practices of Continuous Delivery, an approach pioneered by companies like Facebook, Flickr and Thoughtworks, that aims to make it possible for an organization to deliver frequently (weekly, daily or even hourly) and confidently. It uses idea -> live (the time from idea being conceived until the…
30 Aug 2012
Later this day I switched over to another task. TeamCity provides a fully functional NuGet repository server. My plan was migrating our own libraries and some 3rd party libraries from a direct dependency (binary file or as Git submodule) inside the project Git repositores to a system like Maven or Ivy […] The post NuGet and TeamCity: Repository for third…
WSUS: Moving from Windows Internal Database to external Microsoft SQL Server 2008 and receiving “Token-based server access validation failed with an infrastructure error”
SchakkoToday I had to move the WSUS internal database to one of our backend database servers. Microsoft has a good instruction how to do this, nevertheless I ran into a problem. Microsoft SQL Server 2008 did not allow me to add the machine account of our WSUS frontend server (let […] The post WSUS: Moving from Windows Internal Database to…