Dan Luu

28 Oct 2024 29 min read

There's a common narrative that Microsoft was moribund under Steve Ballmer and then later saved by the miraculous leadership of Satya Nadella. This is the dominant narrative in every online discussion about the topic I've seen and it's a commonly expressed belief "in real life" as well. While I don't have anything negative to say about Nadella's leadership in this…

11 Aug 2024 24 min read

About eight years ago, I was playing a game of Codenames where the game state was such that our team would almost certainly lose if we didn't correctly guess all of our remaining words on our turn. From the given clue, we were unable to do this. Although the game is meant to be a word guessing game based on…

16 Jun 2024 34 min read

There've been regular viral stories about ML/AI bias with LLMs and generative AI for the past couple years. One thing I find interesting about discussions of bias is how different the reaction is in the LLM and generative AI case when compared to "classical" bugs in cases where there's a clear bug. In particular, if you look at forums or…

26 May 2024 137 min read

From 2011-2012, the FTC investigated the possibility of pursuing antitrust action against Google. The FTC decided to close the investigation and not much was publicly known about what happened until Politico released 312 pages of internal FTC memos that from the investigation a decade later. As someone who works in tech, on reading the memos, the most striking thing is…

16 Mar 2024 52 min read

In 2017, we looked at how web bloat affects users with slow connections. Even in the U.S., many users didn't have broadband speeds, making much of the web difficult to use. It's still the case that many users don't have broadband speeds, both inside and outside of the U.S. and that much of the modern web isn't usable for people…

18 Feb 2024 241 min read

If I ask myself a question like "I'd like to buy an SD card; who do I trust to sell me a real SD card and not some fake, Amazon or my local Best Buy?", of course the answer is that I trust my local Best Buy1 more than Amazon, which is notorious for selling counterfeit SD cards. And if…

7 Feb 2024 12 min read

On large platforms, it's impossible to have policies on things like moderation, spam, fraud, and sexual content that people agree on. David Turner made a simple game to illustrate how difficult this is even in a trivial case, No Vehicles in the Park. If you haven't played it yet, I recommend playing it now before continuing to read this document.…

29 Jan 2024 44 min read

This is a set of notes on the Quinn Emanuel report on Cruise's handling of the 2023-10-02 accident where a Cruise autonomous vehicle (AV) hit a pedestrian, stopped, and then started moving again with the pedestrian stuck under the bottom of the AV, dragging the pedestrian 20 feet. After seeing some comments about this report, I read five stories on…

25 Jan 2024 18 min read

There's a class of comment you often see when someone makes a popular thread on Mastodon/Twitter/Threads/etc., that you also see on videos that's basically "Why make a Twitter thread? This would be better as a blog post" or "Why make a video? This would be better as a blog post". But, these comments are often stronger in form, such as:…

30 Dec 2023 87 min read

In The birth & death of search engine optimization, Xe suggests Here's a fun experiment to try. Take an open source project such as yt-dlp and try to find it from a very generic term like "youtube downloader". You won't be able to find it because of all of the content farms that try to rank at the top for…

11 Dec 2022 6 min read

This is a transcription of videos Elon Musk's appearance on stage with Dave Chapelle using OpenAI's Whisper model with some manual error corrections and annotations for crowd noise. As with the Exhibit H Twitter text message release, there are a lot of articles that quote bits of this, but the articles generally missing a lot of what happened and often…

1 Oct 2022 59 min read

This is a scan/OCR of Exhibits H and J from the Twitter v. Musk case, with some of the conversations de-interleaved and of course converted from a fuzzy scan to text to make for easier reading. I did this so that I could easily read this and, after reading it, I've found that most accountings of what was said are,…

12 Sept 2022 149 min read

I've been reading a lot of predictions from people who are looking to understand what problems humanity will face 10-50 years out (and sometimes longer) in order to work in areas that will be instrumental for the future and wondering how accurate these predictions of the future are. The timeframe of predictions that are so far out means that only…

6 Apr 2022 9 min read

Wave is a $1.7B company with 70 engineers1 whose product is a CRUD app that adds and subtracts numbers. In keeping with this, our architecture is a standard CRUD app architecture, a Python monolith on top of Postgres. Starting with a simple architecture and solving problems in simple ways where possible has allowed us to scale to this size while…

14 Mar 2022 39 min read

There's a cocktail party version of the efficient markets hypothesis I frequently hear that's basically, "markets enforce efficiency, so it's not possible that a company can have some major inefficiency and survive". We've previously discussed Marc Andreessen's quote that tech hiring can't be inefficient here and here: Let's launch right into it. I think the critique that Silicon Valley companies…

21 Feb 2022 28 min read

[Click to collapse / expand section on sports] Here are some notes from talent scouts: Recruit A: ... will be a real specimen with chance to have a Dave Parker body. Facially looks like Leon Wagner. Good body flexibility. Very large hands. Recruit B: Outstanding physical specimen – big athletic frame with broad shoulders and long, solid arms and leg.…

2 Feb 2022 33 min read

This was co-authored with Yao Yue This is a collection of information on severe (SEV-0 or SEV-1, the most severe incident classifications) incidents at Twitter that were at least partially attributed to cache from the time Twitter started using its current incident tracking JIRA (2012) to date (2022), with one bonus incident from before 2012. Not including the bonus incident,…

2 Feb 2022 30 min read

You don't have to be at a party to see this phenomenon in action, but there's a curious thing I regularly see at parties in social circles where people value intelligence and cleverness without similarly valuing on-the-ground knowledge or intellectual rigor. People often discuss the standard trendy topics (some recent ones I've observed at multiple parties are how to build…

18 Dec 2021 23 min read

This is an excerpt from an internal document David Mackey and I co-authored in April 2019. The document is excerpted since much of the original doc was about comparing possible approaches to increasing efficency at Twitter, which is mostly information that's meaningless outside of Twitter without a large amount of additional explanation/context. At Twitter, most CPU bound services start falling…

13 Dec 2021 43 min read

I see a lot of essays framed as writing advice which are actually thinly veiled descriptions of how someone writes that basically say "you should write how I write", e.g., people who write short posts say that you should write short posts. As with technical topics, I think a lot of different things can work and what's really important is…

6 Dec 2021 13 min read

This is a pseudo-transcript (actual words modified to be more readable than a 100% faithful transcription) of a short lightning talk I did at Twitter a year or two ago, on pitfalls of how we use latency metrics (with the actual service names anonymized per a comms request). Since this presentation, significant progress has been made on this on the…

22 Nov 2021 7 min read

Here's a list of errors on this blog that I think were fairly serious. While what I think of as serious is, of course, subjective, I don't think there's any reasonable way to avoid that because, e.g., I make a huge number of typos, so many that the majority of acknowledgements on many posts are for people who e-mailed or…

15 Nov 2021 17 min read

One of the most common mistakes I see people make when looking at data is incorrectly using an overly simplified model. A specific variant of this that has derailed the majority of work roadmaps I've looked at is treating people as interchangeable, as if it doesn't matter who is doing what, as if individuals don't matter. Individuals matter. A pattern…

8 Nov 2021 21 min read

Three major tools that companies have to influence behavior are incentives, process, and culture. People often mean different things when talking about these, so I'll provide an example of each so we're on the same page (if you think that I should be using a different word for the concept, feel free to mentally substitute that word). Getting people to…

21 Oct 2021 25 min read

People frequently1 think that I'm very stupid. I don't find this surprising, since I don't mind if other people think I'm stupid, which means that I don't adjust my behavior to avoid seeming stupid, which results in people thinking that I'm stupid. Although there are some downsides to people thinking that I'm stupid, e.g., failing interviews where the interviewer very…

18 Oct 2021 18 min read

It's common to see people advocate for learning skills that they have or using processes that they use. For example, Steve Yegge has a set of blog posts where he recommends reading compiler books and learning about compilers. His reasoning is basically that, if you understand compilers, you'll see compiler problems everywhere and will recognize all of the cases where…

15 Oct 2021 24 min read

A common topic of discussion among my close friends is where the bottlenecks are in our productivity and how we can execute more quickly. This is very different from what I see in my extended social circles, where people commonly say that velocity doesn't matter. In online discussions about this, I frequently see people go a step further and assign…

29 Sept 2021 9 min read

An alternate title for this post might be, "Twitter has a kernel team!?". At this point, I've heard that surprised exclamation enough that I've lost count of the number times that's been said to me (I'd guess that it's more than ten but less than a hundred). If we look at trendy companies that are within a couple factors of…

27 Aug 2021 24 min read

A question I get asked with some frequency is: why bother measuring X, why not build something instead? More bluntly, in a recent conversation with a newsletter author, his comment on some future measurement projects I wanted to do (in the same vein as other projects like keyboard vs. mouse, keyboard, terminal and end-to-end latency measurements), delivered with a smug…

29 Dec 2020 32 min read

In the classic 1986 essay, No Silver Bullet, Fred Brooks argued that there is, in some sense, not that much that can be done to improve programmer productivity. His line of reasoning is that programming tasks contain a core of essential/conceptual1 complexity that's fundamentally not amenable to attack by any potential advances in technology (such as languages or tooling). He…

30 Jun 2020 34 min read

Any time you have a benchmark that gets taken seriously, some people will start gaming the benchmark. Some famous examples in computing are the CPU benchmark specfp and video game benchmarks. With specfp, Sun managed to increase its score on 179.art (a sub-benchmark of specfp) by 12x with a compiler tweak that essentially re-wrote the benchmark kernel, which increased the…

2 Jun 2020 8 min read

This is an archive of an old pseudonymously written post from the 90s from someone whose former pseudonym seems to have disappeared from the internet. I see that Star Trek: Voyager has added a new character, a Borg. (From the photos, I also see that they're still breeding women for breast size in the 24th century.) What ticked me off…

31 May 2020 18 min read

A lot of people seem to think that distributed tracing isn't useful, or at least not without extreme effort that isn't worth it for companies smaller than FB. For example, here are a couple of public conversations that sound like a number of private conversations I've had. Sure, there's value somewhere, but it costs too much to unlock. I think…

30 May 2020 18 min read

We spent one day1 building a system that immediately found a mid 7 figure optimization (which ended up shipping). In the first year, we shipped mid 8 figures per year worth of cost savings as a result. The key feature this system introduces is the ability to query metrics data across all hosts and all services and over any period…

11 Mar 2020 11 min read

I've been comparing notes with people who run corporate engineering blogs and one thing that I think is curious is that it's pretty common for my personal blog to get more traffic than the entire corp eng blog for a company with a nine to ten figure valuation and it's not uncommon for my blog to get an order of…

3 Mar 2020 19 min read

My hobby: opening up McIlroy’s UNIX philosophy on one monitor while reading manpages on the other. The first of McIlroy's dicta is often paraphrased as "do one thing and do it well", which is shortened from "Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new 'features.'" McIlroy's…

18 Feb 2020 16 min read

If you read any personal finance forums late last year, there's a decent chance you ran across a question from someone who was desperately trying to lose money before the end of the year. There are a number of ways someone could do this; one commonly suggested scheme was to buy put options that were expected to expire worthless, allowing…

7 Feb 2020 31 min read

Reaching 95%-ile isn't very impressive because it's not that hard to do. I think this is one of my most ridiculable ideas. It doesn't help that, when stated nakedly, that sounds elitist. But I think it's just the opposite: most people can become (relatively) good at most things. Note that when I say 95%-ile, I mean 95%-ile among people who…

5 Jan 2020 25 min read

When I ask people at trendy big tech companies why algorithms quizzes are mandatory, the most common answer I get is something like "we have so much scale, we can't afford to have someone accidentally write an O(n^2) algorithm and bring the site down"1. One thing I find funny about this is, even though a decent fraction of the value…

12 Jul 2019 29 min read

This is a psuedo-transcript for a talk given at Deconstruct 2019. To make this accessible for people on slow connections as well as people using screen readers, the slides have been replaced by in-line text (the talk has ~120 slides; at an average of 20 kB per slide, that's 2.4 MB. If you think that's trivial, consider that half of…

19 Feb 2019 26 min read

A recurring discussion in Overwatch (as well as other online games) is whether or not women are treated differently from men. If you do a quick search, you can find hundreds of discussions about this, some of which have well over a thousand comments. These discussions tend to go the same way and involve the same debate every time, with…

28 Mar 2018 366 min read

This is an archive of the original "fsyncgate" email thread. This is posted here because I wanted to have a link that would fit on a slide for a talk on file safety with a mobile-friendly non-bloated format. From:Craig Ringer <craig(at)2ndquadrant(dot)com> Subject:Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS Date:2018-03-28 02:23:46 Hi…

24 Dec 2017 22 min read

I've had this nagging feeling that the computers I use today feel slower than the computers I used as a kid. As a rule, I don’t trust this kind of feeling because human perception has been shown to be unreliable in empirical studies, so I carried around a high-speed camera and measured the response latency of devices I’ve run into…

21 Nov 2017 32 min read

A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated. A contentious example of this is the claim that companies can’t be discriminating because the market is too competitive to tolerate discrimination. A less contentious example is that when you see a…

12 Nov 2017 10 min read

It's common knowledge that Android device tend to be more out of date than iOS devices, but what does this actually mean? Let’s look at android marketshare data to see how old devices in the wild are. The x axis of the plot below is date, and the y axis is Android marketshare. The share of all devices sums to…

9 Nov 2017 12 min read

About once a month, an app that I regularly use will change its UI in a way that breaks muscle memory, basically tricking the user into doing things they don’t want. Zulip In recent memory, Zulip (a slack competitor) changed its newline behavior so that ctrl + enter sends a message instead of inserting a new line. After this change,…

23 Oct 2017 20 min read

We’re going to reproduce some results from papers on filesystem robustness that were written up roughly a decade ago: Prabhakaran et al. SOSP 05 paper, which injected errors below the filesystem and Gunawi et al. FAST 08, which looked at how often filesystems failed to check return codes of functions that can return errors. Prabhakaran et al. injected errors at…

16 Oct 2017 17 min read

If you look at “gaming" keyboards, a lot of them sell for $100 or more on the promise that they’re fast. Ad copy that you’ll see includes: a custom designed keycap that has been made shorter to reduce the time it takes for your actions to register 8x FASTER - Polling Rate of 1000Hz: Response time 0.1 milliseconds Wield the…

23 Aug 2017 24 min read

This is a pseudo-transcript for a talk on branch prediction given at Two Sigma on 8/22/2017 to kick off "localhost", a talk series organized by RC. How many of you use branches in your code? Could you please raise your hand if you use if statements or pattern matching? Most of the audience raises their hands I won’t ask you…

9 Aug 2017 16 min read

I recently had a problem where part of the solution was to do a series of pointer accesses that would walk around a chunk of memory in pseudo-random order. Sattolo's algorithm provides a solution to this because it produces a permutation of a list with exactly one cycle, which guarantees that we will reach every element of the list even…

18 Jul 2017 13 min read

There’s a great MSR demo from 2012 that shows the effect of latency on the experience of using a tablet. If you don’t want to watch the three minute video, they basically created a device which could simulate arbitrary latencies down to a fraction of a millisecond. At 100ms (1/10th of a second), which is typical of consumer tablets, the…

13 Jun 2017 13 min read

Which is faster, keyboard or mouse? A large number of programmers believe that the keyboard is faster for all (programming-related) tasks. However, there are a few widely cited webpages on AskTog which claim that Apple studies show that using the mouse is faster than using the keyboard for everything and that people who think that using the keyboard is faster…

7 Jun 2017 25 min read

I often talk to startups that claim that their compensation package has a higher expected value than the equivalent package at a place like Facebook, Google, Twitter, or Snapchat. One thing I don’t understand about this claim is, if the claim is true, why shouldn’t the startup go to an investor, sell their options for what they claim their options…

8 Feb 2017 19 min read

A couple years ago, I took a road trip from Wisconsin to Washington and mostly stayed in rural hotels on the way. I expected the internet in rural areas too sparse to have cable internet to be slow, but I was still surprised that a large fraction of the web was inaccessible. Some blogs with lightweight styling were readable, as…

23 Oct 2016 67 min read

HN comments are terrible. On any topic I’m informed about, the vast majority of comments are pretty clearly wrong. Most of the time, there are zero comments from people who know anything about the topic and the top comment is reasonable sounding but totally incorrect. Additionally, many comments are gratuitously mean. You'll often hear mean comments backed up with something…

16 Oct 2016 26 min read

There are a lot of “12 CS books every programmer must read” lists floating around out there. That's nonsense. The field is too broad for almost any topic to be required reading for all programmers, and even if a topic is that important, people's learning preferences differ too much for any book on that topic to be the best book…

9 Oct 2016 19 min read

Joel Spolsky has a classic blog post on "Finding Great Developers" where he popularized the meme that great developers are impossible to find, a corollary of which is that if you can find someone, they're not great. Joel writes, The great software developers, indeed, the best people in every field, are quite simply never on the market. The average great…

3 Oct 2016 9 min read

I can't think of a single large software company that doesn't regularly draw internet comments of the form “What do all the employees do? I could build their product myself.” Benjamin Pollack and Jeff Atwood called out people who do that with Stack Overflow. But Stack Overflow is relatively obviously lean, so the general response is something like “oh, sure…

27 Sept 2016 18 min read

Developer compensation has skyrocketed since the demise of the Google et al. wage-suppressing no-hire agreement, to the point where compensation rivals and maybe even exceeds compensation in traditionally remunerative fields like law, consulting, etc. In software, "senior" dev salary at a high-paying tech company is $350k/yr, where "senior" can mean "someone three years of out school" and it's not uncommon…

12 Sept 2016 49 min read

Tavish Armstrong has a great document where he describes how and when he learned the programming skills he has. I like this idea because I've found that the paths that people take to get into programming are much more varied than stereotypes give credit for, and I think it's useful to see that there are many possible paths into programming.…

5 Aug 2016 23 min read

Do concurrency bugs matter? From the literature, we know that most reported bugs in distributed systems have really simple causes and can be caught by trivial tests, even when we only look at bugs that cause really bad failures, like loss of a cluster or data corruption. The filesystem literature echos this result -- a simple checker that looks for…

18 Apr 2016 22 min read

This is one of those “N technical things every programmer must read” lists, except that “programmer” is way too broad a term and the styles of writing people find helpful for them are too different for any such list to contain a non-zero number of items (if you want the entire list to be helpful to everyone). So here's a…

11 Apr 2016 29 min read

The book starts with a story about a time Margaret Hamilton brought her young daughter with her to NASA, back in the days of the Apollo program. During a simulation mission, her daughter caused the mission to crash by pressing some keys that caused a prelaunch program to run during the simulated mission. Hamilton submitted a change request to add…

21 Mar 2016 18 min read

An acquaintance of mine, let’s call him Mike, is looking for work after getting laid off from a contract role at Microsoft, which has happened to a lot of people I know. Like me, Mike has 11 years in industry. Unlike me, he doesn't know a lot of folks at trendy companies, so I passed his resume around to some…

1 Mar 2016 207 min read

These are archived from the now defunct su3su2u1 tumblr. Since there was some controversy over su3su2u1's identity, I'll note that I am not su3su2u1 and that hosting this material is neither an endorsement nor a sign of agreement. Harry Potter and the Methods of Rationality full review I opened up a bottle of delicious older-than-me scotch when Terry Pratchett died,…

1 Mar 2016 37 min read

These are archived from the now defunct su3su2u1 tumblr. A Roundabout Approach to Quantum Mechanics This will be the first post in what I hope will be a series that outlines some ideas from quantum mechanics. I will try to keep it light, and not overly math filled- which means I’m not really teaching you physics. I’m teaching you some…

24 Jan 2016 22 min read

Perf is probably the most widely used general purpose performance debugging tool on Linux. There are multiple contenders for the #2 spot, and, like perf, they're sampling profilers. Sampling profilers are great. They tend to be easy-to-use and low-overhead compared to most alternatives. However, there are large classes of performance problems sampling profilers can't debug effectively, and those problems are…

10 Jan 2016 19 min read

2015 was a pretty good year for Intel. Their quarterly earnings reports exceeded expectations every quarter. They continue to be the only game in town for the serious server market, which continues to grow exponentially; from the earnings reports of the two largest cloud vendors, we can see that AWS and Azure grew by 80% and 100%, respectively. That growth…

29 Dec 2015 24 min read

Have you ever mentioned something that seems totally normal to you only to be greeted by surprise? Happens to me all the time when I describe something everyone at work thinks is normal. For some reason, my conversation partner's face morphs from pleasant smile to rictus of horror. Here are a few representative examples. There's the company that is perhaps…

17 Dec 2015 29 min read

There's a meme that's been going around for a while now: you should join a startup because the money is better and the work is more technically interesting. Paul Graham says that the best way to make money is to "start or join a startup", which has been "a reliable way to get rich for hundreds of years", and that…

12 Dec 2015 22 min read

I haven't used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox. Pine, Eudora, and outlook have all corrupted my inbox, forcing me to restore from backup. How is it that desktop mail clients are less reliable than gmail, even though my gmail account not…

27 Nov 2015 20 min read

Jeff Atwood, perhaps the most widely read programming blogger, has a post that makes a case against using ECC memory. My read is that his major points are: Google didn't use ECC when they built their servers in 1999 Most RAM errors are hard errors and not soft errors RAM errors are rare because hardware has improved If ECC were…

23 Nov 2015 11 min read

In 1999, Butler Lampson gave a talk about the past and future of “computer systems research”. Here are his opinions from 1999 on "what worked". Yes Maybe No Virtual memory Parallelism Capabilities Address spaces RISC Fancy type systems Packet nets Garbage collection Functional programming Objects / subtypes Reuse Formal methods RDB and SQL Software engineering Transactions RPC Bitmaps and GUIs…

1 Nov 2015 10 min read

Hardware performance “obviously” affects software performance and affects how software is optimized. For example, the fact that caches are multiple orders of magnitude faster than RAM means that blocked array accesses give better performance than repeatedly striding through an array. Something that's occasionally overlooked is that hardware performance also has profound implications for system design and architecture. Let's look at…

4 Oct 2015 13 min read

Typical server utilization is between 10% and 50%. Google has demonstrated 90% utilization without impacting latency SLAs. Xkcd estimated that Google owns 2 million machines. If you estimate an amortized total cost of $4k per machine per year, that's $8 billion per year. With numbers like that, even small improvements have a large impact, and this isn't a small improvement.…

30 Sept 2015 8 min read

Every once in awhile, you hear a story like “there was a case of a 1-Gbps NIC card on a machine that suddenly was transmitting only at 1 Kbps, which then caused a chain reaction upstream in such a way that the performance of the entire workload of a 100-node cluster was crawling at a snail's pace, effectively making the…

31 Aug 2015 8 min read

I try to avoid making predictions1. It's a no-win proposition: if you're right, hindsight bias makes it look like you're pointing out the obvious. And most predictions are wrong. Every once in a while when someone does a review of predictions from pundits, they're almost always wrong at least as much as you'd expect from random chance, and then hindsight…

20 Aug 2015 10 min read

I love reading postmortems. They're educational, but unlike most educational docs, they tell an entertaining story. I've spent a decent chunk of time reading postmortems at both Google and Microsoft. I haven't done any kind of formal analysis on the most common causes of bad failures (yet), but there are a handful of postmortem patterns that I keep seeing over…

31 May 2015 5 min read

If you've followed any tech news aggregator in the past week (the week of the 24th of May, 2015), you've probably seen the story about how SourceForge is taking over admin accounts for existing projects and injecting adware in installers for packages like GIMP. For anyone not following the story, SourceForge has a long history of adware laden installers, but…

27 May 2015 3 min read

TIL that Bell Labs and a whole lot of other websites block archive.org, not to mention most search engines. Turns out I have a broken website link in a GitHub repo, caused by the deletion of an old webpage. When I tried to pull the original from archive.org, I found that it's not available because Bell Labs blocks the archive.org…

25 May 2015 6 min read

Boring languages are underrated. Many appear to be rated quite highly, at least if you look at market share. But even so, they're underrated. Despite the popularity of Dan McKinley's "choose boring technology" essay, boring languages are widely panned. People who use them are too (e.g., they're a target of essays by Paul Graham and Joel Spolsky, and other people…

17 May 2015 8 min read

Here's a conversation I keep having: Someone: Did you hear that Facebook/Google uses a giant monorepo? WTF! Me: Yeah! It's really convenient, don't you think? Someone: That's THE MOST RIDICULOUS THING I've ever heard. Don't FB and Google know what a terrible idea it is to put all your code in a single repo? Me: I think engineers at FB…

4 May 2015 5 min read

Why are people so concerned with hardware power consumption nowadays? Some common answers to this question are that power is critically important for phones, tablets, and laptops and that we can put more silicon on a modern chip than we can effectively use. In 2001 Patrick Gelsinger observed that if scaling continued at then-current rates, chips would have the power…

29 Mar 2015 17 min read

It's really common to see claims that some meme is backed by “studies” or “science”. But when I look at the actual studies, it usually turns out that the data are opposed to the claim. Here are the last few instances of this that I've run across. Dunning-Kruger A pop-sci version of Dunning-Kruger, the most common one I see cited,…

10 Mar 2015 22 min read

I've been reading a lot about software testing, lately. Coming from a hardware background (CPUs and hardware accelerators), it's interesting how different software testing is. Bugs in software are much easier to fix, so it makes sense to spend a lot less effort spent on testing. Because less effort is spent on testing, methodologies differ; software testing is biased away…

7 Mar 2015 8 min read

I've been hearing this question a lot lately, and when I do, it reminds me how much I don't know. Here are some questions this question brings to mind. How does a keyboard work? Why can’t you press an arbitrary combination of three keys at once, except on fancy gaming keyboards? That implies something about how key presses are detected/encoded.…

5 Mar 2015 10 min read

Most real-world problems are big enough that you can't just head for the end goal, you have to break them down into smaller parts and set up intermediate goals. For that matter, most games are that way too. “Win” is too big a goal in chess, so you might have a subgoal like don't get forked. While creating subgoals makes…

15 Feb 2015 13 min read

There's an ongoing debate over whether "AI" will ever be good enough to displace humans and, if so, when it will happen. In this debate, the optimists tend to focus on how much AI is improving and the pessimists point to all the ways AI isn't as good as an ideal human being. I think this misses two very important…

3 Feb 2015 11 min read

It's generally accepted that any piece of software could be compromised with a backdoor. Prominent examples include the Sony/BMG installer, which had a backdoor built-in to allow Sony to keep users from copying the CD, which also allowed malicious third-parties to take over any machine with the software installed; the Samsung Galaxy, which has a backdoor that allowed the modem…

24 Jan 2015 12 min read

Does it make sense for me to run ads on my blog? I've been thinking about this lately, since Carbon Ads contacted me about putting an ad up. What are the pros and cons? This isn't a rhetorical question. I'm genuinely interested in what you think. Pros Money Hey, who couldn't use more money? And it's basically free money. Well,…

11 Jan 2015 41 min read

This is a response to the following question from David Albert: My mental model of CPUs is stuck in the 1980s: basically boxes that do arithmetic, logic, bit twiddling and shifting, and loading and storing things in memory. I'm vaguely aware of various newer developments like vector instructions (SIMD) and the idea that newer CPUs have support for virtualization (though…

28 Dec 2014 27 min read

Here's a language that gives near-C performance that feels like Python or Ruby with optional type annotations (that you can feed to one of two static analysis tools) that has good support for macros plus decent-ish support for FP, plus a lot more. What's not to like? I'm mostly not going to talk about how great Julia is, though, because…

17 Dec 2014 7 min read

How much overhead should we expect from enabling integer overflow checks? Using a compiler flag or built-in intrinsics, we should be able to do the check with a conditional branch that branches based on the overflow flag that add and sub set. Code that looks like add %esi, %edi should turn into something like add %esi, %edi jo <handle_overflow> Assuming…

4 Dec 2014 14 min read

Let's write a malloc and see how it works with existing programs! This is basically an expanded explanation of what I did after reading this tutorial by Marwan Burelle and then sitting down and trying to write my own implementation, so the steps are going to be fairly similar. The main implementation differences are that my version is simpler and…

1 Dec 2014 15 min read

Public discussions of discrimination in tech often result in someone claiming that discrimination is impossible because of market forces. Here's a quote from Marc Andreessen that sums up a common view1. Let's launch right into it. I think the critique that Silicon Valley companies are deliberately, systematically discriminatory is incorrect, and there are two reasons to believe that that's the…

24 Nov 2014 6 min read

I was curious what different people worked on in Linux, so I tried grabbing data from the current git repository to see if I could pull that out of commit message data. This doesn't include history from before they switched to git, so it only goes back to 2005, but that's still a decent chunk of history. Here's a list…

18 Nov 2014 15 min read

If I had to guess, I'd say I probably work around hundreds of bugs in an average week, and thousands in a bad week. It's not unusual for me to run into a hundred new bugs in a single week. But I often get skepticism when I mention that I run into multiple new (to me) bugs per day, and…

17 Nov 2014 9 min read

I've seen all these studies that show how a 100ms improvement in page load time has a significant effect on page views, conversion rate, etc., but I'd never actually tried to optimize my site. This blog is a static Octopress site, hosted on GitHub Pages. Static sites are supposed to be fast, and GitHub Pages uses Fastly, which is supposed…

10 Nov 2014 6 min read

I've noticed that builds are broken and tests fail a lot more often on open source projects than on “work” projects. I wasn't sure how much of that was my perception vs. reality, so I grabbed the Travis CI data for a few popular categories on GitHub1. For reference, at every place I've worked, two 9s of reliability (99% uptime)…

7 Nov 2014 38 min read

There are some pretty strong statements about types floating around out there. The claims range from the oft-repeated phrase that when you get the types to line up, everything just works, to “not relying on type safety is unethical (if you have an SLA)”1, "It boils down to cost vs benefit, actual studies, and mathematical axioms, not aesthetics or feelings",…

5 Nov 2014 8 min read

The latest version of the Intel manual has a couple of new instructions for non-volatile storage, like SSDs. What's that about? Before we look at the instructions in detail, let's take a look at the issues that exist with super fast NVRAM. One problem is that next generation storage technologies (PCM, 3d XPoint, etc.), will be fast enough that syscall…

3 Nov 2014 8 min read

Once upon a time, my computer architecture professor mentioned that using a random eviction policy for caches really isn't so bad. That random eviction isn't bad can be surprising — if your cache fills up and you have to get rid of something, choosing the least recently used (LRU) is an obvious choice, since you're more likely to use something…

3 Nov 2014 4 min read

This is an off-the-cuff comment for Hacker School's Paper of the Week Read Along series for Out of the Tar Pit. I find the idea itself, which is presented in sections 7-10, at the end of the paper, pretty interesting. However, I have some objections to the motivation for the idea, which makes up the first 60% of the paper.…

19 Oct 2014 13 min read

Every once in a while, I hear how intrinsics have improved enough that it's safe to use them for high performance code. That would be nice. The promise of intrinsics is that you can write optimized code by calling out to functions (intrinsics) that correspond to particular assembly instructions. Since intrinsics act like normal functions, they can be cross platform.…

14 Aug 2014 51 min read

UNITED STATES DISTRICT COURT NORTHERN DISTRICT OF CALIFORNIA SAN JOSE DIVISION IN RE: HIGH-TECH EMPLOYEE ANTITRUST LITIGATION THIS DOCUMENT RELATES TO: ALL ACTIONS Case No.: 11-CV-02509-LHK ORDER DENYING PLAINTIFFS' MOTION FOR PRELIMINARY APPROVAL OF SETTLEMENTS WITH ADOBE, APPLE, GOOGLE, AND INTEL Before the Court is a Motion for Preliminary Approval of Class Action Settlement with Defendants Adobe Systems Inc. ("Adobe"),…

14 Aug 2014 16 min read

This is an archived USENET post from John Cooley on a competitive comparison between VHDL and Verilog that was done in 1997. I knew I hit a nerve. Usually when I publish a candid review of a particular conference or EDA product I typically see around 85 replies in my e-mail "in" box. Buried in my review of the recent…

6 Apr 2014 5 min read

I can't remember the last time I went a whole day without running into a software bug. For weeks, I couldn't invite anyone to Facebook events due to a bug that caused the invite button to not display on the invite screen. Google Maps has been giving me illegal and sometimes impossible directions ever since I moved to a small…

23 Mar 2014 6 min read

Editing binaries is a trick that comes in handy a few times a year. You don't often need to, but when you do, there's no alternative. When I mention patching binaries, I get one of two reactions: complete shock or no reaction at all. As far as I can tell, this is because most people have one of these two…

9 Mar 2014 4 min read

Last week, Quartz published an article titled “There is no gender gap in tech salaries”. That resulted in linkbait copycat posts all over the internet, from obscure livejournals to Smithsonian.com. The claims are awfully strong, considering that the main study cited only looked at people who graduated with a B.S. exactly one year ago, not to mention the fact that…

5 Mar 2014 5 min read

In 1983, at the University of Wisconsin, Dina Bitton, David DeWitt, and Carolyn Turbyfill created a database benchmarking framework. Some of their results included (lower is better): Join without indices table {border-collapse:collapse;margin:0px auto;}table,th,td {border: 1px solid black;}td {text-align:center;} system joinAselB joinABprime joinCselAselB U-INGRES 10.2 9.6 9.4 C-INGRES 1.8 2.6 2.1 ORACLE > 300 > 300 > 300 IDMnodac > 300…

8 Feb 2014 9 min read

In the fall of 2000, I took my first engineering class: ECE 352, an entry-level digital design class for first-year computer engineers. It was standing room only, filled with waitlisted students who would find seats later in the semester as people dropped out. We had been warned in orientation that half of us wouldn't survive the year. In class, We…

9 Jan 2014 4 min read

Dear David, I'm afraid my off the cuff response the other day wasn't too well thought out; when you talked about taking calc III and linear algebra, and getting resistance from one of your friends because "wolfram alpha can do all of that now," my first reaction was horror-- which is why I replied that while I've often regretted not…

2 Jan 2014 6 min read

Here's the graph of a toy benchmark1 of page-aligned vs. mis-aligned accesses; it shows a ratio of performance between the two at different working set sizes. If this benchmark seems contrived, it actually comes from a real world example of the disastrous performance implications of using nice power of 2 alignment, or page alignment in an actual system2. Except for…

13 Dec 2013 4 min read

Earlier this year, I interviewed with a well-known tech startup, one of the hundreds of companies that claims to have harder interviews, more challenging work, and smarter employees than Google1. My first interviewer, John, gave me the standard tour: micro-kitchen stocked with a combination of healthy snacks and candy; white male 20-somethings gathered around a foosball table; bright spaces with…

10 Nov 2013 8 min read

In CPU design, most successful teams have a fairly long lineage and rely heavily on experienced engineers. When we look at CPU startups, teams that have a successful exist often have a core team that's been together for decades. For example, PA Semi's acquisition by Apple was a moderately successful exit, but where did that team come from? They were…

27 Oct 2013 2 min read

What's the first thing you do when you find a bug or see a missing feature in an open source project? Check out the project page and submit a patch! Oh. Maybe their message is so encouraging that they get hundreds of pull requests a week, and the backlog isn't that bad. Maybe not. Giant sucker than I am, I…

4 Oct 2013 6 min read

You ever notice that there's this funny threshold for getting to the front page on sites like HN? The exact threshold varies depending on how much traffic there is, but, for articles that aren't wildly popular, there's this moment when the article is at N-1 votes. There is, perhaps, a 60% chance that the vote will come and the article…

15 Sept 2013 5 min read

Troll? That's how people write Verilog1. At my old company, we had a team of formal methods PhD's who wrote a linter that typechecked our code, based on our naming convention. For our chip (which was small for a CPU), building a model (compiling) took about five minutes, running a single short test took ten to fifteen minutes, and long…

7 Sept 2013 9 min read

Verilog is the most commonly used language for hardware design in America (VHDL is more common in Europe). Too bad it's so baroque. If you ever browse the Verilog questions on Stack Overflow, you'll find a large number of questions, usually downvoted, asking “why doesn't my code work?”, with code that's not just a little off, but completely wrong. Lets…

1 Sept 2013 1 min read

About The Blog This started out as a way to jot down thoughts on areas that seem interesting but underappreciated. Since then, this site has grown to the point where it gets millions of hits a month and I see that it's commonly cited by professors in their courses and on stackoverflow. That's flattering, but more than anything else, I…

5 Mar 2013 23 min read

this is an archive of an old article by John Carmack which seems to have disappeared off of the internet Abstract Virtual reality (VR) is one of the most demanding human-in-the-loop applications from a latency standpoint. The latency between the physical movement of a user’s head and updated photons from a head mounted display reaching their eyes is one of…

12 Feb 2013 16 min read

This is a transcript of the Kara Swisher / Jack Dorsey interview from 2/12/2019, made by parsing the original Tweets because I wanted to be able to read this linearly. There's a "moment" that tries to track this, but since it doesn't distinguish between sub-threads in any way, you can't tell the difference between end of a thread and a…

23 Mar 2012 26 min read

This is an archive of the Jonathan Shapiro's "Retrospective Thoughts on BitC" that seems to have disappeared from the internet; at the time, BitC was aimed at the same niche as Rust Jonathan S. Shapiro shap at eros-os.org Fri Mar 23 15:06:41 PDT 2012 By now it will be obvious to everyone that I have stopped work on BitC. An…

1 Jan 2010 59 min read

This is an archive of an old Google Buzz conversation (circa 2010?) on a variety of topics, including whether or not it's inevetible that a closed platform will dominate social. Piaw: Social networks will be dominated primarily by network effects. That means in the long run, Facebook will dominate all the others. Rebecca: ... which is also why no one…

1 Jan 2010 52 min read

This is an archive of an old Google Buzz conversation on MIT vs. Stanford and Silicon Valley vs. Boston There's no reason why the Boston area shouldn't be as much a hotbed of startups as Silicon Valley is. By contrast, there are lots of reasons why NYC is no good for startups. Nevertheless, Paul Graham gave up on the Boston…

31 May 2008 19 min read

This is an archive of some posts in a forum thread titled "Beware of Bioware" in a now defunct forum, with comments from that forum as well as blog comments from a now defunct blog that archived that made the first attempt to archive this content. The original posts were deleted shortly after being posted, replaced with "big scary company…

16 Nov 2007 13 min read

This is an archive of Dan Weinreb's comments on Symbolics and Lisp machines. Rebuttal to Stallman’s Story About The Formation of Symbolics and LMI Richard Stallman has been telling a story about the origins of the Lisp machine companies, and the effects on the M.I.T. Artificial Intelligence Lab, for many years. He has published it in a book, and in…

1 Feb 2006 9 min read

Archived from an unknown source. Possibly Gravitron? In regards for history: Chapter #1 (Around) December 1995 is when it all started. Rod Humble wished to create something like Air Warrior but online, he approached Virgin Interactive Entertainment with the idea and they replied with something along the lines of "here's the cash, good luck". Rod called Jeff Petersen and asked…