In early 2023, Slack faced a foundational challenge: serving Large Language Models (LLMs) at enterprise scale with the security, reliability, and performance our customers expect. Over three years, we evolved from basic infrastructure to orchestrating a sophisticated multi-cloud architecture. We didn’t just want shiny new models; we needed a system resilient to regional outages and…
#uncategorized
464 posts (showing latest 300 - use search for the rest)
28 May
13 May
I play golf. I am not good at golf. But I have a Garmin Approach R10 launch monitor, a Python interpreter, and too much free time, so naturally I spent way more time building a dashboard to analyze my swing data than I did actually swinging a club. The result is jgamblin/golf, a self-hosted analytics pipeline that turns your Garmin…
5 May
Excerpt By 2024, Slack’s data platform had accumulated 700+ SSH-based operators orchestrating critical data pipelines. We’re talking daily search indexing that processed terabytes of data, analytics jobs powering business intelligence, the whole shebang. Every single one of these jobs required direct SSH access to production AWS Elastic MapReduce (EMR) clusters. We had a massive security…
18 Apr
I spend a significant amount of my time thinking about EPSS, CVSS, and the inherent gaps in how we prioritize vulnerabilities. We all know the drill: a 9.8 CRITICAL that remains unexploited shouldn’t jump the line ahead of a 7.5 HIGH that is being actively used in the wild. Closing that gap between theoretical severity and actual exploitability is why…
13 Apr
Excerpt In complex, long-running agentic systems, maintaining alignment and coherent reasoning between agents requires careful design. In this second article of our series, we explore these challenges and the mechanisms we built to keep teams of agents working productively over long time spans. We present a range of complementary techniques that balance the conflicting requirements…
31 Mar
The Problem: Legacy Tooling and Its Limitations Currently, Slack utilizes a hybrid approach to network measurement, incorporating both internal (such as traffic between AWS Availability Zones) and external (monitoring traffic from the public internet into Slack’s infrastructure) solutions. These tools comprise a combination of commercial SaaS offerings and custom-built network testing solutions developed by our…
19 Mar
Introduction 🔔 At Slack, notifications are how teams stay in the loop, but they can also become overwhelming when not designed with intention. Our goal was to make staying informed feel effortless. We set out to rebuild one of Slack’s most complicated systems from the ground up by bringing calm, consistency, and clarity to the…
12 Feb
The Interop Project is a cross-browser initiative to improve web compatibility in areas that offer the most benefit to both users and developers. The group, including Apple, Google, Igalia, Microsoft, and Mozilla, takes proposals of features that are well defined in a sufficiently stable web standard, and have good test suite coverage. Then, we come […] The post Launching Interop…
22 Jan
Scott Hanselman, the Microsoft VP for developer community outreach and general all around good guy was kind enough to ask me to be on his podcast, HanselMinutes, to talk about my upcoming book. I was delighted to spend a few … Continue reading →
1 Jan
2025 set a new baseline with 48,185 published CVEs. While the sheer volume is climbing, the median CVSS score remained surprisingly stable. We are seeing a distinct shift toward web application flaws (specifically in the CMS ecosystem) and a wider distribution of vendors, proving that vulnerabilities are spreading deeper into the supply chain. This massive growth is exactly why I…
1 Dec 2025
Slack’s Security Engineering team is responsible for protecting Slack’s core infrastructure and services. Our security event ingestion pipeline handles billions of events per day from a diverse array of data sources. Reviewing alerts produced by our security detection system is our primary responsibility during on-call shifts. We’re going to show you how we’re using AI…
19 Nov 2025
Background A Voluntary Product Accessibility Template (VPAT) is a document that outlines how well a product aligns with accessibility (a11y) standards. Its primary purpose is to inform customers about a product’s a11y features, enabling them to make informed decisions before purchasing software. At Slack, we conducted a VPAT by a third party a11y vendor in…
17 Nov 2025
Hey everyone, I first want to say a heartfelt thank you so much for the warm comments and supportive feedback I’ve gotten since I announced that I’m in process of writing Fabulous Adventures In Data Structures And Algorithms. It means … Continue reading →
6 Nov 2025
We manage the build pipeline that delivers Quip and Slack Canvas’s backend. A year ago, we were chasing exciting ideas to help engineers ship better code, faster. But we had one huge problem: builds took 60 minutes. With a build that slow, the whole pipeline gets less agile, and feedback doesn’t come to engineers until…
30 Oct 2025
Hey everyone, I’ve finally turned this blog into a book! Or, rather, I am deep in the process of turning this blog into a book. Yes, it’s about 20 years, give or take, after I first thought of doing so, … Continue reading →
23 Oct 2025
Last year, I wrote a blog post titled Advancing Our Chef Infrastructure, where we explored the evolution of our Chef infrastructure over the years. We talked about the shift from a single Chef stack to a multi-stack model, and the challenges that came with it – from updating how we handle cookbook uploads to navigating…
7 Oct 2025
It’s mid 2023 and we’ve identified some opportunities to improve our reliability. Fast forward to January 2025. Customer impact hours are reduced from the peak by 90% and continuing to trend downward. We’re a year and half into the Deploy Safety Program at Slack, improving the way we deploy, uplifting our safety culture and continuing…
12 Sept 2025
[...] Read More... The post Read Meta’s 2025 Sustainability Report appeared first on Engineering at Meta.
4 Sept 2025
As cyberattacks evolve to unprecedented levels of sophistication and speed, the time gap between breach detection and response has never been more critical. Traditional security approaches often operate reactively, identifying compromises only after damage has occurred. This delay grants attackers a tactical advantage, forcing security teams to focus on damage assessment and remediation rather than…
14 Aug 2025
I’m incredibly excited to finally share something I’ve been pouring my heart into at RogoLabs. For those of you who caught my talk at BSidesLV, you got a sneak peek, but today it’s official: CNAScorecard.org is live! For years, the CVE program has been our shared language for identifying vulnerabilities. But lately, we’ve all felt the growing pains. We’re seeing…
31 Jul 2025
We’re providing free CI/CD security audits for BEAM projects to help open-source maintainers catch issues early and stay secure. The post Supporting the BEAM Community with Free CI/CD Security Audits appeared first on Erlang Solutions.
30 Jul 2025
It’s that time of year again! The first week of August means my annual trip to the desert for “Security Summer Camp”—the whirlwind of BSides Las Vegas, Black Hat, and DEF CON. It’s always an exhausting but amazing week, and I can’t wait to dive in, catch up with everyone, and talk about what I’ve been working on. This year,…
5 Jun 2025
Avoid common startup tech mistakes that slow growth. Build a stack that scales from day one. The post Avoiding Common Startup Tech Mistakes appeared first on Erlang Solutions.
17 Apr 2025
Catch up on the latest from Erlang Solutions. This blog round-up covers key tech trends, including big data, digital wallets, IoT security, and more. The post Erlang Solutions’ Blog round-up appeared first on Erlang Solutions.
14 Apr 2025
In the world of DevOps and Developer Experience (DevXP), speed and efficiency can make a big difference on an engineer’s day-to-day tasks. Today, we’ll dive into how Slack’s DevXP team took some existing tools and used them to optimize an end-to-end (E2E) testing pipeline. This lowered build times and reduced redundant processes, saving both time…
27 Mar 2025
Oleg Ivanov reflects on his transition from Ruby to Elixir, highlighting key lessons and mindset shifts. The post My Journey from Ruby to Elixir: Lessons from a Developer appeared first on Erlang Solutions.
7 Mar 2025
Many don’t know that “Slack” is in fact a backronym—it stands for “Searchable Log of all Communication and Knowledge”. And these days, it’s not just a searchable log: with Slack AI, Slack is now an intelligent log, leveraging the latest in generative AI to securely surface powerful, time-saving insights. We built Slack AI from the…
12 Feb 2025
DORA is in effect, making compliance essential for fintech firms. This guide covers key requirements, risks, and next steps. The post DORA Compliance: What Fintech Businesses Need to Know appeared first on Erlang Solutions.
6 Feb 2025
This is the second part of a two three four-part series, which covers some recent results on “verifiable computation” and possible pitfalls that could occur there. This post won’t make much sense on its own, so I urge you to start with the first part. In the previous post we introduced a handful of concepts, … Continue reading How to…
23 Jan 2025
This guide unpacks the essentials of digital wallets, including their benefits, market trends, and implications for businesses looking to stay ahead. The post Understanding Digital Wallets appeared first on Erlang Solutions.
7 Jan 2025
At Slack, customer love is our first priority and accessibility is a core tenet of customer trust. We have our own Slack Accessibility Standards that product teams follow to guarantee their features are compliant with Web Content Accessibility Guidelines (WCAG). Our dedicated accessibility team supports developers in following these guidelines throughout the development process. We…
5 Jan 2025
2024 brought unprecedented growth in CVE data, so I figured it would be appropriate to start the new year by exploring these statistics and highlighting some of the more intriguing data points. CVEs By The Numbers We ended 2024 with 40,009 published CVEs, up over 38% from the 28,818 CVEs published in 2023. CVEs By Month Month CVEs Percentage January…
19 Dec 2024
Yesterday I wanted to replace my 2 TByte SSD with a 4 TByte model as I was running out of hard drive space. My last upgrade two years ago from 1 TByte to 2 TByte went smoothly: I cloned the old SSD to the new SSD and then extended the […] The post Migrating from 2 TByte SSD with MBR…
16 Dec 2024
Overview The past few months have been exciting times for Slack’s CI infrastructure. After years of developer frustration with Jenkins (everything from security issues to downtime to generally poor UX) internal pressure led us to move a majority of Slack’s CI jobs from Jenkins to GitHub Actions. My intern project at Slack this summer involved…
10 Dec 2024
“A complex system can fail in an infinite number of ways.” -“Systemantics” by John Gall Incidents are stressful but inevitable. Even services designed for availability will eventually encounter a failure. Engineers naturally find it daunting to defend their systems against the “infinite number of ways” things can go wrong. Our team found ourselves in…
9 Dec 2024
What are Slack Audit Logs? Like many Software as a Service (SaaS) offerings, Slack provides audit logs to Enterprise Grid customers that record when entities take an action on the platform. For example, when a user logs in, when a user updates their profile, when an app downloads a file, etc. The actual list of…
18 Nov 2024
Introduction Slack handles a lot of log data. In fact, we consume over 6 million log messages per second. That equates to over 10 GB of data per second! And it’s all stored using Astra, our in-house, open-source log search engine. To make this data searchable, Astra groups it by time and splits the data…
14 Nov 2024
Incident Management takes time Incidents need responders that are trained and experienced. At Slack, training is a foundation of our incident management program. Self-service training and live courses based mainly on prepared content are one piece of the puzzle, but there can be a missing piece in many organizations. How can staff get practical experience…
8 Nov 2024
Background and motivation In the fast-paced world of software development, having the right tools can make all the difference. At Slack, we’ve been working on a set of AI-powered developer tools that are saving 10,000+ hours of developer time yearly, while meeting our strictest requirements for security, data protection, and compliance. In this post, we’ll…
30 Oct 2024
The Common Vulnerabilities and Exposures (CVE) program, launched in late October 1999, has not only marked its presence but has become a pivotal force in shaping how we perceive and manage cybersecurity threats. A Journey Through Time The CVE program emerged as a beacon, standardizing how vulnerabilities are identified, shared, and mitigated. From its inception with just 321 entries, it…
28 Oct 2024
Introduction Large language models are fantastic tools for unstructured text, but what if your text doesn’t fit in the context window? Bazaarvoice faced exactly this challenge when building our AI Review Summaries feature: millions of user reviews simply won’t fit into the context window of even newer LLMs and, even if they did, it would […]
10 Oct 2024
We’ve been working to bring components of Quip’s technology into Slack with the canvas feature, while also maintaining the stand-alone Quip product. Quip’s backend, which powers both Quip and canvas, is written in Python. This is the story of a tricky bug we encountered last July and the lessons we learned along the way about…
Fintech open source is transforming the industry, offering flexible, scalable, and cost-effective solutions for businesses looking to innovate and stay competitive. The post Why Open Source Technology is a Smart Choice for Fintech Businesses appeared first on Erlang Solutions.
23 Sept 2024
After a lot of hard work, you’ve landed that coveted internship. Now comes the next big challenge: delivering a meaningful project over the summer. Leading a project independently is an opportunity to sharpen your skills, demonstrate your capabilities, and experience personal growth. As you drive the project on your own, the support from your mentor…
17 Sept 2024
At Slack, we manage tens of thousands of EC2 instances that host a variety of services, including our Vitess databases, Kubernetes workers, and various components of the Slack application. The majority of these instances run on some version of Ubuntu, while a portion operates on Amazon Linux. With such a vast infrastructure, the critical question…
9 Sept 2024
Here’s how generative AI is revolutionising healthcare- paving the way for more efficient, patient-centric care. The post How Generative AI is Transforming Healthcare appeared first on Erlang Solutions.
30 Aug 2024
“What are your goals for this quarter?” It’s the question every manager asks, and one that often prompts a flurry of technical objectives and project milestones. Jumping into this internship, I knew my answer. I wanted to practice making informed decisions on my project, since that was one of the challenges I faced last summer.…
31 Jul 2024
All software is built atop a core set of assumptions. As new code is added and new use-cases emerge, software can become unmoored from those assumptions. When this happens, a fundamental tension arises between revisiting those foundational assumptions—which usually entails a lot of work—or trying to support new behavior atop the existing architecture. The latter…
4 Jul 2024
This article explores how Erlang and Elixir programming languages support business outsourcing by offering superior security, scalability, and flexibility, ultimately helping companies achieve their strategic goals. The post The Strategic Advantage of Outsourcing with Erlang and Elixir appeared first on Erlang Solutions.
2 Jul 2024
Unlocking Efficiency and Performance: Navigating the Spark 3 and EMR 6 Upgrade Journey at Slack
SlackSlack Data Engineering recently underwent data workload migration from AWS EMR 5 (Spark 2/Hive 2 processing engine) to EMR 6 (Spark 3 processing engine). In this blog, we will share our migration journey, challenges, and the performance gains we observed in the process. This blog aims to assist Data Engineers, Data Infrastructure Engineers, and Product…
28 Jun 2024
At Slack, we’re committed to security that goes beyond the ordinary. We continuously strive to earn and maintain user trust by safeguarding critical components integral to every user’s experience. From passwords to session cookies, and tokens to webhooks, we prioritize protecting everything essential to how users log into the platform and remain authenticated. Through proactive…
24 Jun 2024
Slack uses cookies to track session states for users on slack.com and the Slack Desktop app. The ever-present cookie banners have made cookies mainstream, but as a quick refresher, cookies are a little piece of client-side state associated with a website that is sent up to the web server on every request. Websites use this…
15 Jun 2024
In a previous blog post—A Simple Kubernetes Admission Webhook—I discussed the process of creating a Kubernetes webhook without relying on Kubebuilder. At Slack, we use this webhook for various tasks, like helping us support long-lived Pods (see Supporting Long-Lived Pods), and today, I delve once more into the topic of long-lived Pods, focusing on our…
30 May 2024
The Last 100+ Days The NVD posted the notice below on its webpage in mid-February. Since then, nearly 13,000 CVEs have not been enriched with CWE, CVSS, and CPE data. The vulnerability management community was told that it would be addressed at Vulncon this year. At the conference, we were told the enrichment would restart “in the next couple of…
22 May 2024
Balancing Old Tricks with New Feats: AI-Powered Conversion From Enzyme to React Testing Library at Slack
SlackUpdate (October 2024): In response to numerous requests from external developers, we have open-sourced a version of our Enzyme to React Testing Library (RTL) conversion tool. You can now find it on npm, along with detailed instructions on how to integrate and use it in your projects. In the world of frontend development, one thing remains…
25 Apr 2024
At first glance, it may sound absurd. Here we have technical debt, a purely engineering problem, as technical as it can get, and another area,... The post Technical debt and HR – what do they have in common? appeared first on Erlang Solutions.
18 Apr 2024
At Slack, we’ve long been conservative technologists. In other words, when we invest in leveraging a new category of infrastructure, we do it rigorously. We’ve done this since we debuted machine learning-powered features in 2016, and we’ve developed a robust process and skilled team in the space. Despite that, over the past year we’ve been…
22 Feb 2024
Elixir's syntax, core features and history - tap into expert insights to unleash its full potential. Discover programming language that is celebrated for its fault-tolerance and concurrency features. The post What is Elixir? appeared first on Erlang Solutions.
18 Jan 2024
Most of Slack runs on a monolithic service simply called “The Webapp”. It’s big – hundreds of developers create hundreds of changes every week. Deploying at this scale is a unique challenge. When people talk about continuous deployment, they’re often thinking about deploying to systems as soon as changes are ready. They talk about microservices…
5 Jan 2024
Every year, I get asked, “How many CVEs do you think will be published this year?“ I am always willing to take a guess, but last year, I read Time Series Forecasting in Python. As I started to read more about the Kalman Filter, I figured it would work great for predicting CVE growth, so I built a simple model…
3 Jan 2024
2023 marked another year of record growth in CVE data, and I thought it fitting to kick off the new year by delving into these statistics and showcasing some of the more interesting data points. CVEs By The Numbers We ended 2023 with 28,902 published CVEs, up over 15% from the 25,081 CVEs published in 2022. On average, there were…
12 Dec 2023
We are heavy users of Amazon Compute Compute Cloud (EC2) at Slack — we run approximately 60,000 EC2 instances across 17 AWS regions while operating hundreds of AWS accounts. A multitude of teams own and manage our various instances. The Instance Metadata Service (IMDS) is an on-instance component that can be used to gain an…
5 Dec 2023
Slack users have more power than ever to automate routine tasks and processes, saving themselves time each day. Workflow Builder, a task automation tool built into Slack, has continued to improve since its launch back in 2019. Along with various new steps and triggers, we built a new sidebar section for all available workflow steps.…
28 Nov 2023
Slack Connect, AKA shared channels, allows communication between different Slack workspaces, via channels shared by participating organizations. Slack Connect has existed for a few years now, and the sheer volume of channels and external connections has increased significantly since the launch. The increased volume introduced scaling problems, but also highlighted that not all external connections…
27 Nov 2023
Introduction Background .NET is an ecosystem of frameworks, runtimes, and languages for building and running a wide range of applications on a variety of platforms and devices. The .NET Framework was initially released in the early 2000s as Microsoft’s implementation of the Common Language Infrastructure (CLI) specification. In 2016, Microsoft released .NET Core, the first […]
9 Oct 2023
Introduction Ever wondered what it’s like to intern as a software engineer at Slack? Picture yourself on the famous Ohana floor—the 61st floor of the Salesforce Tower in San Francisco— it is one of many privileges we had as interns. Not only did our experience with Slack’s Data Engineering team let us step onto the…
6 Sept 2023
Embarking on a journey Stepping out of SFO with the familiarity of the fogginess of the city, my story at Slack unfolds once again. As a return intern, I found myself prepped for another exciting summer, and this opportunity encompassed a renewed sense of anticipation — a mix between known pathways and new adventures. Returning…
28 Aug 2023
Slack handles billions of inbound network requests per day, all of which traverse through our edge network and ingress load balancing tiers. In this blog post, we’ll talk about how a request flows — from a Slack’s user perspective — across the vast ether of the network to reach AWS and then Slack’s internal…
22 Aug 2023
Summary In recent years, cellular architectures have become increasingly popular for large online services as a way to increase redundancy and limit the blast radius of site failures. In pursuit of these goals, we have migrated the most critical user-facing services at Slack from a monolithic to a cell-based architecture over the last 1.5 years.…
2 Aug 2023
Hacker Summer Camp, as it is colloquially known, is three security conferences that are all next week in Las Vegas. The three conferences that makeup Security Summer Camp are: While preparing for these conferences, I dug through their schedules and picked out the talks I was most interested in catching. BSides Las Vegas BSides Las Vegas is back with a…
26 Jul 2023
Customer-first: Moving from Hero Engineering to Reliability Engineering From the beginning, Slack has always had a strong focus on the customer experience, and customer love is one of our core values. Slack has grown from a small team to thousands of employees over the years and this customer love has always included a focus on…
20 Jul 2023
In our newest blog post, we delve into the game-changing potential of the Internet of Things (IoT) in supply chain management. The post How IoT is Revolutionising Supply Chain Management appeared first on Erlang Solutions.
14 Jul 2023
In the latest follow-up article by Oleg Tarasenko, he shares his further findings towards streamlining the scraping process with Crawly YML. The post Effortlessly Extract Data from Websites with Crawly YML appeared first on Erlang Solutions.
3 Jul 2023
With the first half of 2023 over, I figured I would take some time and review the data and highlight some of the most interesting data points so far this year. This GitHub repo contains the code for all the data and graphs this blog uses. By The Numbers So far this year, there have been 14,129 published CVEs. On…
29 Jun 2023
As businesses are more focused on measuring Environmental, Social and Governance (ESG), it's time to consider the value behind green coding practices. The post The Business Value Behind Green Coding appeared first on Erlang Solutions.
27 Jun 2023
Discover the transformative power of Industry 4.0. Explore the benefits and versatility Erlang and Elixir provides to tackle these new and existing protocols. The post IoT Complexity Made Simple with the Versatility of Erlang and Elixir appeared first on Erlang Solutions.
12 Jun 2023
The waiting list for Early Bird RabbitMQ Summit tickets is now available. The post Sign up for the RabbitMQ Summit Waiting List appeared first on Erlang Solutions.
9 Jun 2023
Introduction Process Injection is a popular technique used by Red Teams and threat actors for defense evasion, privilege escalation, and other interesting use cases. At the time of this publishing, MITRE ATT&CK includes 12 (remote) process injection sub-techniques. Of course, there are numerous other examples as well as various and sundry derivatives. Recently, I was […]
1 Jun 2023
Ever wondered the impact ChatGPT can have on your Elixir code? The post How ChatGPT improved my Elixir code. Some hacks are included. appeared first on Erlang Solutions.
28 Apr 2023
(cover image from ThisisEngineering RAEng) Let’s face it: software is easier to write than maintain. This is why we, as software engineers, prefer to just “rip it out and start over” instead of trying to understand what another developer (or our past self) was thinking. We seem to have collectively forgotten that “programs must be […]
11 Apr 2023
Did you know that ground stations transmit signals to satellites 22,236 miles above the equator in geostationary orbits, and that those signals are then beamed down to the entire North American subcontinent? Satellite radios today serve hundreds of channels across 9,540,000 square miles. Unless you’re working at a secret military facility, deep underground, you can…
4 Apr 2023
Notifications are a key aspect of the Slack user experience. Users rely on timely notifications of mentions and DMs to keep on top of important information. Poor notification completeness erodes the trust of all Slack users. Notifications flow through almost all the systems in our infrastructure. As illustrated in Figure 1 below, a notification request…
3 Apr 2023
Reference Rot (also called linked rot) is when hyperlinks, over time, cease to point to their originally targeted file, web page, or server due to that resource being relocated to a new address or permanently unavailable. Tod Beardsley from the CVE board gave a talk at the 2023 CVE Global Summit called ‘Link Rot: The Problem and Archiving for Posterity‘…
21 Mar 2023
I wanted to implement concise “pattern matching” in Python, a language which unlike C#, F#, Scala, and so on, does not have any pattern matching built in. Logically a pattern is just a predicate: a function which takes a value … Continue reading →
This blog post discusses the strategies that Slack uses to manage the lifecycle (development, support, and eventual retirement) of infrastructure projects, through the lens of the migration through three successive internal “platform” offerings. Our challenges Circa 2020, our Cloud Engineering team (now evolved into multiple teams responsible for narrower aspects) was responsible for managing our…
8 Mar 2023
3.18.0 Release with Support for more Diagnostics, SQL/JSON, Oracle Associative Arrays, Multi dimensional Arrays, R2DBC 1.0
jOOQDiagnosticsListener improvements A lot of additional diagnostics have been added, including the automated detection of pattern replacements, helping you lint your SQL queries irrespective of whether you’re using jOOQ to write your SQL, or if you’re using it as a JDBC / R2DBC proxy for an existing application. A lot of these diagnostics are available … Continue reading 3.18.0 Release…
23 Feb 2023
Before getting into the details of how my combinator-inspired source code transformation system works, I should say first, what is a general overview of the system? and second, why did I build it at all? In my experience, a typical … Continue reading →
8 Feb 2023
How do we write a compiler in a typical general-purpose line-of-business OO programming language such as Python, C#, Java, and so on? Compilers are programs, so we could make the question more general: how do we write programs? The basic … Continue reading →
3 Feb 2023
The European starling is a lovely looking bird, though territorial, noisy and aggressive up close. Unfortunately, they are very invasive in North America. Most of the hundreds of millions of European starlings now living in the Americas can be found … Continue reading →
1 Feb 2023
In the autumn of last year my friend Joan and I went on a little trip up to the Skagit valley north of Seattle to photograph birds of prey; I managed to get a blurry but recognizable shot of this … Continue reading →
After Duplo modularization, we noticed that the task producing a transitive R class was taking a significant amount of time to execute. To eliminate this task altogether, and since the non-transitive R class is advertised to have up to 40% incremental build time improvement, we decided to migrate our codebase to use it. If you’re not…
30 Jan 2023
Reader “Joel” had an insightful comment on the first part of this series which I thought deserved a short episode of its own. Recall that we proved the theorem “if a compositional forest contains a mockingbird then every bird in … Continue reading →
17 Jan 2023
For the next part in my Bean Machine retrospective to make sense I’ll need to make a short digression. In looking back on the almost 20 years I’ve been blogging, it is surprising to me that I’ve only briefly alluded … Continue reading →
5 Jan 2023
Happy New Year all! Last time I briefly described the basic strategy of the Beanstalk compiler: transform the source code of each queried or observed function (and transitively their callees) into an equivalent program which partially evaluates the model, accumulating a graph as it goes. … Continue reading →
1 Jan 2023
2022 was a record-breaking growth year for CVE data, and I figured it would be a great way to start the new year by going through the data and highlighting some of the most interesting data points. All the data and graphs used in this blog are available in this GitHub repo. CVEs By The Numbers We ended 2022 with…
20 Dec 2022
Let’s take another look at the “hello world” example and think more carefully about what is actually going on: There’s a lot going on here. Let’s start by clearing up what the returned values of the random variables are. It … Continue reading →
14 Dec 2022
I’ll get back to Bean Machine and Beanstalk in the next episode; today, a brief diversion to discuss a general principle of language design and congratulate some of my former colleagues. Back when we were all at Waterloo, a bunch … Continue reading →
9 Dec 2022
Did I actually build a compiler? Yes and no. Traditionally we think of a compiler as a program which takes as its input the text of a program written in one language (C#, say), and produces as its output an … Continue reading →
7 Dec 2022
Introducing Beanstalk Last time I introduced Bean Machine Graph, a second implementation of the PPL team’s Bayesian inference algorithm. We can compare and contrast the two implementations: In short, the BMG user experience is comparatively not a great experience for … Continue reading →
5 Dec 2022
Introducing Bean Machine Graph Bean Machine has many nice properties: I’m not going to go into details of how Bean Machine proper implements inference, at least not at this time. Suffice to say that the implementation of the inference algorithms … Continue reading →
2 Dec 2022
As I mentioned in the previous episode, the entire Bean Machine team was dissolved; some team members were simply fired, others were absorbed into other teams, and some left the company. In this series I’m going to talk a bit … Continue reading →
30 Nov 2022
It’s been almost two years since my last update here. A lot has happened. I hope you all are continuing to weather the ongoing multiple global pandemics and other anthropogenic crises. Apologies that this is so long; I didn’t have … Continue reading →
16 Nov 2022
The National Vulnerability Database plays a vital role in the CVE publication process that many people may overlook or not know they are responsible for. After MITRE publishes a CVE, the NVD enriches it with data points that make it actionable by security companies and professionals. Some of these data points include:CWECVSS 3.1 Base ScoreCPE I was recently asked how…
29 Oct 2022
For a recent project, I needed all the NVD CVE and EPSS data in Elasticsearch and couldn’t find an easy way to do it, so I built CVElk. CVElk quickly builds a local Elastic Stack using docker compose with the help of a simple shell and python script. Philipp Krenn from Elastic also contributed an updated dashboard to the project…
25 Oct 2022
At Slack, we use Terraform for managing our Infrastructure, which runs on AWS, DigitalOcean, NS1, and GCP. Even though most of our infrastructure is running on AWS, we have chosen to use Terraform as opposed to using an AWS-native service such as CloudFormation so that we can use a single tool across all of our…
3 Oct 2022
Internships are a great opportunity for companies to invest in great talent and train future engineers. It is important to prepare a good mentorship plan for the interns, so that they make the best use of their time and acquire experiences that will help make them full-time employees in the future. The first step is…
20 Sept 2022
You grinded LeetCode, nailed the interview process, and got an internship at an amazing company. Congrats! But now it’s week three of your internship, you have no idea how anything works, and you’ve written one line of code in the past two days. How do you ask for help? Whom do you ask for help?…
13 Sept 2022
An internship at Slack is an exciting opportunity to learn new skills, meet other engineers, and build cool stuff. This was the reality for three interns on the Data Engineering team this summer. Throughout our time in this flex-work environment, we got to experience both the wide reach of the virtual environment and the benefits…
6 Sept 2022
Slack, as a product, presents many opportunities for recommendation, where we can make suggestions to simplify the user experience and make it more delightful. Each one seems like a terrific use case for machine learning, but it isn’t realistic for us to create a bespoke solution for each. Instead, we developed a unified framework we…
31 Aug 2022
Our build platform is an essential piece of delivering code to production efficiently and safely at Slack. Over time it has undergone a lot of changes, and in 2021 the Build team started looking at the long-term vision. Some questions the Build team wanted to answer were: When should we invest in modernizing our build…
22 Aug 2022
Introduction Last year, I blogged about Investigating .NET CLR Usage Log Tampering Techniques For EDR Evasion. In that part 1 post, we covered: Recently, I revisited the research topic to close the loop on some outstanding research and figured I would share. In this post, we’ll recap .NET Usage Logs, highlight two other tampering techniques, […]
19 Aug 2022
What happens when your distributed service has challenges with stampeding herds of internal requests? How do you prevent cascading failures between internal services? How might you re-architect your workflows when naive horizontal or vertical scaling reaches their respective limits? These were the challenges facing Slack engineers during their day-to-day development workflows in 2020. Multiple internal…
27 Jul 2022
Security Summer Camp, as it is colloquially known, is three security conferences that occur during the same week in Las Vegas. The three conferences that make up Security Summer Camp are: BSides Las Vegas Blackhat USA DEF CON While preparing for these conferences, I dug through their schedules and picked out the talks I was interested in catching. BSides Las…
21 Jul 2022
Every codebase starts off small and modern. While it’s still small, the team can easily keep it up-to-date with current standards, upgrade libraries, and handle any code hygiene issues that may arise. Updating the API of a framework you built is easy when it’s only called in a handful of places. However, as the codebase…
28 Jun 2022
In this article, “remote development environments” refer to AWS EC2 instances where engineers make code changes and can see a running Slack application with those changes. For years, engineers at Slack isolated and tested their changes by running microcosms of the Slack application on their local computers. This was difficult for many reasons: it involved…
7 Jun 2022
Last September, Slack released Clips, allowing users to capture video, audio, and screen recordings in messages to help distributed teams connect and share their work. We’ve continued iterating on Clips since its release, adding thumbnail selection, background blur, and most recently, background image replacement. This blog post provides a deep dive into our implementation of…
13 May 2022
Covid restrictions are starting to be relaxed, so I am beginning to feel like Willie and am getting on the road again, and in the next six weeks, I am attending and presenting at these four amazing events. BSidesKy I am amazingly excited to attend bsides.ky in the Cayman Islands at the end of May, where I will be leading…
4 May 2022
Every now and then I run across a use case for the arcane NATURAL JOIN SQL operator, and I’m even more delighted when I can make that a NATURAL FULL JOIN. A few past blog posts on the subject include: Use NATURAL FULL JOIN to compare two tables in SQL Impress Your Coworkers With the … Continue reading A Quick…
In the first two posts about the Duplo initiative, we described why we decided to revamp our mobile codebases, the initial phase to clean up tech debt, and our efforts to modularize our iOS and Android codebases (post 1, post 2). In this final post, we will discuss the last theme of the Duplo initiative,…
29 Apr 2022
Building load test infrastructure is tricky and poses many questions. How can we identify performance regressions in newly deployed builds, given the overhead of spinning up test clients? To gather the most representative results, should we load test at our peak hours or when there’s a lull? How do we incentivize engineers to invest time…
26 Apr 2022
By Laura Nolan, with contributions from Glen D. Sanford, Jamie Scheinblum, and Chris Sullivan. Assessing conditions Slack experienced a major incident on February 22 this year, during which time many users were unable to connect to Slack, including the author — which certainly made my role as Incident Commander more challenging! This incident was a…
5 Apr 2022
At Slack, the goal of the Mobile Developer Experience Team (DevXp) is to empower developers to ship code with confidence while enjoying a pleasant and productive engineering experience. We use metrics and surveys to measure productivity and developer experience, such as developer sentiment, CI stability, time to merge (TTM), and test failure rate. The DevXp…
2 Apr 2022
Yes, you read that correctly – “Dynamic Pinvoke” as in “Dynamic Platform Invoke” Background Recently, I was browsing through Microsoft documentation and other blogs to gain a better understanding of .NET dynamic types and objects. I’ve always found the topic very interesting mainly due to its relative obscurity and the offensive opportunities for defensive evasion. […]
28 Mar 2022
In the first post about the Duplo initiative, we discussed the reasons for launching a project to revamp Slack’s mobile codebases, and what we accomplished in Duplo’s initial Stabilization phase. This post will explore modularization, and then there will be a third post to describe how we modernized our codebase and the overall results of…
9 Mar 2022
According to a recent Thoughtworks radar, “the industry is increasingly gaining experience with platform engineering product teams that create and support internal platforms.” They caveated this with a piece of advice: “When creating a platform, it’s critical to have clearly defined customers and products that will benefit from it rather than building in a vacuum.”…
18 Feb 2022
In 2021, we changed developer testing workflows for Webapp, Slack’s main monorepo, from predominantly testing before merging to a multi-tiered testing workflow after merging. This changed our previous definition of safety and developer workflows between testing and deploys. In this project, we aimed to ensure frequent, reliable, and high-quality releases to our customers for a…
11 Feb 2022
In this article, I will talk about how Slack uses Kafka, and how a small-but-mighty team built and operationalized a self-driving Kafka cluster over the last four years to run at scale. Kafka is used at Slack as a pub-sub system, playing an essential role in the all-important Job Queue, our asynchronous job execution framework…
12 Jan 2022
When do you need to overhaul a large code base to address tech debt? What is the best way to address widespread inconsistencies and outdated patterns? How can you make significant architectural improvements to a complex application while still continuing to ship features? These were questions we grappled with at the beginning of 2020, when…
30 Dec 2021
I have spent a lot of time this year working with CVE data and most of that time in Jupyter notebooks. Over the holiday season, I decided to build a website from these notebooks using Github Actions, Github Pages and NBConvert. CVE.ICU ended up being the end product, and here is the source code. It is still an early work…
14 Dec 2021
While adding a recent feature to our Kubernetes compute platform, we had the need to mutate newly-created pods based on annotations set by users. The mutation needed to follow simple business rules, and didn’t need to keep track of any state. Surely there must be a canonical solution to this simple problem? Well, sort of.…
2 Dec 2021
Over 70% of the files uploaded on Slack are images, and over 75% of those images are screenshots. What this tells us is that though images are ephemeral, screenshots are often used as a quick way to provide extra detail and context, and typically gain a high level of engagement over a short time period.…
29 Nov 2021
On September 30th 2021, Slack had an outage that impacted less than 1% of our online user base, and lasted for 24 hours. This outage was the result of our attempt to enable DNSSEC — an extension intended to secure the DNS protocol, required for FedRAMP Moderate — but which ultimately led to a series of…
10 Nov 2021
We use plenty of open source tools at Slack and we’ve benefited immensely from the wider Android, Kotlin, and Gradle communities. We also try to be good citizens by giving back. This includes things like sponsoring the Kotlin Lang Slack, contributions to projects we use like Anvil and Insetter, sharing projects of our own like…
20 Oct 2021
About a year ago, I wrote a blog post called Building the Next Evolution of Cloud Networks at Slack. In it, we discussed how Slack’s AWS infrastructure has evolved over the years and the pain points that drove us to spin up a brand-new network architecture redesign project called Whitecastle. If you have not had…
8 Oct 2021
Introduction It is always fun to reexplore previously discovered techniques or pick back on old research that was put on the wayside in hopes to maybe finding something new or different. Recently, I stood up an ESXi server at home and decided to take a quick peak at the VMware directory structure after installing the […]
7 Oct 2021
Slack is an integral part of where work happens for teams across the world, and our work in the Core Development Engineering department supports engineers throughout Slack that develop, build, test, and release high-quality services to Slack’s customers. In this article, we share how teams at Slack evolved our internal tooling and made infrastructure bets.…
7 Sept 2021
On July 21st, 2021, Slack officially became a part of Salesforce at the price tag of $27.7 billion. This was undoubtedly Slack’s most significant event in the past year. As a frontend intern on Slack’s Customer Acquisition team this summer, I had the once-in-a-lifetime opportunity to be directly involved with announcing the closing of one…
1 Sept 2021
Agile development methods can bolster company culture and empower teams to move quickly, with a focus on frequently adding value for customers. Whether you are a program manager, game developer, event planner, or architect, within businesses where change is constant, it’s key to have flexibility, and that’s where agile shines. While there are a variety…
25 Aug 2021
We recently rolled out support for Conversation Bubbles for DMs and Group DMs on Android 11. In case you’re not familiar with Conversation Bubbles, take a look at the video below. Basically, they are a way to pop out a conversation from a notification into a Bubble that will draw over other apps, making multitasking…
17 Aug 2021
Reinventing how the world does work inevitably creates a lot of data. Each year, Slack’s scale has increased and the volume of data ingested and stored has kept pace. To make it possible to understand relationships within our data, we’ve invested heavily in an automated data lineage framework. This facilitates producer/consumer coordination, improves risk mitigation,…
7 Aug 2021
TL;DR Intel Driver & Support Assistant (DSA) is a driver and software update utility for Intel components. DSA version 20.8.30.6 (and likely prior) is vulnerable to a local privilege escalation reparse point bug. An unprivileged user has nominal control over configuration settings within the web-based interface. This includes the ability to configure the folder location […]
28 Jul 2021
With the release of Slack Connect, people can now collaborate both with internal employees and external organizations in the same channel. To make this as smooth as possible, Slack does predictive email analysis to classify and recommend the best way for a user to work with people they want to collaborate with. To accomplish this,…
23 Jul 2021
In a Study in Scarlet, Sherlock Holmes said, “It is a capital mistake to theorize before one has data,” which is one of my favorite Sherlock quotes. For the last month or so, my team has been dealing with missing CPE data points in the Mitre CPE data, and it finally forced me to set down and put together a…
20 Jul 2021
Note: This article assumes some familiarity with Dagger, Anvil, and Kotlin. We use Dagger heavily in the Slack Android app for compile-time dependency injection. It’s powerful, flexible, supports basic Kotlin idioms, and allows for advanced dependency injection patterns with less boilerplate. It’s not without its sharp edges though. It slows down our builds with kapt,…
17 Jul 2021
I was recently asked if I had ever thought about trying to predict CVE growth. I had not, or really didn’t even know where to start, but after some research, I found the Prophet project that is a forecasting algorithm open-sourced by Facebook and uses the GAM family of algorithms. Using prophet with the NVD data in a Jupyter notebook…
13 Jul 2021
At Slack, we believe that designing an optimal keyboard experience is key to delivering a best-in-class product for all our customers. However, despite our design system components being individually accessible, we heard from keyboard users that we were still missing focus transitions in their end-to-end user experience. Non-sighted users who relied on a screenreader constantly…
3 Jun 2021
Artwork courtesy of the Jenkins project. At Slack we manage a sophisticated Jenkins infrastructure to continuously build and test our mobile apps before release. We have hundreds of jobs running in a variety of different environments. One day something very odd happened — our Jenkins UI stopped working although the jobs continued to run. This…
30 May 2021
Background As discussed in this previous post, Microsoft has provided valuable (explicit and implicit) insight into the inner workings of the functional components of the .NET ecosystem through online documentation and by open-sourcing .NET Core. .NET, in general, is a very powerful and capable development platform and runtime framework for building and running .NET managed […]
24 May 2021
Controlling which users are able to take which actions is no simple task. Building this into Slack has always been an interesting challenge. In large enterprise organizations, the standard types of roles we offered to customers were too broad, and delegating a generic admin role can grant someone with too much power — what if…
23 Apr 2021
Complex systems are difficult to reason about at scale; we often can’t accurately extrapolate system behavior and performance, so we need to derive that data empirically. We use load testing to do just that: find the limits of our systems and weed out bugs at a large scale in a controlled environment. Slack is a…
6 Apr 2021
The first quarter of 2021 has been a busy quarter for the Project Zero (P0) team as they announced 16 “in the wild” zeros days. That is one new announcement a week on average. This is great for driving news cycles or if you’re in marketing and need some FUD to help sales. This isn’t so great if you are…
16 Mar 2021
Introduction In recent years, there have been numerous published techniques for evading endpoint security solutions and sources such as A/V, EDR and logging facilities. The methods deployed to achieve the desired result usually differ in sophistication and implementation, however, effectiveness is usually the end goal (of course, with thoughtful consideration of potential tradeoffs). Defenders can […]
28 Dec 2020
That was the simple question I asked myself on Saturday morning, thinking the answer would likely be simple to find. It wasn’t and ended up 48 hours later with me building this jupyter notebook to find out. I really thought it would be as easy as pulling down the NVD data feeds and running a simple nvd['Published'].value_counts().head(10) to find out…
17 Dec 2020
I monitor the @CVENew Twitter feed to keep up with any interesting new vulnerabilities that are released. On December 11th CVE-2020-29589 was published claiming that “the kapacitor Docker images through 1.5.0-alpine contain a blank password for the root user” and that it has a CVSS score of 9.8. This CVE was just a re-report of CVE-2019-5021, which I researched last…
15 Dec 2020
Since I’m staying home all day due to the ongoing pandemic emergency, I’ve decided to document all the different species of birds that arrive in my yard. I am not a great bird photographer but I am enjoying practicing every … Continue reading →
14 Dec 2020
If you receive the error ERR_CONNECTION_RESET on one website only, it is highly likely that something went wrong on the server side. Receiving random ERR_CONNECTION_RESETs A few weeks ago one of my colleagues complained to me: Sometimes, when he was using the WordPress theme editor, he either received an empty […] The post Diagnosing and fixing an ERR_CONNECTION_RESET error in…
11 Dec 2020
I joined Kenna Security two years ago as their Principal Security Engineer not long after my friend JCran joined as the Head of Research. In the last two years, while building the security team, I have stayed deeply involved with the research team, and from time to time, some of that research was made public: Fifth of Docker Containers Have…
2 Nov 2020
Exploring the WDAC Microsoft Recommended Block Rules (Part II): Wfc.exe, Fsi.exe, and FsiAnyCpu.exe
BohopsIntroduction In Part One, I blogged about VisualUiaVerifyNative.exe, a LOLBIN that could be used to bypass Windows Defender Application Control (WDAC)/Device Guard. The technique used for circumventing WDAC was originally discovered by Lee Christensen, however, it was not previously disclosed like a handful of others on the Microsoft Recommended Block Rules list. If you are […]
29 Oct 2020
Earlier this week I was looking for an old photo, and while browsing I came across a photo I took of my whiteboard in my office at Microsoft in 2004. Or rather, it was two photos; I’ve crudely stitched them … Continue reading →
15 Oct 2020
Introduction If you have followed this blog over the last few years, many of the posts focus on techniques for bypassing application control solutions such as Windows Defender Application Control (WDAC)/Device Guard and AppLocker. I have not been blogging as much lately but wanted to get back into the rhythm and establish a similar theme […]
23 Sept 2020
The final part of my Life series is still in the works but I need to interrupt that series with some exciting news. The new programming language I have been working on for the last year or so has just … Continue reading →
8 Sept 2020
Episode 34 will be delayed again — sorry! — because once again the time I had set aside for writing this weekend got consumed by a real-world task that could not wait. (I will try for Thursday of this week.) … Continue reading →
31 Aug 2020
Episode 34 of my ongoing series will be slightly delayed because I spent the time on the weekend I normally spend writing instead rebuilding one of my backyard fences. I forgot to take a before picture, but believe me, it … Continue reading →
27 Aug 2020
Github Actions was launched last November and it has taken a little while to mature but it has recently got to the point where you can build a fairly robust application security pipeline using Github actions. In most of my projects, I can run a Linter, an SCA, a SAST and DAST tool aginst my code daily using open source…
21 Aug 2020
Mozilla announced some general changes in our investments and we would like to outline how they will impact our MDN platform efforts moving forward. It hurts to make these cuts, and it’s important that we be candid on what’s changing and why. The post An Update on MDN Web Docs appeared first on Mozilla Hacks - the Web developer blog.
Part 33 of my ongoing series is coming but I did not get all the code written that I wanted to this week, so it will be delayed. In the meanwhile: Living in Canada as a child, of course I … Continue reading →
14 Aug 2020
Normally this time of year I would be visiting friends and family in Canada, but obviously that’s impossible right now. Instead we took a long weekend at a rental on Bainbridge Island and strolled around some parks in a socially … Continue reading →
15 Jul 2020
I went out to Shilshole Bay Marina Tuesday night to get a few photos of the comet; it is quite spectacular! If you’re going stargazing this week, bring binoculars, look to the northwest about an hour after sunset, below the … Continue reading →
18 May 2020
I have been spending a lot of time over the last few weeks looking at the OSQuery to get a better understanding of what it can do since it seems every major security tool from Sophos to Cisco to CarbonBlack is building it into their product. I have also been looking at Juypter notebooks for machine learning and data science…
12 May 2020
Introduction Lateral movement techniques in the wonderful world of enterprise Windows are quite finite. There are only so many techniques and variations of those techniques that attackers use to execute remote commands and payloads. With the rise of PowerShell well over a decade ago, most ethical hackers may agree that Windows Remote Management (WinRM) became […]
10 Apr 2020
Welcome to yet another working-from-home pandemic episode of Fun For Friday Fabulous Adventures. Over the past while I’ve gradually been looking for music, movies and games I enjoyed as a teenager and seeing how they hold up. So I am … Continue reading →
27 Mar 2020
A student who I used to tutor in CS occasionally sent me a meme yesterday which showed “NEW GRAD vs SENIOR DEVELOPER”; the new grad is all caps yelling NO! YOU CAN’T JUST USE BRUTE FORCE HERE! WE NEED TO … Continue reading →
25 Mar 2020
Did you know you can easily turn any video from Youtube into a background for Zoom (Version 4.6.4+) using a simple command-line tool called Youtube-DL. One of my favorite videos is The Traveling Bird Feeder so I will use it for this example. Install Youtube-dl: brew install youtube-dl Then fingerprint the video: youtube-dl -F https://www.youtube.com/watch?v=vu72ja_mGME Then download any video larger…
24 Mar 2020
I don’t enjoy politics, I don’t know enough about it, and my privilege greatly insulates me from its negative effects, and so I don’t talk about it much on this blog. My intention in creating the blog lo these decades … Continue reading →
20 Mar 2020
Good Friday afternoon all and welcome to this working-from-home-and-obsessively-washing-hands edition of FAIC. I am posting today from my recently-transformed spare room which is now apparently my office. Scott Hanselman started a great twitter thread of techies showing off their home … Continue reading →
12 Mar 2020
Sophos’ access points are very sensitive with PoE enabled cable connections. If your access point gets disconnected, disable PoE for the given Ethernet port. As a long time reader of my blog, you might know that I am struggling a lot with curious issues like broken L2TP/IPSec VPN connections. This […] The post Sophos UTM and AP15: fixing “ll_read: dead…
27 Feb 2020
My manager and I got off on a tangent in our most recent one-on-one on the subject of the durability of design mistakes in programming languages. A particular favourite of mine is the worst of the operator precedence problems of … Continue reading →
24 Feb 2020
Well this is a first. Twitter user Plazmaz brought a scam github repository and web site to my attention; see his thread on Twitter for details. It’s a pretty obviously fake site, and there is some evidence in the metadata … Continue reading →
23 Feb 2020
Last summer I launched vulnerablecontainers.org to help shed light on the number of vulnerabilities in the 1,000 most popular containers on docker hub. While it was an interesting project, right after I launched the project I had multiple people ask if it was able to scan other public containers. Initially, it wasn’t but I wanted to offer the ability, so…
18 Feb 2020
With the RSA Conference less than a week away I figured I would spend a few minutes and write a quick post about what I am excited to see this year in San Francisco. Not At RSA Like most security conferences these days while the conference itself is the reason I go the auxiliary events end up providing a majority…
14 Feb 2020
Introduction Microsoft Teams Rooms (MTR), formerly known as Skype Room System and Lync Room Systems, is the latest and greatest solution from Microsoft for managing online collaborative meetings. In many businesses across the globe, a Teams Rooms console (“Teams console”) is the lifeblood of the conference room. The console typically consists of a supported computer […]
24 Jan 2020
While watching the first episode of the new Star Trek series just now I noticed a nice little Easter egg: Admiral Picard (retired) apparently has the same 1982 science fiction book club edition of The Complete Robot handy on his … Continue reading →
16 Jan 2020
One of my personal projects this year is to understand and build a SLAM (Simultaneous localization and mapping) robot. To get started I bought the Xaxxon OpenLidar and after a few struggles getting it to work correctly in a VM I finally did and decided to throw together my build notes for future reference. Virtual Platform While I would have…
5 Jan 2020
This is part five of a series on the Random Oracle Model. See here for the previous posts: Part 1: An introduction Part 2: The ROM formalized, a scheme and a proof sketch Part 3: How we abuse the ROM to make our security proofs work Part 4: Some more examples of where the ROM … Continue reading What is…
30 Dec 2019
One last post for this decade. There has been some discussion on tech twitter lately on the subject of whether it is possible to be “successful” in the programming business without working long hours. I won’t dignify the posts which … Continue reading →
26 Dec 2019
As the 2010s come to an end I started to think about what security stories from the last ten years changed how we think about security in this decade and the next. While this list is in no way complete these are the ten stories that I think had a lasting impact on security in the last decade and the…
12 Dec 2019
I had a new years resolution to Read More Books this past year and actually read around 20 books this year. Out of those books here is a quick list of some of my favorites from the past year that I really enjoyed. Stillness Is the Key This book was probably one of the most impactful books I read this…
11 Dec 2019
You might recall that before my immensely long series on ways we could make C# a probabilistic programming language, I did a short series on how we can automatically computed the exact derivative in any direction of a real-valued function … Continue reading →
6 Dec 2019
I spent the last week at AWS re:Invent 2019 in Las Vegas with over 65,000 other AWS users. This conference is always jammed packed with announcements and interesting discussions with people both inside and outside of my normal security bubble. Overall I really enjoy this conference even though it is ridiculously large and I spent over 6 hours on the…
14 Nov 2019
CVE-2019-1378: Exploiting an Access Control Privilege Escalation Vulnerability in Windows 10 Update Assistant (WUA)
BohopsIntroduction Windows 10 is an incredibly feature rich Operating System (OS). In the last four years, the innovative folks at Microsoft have continued to introduce and expand functionality as well as improve and integrate security features in its flagship OS. On the second Tuesday of each month, many of us that live in the Windows […]
12 Nov 2019
This week I gave a talk on Hacking Holiday Lights at Kenna Security and here is the promised accompanying blog that outlines the hardware and software I demoed for easy reference for anyone who wants to build their own holiday lights. Controller Boards I looked at a bunch of different boards that ended up having a variety of technical hurdles…
8 Nov 2019
Source code for this episode is here. Welcome to this special bonus episode of Fixing Random, the immensely long blog series where I discuss ways to add probabilistic programming features into C#. I ran into an interesting problem at work … Continue reading →
23 Oct 2019
I have been meaning to look at Cartography since I saw their talk at BSidesSF last year and I finally had a chance to start looking at it today. One of the first things I noticed was that is was not containerized so I built a quick container for it and decided to document my progress here. Prerequisites AWS CLI…
22 Oct 2019
This is a short example of how to use SUNDIALS to solve a simple partial differential equation in Haskell via the hmatrix-sundials library. The example is taken from the C examples that come with the SUNDIALS source. Here’s the full blog. I’ll give a better URL soonish.
24 Sept 2019
Edward Snowden recently released his memoirs. In some parts of the Internet, this has rekindled an ancient debate: namely, was it all worth it? Did Snowden’s leaks make us better off, or did Snowden just embarass us and set back U.S. security by decades? Most of the arguments are so familiar that they’re boring at … Continue reading Looking back…
19 Sept 2019
I just spent a day and a half recovering my Github account after the code in my 2FA application stopped working for authentication. GitHub has a good support article on how to recover your account that has this ominous warning on it: Warning: For security reasons, GitHub Support may not be able to restore access to accounts with two-factor authentication…
19 Aug 2019
[*] Introduction .NET Core is an open-source, cross-platform framework for building and running applications. The framework was introduced in 2014 as the (eventual) successor to the ever-popular .NET Framework. .NET Core runs on Windows, *Nix, and MacOS operating systems. The .NET Core management tool, DotNet (dotnet.exe), potentially offers an untapped attack surface on Windows when […]
16 Aug 2019
My friend Larry from the previous episode mentioned to me that a group of several male and female belted kingfishers had been spotted at the river; I’d never seen kingfishers at our little river before and I wanted to get … Continue reading →
15 Aug 2019
I enjoy photographing dragonflies and damselflies; this year I got some pretty reasonable shots of common blue damselflies, white-faced meadowhawks, a twelve-spotted skimmer, and my favourite, ebony jewelwings. It can be hard to get these little guys in focus, but … Continue reading →
14 Aug 2019
Today, I have a Mystery Of The Unknown for you to solve. Unlike most of the puzzlers on this blog, I don’t know the answer. UPDATE: Mystery solved! See below. On August 4th at about 20 minutes past 10 PM … Continue reading →
13 Aug 2019
I’m back from my annual vacation where I fly south to Canada and take way too many photos. As with all my hobbies, I’m not a very good nature photographer but I do enjoy it, and this year was particularly … Continue reading →
27 Jul 2019
All right, let’s finish this thing off! First, I want to summarize, second I want to describe a whole lot of interesting stuff that I did not get to, and third, I want to give a selection of papers and … Continue reading →
15 Jul 2019
Let’s sum up the last few episodes: Suppose we have a distribution of doubles, p, and a function f from double to double. We often want to answer the question “what is the average value of f when it is given samples … Continue reading →
8 Jul 2019
Last time on FAIC we were attacking our final problem in computing the expected value of a function f applied to a set of samples from a distribution p. We discovered that we could sometimes do a “stretch and shift” of … Continue reading →
About once a month I need a Kali VM to use for an hour or so, and I am terrible at keeping a VM up-to-date, so this weekend I took a few hours and built a tool to download automatically, provision and update a Kali Linux VM in Virtualbox. All the code for this project is in this Github Project.…
2 Jul 2019
Recently I have been working on a project to use the Trivy container scanner to scan large swath of containers for open vulnerabilities that I wanted to quickly post here. There is a full blog about the project here on the Kenna site. Here are some of the pages I have built out so far: Top 1000 Popular Containers Scanned…
1 Jul 2019
Last time on FAIC we finally wrote a tiny handful of lines of code to correctly implement importance sampling; if we have a distribution p that we’re sampling from, and a function f that we’re running those samples through, we can compute … Continue reading →
24 Jun 2019
One more time! Suppose we have our nominal distribution p that possibly has “black swans” and our helper distribution q which has the same support, but no black swans. We wish to compute the expected value of f when applied to samples … Continue reading →
17 Jun 2019
Last time on FAIC we deduced the idea behind the “importance sampling” technique for determining the average value of a function from double to double — call it f — when it is applied to samples from a possibly-non-normalized weighted distribution of … Continue reading →
10 Jun 2019
Last time on FAIC we implemented a better technique for estimating the expected value of a function f applied to samples from a distribution p: Compute the total area (including negative areas) under the function x => f(x) * p.Weight(x) … Continue reading →
3 Jun 2019
Last time on FAIC I showed why our naïve implementation of computing the expected value can be fatally flawed: there could be a “black swan” region where the “profit” function f is different enough to make a big difference in … Continue reading →
30 May 2019
I’m continuing with my project to port over, reformat and update a decade of old blog posts. Today, a few days in mid-October 2003; this is still my second month of blogging and I am writing at what I would … Continue reading →
28 May 2019
Last time on FAIC we reviewed the meaning of “expected value”: when you get a whole bunch of samples from a distribution, and a function on those samples, what is the average value of the function’s value as the number … Continue reading →
20 May 2019
Last time in this series we saw that we could compute a continuous posterior distribution when given a continuous prior and a discrete likelihood function; I hope it is clear how that is useful, but I’d like to switch gears … Continue reading →
16 May 2019
We’ll get back to stochastic programming soon; I wanted to do a quick post about some updates to my earlier series on anti-unification. As I noted in the final part of that series, I spent a few months in 2018 … Continue reading →
13 May 2019
Last time on FAIC I posed and solved a problem in Bayesian reasoning involving only discrete distributions, and then proposed a variation on the problem whereby we change the prior distribution to a continuous distribution, while preserving that the likelihood … Continue reading →
10 May 2019
[It is] a spectacular vindication of the principle that each individual coin spun individually is as likely to come down heads as tails and therefore should cause no surprise that each individual time it does. Thus Guildenstern (or is it … Continue reading →
6 May 2019
Last time on FAIC we implemented a technique for sampling from a non-normalized target PDF: Find an everywhere-larger helper PDF that we can sample from. Sample from it. Accept or reject the sample via a coin flip with the ratio … Continue reading →
4 May 2019
Introduction Last week, I presented COM Under The Radar: Circumventing Application Control Solutions at BsidesCharm 2019. In the presentation, I briefly discussed COM and highlighted a few techniques for bypassing Windows application control solutions. One of those techniques takes advantage of an issue with catalog hygiene where old code often remains signed in updated versions […]
2 May 2019
I Recently complete a basic ncurses based terminal application for minglinng with bits. I’m working low level most of the times and need to tinker with bits, masks and to translate between number bases (hex to dec, and vice verse). As I’m working 99% of my time in a terminal, I found it very annoying […]
Last time on FAIC we went through a loose, hand-wavy definition of what it means to have a “weighted” continuous distribution: our weights are now doubles, and given by a Probability Distribution Function; the probability of a sample coming from … Continue reading →
1 May 2019
I’m continuing in my efforts to move and update all my old content from my MSDN blog to ericlippert.com. Today, posts from early October of 2003. In, out, in-out, make up your mind already The late-binding code designed for OLE … Continue reading →
29 Apr 2019
We’ve been mostly looking at small, discrete distributions in this series, but we started this series by looking at continuous distributions. Now that we have some understanding of how to solve probability problems on simple discrete distributions and Markov processes, … Continue reading →
Are you looking to develop your own application on top of the Bazaarvoice Response API? Well, we got something for you. The Response API Demo App is a simple Node-React application which demonstrates how to use Response API in conjunction with our 3-legged OAuth2 API. It is recommended to go through the Developer Portal and […]
26 Apr 2019
Last time on FAIC we implemented the Markov process distribution, which is a distribution over state sequences, where the initial state and each subsequent state is random. There are lots of applications of Markov processes; a silly one that I’ve … Continue reading →
23 Apr 2019
[Code for this episode is here.] So far in this series we’ve very briefly looked at continuous distributions on doubles, and spent a lot of time looking at discrete distributions with small supports. Let’s take a look at a completely … Continue reading →
22 Apr 2019
I had the chance to attend LoCoMoCoSec this year and had a fantastic time. It was a well-run conference that was extremely focused on being friendly for families and being inclusive of the diverse group of people who make up our community. It also doesn’t hurt that it was in one of the most beautiful places I have ever seen.…
17 Apr 2019
So… I’ve got good news and bad news. The good news is: I’ve described an interface for discrete probability distributions and implemented several distributions. I’ve shown how projecting a distribution is logically equivalent to the LINQ Select operator. I’ve shown … Continue reading →
15 Apr 2019
[Code for this episode is here.] Last time in this series I left you with several challenges for improving our DSL for imperative probabilistic workflows. But first, a puzzle: Question One: You are walking down the street when you see … Continue reading →
12 Apr 2019
I’m continuing my efforts to port over and update my old blog content. The previous episode is here. We’re still in the first few weeks of me blogging; I was pumping out articles at a rate I now consider to … Continue reading →
11 Apr 2019
Last time in this series I proposed a stripped-down DSL for probabilistic workflows. Today, let’s see how we could “lower” it to ordinary C# 7 code. I’ll assume of course that we have all of the types and extension methods that … Continue reading →
9 Apr 2019
Thanks again to the good people at Microsoft who have kept my old blog alive for now; my plan is to port the articles from the old site over, and then they will redirect from the old URLs to the … Continue reading →
8 Apr 2019
Without further ado, here’s my proposed stripped-down C# that could be a DSL for probabilistic workflows; as we’ll see, it is quite similar to both enumerator blocks from C# 2 and async/await from C# 5. (Code for this episode can … Continue reading →
4 Apr 2019
I’ve got no code for you this time; instead here are some musings about language design informed by our discussion so far. One of the most important questions to ask when designing a language feature is: what should the balance … Continue reading →
2 Apr 2019
Before that silly diversion I mentioned that we will be needing the empty distribution; today, we’ll implement it. It’s quite straightforward, as you’d expect. [Code for this episode is here.] public sealed class Empty<T> : IDiscreteDistribution<T> { public static readonly Empty<T> Distribution = new Empty<T>(); private Empty() { } public T Sample() => throw new Exception(“Cannot sample from empty distribution”);…
1 Apr 2019
I just thought of a really cute application of the stochastic workflow technology we’ve been working on; most of the series has already been written but it fits in here, so I’m going to insert this extra bonus episode. We’ll … Continue reading →
I talk and chat to a lot with customers, prospects, or just entrepreneurs and business owners in various online and offline groups. There is one question that keeps being asked over and over; I often reply on-the-spot and I always The post Should You Outsource Your Core App or Software? appeared first on FullStack - Ofer Zelig's Blog.
22 Feb 2019
With the 2019 RSA Conference fastly approaching I thought I would take a few minutes and put together a quick list of what I am excited to see this year. Sunday BSides San Francisco How to Build an Application Security Program (Presenting) Automating Web Application Bug Hunting (Presenting With @JCran) Monday RSAC Innovation Sandbox Contest CSA Summit BSides San Francisco…
26 Jan 2019
Bundle Audit is a great tool to check if the Ruby Gems used in your project have any known vulnerabilities. Most DevOps teams I know run this tool against their builds in their CI/CD process when deploying. This can mean that code that is not updated often can have vulnerable gems unless you have a way to continually monitor your…
10 Jan 2019
Introduction Greetings, Everyone! It has been several months since I’ve blogged, so it seems fitting to start the New Year off with a post about two topics that I thoroughly enjoy exploring: Application Control/Application Whitelisting (AWL) and the Component Object Model (COM). As the title suggests, I stumbled upon a technique for bypassing Microsoft Application […]
17 Dec 2018
The past few years have been an amazing time for the deployment of encryption. In ten years, encrypted web connections have gone from a novelty into a requirement for running a modern website. Smartphone manufacturers deployed default storage encryption to billions of phones. End-to-end encrypted messaging and phone calls are now deployed to billions of users. While this … Continue…
15 Dec 2018
I have developed a bad habit of picking up vanity domain names and not really doing much with them. Last month at AWS Re:Invent I picked up ServerlessSecurity.org and really wanted to do something with it but didn’t feel like maintaining, or paying for, a VPS so after doing some looking around I found that is was possible to point…
10 Dec 2018
Here is a list of my favorite security books from 2018 if you are looking for that last minute gift or have some extra time around the holidays to catch up on some reading. The GCHQ Puzzle Book 2 I just got The GCHQ Puzzle Book 2, and like the original, it has quickly become the book that I always…
1 Dec 2018
I spent this last week in Las Vegas attending AWS Re:Invent. This event is mind-numbingly massive with classes happening at 4 or 5 hotels all over the strip. I personally spent over an hour every day on their (nice but extremely slow) shuttle buses between the MGM Grand, Aria and the Sands Expo Center. It would be impossible to see…
8 Nov 2018
I have started using the Burp Suite 2.0 beta full time recently, and some of the new features I knew I wanted to explore more was the API and the CI Integration. I took a few hours this last week and built a small POC shell script that will scan a website and open Github Issues for all findings. Here…
30 Oct 2018
Introduction I have always been a fan of Google Products, so when they announced the Google Home Hub, I ordered one. Once I got the Hub on my network I scanned it and it returned the following: Nmap scan report for hubHost is up (0.046s latency).Not shown: 995 closed portsPORT STATE SERVICE8008/tcp open http8009/tcp open ajp138443/tcp open https-alt9000/tcp open cslistener10001/tcp…
30 Aug 2018
The new rest API in Burp 2.0 it is going to be amazing but it will allow things like this 9 line shell script I wrote this morning that will grab all public bounty sites from @arkadiyt’s bounty-targets-data repo and kick off a full scan. https://gist.github.com/jgamblin/c22c0791af7572280d7fd569141650fe I almost didn’t post this blog because I *think* this script is, in general,…
28 Aug 2018
I spend a lot of time working with MacOS and I have noticed that out of the box the operating system has some basic security settings that are not enabled by default so I have built a small script that automates configuring these. It does the following: Requires Password Immediately After Sleep. Turns On Firewall. Enables Stealth Mode. Disables Remote…
18 Aug 2018
TL;DR There are several ways that attackers can leverage COM hijacking to influence evasive loading and hidden persistence. A few examples include CLSID (sub)key abandonment referencing, key overriding, and key linking. There are several programs and utilities that can invoke COM registry payloads including Rundll32.exe, Xwizard.exe, Verclsid.exe, Mmc.exe, and the Task Scheduler. In the traditional […]
4 Aug 2018
TL;DR An Office XML (.xml) document can call a remote XSL stylesheet over SMB. If this occurs against an attacker controlled server, the net-NTLM authentication hash (challenge/response) of that user is revealed. Operationally, an attacker could crack this offline or leverage a relay technique for remote command execution (if privileged and on-net). There are possible […]
27 Jul 2018
For over a year this blog has failed to deliver on an essential promise — that there would someday be pictures of dachshunds. Today we deliver. This is Callie (short for Calliope) working her way through a bit of summer crypto reading: But sometimes that’s exhausting and you’ve gotta take a break. A visit from … Continue reading Friday Dachshund…
20 Jul 2018
This continues the post from Part 1. Note that this is a work in progress, and may have some bugs in it 🙂 I’ll try to patch them up as I go along. In the previous post I discussed the problem of building CCA-secure public key encryption. Here’s a quick summary of what we discussed … Continue reading Wonk post:…
15 Jul 2018
Post available from my new site. Sadly WordPress doesn’t allow me to render the html exported by a Jupyter notebook.
28 Jun 2018
TL;DR Vendors are notorious for including and/or leaving behind Registry artifacts that could potentially be abused by attackers for lateral movement, evasion, bypass, and persistence. CLSIDs subkeys (LocalServer32 and InprocServer32) can be enumerated to discover abandoned binary references. Interestingly, CLSIDs can be called (‘invoked’) with this command: rundll32.exe -sta {CLSID} Defensive recommendations – clean up […]
17 May 2018
TL;DR. No. Or keep reading if you want. On Monday a team of researchers from Münster, RUB and NXP disclosed serious cryptographic vulnerabilities in a number of encrypted email clients. The flaws, which go by the cute vulnerability name of “Efail”, potentially allow an attacker to decrypt S/MIME or PGP-encrypted email with only minimal user interaction. By … Continue reading…
7 May 2018
Recently I have noticed that companies that use Google Suite have a fairly common misconfiguration that is making their internal groups public. In some cases it is just the name of the groups but in some extreme cases the content of the posts are public. Testing for this misconfiguration on your domain is as easy as looking at: https://groups.google.com/a/%yourdomain.tld%/forum/#!forumsearch/ Google…
28 Apr 2018
TL;DR This post discusses an alternate DCOM lateral movement discovery and payload execution method. The primary gist is to locate DCOM registry key/values that point to the path of a binary on the ‘remote’ machine that does not exist. This example method is likely to work if mobsync.exe is not in \\target\admin$\system32\, which is default […]
21 Apr 2018
In general I try to limit this blog to posts that focus on generally-applicable techniques in cryptography. That is, I don’t focus on the deeply wonky. But this post is going to be an exception. Today, I’m going to talk about a topic that most “typical” implementers don’t — and shouldn’t — think about. Specifically: … Continue reading Wonk post:…
7 Apr 2018
Over the past several years I’ve been privileged to observe two contradictory and fascinating trends. The first is that we’re finally starting to use the cryptography that researchers have spent the past forty years designing. We see this every day in examples ranging from encrypted messaging to phone security to cryptocurrencies. The second trend is … Continue reading Hash-based Signatures:…
26 Mar 2018
[Source: blog.microsoft.com] Introduction Not long ago, I blogged about Vshadow: Abusing the Volume Shadow Service for Evasion, Persistence, and Active Directory Database Extraction. This tool was quite interesting because it was yet another utility to perform volume shadow copy operations, and it had a few other features that could potentially support other offensive use cases. […]
17 Mar 2018
Abusing Exported Functions and Exposed DCOM Interfaces for Pass-Thru Command Execution and Lateral Movement
BohopsBackground Last Wednesday, I had some down time so I decided to hunt around in \System32 to see if I could find anything of potential interest. I located a few DLL files that shared an interesting export function called OpenURL: While looking for a quick win, I wanted to see if anything could be invoked […]
10 Mar 2018
Introduction Two weeks ago, I blogged about several “pass-thru” techniques that leveraged the use of INF files (‘.inf’) to “fetch and execute” remote script component files (‘.sct’). In general, instances of these methods could potentially be abused to bypass application whitelisting (AWL) policies (e.g. Default AppLocker policies), deter host-based security products, and achieve ‘hidden’ persistence. […]
5 Mar 2018
I am a fan of Kali Linux and AWS so I love the fact that they have an official AMI. While spinning up a Kali instance in AWS is fairly easy, I had a long flight today so I wrote a script that will spin up a Kali instance in about 60 seconds. The script does the following: Builds a…
26 Feb 2018
Introduction Over the last few weeks, I researched and tested a few interesting namespaces/methods documented on various Microsoft/MSDN sources that dealt with executing various COM scripts/scriptlets (e.g. VBscript, Jscript, etc.). In particular, I was curious to see if there were potentially new ways to invoke remote scripts (ActiveX Objects) by leveraging some of the great […]
10 Feb 2018
Vshadow: Abusing the Volume Shadow Service for Evasion, Persistence, and Active Directory Database Extraction
Bohops[Source: blog.microsoft.com] What is Vshadow? Vshadow (vshadow.exe) is a command line utility for managing volume shadow copies. This tool is included within the Windows SDK and is signed by Microsoft (more on this later). Vshadow has a lot of functionality, including the ability to execute scripts and invoke commands in support of volume shadow snapshot […]
31 Jan 2018
Introduction Visual Studio Tools for Office (VSTO) “is a set of development tools available in the form of a Visual Studio add-in (project templates) and a runtime that allows Microsoft Office 2003 and later versions of Office applications to host the .NET Framework Common Language Runtime (CLR) to expose their functionality via .NET” (Wikipedia). For […]
23 Jan 2018
(Image Source: blogs.technet.microsoft.com) Introduction A few weeks ago, I wrote about Executing Commands and Bypassing AppLocker with PowerShell Diagnostic Scripts. Overall, it was a viable technique that allowed for the loading of .NET/C# assemblies. However, PowerShell Constraint Language Mode proved to be a viable mechanism for defeating this technique if strictly enforced by UMCI/system policies […]
7 Jan 2018
Introduction Last week, I was hunting around the Windows Operating System for interesting scripts and binaries that may be useful for future penetration tests and Red Team engagements. With increased client-side security, awareness, and monitoring (e.g. AppLocker, Device Guard, AMSI, Powershell ScriptBlock Logging, PowerShell Constraint Language Mode, User Mode Code Integrity, HIDS/anti-virus, the SOC, etc.), […]
5 Jan 2018
Recently while working on a project I wanted to run OWSAP Dependency Check against a Github Organization to find any out of date frameworks but I couldn’t find an easy way to do it so I built a tool. Right now it will check Node and Ruby applications and put all the out of date frameworks in a single CSV.…
29 Dec 2017
On Friday, January 6th 2017 I walked into the first Yoga class of my life at YogaSol as part of fulfilling a new years resolution. I was in the best shape of my life. I was running, swimming and lifting weights multiple times a week. I weighed 165 pounds and was at 9% body fat. I was also really stressed…
19 Dec 2017
Yesterday, David Benjamin posted a pretty esoteric note on the IETF’s TLS mailing list. At a superficial level, the post describes some seizure-inducingly boring flaws in older Canon printers. To most people that was a complete snooze. To me and some of my colleagues, however, it was like that scene in X-Files where Mulder and Scully finally learn … Continue…
2 Dec 2017
ClickOnce (Twice or Thrice): A Technique for Social Engineering and (Un)trusted Command Execution
BohopsWhat is ClickOnce? ClickOnce is a “a Microsoft technology that enables the user to install and run a Windows-based smart client application by clicking a link in a web page” [Wikipedia]. Included as a component within the .NET Framework, ClickOnce allows a developer to create a web-enabled installer package for their (C#) Visual Studio project. […]
Yesterday in Paris, I gave the closing keynote at the dotJS conference. I’ve had the privilege to speak at dotJS every other year since 2013. Click above for a PDF of my slides (sorry, I used Keynote for several reasons, and its generated HTML is huge and not likely to work well with WP). Long-timer … Continue reading "My dotJS…
Introduction Active Directory (AD) Trusts have been a hot topic as of late. @harmj0y posted a recent entry about domain trusts [A Guide to Attacking Domain Trusts]. It provides a great understanding of how AD trusts actually work, so be sure to check that out as a primer for this post. In this blog entry, […]
26 Nov 2017
It is now more than 6 years since I started blogging about software development. It has been a great experience, and I thought I would reflect on what I have learnt. So here are my reasons for writing about programming, … Continue reading →
16 Oct 2017
The big news in crypto today is the KRACK attack on WPA2 protected WiFi networks. Discovered by Mathy Vanhoef and Frank Piessens at KU Leuven, KRACK (Key Reinstallation Attack) leverages a vulnerability in the 802.11i four-way handshake in order to facilitate decryption and forgery attacks on encrypted WiFi traffic. The paper is here. It’s pretty easy to read, … Continue…
25 Sept 2017
I wrote about OTOY over four years ago, in “Today I Saw The Future”. Since then, I have been inspired by the commitment of the founders Jules Urbach and Alissa Grainger to the vision that Jules enunciates: “… to render and remix simulated reality as effortlessly as the web did for text and digital media. … Continue reading "The Render…
4 Sept 2017
Last November I hacked together a script that continually monitored your network and sent a slack alert when something change. It worked but I was never 100% happy with it so I spent some time this weekend and rewrote it so that is hopefully more user friendly and functional. Some changes in this version includes the ability to set timeouts…
24 Aug 2017
I was working on a project recently and was asked if it was possible to stop users from setting common passwords. Using the pam_cracklib module and @DanielMiessler common passwords list it is as simple as these 3 commands: sudo apt-get install libpam-cracklib -y sudo wget https://raw.githubusercontent.com/danielmiessler/SecLists/master/Passwords/10_million_password_list_top_1000000.txt /usr/share/dict/ -O /usr/share/dict/million.txt sudo create-cracklib-dict /usr/share/dict/million.txt Seriously
19 Aug 2017
Mod_Security is the most widely known and used server based Web Application Firewall but I had not had a chance to play with it so I decided to take sometime this weekend to build a website (modsec.handsonhacking.org) to test it. Here is a small walk through on how I did it. Base Server Install: I used AWS Lightsail to build…
14 Aug 2017
One of the things that even the new MacOS beta is missing is MAC Address Randomization on boot. After spending a few hours working on it I put together this completely hack-y solution that uses Spoof and an automator Script saved as an application. Here is how I configured it: Install Spoof Open Automator Select “Application” Add “Run Applescript” Copy…
6 Aug 2017
This is a list of Israeli websites that stores user password as plain text. For anyone who don’t understand why you should care see this. http://www.winwin.co.il http://www.am-oved.co.il A mail to the web admin was sent, I’ll update if they’ve changed their ways.
14 Jul 2017
Security summer camp is about a week away so I spent some time this afternoon trying to figure out what talks and events I want to make sure I attend. BSides Las Vegas: A Day in the Life of a Product Security Incident Response Manager From SOC to CSIRT Hadoop Safari : Hunting For Vulnerabilities Introduction to Reversing and Pwning…
3 Jul 2017
I recently saw this SSH/HTTP(S) multiplexer on Github and tweeted that it looked amazing: An amazingly cool tool to run a webserver and a ssh on the same port: https://t.co/Z2eel3aIq5 — Jerry Gamblin (@JGamblin) July 2, 2017 A couple of people responded that you should be able to do the samething with HAProxy or something similar but my experience with…
2 Jul 2017
One of the saddest and most fascinating things about applied cryptography is how little cryptography we actually use. This is not to say that cryptography isn’t widely used in industry — it is. Rather, what I mean is that cryptographic researchers have developed so many useful technologies, and yet industry on a day to day basis barely uses … Continue…
11 Jun 2017
I love OWASP (I wanted to get that out of the way) but they let their TLS certificate expire yesterday: Should it have happened to an organization whose whole goal is to secure web applications? No. There are a million reasons why their TLS certificate could have expired and plenty of reasons it shouldn’t have (OWASP uses letsencrypt for their…
5 Jun 2017
A while ago, I published a blog post that presented a tutorial overview of how to use Jmeter for load testing a typical RESTful API. This post builds upon that original post with handy information on some updated reporting features of Jmeter as well a quick dive into how you can better propagate your load […]
29 May 2017
Have you ever wanted to control a vast medium small network of Honeypots but only had an hour and about $40 a month to spend on your project? So did I! So with the help of Digital Ocean and Anomali‘s Modern Honey Network we can now do it! For a basic distributed Cowrie network you will need: 1 – $20…
28 May 2017
Many software developers have a tendency to avoid talking to people. They would rather just rely on written communication in chats, email or issue tracker tickets. However, talking to people more can make them more effective as software developers. Here … Continue reading →
16 May 2017
In the last couple of years the Anti-Vaccination crowd in the United States has started to make inroads with more and more people deciding that the perceived risk of the vaccination outweighs the known risk of the disease. When you ask them why they dont vaccinatie they always have anecdotal evidence of how the vaccination could hurt them, how they…
4 May 2017
As I continue to try to learn R, I am trying to build tools that other people might find useful. Tonight with the help of Bob Rudis I built a script that will find domains with a keyword in it from DomainPunch, do a geoip lookup and map it if it is online. Since it is time to start thinking…
30 Apr 2017
Since I have started looking at the Umbrella DNS Popularity List I was interested in seeing how much the data changes day to day. I fired up RStuido and wrote some terrible code but finally got it to work with some help. Yesterday there were 80937 new DNS names on the list that were not on the list the day…
18 Apr 2017
Introduction Tribbles originate from the planet Iota Geminorum IV and, according to Dr. McCoy, are born pregnant. No further details are given but we can follow Gurtin and MacCamy (1974) and perhaps recover some of what happens on the Enterprise. Of course, age-dependent population models are of more than fictional use and can be applied, for … Continue reading Trouble…
17 Apr 2017
I am a huge fan of Tim Tomes and his Burp Suite Configuration Suggestions blog post. The problem is that I only use Burp a couple times a month and end up facing this screen and have to re-configure burp on every launch: So I built burpsettings.json that: Disables Browsers XSS Protection Disables Burp Collaborator Server Disables Intercept by Default…
13 Apr 2017
Today I was asked if it was possible to generate a list of domain names registered everyday with a keyword in the record (company name, city, trademark, etc). There are a few paid services that do this and domainpunch.com has a web based tool that will do this but I wanted to automate it so I could use it with…
29 Mar 2017
I am reading a book called “The Art of Authenticity” and in the book over a couple of chapters it talks about understanding what makes strong leaders and deciding who you should follow. I have pulled these 10 questions out of those chapters: What was your first leadership role? When you think about the process of becoming the leader that…
27 Mar 2017
The frequency in which data leaks and breaches appear makes it easy to forget how severe some of them are. It seems like every 2 days we hear about a new data breach; we’ve got people like Troy Hunt who The post Handling Data Breaches, Or: The Art of Fixing and Saying “Sorry” appeared first on FullStack - Ofer Zelig's…
21 Feb 2017
Many thanks to Pressable for the theme porting and ongoing hosting, and to w0ts0n for help with the transfer. I’ll blog again soon.
9 Feb 2017
The RSA conference starts next week and lets be honest it is becoming known as a stuffy management conference with very little useful technical information but if you know where to look you can take some deep dives. I have put together a quick guide of some amazing talks and events I am looking forward to. Talks: BSidesSF – Coming…
27 Jan 2017
So many folks are wonder what they need to do to make a career of User Experience Design. As someone who interviewed many designers before, I’d say the only gate between you and a career in UX that really matters is your portfolio. Tech moves too fast and is too competitive to worry about tenure […]