In the last blog post, I investigated why my Noda Time tests on Travis were running much slower than those on AppVeyor. I resolved a lot of the problem just by making sure I was running release builds on Travis. That left a single test which takes up about half of the test time though: … Continue reading Diagnosing a…
19 Aug 2017
18 Aug 2017
Tech preview: Get your own Kubernetes container orchestration platform in cPouta
CSC - IT Center For Science - Cloud TeamIntroduction the container orchestration chapter of the Pouta user guide. This is a technology preview feature for now, so there are some limitations that you should be aware of. Those are listed at the end of the documentation page. Awesome-Kubernetes list of curated links about the topic. There you will find many introductory articles on Kubernetes, links to various related…
We’re releasing two new OpenAI Baselines implementations: ACKTR and A2C. A2C is a synchronous, deterministic variant of Asynchronous Advantage Actor Critic (A3C) which we’ve found gives equal performance. ACKTR is a more sample-efficient reinforcement learning algorithm than TRPO and A2C, and requires only slightly more computation than A2C per update.
Event Gateway is the backbone of your serverless architectures. React to any event, with any function, on any cloud.
17 Aug 2017
How to build a dynamically-updating progress bar in JavaScript that updates in real time as status updates are streamed to it.
I’ve previously blogged about a case where tests on .NET Core on Linux were much, much slower than the same tests on Windows. Today I’ve got a very similar problem – but I suspect the cause isn’t going to be the same. This is my reasonably raw log – skip to the end for a … Continue reading Diagnosing slow…
For a long time, I’ve believed that diagnostic skills are incredibly important for software engineers, and often poorly understood. The main evidence I see of poor diagnostic skills is on Stack Overflow: “I have a program that does 10 things, and the output isn’t right. Please fix.” “I can’t post a short but complete program, … Continue reading Diagnostics everywhere!…
Enhancements, bug fixes and more added in the Serverless Framework v1.20 release.
16 Aug 2017
Our Dota 2 result shows that self-play can catapult the performance of machine learning systems from far below human level to superhuman, given sufficient compute. In the span of a month, our system went from barely matching a high-ranked player to beating the top pros and has continued to improve since then. Supervised deep learning systems can only be as…
Jane Street is looking to hire an engineer with experience in both software and hardware design to work on FPGA-based applications, and on tools for creating such applications.
The top Twitter accounts you should follow to stay on top of serverless, event-driven architecture, and other tech news.
15 Aug 2017
Welcome to the jOOQ Tuesdays series. In this series, we’ll publish an article on the third Tuesday every other month where we interview someone we find exciting in our industry from a jOOQ perspective. This includes people who work with SQL, Java, Open Source, and a variety of other related topics. I’m very excited to … Continue reading jOOQ Tuesdays:…
Displaying a list of items is a challenge you will encounter in most web applications. When using a view layer library such as React, you only have to iterate over the list of items and return elements. However, often you want a couple of more features such as filtering, sorting or pagination. Not every list […]
This is just a brief post that I’m hoping may help some people migrate to use .NET Core 2.0 SDK on Travis. TL;DR: see the end of the post for a sample configuration. Yesterday (August 15th), .NET Core 2.0 was fully released. Wonderfully, Travis already supports it. You just need dotnet: 2.0.0 in your YAML … Continue reading Using .NET…
Have you ever found a Rails topic that didn’t make any sense to you? Like, you thought you knew it, so you wrote some code, and something completely different happened? Or you know you don’t understand, but you kind of know enough to get by, except you spend so much time fighting edge cases that you could have been an…
Highlights of our recent work to improve our cloud-based monitoring and alerting pipeline.
We all know how inheritance works and implemented some kind of class hierarchy at least few times during our career. Some of us know already that inheritance is not the silver bullet. Some of us know that inheritance must not be overused and considered with caution. Now I’m going to show you how choosing the quick win might cost you…
14 Aug 2017
One of the things that even the new MacOS beta is missing is MAC Address Randomization on boot. After spending a few hours working on it I put together this completely hack-y solution that uses Spoof and an automator Script saved as an application. Here is how I configured it: Install Spoof Open Automator Select “Application” Add “Run Applescript” Copy…
Last week I was in Boston for the Geometry of Redistricting workshop. It was an optimistic gathering of over 500 mathematicians, computer scientists, lawyers, policy makers, teachers, and interested people of all stripes. There was a ton of information in the talks and subsequent discussions. I’ll try to distill the main ideas and avenues for research as best I can.…
Intern season is coming to a close, and it’s a nice time to look back (as I’ve done in previous years) and review some of what the interns did while they were here. The dev intern program has grown considerably, with almost 40 dev interns between our NY, London, and Hong Kong offices.
5 must-see AWS Learning Resources
11 Aug 2017
We’ve created a bot which beats the world’s top professionals at 1v1 matches of Dota 2 under standard tournament rules. The bot learned the game from scratch by self-play, and does not use imitation learning or tree search. This is a step towards building AI systems which accomplish well-defined goals in messy, complicated situations involving real humans.
Shadow DOM is a fairly recent-ish spec that gives you DOM tree encapsulation – it’s one of the superhero lions in the Voltron of specs called “Web Components”. Web Components let you create reusable, self-contained components in JavaScript; the Shadow DOM bit makes sure that the CSS and markup you bundle with your implementation is encapsulated, hiding the implementation details…
Level up your serverless ops game with a walkthrough on CloudWatch metrics and alarms
10 Aug 2017
This is an endless GIF that always shows the current time in UTC: Source Code (From reports it doesn’t seem to work on Safari, other browsers should be fine.) time.gif is written in Haskell and works by dynamically generating each frame of the GIF and slowly feeding them over the HTTP connection. There is no guarantee that this GIF shows…
Anticipating change on behalf of the serverless enterprise.
9 Aug 2017
I’ve got a few speaking engagements coming up that I thought it might be worth publicising a bit further. They’re all within just over a week of each other, which is going to be somewhat tiring, but… Here they are, in chronological order: Progressive .NET 2017 Progressive .NET 2017 will be held on September 13th-15th … Continue reading Upcoming speaking…
This is an experience report about a gotcha in Go that catches every Go programmer at least once. The following program is extracted from a larger version that caused my co-workers to lose several hours today. package mainimport "fmt"type T struct{}func (t T) F() {}type P interface { F()}func newT() *T { return new(T) }type […]
I recently had a problem where part of the solution was to do a series of pointer accesses that would walk around a chunk of memory in pseudo-random order. Sattolo's algorithm provides a solution to this because it produces a permutation of a list with exactly one cycle, which guarantees that we will reach every element of the list even…
Making it easy to think about events in a serverless world.
If you didn't know, in Safari's private mode both localStorage and sessionStorage are not working. To be exact, Safari sets storage's limit to 0, so you can't write anything to it. I keep forgetting this, until QA people report it at some point. So I quickly wrote a small facade for it, which fails silently in this case. That means…
8 Aug 2017
I always wanted to know how stargazers of my repos increased over time.
7 Aug 2017
How to use Functions to ingest, store and analyze IoT analytics in real time with the mnubo BLOCK.
Your cheatsheet to a seamless migration.
6 Aug 2017
This is a list of Israeli websites that stores user password as plain text. For anyone who don’t understand why you should care see this. http://www.winwin.co.il http://www.am-oved.co.il A mail to the web admin was sent, I’ll update if they’ve changed their ways.
4 Aug 2017
Handling Python dependencies in your Lambda functions can be a pain. Here's how I handle Python packaging with Serverless for dev/prod parity.
A roundup of links and videos from the serverless ecosystem that caught our attention this week.
3 Aug 2017
RL-Teacher is an open-source implementation of our interface to train AIs via occasional human feedback rather than hand-crafted reward functions. The underlying technique was developed as a step towards safe AI systems, but also applies to reinforcement learning problems with rewards that are hard to specify.
Enhancements, bug fixes and more added in the Serverless Framework v1.19 release.
2 Aug 2017
Keeping with the serverless pulse.
1 Aug 2017
With spring 4.2 (released more than one year ago) serious improvements regarding embedded events were made. You probably already know it, but I’ve never had a chance to properly investigate it. Lately, when digging into code base of the system I’m currently working on I got an opportunity to see it in action and after quick glimpse, I decided to…
31 Jul 2017
Easy Approach to Requirements Syntax and the segue to Behavior Driven Development
Henrik Lau ErikssonI was attending a conference six months ago and listened to a talk about quality. During the talk, I was introduced to EARS — Easy Approach to Requirements Syntax. This way of writing requirements struck a chord with me, given my prior experience reading and writing requirement specifications. When doing agile development, we write User Stories and define Acceptance Criteria.…
The ones who started it all.
28 Jul 2017
A roundup of links and videos from the serverless ecosystem that caught our attention this week.
Keeping with the serverless pulse.
If you’re a regular visitor to this blog, you might be aware that we have been transitioning to a microservices based architecture over the past four to five years, as we have shared insights into the process and the related challenges on multiple occasions. To recap, adopting a microservices architecture has allowed us to regain team autonomy by breaking up…
27 Jul 2017
I recently gave a talk at DevOps Days (slides) and it had a pretty great response. I’m still pretty care-mad about the topics it covered so I figured I would turn some key points from it into a blog post. The overall outline of the talk covered the past, present, and future of usable security. Let’s start with the past.…
We’ve found that adding adaptive noise to the parameters of reinforcement learning algorithms frequently boosts performance. This exploration method is simple to implement and very rarely decreases performance, so it’s worth trying on any problem.
Clever Instant Login makes it easy for students to log in to their learning applications, saving valuable instructional time. By using the widely-deployed OAuth 2 protocol, our team tries to save valuable development time and make it easy for our app customers to create integrations. OAuth 2 has been a fairly smooth road, but we […] The post Clever Instant…
A step-by-step look at the development of a serverless application.
26 Jul 2017
.content img:not(.emoji) { border: 1px solid #efefef; margin: 24px 0; } The Meownica Web App Workflow™ goes like this: Write bad code until the file is too long Refactor that code into some web components Repeat steps 1-2 until done Realize you forgot to do the PWA dance, so your app is scoring 45 on Lighthouse Make it into a…
When we left jQuery behind and embraced modern JavaScript frameworks, we thought we would never touch DOM directly again. Well that is not entirely true. There are a lot of cases when you need to get some DOM element size. For element's dimensions .offsetWidth and .offsetHeight are great way to do it. But one of the other common tasks is…
25 Jul 2017
jOOQ has been around for a while now (since 2009!) and by now we can say we’ve seen quite a bit of things about the SQL and Java languages. Some of our design decisions are particular in the way jOOQ thinks about programming with SQL. These include: Nullability (let’s stop fighting it) Value types (let’s … Continue reading 5 Things…
I’ve been using springboot for some time now, but there was that one thing that bugged me a lot. While writing integration tests with mocks you are forced to use mockito as the mocking library. That’s great and easy to understand if you are not using spock. The problem is that in spock there are better ways to mock stuff……
24 Jul 2017
In recent months, there had been some really exciting news from the MySQL team: (Recursive) Common Table Expressions in MySQL Introducing Window Functions These two SQL standard language features are among the most powerful SQL features that are available from most other databases. I frequently include them in conference talks about SQL (see my article … Continue reading jOOQ 3.10…
Problem: Express a boolean logic formula using polynomials. I.e., if an input variable $ x$ is set to $ 0$, that is interpreted as false, while $ x=1$ is interpreted as true. The output of the polynomial should be 0 or 1 according to whether the formula is true or false as a whole. Solution: You can do this using…
Many dynamic companies find the cash injection provided by business finance is exactly what they need to give their growth plans a kick start, and there are more types of funding available to businesses than ever before.
I'm becoming predictable. Again, I haven't found component I like, so I wrote my own. This time, I've build React component for parallax (on scroll) effects. Check the live demo. It is called Plx, it is open source and available on GitHub and npm. What it does # So far in my career, I've built so many parallax components. Parallax…
22 Jul 2017
A long time ago, someone–I normally attribute this to David Symonds, but I can’t be sure he was the first to say it–said that the reason for adding generics to Go would be the reason for calling it Go 2.0. That is to say, adding generics to the language would be half baked if they […]
21 Jul 2017
A roundup of links and videos from the serverless ecosystem that caught our attention this week.
20 Jul 2017
Java’s visibility rules are tricky at times. Do you know what this will print? It will print (highlight to see the solution): B.x Because: The super type B's members hide the enclosing type C's members, which again hide the static import from A. How can this lead to bugs? The problem isn’t that the above … Continue reading A Curious…
We’re releasing a new class of reinforcement learning algorithms, Proximal Policy Optimization (PPO), which perform comparably or better than state-of-the-art approaches while being much simpler to implement and tune. PPO has become the default reinforcement learning algorithm at OpenAI because of its ease of use and good performance.
Serverless v1.18 - Request parameter support for Lambda Proxy, default value for plugin options added
ServerlessSupport for request parameters when using Lambda Proxy integration and more added in the Serverless Framework v1.18 release.
19 Jul 2017
What is a slug? The term “slug” comes from the world of newspaper production. If you have ever created a simple “Blog” application you have already reinvented it. When you need to access any post at some URL you need to identify it somehow. The simplest idea is to identify them by id, but that does not seem too pretty.…
Learn some tips and tricks as you move to an event-driven architecture
18 Jul 2017
As we all agree, GOTO is evil, right? Relevant XKCD Or even funnier: New Intern Knows Best Of course, GOTO isn’t evil Of course, somewhere deep down in our professional selves, we know that GOTO isn’t evil, it’s just a very basic processor instruction that was available since the early days of assembly code. GOTO … Continue reading Don’t Overdo…
There’s a great MSR demo from 2012 that shows the effect of latency on the experience of using a tablet. If you don’t want to watch the three minute video, they basically created a device which could simulate arbitrary latencies down to a fraction of a millisecond. At 100ms (1/10th of a second), which is typical of consumer tablets, the…
I left the company for which I’ve been working for last 4 years and now it is time to summarize my experiences. It is not the story of shady business nor confession of my sins (not all of it anyway), but rather my thoughts and experiences on working for a long time in the same company from which most of…
17 Jul 2017
Combine Functions + Twitter API to allow users to tweet, delete, and more from an authorized Twitter account, by publishing a message.
We’ve created images that reliably fool neural network classifiers when viewed from varied scales and perspectives. This challenges a claim from last week that self-driving cars would be hard to trick maliciously since they capture images from multiple scales, angles, perspectives, and the like.
15 Jul 2017
After migrating my domain to AWS Route 53 I finally transferred my website to a new Uberspace host which supports Let’s Encrypt. You should be automatically redirected to HTTPS when visiting www.schakko.de. The whole procedure took 2 hours, including setting up the new Uberspace, importing the existing databases and changing […] The post Website moved to new Uberspace with HTTPS…
14 Jul 2017
Security summer camp is about a week away so I spent some time this afternoon trying to figure out what talks and events I want to make sure I attend. BSides Las Vegas: A Day in the Life of a Product Security Incident Response Manager From SOC to CSIRT Hadoop Safari : Hunting For Vulnerabilities Introduction to Reversing and Pwning…
Every now and then, I tweet something like this, just to piss off some clean coders: Call me what you want, but I like the occasional break-out-of-if pic.twitter.com/kQ0BIkijz0 — Lukas Eder (@lukaseder) July 13, 2017 Apart from the obvious trolling factor (why can’t I ever resist?), I do think there’s something thought provoking in such … Continue reading Don’t Extract…
13 Jul 2017
You know JDBC, right? It’s that really easy, concise API that we love to use to work with virtually any database, relational or not. It has essentially three types that you need to care about: Connection Statement (and its subtypes) ResultSet All the other types some sort of utilities. Now, with the above three, we … Continue reading How I…
This is a blog post which literally drove me crazy for a week. After building our mining rig I experienced a bad WiFi connection with high pings, periodically occuring every 30 seconds. Just scroll down to see my – fairly simple – solution. Getting into the mining business A few […] The post Fixing periodically occurring WiFi lags when running…
12 Jul 2017
Vacation time means administration time. I am one of these Uberspace customers whose domain has been registered and managed not by an external DNS registrar but by Uberspace (or Jonas Pasche) itself. Since a few years Uberspace has not been providing this service. Actually this was not a problem and everything […] The post Transferring DNS from Uberspace to AWS…
A few days ago I switched from ethminer to Claymore’s Dual Ethereum Miner because ethminer has problems running multiple instances with multiple GPUs. My blog post How to run same ethminer instance with multiple GPUs is still valid but ethminer simply can’t handle two or more parallel running instances. In […] The post Running multiple Claymore miner instances with different…
MapTiler Desktop 8.0 is here - and it can generate map tiles in OGC GeoPackage format.
11 Jul 2017
Expert tips to ensure your gym blows away the competition and keeps on delivering, day in and day out.
I’ve seen and been in Research & Development (R&D) teams in the past. Sadly I can’t say any of these teams have been successful making a significant impact on the directions company they were serving took. R&D is often mentioned in the context of the military (in 2015, the US Army’s research, development, testing and evaluation (RDT&E), coupled with procurement…
A piecewise regression can model multiple trends in a single data set. Learn how Datadog automates piecewise regression on our timeseries data.
Some time ago I’ve been struggling with mapping hierarchical data structure in angular. Labels hierarchy was complex (like 4 levels deep with multiple parents, multiple children, basically graph like structure with some logic behind it). In the end it was/is still working but that’s the best I can say about it. Read more
10 Jul 2017
tl;dr “Transparent Hugepages” is a Linux kernel feature intended to improve performance by making more efficient use of your processor’s memory-mapping hardware. It is enabled ("enabled=always") by default in most Linux distributions. Transparent Hugepages gives some applications a small performance improvement (~ 10% at best, 0-3% more typically), but can cause significant performance problems, or even apparent memory leaks at…
Last week I attended a software development conference, QCon in New York. Here are my impressions of it, as well as some thoughts on programming conferences in general. For me, there are several reasons to attend developer conferences. You have … Continue reading →
8 Jul 2017
Some time ago I was faced with a task of testing a bash script. At first I decided to use Python unit-tests, however, I was reluctant to bring external technologies to the project. Therefore I had to go with the testing framework written in the notorious bash. Overview of the existing solutions After googling available solutions, I was presented with…
7 Jul 2017
here. anti-affinity server group.
6 Jul 2017
How to control a drone using PubNub and the AT&T IoT StarterKit for real-time device control and monitoring.