#scala
17 posts
24 Jul 2018
25 May 2018
15 May 2018
27 Jul 2017
25 Feb 2016
If you’re programming in more advanced languages like Scala or Ceylon, or even JavaScript, “nested functions” or “local functions” are a very common idiom to you. For instance, you’ll write things like fibonacci functions as such: (Question from Stack Overflow by Aaron Yodaiken) The f() function contains a nested g() function, which is local to … Continue reading (Ab)using Java…
14 Jan 2016
Notice that the examples in this article may be outdated, as Typesafe’s Activator works differently now. The blog post will not be maintained to provide up-to-date Activator examples. We’re very happy to continue our a guest post series on the jOOQ blog by Manuel Bernhardt. In this blog series, Manuel will explain the motivation behind … Continue reading Reactive Database…
6 Jan 2016
I love really Amazon EMR. Over the years it’s grown from being “Hadoop on-demand” to a full-fledged cluster management system for running OSS big-data apps (Hadoop MR of course, but also Spark, Hue, Hive, Pig, Oozie and more). While Hadoop out of the box supports reading from S3, EMR has a proprietary implementation called EMRFS that has some nice features.…
20 Dec 2015
My previous post showed a very simple Scalding workflow. Apache Flink is a real time streaming framework that’s very promising. It also supports running Cascading workflows with very little modification. Surely there must be some way to run a Scalding job on top of Flink? Turns out… YES! In a nutshell Here are the high-level things we need to solve…
I’ve been using Scalding for the last few years and really love how simple it makes writing scalalbe data processing jobs. I think many of the issues beginners have with Scalding relate to project setup. I hope this post simplifies things for people so they can started with less hassle. Building your project with SBT The official getting started guide…
16 Dec 2015
Notice that the examples in this article may be outdated, as Typesafe’s Activator works differently now. The blog post will not be maintained to provide up-to-date Activator examples. We’re very happy to continue our a guest post series on the jOOQ blog by Manuel Bernhardt. In this blog series, Manuel will explain the motivation behind … Continue reading Reactive Database…
3 Dec 2015
Notice that the examples in this article may be outdated, as Typesafe’s Activator works differently now. The blog post will not be maintained to provide up-to-date Activator examples. We’re very happy to announce a guest post series on the jOOQ blog by Manuel Bernhardt. In this blog series, Manuel will explain the motivation behind so-called … Continue reading Reactive Database…
24 Feb 2015
Every framework introduces a new compromise. A compromise that is introduced because the framework makes some assumptions about how you’d like to interact with your software infrastructure. An example of where this compromise has struck users recently is the discussion “Are Slick queries generally isomorphic to the SQL queries?“. And, of course, the answer is: … Continue reading jOOQ vs.…
28 Sept 2014
It’s occasionally useful when writing map/reduce jobs to get a hold of the current filename that’s being processed. There’s a few ways to do this, depending on the version of Spark that you’re using. Spark 1.1.0 introduced a new method on HadoopRDD that makes this super easy: import org.apache.hadoop.io.LongWritable import org.apache.hadoop.io.Text import org.apache.hadoop.mapred.{FileSplit, TextInputFormat} import org.apache.spark.rdd.HadoopRDD // Create the text…
7 Aug 2014
Strange bedfellows: how a web-tier validation framework enables strongly typed, big data pipelines
Ian HummelThe other day I was talking with a colleague about data validation and the Play web framework came up. Play has a nice API for validating HTML form and JSON submissions. This works great when you’re processing small amounts of data from the web-tier of your application. But could that same tech benefit a Big Data team working on a…
1 Aug 2014
So, I’m experimenting with Scala because I want to write a parser, and the Scala Parsers API seems like a really good fit. After all, I can implement the parser in Scala and wrap it behind a Java interface, so apart from an additional runtime dependency, there shouldn’t be any interoperability issues. After a few … Continue reading The 10…
17 Mar 2014
There are a tremendous amount of SQL APIs natively written in Scala. Manuel Bernhardt has summarised a nice collection in his a post. Another collection of Scala SQL APIs can be seen in this Stack Overflow question. One API that we want to focus on in particular is ScalikeJDBC (licensed ASL 2.0), which has recently … Continue reading A SQL…
8 Jan 2014
Several of my colleagues love IntelliJ for coding in Scala. I was pretty happy with Sublime Text 2 (and still use it for Ruby/Python/Shell/whatever) but the lack of code completion was really starting to affect my productivity. I spent way too much time looping through the edit/compile/fix typo cycle. Before I could switch though, I really wanted my fancy arrows…