Note: It looks like videos of the JuliaCon talks were uploaded to Youtube the day this post was finally published!
I was in Cambridge, MA for a few days the other week at JuliaCon, a small conference for the Julia programming language. Julia is a young language (started around 2014 and currently pre-1.0) oriented towards fast numerical computation: matrix manipulation, simulation, optimization, signal analysis, etc. I've done a fair amount of such programming over the years, and it has never felt as elegant or coherent as it could be. The available tools and languages are generally either:
- stuck in the 1980s in terms of programming language features for safety, productivity, and collaboration (eg, Fortran and Matlab)
- expensive proprietary closed-source packages (eg, Matlab and Mathematica)
- general-purpose languages with numerical features either hacked on or in the form of libraries (eg, Python)
There is a lot to be excited about in Julia. It's already pretty fast (leveraging pre-existing JIT tools, hand-tuned matrix and solver libraries, and the LLVM compiler suite) and has contemporary high-level language features (like optional type annotation, polymorphic function dispatch, package management tools, and general systems tools (eg, JSON and HTTP support)) that can make the language more faster to develop in, and easier to read and maintain. I'm personally excited about the progeny of the language: the birthplace of the language is the CSAIL building at MIT, and the spirit of Scheme and the work of Project MAC is sprinkled through the project. One of the big pitches of Julia is that scientists won't need to learn both a productive high-level language (eg, Python) and a low-level performant language (eg, C or Fortran) and interface between the two: Julia has everything all in one place.
All that being said, while I thought I would be working in Julia a lot during my time at the Recurse Center, I've ended up being much more drawn to the Rust language instead. Rust is a general systems language (it's compiled, has stronger typing, and no garbage collection), and not great for interactive numerical exploration, but I've found it a joy to program in: for the most part everything just works the way it says it will. My recent experience with Julia, on the other hand, has been a lot of breakage between library and interpreter versions, poor developer usability (eg, hard to figure out where files should live in a package), and very frustrating import/load times. Though I have to admit that I while I pushed through some frustrations with Rust, I haven't spent that much time with Julia, and may have just been impatient, so take everything I say here with a grain of salt.
With these feels going in, what did I learn at JuliaCon and what do I think of the future of the language now? In the below sections I'll go over the interesting things I saw, then come back to summary at the end.
Programming Language Design
An older research language for numerical computing that I have always been curious about is Fortress, and the leader of that project (Guy Steele, who also worked on the design of the Scheme and Java languages) gave one of the opening keynote speeches at JuliaCon this year. Awesome! I get really excited about inter-generational learning and dialog.
Fortress was a very "mathy" language. The number tower was intended to be "correct" (aka, have the same structure that mathematicians use), physical units were built-in, and some operator precedence was non-transitive. Operators on built-in types (like Integers) could be overloaded, unlike in Java, because Fortress users could apparently be trusted to "preserve algebraic properties". Steele is a proponent of using whitespace (or lack of whitespace) to clarify expressions, sort of like extra parentheses, and enforcing this in the compiler. For example, the following two statements would be equivalent in most languages, but not in Fortress:
a + b*c + d // Clear: Ok
a+b * c+d // Misleading: Compiler Error
This was part of a general effort to allow "whiteboard" style syntax in the language. Fortress code actually has two representations: a plain text Scala-style source code, and a LaTeX-y symbolic math format. Steele also used some font-coloring in his slides to differentiate different types of symbols, which reminded me of the helpful style my undergraduate physics professors would use on the blackboard. I think this effort to adapt the "look and feel" of the language to how the intended audience already writes and communicates is really cool. I wonder if a third syntax format could have been added in a one-to-one manner: that of a general purpose language like Scala or Haskell (both noted as influences to Fortress) to make collaboration with general purpose programming experts easier. Steele mentioned that some efforts to make the syntax more math-like resulted in "contortions", so there is probably more work to be done here.
In my limited experience, Julia has a pretty clean syntax, and allows some math-y unicode characters as operators (like ∈, ≠, etc), but didn't prioritize math-y syntax as much as Fortress. Given the open challenges with formalizing informal whiteboard syntax this may or may not have been a missed opportunity.
The positive lessons learned from Fortress were summarized as being the type system, automatic parallelism (via generators and reducers), the math-y syntax, pretty printing (I assume meaning the LaTeX-y representation), physical units, and forced syntax clarity (aka, forced use of parentheses and whitespace). One issue that come up during implementation was that it was hard to bound the latency and computational complexity of type constraint solving at run-time.
A few other talks touched on language design decisions and features. There was a short "Functional HPC" talk by Erik Schnetter, in which it was pointed out that for some workloads regular old garbage collection can be faster than reference counting: I've become used to thinking of latency and GC pauses as a huge performance problem in systems programming, but for number crunching that isn't as much of an issue, while little reference overheads are (especially if locks or atomic operations are necessary).
Keno Fischer gave an overview of the Gallium debugger, which had some cool features, but is still under development. There are both AST-based and LLVM-based backends for the debugger, which allows stepping at function calls, line-by-line, or expression-by-expression, which is something I hadn't seen before. He demoed stepping through each step of the creation of a matplotlib graph, with the output shown graphically after each step. Neat stuff!
One of my personal interests in Julia would be formalizing the syntax into a machine-readable grammar (eg, EBNF or ABNF). I was lucky enough to run in to Stefan Karpinski during one of the coffee breaks, and he pointed me to the Julia plugin for Eclipse, which already has a partial implementation of a grammar.
A few talks touched on the issue of Nullable datatypes (also called "Maybe" or "Option" types in other languages), particularly for data science and DataFrame-type applications. I only recently encountered Option (and the related Result type) datatypes, in Rust, and can see why people want these so badly, but there doesn't seem to be a simple path forward yet. Rust really leverages these types in function return signatures, a feature which Julia does not have for now; I think I read rumors about them being added in the future, but didn't hear any mention of them here or on the 1.0 feature roadmap.
Numeric Abstraction
One of the big trends I saw was taking advantage of Julia's abstractions around generic operators and arrays to experiment with novel computation strategies. Sometimes this means improving precision (with novel data types and representations), sometimes it means increasing performance (by changing memory layout or distribution, or targeting special hardware), and sometimes it just makes code more elegant or semantic.
For example, Tim Holy gave a talk (titled "To the Curious Incident of the CPU in the run-time") which covered a bunch of nitty-gritty details for implementing wrapper classes that re-shape or re-size Arrays, including sparse arrays.
Lindsey Kuper gave a nice overview of the ParallelAccelerator.jl project, which entirely re-compiles Julia into C++ to get some extra performance from the static full-program compiler. It seems to me that this only makes sense because the Julia language has clean abstractions that the transpiler can take advantage of.
One of my favorite talks from the whole conference was David Sanders' and Luis Benet's talk on ValidatedNumerics ("Precise and rigorous calculations for dynamical systems"). Instead of computing on approximate (rounded) scalars, they compute on intervals of floating point numbers (or in higher dimensions, boxes): at the end of computation the "correct" solution is known to be within the final box, which also gives context as to how much numerical error has accumulated. By defining new types to accomplish this (specifically, DualNumbers), they can re-use any generic code in a relatively performant manner. They also noted that when there is an analytic form to bound the error for all following terms, Taylor expansion approximations can be truncated as soon as the interval error exceeds the error in all following terms. Cool!
Other Fun Stuff
Using Julia as a Quick and Dirty Code Generator: The speaker (Arch Robison) is clearly having way too much fun! He used Julia to output assembly code to get fast (real-time) discrete Fourier transform (DFT) performance for a little video game called "FreqonInvaders". Infectious enthusiasm!
Autonomous driving for RC cars with ROS and Julia: A fun little project doing "Model Predictive Control" on a small model car to do stunts like drifting and slide parking into a tiny space. They achieved about a 10Hz closed-loop control latency, which seems to me like barely enough for this sort of thing, but clearly worked alright. Everything ran on the car itself (no computation on a remote desktop with wireless control or anything like that), with an Odroid ARM Linux system and an Arduino-compatible microcontroller; Julia code using JuMP and other optimization stuff ran on the ARM system. The code and raw data (for analysis) is available on the BARC project website. Super cool, having this stuff being experimented with already means there will be pressure to improve soft-real-time performance in the language itself.
Astrodynamics.jl: Modern Spaceflight Dynamics in Julia: Mostly a bunch of code for doing timebase conversions and interpreting (or calculating) ephemeris data (which is information about where astro bodies like the Moon and planets will be at a given time), but some simple demos of orbital simulation and event detection (eg, perihelion time and position) as well. Would be cool if the ValidatedNumerics stuff was integrated.
GLVisualize: The demos in this talk were really impressive: live editing of mesh vertices, relatively high performance, real-time feedback, etc. There were a bunch of good graphics talks: the GR Framework stuff is really impressive in scope (though maybe not as big a performance boost over Python as hoped), and Vulkan is exciting.
Diversity
It's sad to say, but the gender diversity at the conference was really poor, particularly in contrast to the Recurse Center (where I have spent the past couple months). The women I did meet gave some of the best talks, are crucial contributors to infrastructure, and are generally amazing: more please! Aside from the principle of the thing, there is just something about a giant sea of guys at a tech event that results in a tense group vibe. Everybody I spoke to one-on-one was friendly and we had great conversations, but as a group there was a lot of ice to be broken. In my experience even hitting 10-20% women in attendance can thaw this out, but that's just my anecdotal experience.
I haven't attended, but I hear that PyCon has done a great job improving diversity with careful planning and systemic initiatives.
Overall, I thought the conference was a great group of people and admirably well run. I appreciated the efforts to keep costs low, and everything generally ran on time. Thanks to all the volunteer and MIT staff organizers for their efforts!
Julia 1.0
Stefan Karpinski gave an overview of features and roadmap for getting to Julia 1.0, which I think was a topic close to most attendee's hearts (including mine). I ended up with a huge list of written notes, which I'll summarize below; the punchline was aiming to have a 1.0 release around one year from now. Apparently the one-year goal has been floated in previous years; I'm not sure how wise it is in general to float initial release timelines for a project like this, it seems like it will just "be done when it's done".
Some of the goals that were interesting to me:
- Arrays: might refactor Arrays to have a separate backing abstraction of "Buffers" with arrays on top (apparently Lua and Torch do this).
- Strings: move full Unicode support out of core language (Base) and into a
package. The
@printfmacro will be refactored into a function. To my surprise, currently Strings are implemented as an Array! This has a relatively large overhead for each string (72 bytes). - Modularity and Package infrastructure: currently a mess (I agree),
import,usingandexportwill be refactored. - Compiler: add non-pthreads multithreading; better static compilation; ability
to define a
main()function and get a standalone script or binary; ability to redefine functions and have the changes propagate (cache invalidation problem); stabilize intermediate representations. Seems like a lot! - Optimizations: faster garbage collection, more auto-vectorization (eg, for vector floating point units), improve globals performance. Might pull in part of ParallelAccelerator?
I'm a little nervous how many of these goals are big open questions instead of just implementation tasks. I wish there was a more healthy way to experiment with new features and refactoring without breaking everything or committing to a long-term stable API; I think other languages have settled into good patterns for this kind of development, though maybe they needed to go through a difficult 1.0 process first. It was mentioned that 0.6 would be the last of the 0.x series of releases and considered 1.0-alpha, and that from 1.x and on things should generally be backwards compatible.
Separate from Stefan's talk, there was a short overview of progress on the next iteration of the Julia package and dependency manager, called Pkg3. The goals were described as "a mash-up of virtualenv and cargo": virtualenv is a tool for isolating per-application dependencies and toolchains in Python, and Cargo is is the Rust dependency manager and build tool (which is also used in a per-application fashion). Pkg3 sounds like it will have a concept of distinct "global" (meaning system-wide?) installations and "local" (eg, per-project or per-directory) installations and name-spacing. The naming could use some work, as "global" and "local" are pretty overloaded, but I think they are chasing the right goals. Reproducibility (both for binary generation and data/experiment reproduction), lock files (which lock in known-good versions of dependencies a la Cargo), and other concepts that I care about were also thrown around. I didn't catch all the details (and I'm not sure how much has been worked out and implemented yet), but after my experiences with Elm and Rust, and the current state of packaging for Julia, I'm excited for Pkg3!
Overall Julia Feels
There is sort of an explosion of ideas and experiments going on. It feels sort of like what the Ruby community maybe went through with web frameworks, or the web community did with languages that compile to Javascript: ambitious ideas, which may have been on the back-burner for some time, can finally be prototyped quickly and tested in a mostly-real-world environment, and everybody is excited to try it out and demo their creations.
One of the sponsors said:
"there is something quite good about not feeling bad about programming"
and that seemed representative of the current state of Julia. It seems undeniable that the language is less painful for developing performant numerical code than the previous generation of languages and library wrappers.
Perhaps because of this enthusiasm and froth of ideas, I'm a little worried that the foundations of Julia (the language and the ecosystem) have not yet had time to fully bake. The more demos and experiments that get implemented, and the more popular they become, the more delicate it becomes to make hard decisions about language syntax and features. I think people want stability and promised features yesterday, but these things take time and reflection. My feelings right now is that it doesn't really matter. The enthusiasm for a language like Julia is proven and growing. Julia itself might end up being the first try that gets thrown away in a decade or two, but in the end we'll end up with something which is both exciting and robust.