Pre-Conj Interview: Zach Oakes

Introduction

I had a conversation with Zach Oakes about game development in Clojure. He is giving a talk at Clojure/conj. Read the background to his talk.

Interview

LispCast: Briefly, how did you get into Clojure?

Zach Oakes: Being hosted on the JVM was my original draw to Clojure. I wanted to hack on an anonymous networking project called I2P, which is written in Java, but I wanted to use a more enjoyable language. Clojure's interop made it effortless, and later on I came to enjoy the other aspects of Clojure as well.

LC: How long have you been into games programming?

ZO: Games were how I originally got into programming as a teenager. However, I had a ten-year period where I did nothing but crypto software. I only got back into games last year, and since Clojure was my favorite language by then, it was inevitable that I would start using it for game development.

LC: What is the main concept of play-clj? Is it based on immutable game states, or something else?

ZO: The main technical concept is that most of your game should just be list transformations, so your code mostly consists of using normal Clojure functions to transform your entities. It definitely encourages immutability, but since it builds on top of a Java framework, there are times when you will need to mutate things.

LC: What is the advantage of using list transformations instead of mutating game objects? Is this a drastic departure from normal game architectures?

ZO: The advantage of functional programming in games is the same as in other kinds of software; the code becomes simpler to understand, and parallelism becomes trivial. I think this is very unusual for game development. Many games keep entities as class data members that the methods mutate directly, which quickly becomes a mess.

That said, I think the REPL is by far the biggest reason to use Clojure for game development. A disciplined programmer might be able to avoid the aforementioned mess, but it's hard to replicate the runtime modifiability of Clojure if your language wasn't built for it, and I think it has a big effect on how you make your game.

LC: What kind of response from game programmers have you had to using functional programming in games?

ZO: So far the response to functional game development has been tepid. They are understandably conservative about their technical choices due to the performance sensitivity of games, but nonetheless they are slowly adopting functional idioms in their non-functional languages. The next step is to get them to use a functional language. I think it's up for grabs, and the languages that are hosted and provide an excellent REPL will have the advantage.

LC: Can you tell us a little about play-clj? Is it based on a Java framework?

ZO: Yes, play-clj is essentially a big wrapper around libGDX, a popular Java framework. I used it to teach Java game development, and eventually made play-clj so I could use Clojure instead. It is a fairly thick wrapper and hopefully a nice teaching tool.

The core of the library is defscreen, a macro in which you provide functions that will be run during various events like "on-show" and "on-key-down". Making a game consists primarily of transforming your entities in response to these events.

There is always a design compromise when you build on top of a large object-oriented framework, but I think it is crucial to leverage existing work to make it practical to use a functional language to build games worth shipping.

LC: Yeah, I agree. Game frameworks have a lot of work put into them. Was it a challenge combining OO/mutable state with the Clojure functional approach?

ZO: It's definitely a challenge, and since this is the first library I've ever made I have been learning as I go along. I took some inspiration from neko, a Clojure wrapper for the Android API, since it is doing something similar. I don't try to hide mutations; most of the library is just a series of macros that allow you to use the underlying Java classes with a nicer syntax.

That said, you can at least keep your core game logic free of side-effects by deferring those mutations until later. In my play-clj games, I typically first map over my entities with a series of pure functions, then afterwards I do various side-effecting things like playing sounds and rendering. This allows me to trivially parallelize the pure portion of my code while also doing the necessary side-effecting things.

LC: That's pretty cool. I've always wanted to program games, but every time I start, I get lost in a forest of spaghetti code. Have you found that a functional approach makes that better?

ZO: I think it definitely imposes discipline on your game's code. It is easy in Java to make your entities and other objects data members in a class and mutate them from methods without even explicitly passing them as arguments, which is an easy path to spaghetti code. In play-clj, entities are records so transforming them requires explicitly passing them to functions and returning new ones at the end.

I would say that play-clj also imposes an additional degree of structure as well. It has its own built-in system for managing state, including its own entity system, which is exposed to you in the defscreen macro. I think this causes play-clj games to essentially have the same general design. The downside is that it makes a lot of design decisions for you, so more advanced Clojure programmers may desire a thinner wrapper.

LC: What resources could a beginner use beforehand to make the most of your talk?

ZO: I wrote a tutorial for play-clj that people could check out if they'd like to see how it works:

https://github.com/oakes/play-clj/blob/master/TUTORIAL.md

There are also some example games they could play to see it in action:

https://github.com/oakes/play-clj-examples

LC: Where can people follow your adventures online?

ZO: I don't do much social networking, unless Reddit/HN count. I suppose the best way to see what I'm up to is follow my Github!

LC: One last question: If Clojure were a Star Trek character from the original series, what character would it be?

ZO: I suppose Clojure would be all the redshirts collectively. It sounds like a bad thing, but when they died they were always replaced, so I guess that means Clojure will live forever.

LC: Thanks for the informative interview. It was a pleasure.

*include: templates/pre-conj.html

You might also like

Pre-Conj Interview: Colin Fleming

Introduction

The next interview is with Colin Fleming. He is the developer of Cursive Clojure. His talk at Clojure/conj will be about that. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Colin Fleming: I got into Clojure bit by bit, starting about four years ago. I was working in Java on highly concurrent transaction processing systems and the promise of a language that would make that easier was very appealing. I was also playing with Scheme at the time so it seemed like a good fit since I already knew the JVM really well.

I’ve used IntelliJ for Java for years now and I wanted to use it for Clojure too, so I installed La Clojure, which was the only option at the time. But I quickly got frustrated with its limitations and started fixing it up, porting the parts I worked on to Clojure as I went. That was really my first real Clojure project, which in retrospect wasn’t a great choice - it was a tough project to get started on. But it evolved into Cursive, which I’m very happy with, so it worked out well in the end.

LC: Developing an IDE is quite a big task. And the web site indicates that you will charge for it. What has the response been? How many users do you have in the Early Access Program?

CF: Right, it’s definitely a big task. Initially Cursive started as a hobby, just scratching my own itch. But I quickly realised that if I was going to develop all the functionality I wanted to add it was at least a full-time job, probably more. I had just taken some time off after leaving my previous job at Google and I thought I would see if there was appetite in the Clojure community for a professionally-supported development environment, and the initial response on the Clojure mailing list was pretty positive.

I released the first public beta version almost a year ago now, and the response has really been fantastic. Quite a few people have asked me when they’ll be able to pay for it so they can continue to support development, which is really nice to hear - I’m really happy people like Cursive so much. It’s hard to tell exactly how many people are using it, but each version tends to get around 10,000 downloads. There’s quite a long tail so it depends how long each version is out for, but generally there are around 2,000 downloads within 24 hours of making a release so I guess that’s roughly the number of people using it daily, or at least have it installed in the copy of IntelliJ that they use daily.

LC: I use Emacs but I can't really recommend it to someone who just wants to try Clojure. Does Cursive fit that bill and if so, how?

CF: Sure, I think Cursive is definitely easier to get started with than Emacs. In general IntelliJ’s interface is more like a standard application than Emacs, that familiarity helps a lot. And I try to make sure that Cursive is easy to get started with and to minimise the yak shaving required to open a Leiningen project and a REPL. There are still some outstanding things to do there, a big one has been that Cursive didn’t ship with default keybindings assigned - that’s fixed in the next release which should be out shortly. Cursive also has paredit turned on by default for Clojure which can confuse newbies, but it’s easy to toggle. And Cursive still doesn’t have a great story for a Clojurescript REPL, which is always a big pain point for people getting started - I’m hoping to fix this soon. Light Table really did a great job on that one.

Cursive has a user guide with a lot of screenshots that should hopefully make it easier for people to get started. It’s not 100% up to date with all the latest features, but it should help people to get going. I think having one definitive source of friendly documentation really helps at the beginning. I’d actually like to have an interactive tutorial in the IDE that people could run that would show them how to use the editor and possibly the basics of Clojure too, but that’s looking like a v2 feature at this point.

LC: I've always liked IDE's for their ability to smooth over the rough parts of a language. And tooling often trumps terseness in a language with less tooling. What are some of the trouble-spots you see in Clojure, from a beginner's perspective, that haven't been addressed with tools?

CF: The classic one that a lot of people seem to struggle with is the stack traces - I’ve never found them to be too difficult, but I worked with Java for years so I know what they look like. It’s easy to forget how difficult things like that are for people new to the JVM. Leiningen has done a great job of hiding things like the class path and dependency management as well, which can be really tough.

The other thing that I’d really love to see improved is the error messages. Since Clojure macro forms are essentially parsed by hand some of the error messages are really impenetrable. For example, something that might be a common newbie error like putting an (:import) clause ahead of the docstring in an ns form will give you a "java.lang.Character cannot be cast to clojure.lang.Named” error, and will also frequently attribute it to the wrong namespace - these sorts of errors can be really discouraging for someone trying to get started. Cursive internally uses something similar to the seqex library (from Chris Houser and Jonathan Clagget’s great Illuminated Macros talk) for its parsing, I’d love to see libraries like that being used in Clojure core.

Cursive doesn’t actually mark a lot of these errors in the editor yet, but I’d like to mark as many as possible. In particular, I’ll be able to mark things like function invocations with the wrong number of args, non-gensym’ed variables inside syntax-quote and many of the problems detected by tools like Kibit and Eastwood. I hope that’ll make Clojure less frustrating for newbies and more experienced folk alike.

LC: There's a bunch of stuff I'd like to ask you about. Let's talk about the practical side of developing an IDE. You say it's a plugin for IntelliJ. How much of the IntelliJ system are you able to leverage, how much is custom, and how much is a hack to fit the Clojure peg in the Java hole?

CF: Right, it’s currently only available as a plugin to IntelliJ. JetBrains actually put a lot of work into making their base platform reusable so I’ll be able to create a standalone IDE as well, as JetBrains have done with Redmine for Ruby, or PyCharm for Python. But it’s a reasonable amount of work and probably adds some support load with cross-platform issues so it hasn’t been a focus for me yet.

The IntelliJ infrastructure is really great - it’s certainly saved me a huge amount of work being able to build on top of it. It’s very impressive how well abstracted it is, I’ve encountered very few places where I’ve had serious roadblocks from IntelliJ itself due to the nature of Clojure. Most of the real difficulties are just intrinsic to how flexible the syntax of Clojure is, they’re not problems with IntelliJ. Obviously all the base IDE functionality comes for free (project management, VCS support, indexing infrastructure, debugger and so forth) and the Java support is very easy to hook into which is important for Clojure. The biggest part that’s totally custom is the symbol resolution, which generally has to be written for each language anyway.

Cursive also supports Clojurescript, and that’s been more challenging since the Javascript support in IntelliJ is only available in the paid Ultimate Edition and is closed source. Cursive currently implements fairly primitive Javascript support itself which is just enough for Clojurescript, but it would be nice to be able to integrate that better with the existing support - I’m hoping to in the future.

The part that is very different in Cursive from a more standard IntelliJ language plugin is that it all has to be extensible since everything in Clojure is based on macros, even core language features. People will eventually have to be able to add support for the libraries they use or for the forms they define in internal projects so Cursive is internally based around an extension API that I’ll open up at some point soon. More conventional languages generally have a fixed syntax, even if it’s very flexible like Ruby for example, so this isn’t an issue for them but it’s essential for Clojure.

LC: In the talk description, you mention Rename Refactoring. It's not something I can say I miss in Emacs, but perhaps I don't know what I'm missing. Do people make use of that and other refactorings?

CF: Oh yes, I’d definitely argue that you don’t know what you’re missing out on! The two operations I use a lot are rename and find usages - I’d struggle to live without them now, especially find usages. I also recently added refactorings to extract and inline let bindings, which are really useful - people definitely use them a lot. I’m planning to add support soon for extracting and inlining functions, as well as introducing parameters which I use a lot in Java. Refactoring in Clojure will never be as accurate as in Java, but it’s still very useful. Things like marking unused local variables is also something which sounds trivial but really catches a lot of bugs while you’re editing.

I think this is the main difference between something like IntelliJ and something like Emacs, actually. When I’m working on Java code in IntelliJ the refactorings and all the small operations they provide as intentions are so good that I really feel like I’m working directly on my program, not on text. We’ve all experienced that flow state when programming when you’re totally absorbed in what you’re doing and you lose track of time, whether you’ve eaten or need to go to the bathroom or whatever. IntelliJ makes it much easier for me to achieve that state because it provides a huge number of code manipulations that directly reflect what I want to do to my program, without me then having to muck around with text. I’ve seen comments online along the lines of “IntelliJ is great but I type so much faster in Emacs/vi/whatever”. I’d argue that approach is totally wrong - I don’t want to avoid typing because I’m lazy, I want to avoid typing because text manipulation is the wrong level of abstraction. Cursive is a long way from providing that experience, partly because of Clojure’s language characteristics and partly because it took teams of dozens of smart people at JetBrains years to achieve it for Java, but that’s always my ultimate aim whether or not I ever actually get there.

LC: What resources would you recommend people make use of before your talk in order to get the most out of it? Anything useful about refactorings or IntelliJ?

CF: I don’t think so, no - hopefully it’ll be pretty self-contained. I’ll be showing the features I’m talking about and explaining some of the implementation, so hopefully it’ll be understandable to everyone. And hopefully it’ll be interesting for everyone too!

LC: Where can people follow your adventures online?

CF: There’s a twitter feed for Cursive and there are also some mailing lists for general Cursive discussion. I’ve also been hanging out more in the #clojure channel on IRC as cfleming in case anyone has questions. I’ve been meaning to blog more about Cursive but so far that’s just one to-do item among many!

LC: Nice. One more question: what is Clojure's favorite breakfast?

CF: Cursive is most frequently powered by Tropical Mango and Nut Crunch, but the true breakfast of the gods is scrambled eggs with little chunks of Venison Pepperoni salami cooked through it.

LC: Cool.

Thanks for the great interview. It was fun.

CF: Thanks Eric, that was fun! I’m looking forward to seeing it, and to catching up at the conj too!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Ashton Kemerling

Introduction

I asked Ashton Kemerling some questions about his talk at Clojure/conj about using generative testing in Clojure. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Ashton Kemerling: The path I took to Clojure is fairly roundabout. During my College years I was very interested in function programming, although ML & Haskell were my favorites back then. I think a large amount of my fascination with them stemmed from the fact that I had more experience than my classmates, my high school had a very rich set of programming courses, and I needed something interesting to play with while taking introduction to OOP courses.

Fast forward a few years and I ended up working at a job using Common Lisp professionally in a legacy web application. The application had a lot of problems due to the condition of CL libraries at the time and we were looking for something to switch to. Clojure was an obvious choice to look at. It's still a lisp, so a lot of our habits would still work, but it fixes the deployment & library issue that CL has while adding a much richer set of data structures. Unfortunately our application relied extensively on mutation (CL doesn't really encourage or discourage any coding style) and OOP, so a conversion was deemed too costly to attempt.

Over the course of the following years I continued to tinker with Clojure on the side. I was convinced it was a great way to do things, but I could never convince anyone to consider using it in their code base. Most places were either too invested emotionally in OOP, or had well tested and working codebases that made a rewrite unwise. So I mostly just hacked on side projects using Clojure over the intervening years without using it professionally. But in the last year or so I realized that Clojure makes a great secondary language for odd-tasks in non-Clojure codebases, which brings me to the current state of using Clojure to test a very large Ruby on Rails and Javascript application.

LC: Can you give a basic outline of how your Clojure program tested the web front end?

AK: We've basically tested in two ways. The first was to simply compile our JS file along with Clojurescript tests and run them in a PhantomJS process similarly to how you'd run Clojurescript or Javascript tests normally. We're still working on this approach, but it's slower to work on that I would have hoped.

The second way was to leverage the fact that Clojure's on the JVM. This involves mixing JDBC, HTTP libraries, and Selenium in novel and exciting ways. This has been the most fruitful way of testing, and what I'll be focusing the most on during my talk. We've used the code I've written as both a tool to hunt down API mistakes, and as a means of narrowing down the reproduction steps on complicated bugs reported by the user.

LC: Can you explain a little more about that? What do you mean by "narrowing down reproduction steps"?

AK: Sure. I'm testing Pivotal Tracker, a very large (~24kloc) Javascript "single page app". With large apps like this there are ways for the application to subtly fall out of sync with the server or process data in a degenerate way.

In particular a lot of times you'll get a report from a user that says something along the lines of "I reloaded and it crashed". You can dig up the logs from that session, but you'll get the last 20 or 30 steps they did before the crash, which doesn't really help you out a whole lot. So we occasionally turn the generative tests towards a type of action (in this case, do things and then refresh) in the hopes that it will help us narrow down to the minimal reproduction steps required to trigger a problem.

In the case that happened last week, we were able to find 2 distinct actions that would provoke the issue reliably after about 20 minutes of test modification and running. Obviously this took hours off of the process required to find the actual root of the bug, since 2 is orders of magnitude easier to work with than 7 or 10 when debugging.

LC: I see. So how is test.check able to reduce the reproduction steps?

AK: Test.Check provides "shrinking". Just as the generators used to create randomized data have the ability to produce more complicated data, they also have the ability to produce less complicated data. Test.Check records all of the failures it finds and attempts to simplify them and find the smallest failing case it can.

LC: A lot of the Clojure/conj participants will be new to Clojure. What resources would you recommend them to make the most of your talk?

AK: They can't go wrong with the test.check README or any of the blog posts mentioned therein. I recommend reading the source directly because it's heavily commented, in particular I recommend the generator source, because the generators represent 90%+ of the test.check API a user will interact with.

Beyond that I have blog posts on my work blog and on my personal blog. Also Reid Draper and I were both on the Cognicast, both of which are based around this subject.

LC: Are there any resources on Selenium and other methods for running frontend tests?

AK: The Clj-Webdriver docs are all I can recommend.

LC: Where can people reach you?

AK: Twitter: @ashton

My blog: ashtonkemerling.com

LC: If Clojure were stranded on a desert island, what one book would it bring?

AK: A boat building book, clearly.

LC: Awesome! Thanks for the interview, it was a pleasure.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Steve Miner

Introduction

I had a conversation about generating test.check generators with Steve Miner. He's giving a talk at Clojure/conj about that. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Steve Miner: I think it was around 2008 when Paul Graham announced his new programming language, Arc. I was reading about Arc, when I came across a comment about Clojure and decided to take a look. I was very impressed with Rich Hickey's intro video. The immutability and concurrency features really resonated with what I was hoping to find in a new language. The Java integration made Clojure a practical language with a huge eco-system of tools and libraries. I was using Java at work and I managed to do a bit of Clojure for a side project, but mostly I was just dabbling. A couple of years later, I decided to work on my own with Clojure full-time.

LC: Can you describe Herbert for those who have never used it? Why would someone be interested?

SM: Herbert is a schema language for edn. The goal is to have a convenient language for describing the shape of Clojure data. I started out with an informal notation that I used for my internal documentation. For example, describing a map with certain required keys and the corresponding value types. I think you can guess what {:name str :num int} means as a schema. It turned out that with a little work, that informal notation could be used as a pattern language with a simple API for testing conformance. A Herbert schema is itself just edn data, open to all your Clojure tools. More recently, I added the ability to generate test.check generators from Herbert schemas, which makes it easy to generate test data.

LC: That sounds nice. Would you mind explaining test.check generators a bit for those who don't know?

SM: Reid Draper ported QuickCheck from Haskell to Clojure, and it became a contrib library called test.check.

Test.check is a property-based testing tool. A property is basically an invariant that should hold true over a range of input values. The test.check library gives you combinators that allow you to define generator functions which create data of specified types with optional constraints. The idea is to think about the whole range of possible inputs that your system should handle. Test.check then automatically tests across a random sample of generated data, probably generating example data that you might not have considered in your typical unit testing. If it finds a failure case where the desired property does not hold, test.check is smart enough to regenerate test cases so as to shrink the failure example to a reasonable size. That helps you isolate the cause of the failure.

Clojure/West had two excellent talks about property-based testing last year. Reid Draper covered test.check, and John Hughes talked about QuickCheck.

LC: So you're able to automatically create generators from your schemas, which can also be used as contracts on your function arguments. Has having both improved your bug rate?

SM: I don't have any numbers, but subjectively I think it's helped. For me, Herbert schemas are primarily documentation tools, which help me to keep track of my data. That being said, I often test schema conformance in preconditions or asserts, especially with new code or when I'm trying to debug a problem.

Of course, I still make errors in specifying schemas and sometimes my properties aren't exactly correct the first time. Particularly with new property-based tests, I have to look carefully at failures in case the bug is actually in the test. My hope is that schema-based generators will make property-based testing easier to use.

Using test.check definitely improves my confidence that I'm finding bugs in testing and avoiding regression errors. It's been a great way to catch bugs in my own Herbert library.

LC: So you have runtime check to make sure the function arguments conform to certain schemas. And you have a generative test that exercises a large space of that schema. Sounds pretty good to me!

But it sounds like you're saying the primary benefit is more for you or other readers of your code. Can you elaborate on that?

SM: My approach started with a notation designed to help me keep Clojure data structures straight in my mind. I wanted something simple and terse, what I called a "whiteboard compatible" notation. My goal was that the notation should look something like the data it was supposed to represent as opposed to code or a type system.

So I began with documentation in mind, and I still think of that as the primary benefit. Once I got the idea of implementing conformance testing against formal schemas, the project became more about the code.

LC: In what other areas do you see schemas playing a part? The first thing that comes to mind is writing core.typed type annotations. Anything else?

SM: There's some conceptual overlap between schemas and type systems, but I see core.typed as a much more ambitious project. Herbert schemas only cover edn data and don't deal with function types, for example. The Datomic database naturally has a schema language, so it would be interesting to see if Herbert could useful for data modeling. In the near term, I plan to extend Herbert so that it supports the Transit datatypes.

LC: What resources would you recommend to a beginner who wanted to make the most of your talk?

SM:

LC: Where can people follow your adventures online?

SM:

LC: One last question: If Clojure were a food, what would it be?

SM: I'll say "pizza" because it's a food that is made by composing fancy toppings on a classic crust. And hackers like it while coding.

LC: Thanks, Steve, for a great interview.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Jeanine Adkisson

Introduction

Jeanine Adkisson was kind enough to answer some questions. Her talk at Clojure/conj will be about variants in Clojure. Read the background to her talk.

Interview

LispCast: How did you get into Clojure?

Jeanine Adkisson: Through work, actually. I'd seen clojure around and studied some of its features from a language design perspective, but I've never really been a lisper, so it hadn't occurred to me to actually use it until we started a migration to datomic at GoodGuide. I admit to being a bit skeptical - I often find that lisps are a bit oversold - but a coworker of mine (@bonkydog) started really gushing about generative testing, core.typed, and the awesomeness that is leiningen, so I decided to give it a closer look.

LC: Your talk is about Variants. Can you explain Variants and how they are useful?

JA: Sure! Variants are a functional design pattern that should be familiar to anyone who's worked in the Haskell/ML family of languages (and to a lesser extent, Erlang). Their main purpose is to provide a structured way to manage polymorphism. In lisps and other single-type languages, they're not that widespread because we don't have to convince a type-checker that our polymorphism works out in the end. But we still have to convice users of our libraries and our doc writers that our functions make sense!

The method of representing variants I'll be talking about came out of some frustrations I was having at porting part of a language over to clojure, and finding it difficult to represent and destructure the trees I needed to process. But I was really impressed at how quickly David Nolen and Ambrose BS got back to me and shipped patches that made this approach possible (and typeable!).

LC: You mean David Nolen patched ClojureScript and Ambrose BS patched core.typed?

I wonder what it means about dynamic languages like Clojure that you can add Variants as a library instead of making it an integral part like in Haskell. What do you think about that?

JA: Yeah, actually it was a bugfix the match macro and the way that core.typed handled vectors that were key. What's even more interesting about variants is that there is really no library to write - all the machinery is already there. The solution isn't even particularly clever or novel - it's already in use in, for example, instaparse. It's just not on people's radars, or they have misconceptions about the approach from their OO training.

I love the fact that you can backport this pattern into a language that's not explicitly built for it. I wouldn't necessarily count that as a strength for dynamic languages in general though - I find a language's strengths often come from ways in which semantics are restricted. Clojure is already a great example of this because it explicitly rejects the use of willy-nilly mutable structures, to great benefit. Sure, you can program that way in Ruby, but the tools are already there in Clojure.

What I find coolest about the Clojure case is the extent to which core.typed has been able to backport types onto a mostly untyped ecosystem. The general problems with dynamic languages are still there, but they have to be made explicit with union types. How many functions surprised you by outputting (U <thing> (Value nil))? But because it's powerful enough to support variants in a lightweight and usable way, I think we now have a tool to deal with one of the most common culprits of surprise nils.

LC: I think the approach is worthy of study. I flipflop between statically typed vs dynamically typed because there are obvious problems with both. How close does your core.match and core.typed solution come to a full-on ML/Haskell-style, integrated solution?

JA: It's pretty comparable, and actually has some advantages. Since it's just vectors and keywords, it's trivial to handle in a generic way, and as long as you're not putting functions or host objects in them it's trivial to serialize and send over the wire. (Haskell and ML can do most of this programming for you with various generic programming extensions, but it's not straightforward or obvious). But yeah, core.typed is capable of refining types based on the tag matching code generated by core.match - a little roundabout, but it gets the job done. The match macro can also match against more than one object at a time, something OO languages constantly struggle with. There's a little more macro work to be done in core.typed to make it really straightforward to do variants, but it's a relatively trivial syntactic transformation.

In terms of typed vs. untyped, I think there's lessons to be learned from both. My instinct is to say that we just haven't really figured out how to make typed systems all that usable. Haskell and ML went a long way towards this, and I think they kind of hit a wall because of their restriction that compilation be decidable. In other words, the compiler is not allowed to infinite-loop - which sounds like a good idea until you line up all the restrictions required to make that happen. In dynamic languages, we don't have a semantic separation between "compile-time" and "run-time", but we still talk about "load-time" vs "run-time". And at load-time you can wield the full turing-complete power of the language. That's why I'm also excited about projects like Agda and Idris, which, while still statically typed and compiled, blur the lines between runtime and compile time - moving away from strict decidability guarantees in exchange for a more expressive system.

LC: A lot of the audience is going to be new to Clojure or they have never heard of variants. Is there a resource that you think would help make the most of their time? Just a little pre-reading or pre-watching so they feel confident with the material?

JA: Yeah, there's unfortunately not a whole lot targeting clojure or other lisps, which is one of the reasons I wanted to give this talk. There's a great section in the ocaml tutorial about them, and Learn You a Haskell covers them as well (although they call them ADTs). The approach I'll be talking about is most like what's used in Erlang, as in the temperature example here.

LC: Where can people follow your adventures online?

JA: Probably the best way to keep up with me is on twitter at @jneen_ (note the final underscore), or on github as @jneen.

LC: One last question: If Clojure were a comic book superhero, what would its superpower be?

JA: Hahaha, I'd probably say a shapeshifter, like most lisps I suppose.

LC: Thanks for an excellent interview. I had fun.

JA: Awesome, thanks!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Lucas Cavalcanti

Introduction

I got in touch with Lucas Cavalcanti. He is giving a talk together at Clojure/conj with Edward Wible, who was unavailable. The background to their talk is available, if you like.

Interview

LispCast: How did you get into Clojure?

Lucas Cavalcanti: I've got into Clojure at Nubank, the company I currently work for, about a year ago. Nubank was already using Clojure, so I learnt it as we evolved the project. I was already familiar with Lisp syntax and knew functional programming very well, so the move to Clojure was not too hard.

LispCast: Can you explain a little about Nubank? And how do you use Datomic?

LC: Nubank is a tech company for financial services that just went public:

We use datomic as our primary database system, taking advantage of the free datomic transaction logs for auditing, and adding more data to improve auditing. We also use the datomic query language (based on datalog) to analyze data, often joining multiple databases, since we have a SOA with each service having its own database.

We also use some datomic capabilities to do database sync with mobile clients and cache management.

LispCast: Your talk's description mentions some "hidden superpowers of Datomic". You mention that the solutions to these problems are generic. Does that mean they can work for any problem?

LC: The solutions we'll provide will have some small assumptions to work, but these assumptions don't depend on the entities types we're saving, just something about the database structure. Thus we said the solutions are generic, and could work for any problem that matches these assumptions.

Also means that we can write just one function that will serve to implement one of that "superpowers" for all entities/endpoints of your system.

LispCast: So a lot of people are new to Clojure, and certainly to Datomic. Can you explain Datomic briefly for those of us who have never used it?

LC: If you know Git or any version control system, Datomic brings to a database many powers you have with Git: Think of each database transaction as a commit that generates another version of the database. So you can get the database value at any point in time, check the changes made by a specific transaction, all the history of a certain entity or even create "branches" of the database.

It works by collecting facts like "the entity 123 has the attribute :speaker/name with value "Lucas Cavalcanti"" and gives a query language based on datalog that, once you get the basic concepts, makes much more sense than any other query language I've worked with.

LispCast: So, is there anything else you think people should read up on to make the most of your talk?

LC: Read the docs: http://docs.datomic.com/index.html

Specially:

LispCast: Where can people follow your adventures online? Where do you socialize?

LC: I guess twitter @lucascs

LispCast: Ok. Last question: If Clojure and Ruby got together, what would they name their baby?

LC: http://en.wikipedia.org/wiki/Platypus

;)

"the (dis)functional duck typing" ;)

LispCast: Awesome! Thanks for a great interview. It was informative.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: David Pick

Introduction

David Pick was kind enough to answer a few questions about himself and his upcoming Clojure/conj talk. You may want to read the background before you begin.

Interview

LispCast: How did you get into Clojure?

David Pick: A friend of my mine suggested I read Purely Functional Data Structures and told me the work in the book was the basis for Clojure's persistent data structures. I really enjoyed it and decided to give Clojure a try.

LC: Braintree must process a lot of very important data, since it's a payments service. What were you using before you developed the Clojure and Kafka system?

DP: I'd say for 98% of our systems we still use Ruby and Postgres, we're simply using Clojure and Kafka as a messaging platform to move data between systems.

LC: What is Kafka?

DP: Kafka is a system for doing publish subscribe messaging, that was built by LinkedIn to have extremely high throughput and replay ability of messages. It stores all messages sent through the system for a rolling window. So consumers simply say give me messages starting with offset n or whatever your most recent message is. This way if a receiving system goes down the consumer can just rewind the stream and replay messages that were missed.

LC: Could you describe the problem you were trying to solve? What were the requirements?

DP: There were actually two problems we were trying to solve. The first was to pull our search tool into a separate system that was more scalable than our current one (the current one just runs sql queries on main database). The second was to improve the accuracy and speed that our data warehouse is updated.

For the infrastructure part of the solution we ended up using Kafka, Zookeeper, and Elasticsearch which are all JVM based so that ended up heavily influencing our decision to use Clojure.

LC: You mentioned in the talk description that Clojure had certain advantages. What tradeoffs did Clojure, or your choice of technologies, bring about?

DP: The biggest tradeoff I think we made with Clojure is familiarity, I was the only one on the team who knew Clojure prior to starting this project. So that has certainly slowed us down. Other than that things have gone surprisingly well.

LC: I'm really fascinated by the engineering efforts to build very reliable and robust systems. Are you going to talk about that?

DP: Yep, that's the plan!

LC: Awesome.

Do you have any resources that would help someone new to Clojure or new to Kafka so that they can make the most of your talk? Some pre-reading or pre-watching materials?

DP: For Clojure I definitely recommend both the Clojure Koans and 4Clojure. For Kafka I'd recommend the introduction on their site.

LC: Where can people follow your adventures online?

DP: Twitter is probably the best place @davidpick.

LC: Who would win in a no-holds-barred fight to the finish, Clojure with an Arduino-controlled laser or a T-Rex?

DP: If Rich is writing the Clojure I give it a fighting chance, otherwise it's the T-Rex for sure.

LC: Awesome! Thanks for a great interview. I look forward to seeing you at the Conj.

DP: Awesome, thanks Eric!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Zach Tellman

Talk: Always Be Composing

Zach Tellman's talk at the conj is about building composable software.

Background

People often boast of the composability of Clojure programs. Compojure claims that it "allows web application to be composed of small, independent parts." Ring itself is based on composing middleware and handlers to build a web server. In fact, you can compose pieces easily with Ring, if those pieces were designed to compose with Ring. But not all libraries compose well with the Ring ecosystem. This talk asks "How can we design libraries that compose well in general?" A good intro to composition in Clojure is Rich Hickey's talk Design, Composition, and Performance.

Why it matters

I often get asked why a certain library in Clojure doesn't have an easy interface like the one in Ruby. The answer, in many cases, is that the Clojure library was designed to compose with other code, while the Ruby library was meant for programmer convenience.

But if I leave it at that, this answer is unsatisfactory. Behind my answer is years of experience telling me that composition beats convenience in the long run. The asker of that question does not have that experience. I would love to have a way to explain why in an analytical way.

If this talk even so much as opens the conversation up about the design space of composition, it would be extremely helpful. But I'm expecting more from Zach Tellman. His talk could have a profound impact similar to Rich Hickey's talks on immutable values and simplicity.

About Zach Tellman

Github - Twitter


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Zach Oakes

Talk: Making Games at Runtime with Clojure

Zach Oakes' talk at the conj is about making games in a dynamic programming environment.

Background

Clojure, and Lisps in general, are famous for dynamic programming. That means being able to modify the program as it is running without restarting it. Wouldn't it be nice to develop games in this way? It seems like the perfect fit, since video games are so visual, you'd want to see what your code changes look like very quickly. This talk will explore this. Nightmod is a development environment written by Zach Oakes to modify a game in one window while it is running in another. It is based on the play-clj library, which he also wrote.

Why it matters

Game programming is a huge industry dominated by static languages with long code-compile-run cycles. Games cost millions of dollars to make, so even small improvements in developer productivity can save lots of money.

But further, games are hugely complex systems that are often on the cutting edge of programming. Functional programming has a lot to say about complex systems, yet there has not been a lot of functional programming in commercial games. The domain is ripe for exploration and discovery.

About Zach Oakes

Github

You can support Zach Oakes at Gratipay.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Paul deGrandis

Talk: Unlocking Data-Driven Systems

Paul deGrandis' talk at the conj is about the design tradeoffs when building your system in a Data-Driven way.

Background

The recommended priority for choosing an implementation strategy is Data > Function > Macro. Macros are made for humans, not machines, so they are the least composable. Functions are composable, but opaque. Their only meaning is their side-effects and their return value. Data, however, can be interpreted in different ways, depending on the context. If one can design it right, a data-driven system can be both easy for a person to read and write and reusable for many problems.

This talk is part of an ongoing discussion about data-driven systems in Clojure. Data-driven design has been around far longer than Clojure has. Paradigms of Artificial Intelligence Programming is the book on the subject, showing its use in Common Lisp. Christophe Grand gave a talk at the Clojure/conj in 2010 that explained in depth why macros should not be the first choice when building DSLs.

Many systems use data as their primary interface. Datalog queries in Datomic are just Clojure data. Prismatic Schemas are data. Leiningen project files are data (with a small macro for human convenience). Data-driven system is related to code-as-data, that all Lisps share. If code is data, why can't data be code? Clojure makes data-driven solutions easy with its variety of literal data structures.

Why it matters

The conversation about how best to design data-driven systems is not over. Active Clojure developers are still experimenting and testing the approach. This talk will show data-driven systems taken to an extreme, where an entire ClojureScript application is represented as data. Many people already agree with the approach and are eagerly awaiting the deeper analysis this talk promises.

About Paul deGrandis

Github - Twitter


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like