Pre-Conj Interview: Steven Yi

Introduction

Steven Yi is next up on the interviews. He will talk at Clojure/conj about music systems in Clojure. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Steven Yi: I think I came across it online about two years ago when it started to get a bit of a buzz. I had long wanted to learn a Lisp, but Common Lisp and Scheme never really clicked when I tried. Clojure was different though, and since I do a fair amount of Java programming, the possibility to integrate Clojure with my JVM music software Blue helped motivate me to get into it.

LC: Can you briefly describe Blue, Pink, and Score?

SY: Blue is a music composition environment, written in Java using the Netbeans RCP. It is a graphical environment with a score timeline, graphical instruments, mixer and effects, and more. It uses Csound as its audio engine and signal processing system. (Disclosure: I'm also one of the core developers of Csound.)

Pink is a Clojure library for building music systems. It features a low-level audio engine, higher-order events, and includes signal generation and processing functions.

Score is a Clojure library for higher-level music work, particularly for generating notes and scores using various strategies. It is designed to be generic and usable with multiple backends. Using Score and Pink together can be particularly effective and expressive, having one language for the full musical stack.

LC: Overtone has been getting a lot of attention lately. How do Pink and Score compare to Overtone?

SY:Ah yes, certainly the elephant in the room. :) I'll certainly address comparisons to Overtone in more detail during the talk. In a nutshell, Overtone builds upon SuperCollider. Of those two, Pink would compare more to SuperCollider than to Overtone, as it is concerned with low-level audio engine design. Score deals with higher-level abstractions in music, many of which are covered in Overtone. However, because of the full-stack Clojure design between Pink and Score, Score can interact with Pink in ways Overtone/SuperCollider can not. Pink/Score are currently not as mature nor as performant as Overtone/SuperCollider. There's a lot of overlaps between the various systems, and use cases and requirements for a user's project will guide them to one system or the other.

Hopefully that's enough to get going! I think the breakdown of music systems design I'm planning for the talk should hopefully make things clear.

LC: Is there any resource that could help someone new to Clojure, or new to music systems, get the most out of your talk?

SY: think it's a tough question about music systems as I don't know of any resources out there that really introduce the subject. Having some experience with Overtone would certainly be useful. I can say I'll probably mention SuperCollider, Csound, Common Lisp Music, Nyquist, and Common Music during the talk:

and perhaps Incudine and OpusModus:

But that's a long list. :P

Actually, the Wikipedia link to Music-N would probably be a good place to get going.

LC: Where can people follow you on your adventures?

SY:

LC: Thanks, Steven, for a great interview! It was a pleasure.

SY: And thank you too!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Bozhidar Batsov

Introduction

Bozhidar Batsov agreed to an interview about his talk at Clojure/conj about CIDER, the Clojure Emacs IDE. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Bozhidar Batsov: I’m very fond of Lisps and I love exploring new programming languages. In 2009 Common Lisp was my favourite Lisp dialect and I was often solving small programming exercises in it. I wanted to do some “real work” in CL, but the ANSI standard had left out basic things like networking, GUI, etc. and relying on distribution-specific extensions didn’t feel right to me. I was working as a Java developer at the time, so when I stumbled upon a news about the release of Clojure 1.0 I was super excited. A practical Lisp seemed like a dream come true to me! I immediately started playing with Clojure and enjoyed it immensely. I was impressed by the immutable data structures, the seamless Java interop and the powerful concurrency primitives. Along the way I developed a deep appreciation for the merits of functional programming and explored a few other functional programming languages.

LC: What is CIDER and how did you wind up as its maintainer?

BB: CIDER is an interactive Clojure development environment for Emacs. It’s pretty similar to SLIME (which I consider CIDER’s spiritual ancestor) for Common Lisp and Geiser for Scheme/Racket. People who are not familiar with Emacs should think of CIDER as a Clojure IDE plugin, that provides a ton of useful (Emacs-flavoured) features like interactive code evaluation, code completion, documentation lookup, code navigation and many others.

CIDER is built on top of the popular nREPL server and replaces SLIME + swank-clojure as the standard way to write Clojure programs in Emacs. A lot of the functionality in implemented in terms of reusable nREPL middleware, which makes the project pretty accessible to Clojurists who are afraid of Emacs Lisp. Portions of CIDER’s middlewares are used in other dev tools like vim’s fireplace and Gorilla REPL.

I was an early adopter of CIDER (which was named nrepl.el back then) as I was pretty excited to use a development environment tailored specifically for Clojure (unlike SLIME). The project felt pretty barebone (compared to SLIME) and my Emacs skill were pretty decent, so I started contributing features and improvements to it (I think the first major thing I contributed were the compilation error highlights). About one year after the project started, the original maintainer Tim King had to step back and handed over the maintenance to me. I’ve been hacking on CIDER pretty actively ever since and I have epic plans for its future! :-)

LC: What is the best resource for someone who wants to get started with CIDER?

BB: I guess that for people familiar with Emacs the best resource would be the project's README itself.

Users who are new to Emacs should probably start with some gentler tutorial like http://clojure-doc.org/articles/tutorials/emacs.html or perhaps http://www.braveclojure.com/basic-emacs/.

While not strictly necessary, I’d encourage people to also have a look at cider-nrepl’s README.

A more extensive user guide (perhaps in texinfo format) and a few screencasts are on the roadmap, but I cannot promise when/if they’ll become a reality. Lots of bugs to squash and features to add prevent me from spending more time on improving the documentation.

LC: What is the best resource for someone (like me) who has migrated from SLIME and might not know all the cool stuff that's in there now?

BB: The README lists all prominent config options and commands (or at least I think it does). The changelog is also a great place to discover new features.

There’s a wiki entry on differences with SLIME, but it hasn’t been updated in a while, so people won’t learn much from it. Guess I should find some time to update it!

I’m always planning to write blog posts and do short screencasts for important new features, but unfortunately I never find the time to do so. Who knows, maybe next year...

LC: Where can people follow your adventures online?

BB: There are plenty of opportunities for people to stalk me online. :-) Here are some of the best options:

LC: One last question: If Clojure were a zoo animal, what would it be and why?

BB: A bear! Bears are (arguably) cute, smart and powerful. They are good at pretty much everything - they can run very fast, climb trees and swim. Does this remind you of a certain programming language?

LC: This was a great interview and I enjoyed it!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Paul deGrandis

Introduction

Paul deGrandis gave a nice interview about his talk at Clojure/conj about data-driven systems. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Paul deGrandis: I was a very early Clojure adopter. I came from Python (and before that a mix of C/C++/Java/Python), but I had always used Common Lisp to prototype ideas. Once I saw Clojure publicly announced, I gave it a shot and instantly identified with the Clojure Trinity: Simplicity, Power, and Focus. I began using it in all of my proof-of-concept work, and as I became confident in the language, integration with the platform, and my own skill set, I also started putting it into production.

LC: Can you define Data-Driven systems briefly?

PdG: Data-driven systems capture their interactions and operations as values - they are one possible conclusion of programming with values. We see this done in Datomic - queries are expressed as data. This concept is easily mapped to other systems - services, client-side applications, and more. Note that is more than just, "data all the things," this is, "data all the thing's things." Another example: Pedestal routes are represented as data ("data all the things"), but imagine if we captured the expression of an entire service as data. Imagine the properties that naturally fall out from that design decision. My talk walks through a real project that pushed this design decision as far as possible, and the steps that other developers can follow to do the same for their own systems.

LC: At Cognitect, they talk about Data > Functions > Macros. Is that what your talk is about?

PdG: My talk, at a very high level is about "Data > Functions > Macros," but it's mostly about modeling entire systems (not just interfaces) as data, why you'd want to, and why it's hard. It definitely challenges the previous notions (established by existing tooling) of what the conclusions of "programming with values" actually are (or could/should be). Tangentially, I'll talk about DSLs, but only in regards to constraining your domain as a means to increase expressiveness.

LC: I'm a big fan of working with data. But I've also seen its dark side. I've seen it pushed too far, or in the wrong area, to where it would have been better to use code. Have you noticed this? How do you avoid that?

PdG: I have indeed noticed it and it's a topic within my talk. Two crucial pieces of any great design (in software or otherwise) are the external constraints imposed by the problem, and the internal constraints imposed by the designer. Both are important, but it seems common that people overlook self-imposing constraints - and one needs them to achieve a really great design. I don't necessarily agree that capturing a system in data can be pushed too far, but I think it can be done without constraint or design intent - which produces a useless system. The crux is knowing that limitation and working within it. Outside of those limitations and constraints, one should fall back to foundation - regular Clojure code, etc.

LC: What is one thing you want someone to be able to do after they watch your talk?

PdG: I want someone to come away with a set of techniques to envision, constrain, and design data-driven systems. I want to create a spark that encourages Clojure developers to look beyond the initial merits of programming with values, and see further-reaching impact within their own systems - how to turn our ecosystem into a set of super powers.

I hope to provide a compelling example of how this approach plays out in production systems, and the general metrics and project impact it has.

LC: What resources would you recommend to a beginner so they could maximize their understanding of your talk?

PdG: My talk touches upon a lot of themes covered in some inspiring and informative talks: Tim Ewald's keynote from last year's Conj (Programming with Hand Tools), Rich Hickey's GOTO conference Keynote (The value of values), and previous Conj talks. I'm still searching for literature and books that I like on the topic, but I haven't hit any that I'd strongly recommend.

LC: Where can people follow your adventures online?

PdG:

LC: If Clojure rode an animal, what animal would it ride?

PdG: A very serious pony.

LC: Thanks for a great interview. It was informative.

PdG: Thanks a ton! I really appreciate all of this!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Anna Pawlicka

Introduction

Anna Pawlicka generously agreed to do an interview about her talk at Clojure/conj. The background to her talk is available, if you like.

Interview

LispCast: How did you get into Clojure?

Anna Pawlicka: During my final year at university I was looking for a contract work or a summer internship and my recruiter suggested a big data startup. Description mentioned Clojure so I solved some 4clojure problems and they sparked my interest. Computer Science course I studied was mostly based on Java so it was refreshing if not challenging to try functional programming. I got a Clojure in Action book, applied for the internship and luckily was offered the job. It was a fun summer: lots of Cascalog queries, friendly Clojure Dojos and interesting talks organised by London Clojure Community. It felt like home and you don’t walk away from that feeling. I stayed for both the language and community.

LC: You've been presenting and blogging a lot about Om. What are you using Om for and why?

AP: We’ve been recently working on a building performance platform at Mastodon C (project is open sourced). It’s built around high volumes of data and big part of it was UI programming. We were looking for something that performs well but isn’t t too complex and Om came out just in time for us to solve this problem. We’ve built the entire UI using Om and we combined it with d3.js and core.async for nice data visualisations and user interaction. I don’t think we’d be able to achieve that using vanilla JavaScript. Sure, there were some bumps along the way but the overall reusability makes it worth a while. Outside of my day job I’m working on a in-browser genetic algorithm testing tool, using Om and Quil, but with all the presenting and other activities I had to pause the development for a while.

LC: I've used React or Om for a few things and it quite simply enables me to develop UI interaction that would be too intricate otherwise. How has your experience with Om been relative to other frameworks?

AP: I haven’t used any frameworks as it’s just easier to mix and match various Clojure/ClojureScript libraries depending on one’s needs. Om was the first library we’ve tried for interacting with React and it suited our needs well enough that we haven’t had the need to try anything else. If you choose a framework, like let’s say Hoplon, you have to go with its programming model and you lose the ability to control every part of your stack.

LC: You say you're using Om with d3.js. Om is a very functional style, while d3 uses mutation pervasively. How does the integration work? How well do they work together?

AP: Fortunately both are driven by data so having d3 visualisations render on your application state change makes perfect sense. I use interop a lot, and although it doesn’t look as nice as pure ClojureScript, it does the job. We can easily replace iteration methods like map and reduce with their ClojureScript alternatives. The only problem is the type coercion between Clojure’s data structures and JavaScript’s Object. It’s not always immediately clear which one should be used where. The perfect option would be to control SVG via Om (or similar library) and use C2’s functions to calculate path data. But I haven’t tried that myself yet. Charts I’ve been working on are quite complex and d3 comes with this gold mine of resources that none of the other visualisation libraries has. I’m not yet done experimenting though.

LC: Can you explain the basic models of Om and d3?

AP: D3 performs data-driven DOM manipulation, e.g. you can build a bar chart out of an array of numbers and when your data changes so will your chart. D3 can manipulate any part of DOM and has lots of helper functions to compute our visualisations. React/Om works on DOM as well - we write functions that translate our data into a DOM, but the difference is that the DOM is virtual. Each time the data changes, React/Om compares the old and new state and computes the minimal set of transformations that will be applied to the real DOM. DOM operations are slow so naturally this virtual representation speeds things up by skipping unnecessary transformations. This means that both Om and d3 want to be in control of the DOM inside our component. To make use of both libraries, you can either have Om take care of everything (but this means we need to write functions to compute paths, scales, etc.), or you could use d3’s scales, projection helpers and path generation while having Om generate and manage SVG, or you can have an empty div rendered by Om and allow d3 to generate and manipulate the SVG inside. The latter is the most common approach, probably because we leave the decision whether to re-render to Om (skipping any insignificant data changes), and when Om decides to render the component, d3 takes care of the rest. We combine the best of both.

My talk will include some of that, with examples, since you can’t have a dashboard without a nice chart :)

LC: I see. It sounds like they work well together. I wouldn't have guessed that.

In the talk you mention live data. How does your system handle that?

AP: I don’t have an actual system that I plan to demo - the talk is based around various ways to create, reuse and combine Om components to display data. Long polling and HTTP streaming are available through HTTP Kit, but I also like how Sente combines that with core.async. But the focus is mainly on the components themselves - how you can mix and match them.

LC: Really cool.

Talks are always limited in time, and there's lots of context that you assume people have (or wish they have). What is something people can learn more about to help them get ready for your talk?

AP: I will try to keep the talk as self contained as possible, but I won’t be able to cover things like the basics of web development with ClojureScript - this would be material for a separate talk. However, more and more people are using ClojureScript so I hope most of the attendees shouldn’t have a problem with that. Those who feel that they need to prepare more will find lots of resources online.

LC: Where can people follow your adventures online?

AP: The usual: Twitter, my blog and GitHub account. I’m also often on #clojure and #clojurescript IRC channels if someone wants to chat.

LC: Ok. One last question: when they make a movie about programming languages, who will play Clojure?

AP: That's a tricky one! I'd say Leonardo DiCaprio since he's one of the best actors yet still without an Oscar (read: very good but underappreciated) :-)

LC: Thanks for a great, informative interview.

AP: Thanks for putting this together. It was my first interview and it was fun :)


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Julian Gamble

Introduction

The next interview is with Julian Gamble. His talk at Clojure/conj is about core.async and ClojureScript. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Julian Gamble: It was about 2007, I was a Java Programmer toying with Rails. I was reading Steve Yegge and his amazing but incendiary blog posts. These had a big theme on languages and expressiveness.

I got curious enough to play with Common Lisp and Racket. I eventually worked through The Little Lisper - and found it hugely inspiring. I also read SICP and Paradigms of Artificial Intelligence Programming and was amazed (coming from Java) at what was possible with functional programming and homoiconicity.

In 2008 I was looking for a way to combine the power of Lisp and my Java background. I had a look at ABCL and Kawa and found they didn't really have much of a community behind them.

Then the Rich Hickey videos started coming out, "Clojure for Java Programmers" etc. Then lots of Java/Lisp people started coming out of the woodwork. I was fascinated by the design choices made and how opinionated the language was. I was delighted by the community that seemed to have come from thin air.

In 2009 Stu Halloway's Programming Clojure book came out and I devoured it. Then I went back and worked through the Little Lisper chapters in Clojure - and found functional programming had a lot more to it than a first-year course for students in the 80s. This meant idiomatic functional programming didn't work like that anymore. (Although the idea of building a Lisp from a few primitives is still very elegant).

In 2011 a Clojure User group started up in Sydney run out of Thoughtworks. This was a fantastic chance to meet up with people who programmed and developed their skills just for the joy of it - rather than just to pay a mortgage.

In 2013 Dave Thomas started running LambdaJam in Australia, and it meant that the functional programming community because more social and more willing to look at each others ideas across languages. It also meant we developed a shared appreciation for the heritage of functional programming. We also developed a deeper appreciation of academic ideas and their linkage back to our practice.

So now in 2014 when Rich Hickey gives a talk on Transducers, and explains that folds are fundamental - I go and read the linked papers from Hutton and Bird. Then I can go and talk about this with my local community of Clojure people. This is something that I wouldn't have done two years ago.

Clojure has been a great ride!

LC: In your talk description, you mention that you will relate core.async to the concurrency semantics we already know. Can you explain this briefly?

JG: The assumption here is that most Clojure people have a Java or C# background. From this background they would have an understanding of how to put process on a thread, similar to how we might use a future in Clojure. Alternatively, Clojure people may have seen how we can pass a function to an agent in Clojure.

Now it is true that core.async go blocks have some very specific queue sleep/wait semantics (CSP) that make it different to other concurrency models. In addition, core.async in JavaScript runs as part of the single-thread model of JavaScript, so it isn't semantically a thread, more of a 'process'.

Given those assumptions, the primitives we find for doing concurrency in core.async are very similar to the concurrency models we have seen in the past in OO languages or in Clojure.

In this talk we'll take a look at an example of taking a well known graphical example in Clojure, port it to ClojureScript and core.async. What we find is that many of the Clojure concurrency paradigms have a natural analogue in ClojureScript and core.async.

LC: Are there any topics to read up on or resources to look at that could help a beginner make the most of your talk?

JG: First thing to look at is David Nolen's 10,000 processes and also this one.

Take a look at the source code and try and understand what it is doing.

Also take a look at Tim Baldridge's 2013 Conj talk at 32:20

Also take a look at the code here: (lines 568-612)

Watch the 1969 version of The Italian Job.

LC: Where can people follow your adventures online?

JG: They can follow me on twitter with @juliansgamble.

Or they can send me an email @ and ask me what I'm up to.

(I've also been known to present at clj-syd on meetup.com - but that is probably only of interest to Sydney readers).

LC: One last question: If Clojure could eat, what would be its favorite food?

JG: Well I'm a huge fan of Dan Friedman's work on The Little Lisper/Schemer - which has frequent food references, such as a peanut butter and jelly sandwich. The book will at random reserve an empty page and state:

THIS SPACE RESERVED FOR JELLY STAINS!

So the answer would have to be PBJ sandwich!

(But for all the British, European, Australian and New Zealand Readers - we all know it as peanut butter and strawberry jam - and some might be puzzled by the combination of these two condiments, but would still appreciate Friedman's book.)

LC: Thanks for a great interview!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Zach Tellman

Introduction

Zach Tellman is the next interview participant. He is giving a talk at Clojure/conj about composition. The background to his talk is available, if you like.

Interview

LispCast: Briefly, how did you get into Clojure?

Zach Tellman: At my first job, the main language was C#. However, I worked with Tom Faulhaber, who is a Common Lisp guy from way back, and at the time was just working on the initial implementation of clojure.pprint. I started playing around with Clojure, and started building Penumbra in mid 2009. I haven't looked back since.

LC: Composition is a very important concept in the Clojure world, yet you claim that many libraries can't compose cleanly. Why is it such a problem?

ZT: Clojure is a young language, and is fairly unique in the way it tries to couple functional idioms with less-than-functional host environments. It also lacks any real organizational principle: a namespace can encompass an enormous amount of functionality (see clojure.core), or a very small, focused piece of functionality (see most of Ring). Both of these allow us enormous flexibility, but it also means that Clojure programmers are without either precedent or obvious affordances when starting out with Clojure. It's understandable that we'd make some missteps.

If we look at the first two or three SQL libraries written for Clojure, it's notable that they all used macros. It became clear after not too long, though, that this was a really limiting approach - the only way to compose with a macro was with more macros. This doesn't mean that macros shouldn't be used, of course, just that we should always be aware of how they constrain the surrounding code. Abstractions that are incredibly useful in application-level code might be irritating and arbitrary in a library.

I think a lot of people begin to innately understand these design considerations after a few years of using Clojure, but I haven't heard anyone try to articulate them in a general form. I guess we'll see how I do at that.

LC: You mention Ring. I think it's a great example of a system with very good composability. Middleware written by different programmers can be combined without problems. Ring achieves this by being a standard protocol. So there is coordination between the programmers in that they both agree to the centralized standard. Are there other patterns besides standards that can allow for composition?

ZT: Ring is useful because it's focused. Modeling HTTP requests as a pure function is a lossy abstraction, but it works for the vast majority of requests. If it wasn't so simple, I don't think it would have had nearly as much adoption (see the n-many mostly unused async extensions to Ring that try to accurately model the entire problem space). But by using Ring, we commit ourselves to its simplified view of the world, as well as its thread-per-connection execution model. This is almost always an acceptable simplification, but it transitively limits everything downstream of it. Now we find ourselves in a situation where the entire ecosystem is predicated on this simplified view of things, and anyone who tries to do something more general has to start over from scratch.

So to actually answer your question, composition can't happen without conventions. But conventions, like macros (which really are just conventions of code structure), constrain and shape the code around them. The tradeoffs and resulting design space are what I want to explore.

LC: You've mentioned that Clojure does not constrain the code. One of the most common questions I answer from Clojure beginners is how to structure their code given the freedom allowed in Clojure. Do you think conventions that support composition could help answer that question?

ZT: I use a lot of Java-land libraries, so I spend a lot of time reading Javadocs. The nice thing about Javadocs is that even if the actual docstrings are minimal, it gives you a full, unambiguous dependency graph between all the different pieces of code. There's no analogue for that in Clojure, which makes reuse a lot harder than it otherwise would be.

I'm not sure what the solution for this is. I think standardizing on a narrower sort of code structure, like the "class" convention in Javascript, isn't that useful or likely to be popular. Standardized (and better) documentation, though, along with tooling to effectively browse it, would be enormously helpful. Formal specifications of data shapes or types, like Schema or core.typed, might be helpful but I don't think they're a necessary precondition.

LC: You mention dependency graph. Have you had a chance to look at CrossClj? Would that kind of thing be helpful? What would it need to be more helpful?

ZT: I've looked at it, it's a good place to start, but raw dependency data isn't useful without the tooling to integrate it into our development process. Light Table and others are playing around in this space, but it's a genuinely hard problem, because it's not a one-size-fits-all visualization problem. I don't have any solutions to offer, but I'm certainly paying attention to how things develop.

LC: So there are going to be a lot of beginners at the conj. Are there any resources that could prepare them to make the most of your talk? Any blog posts or videos that would get them ready to participate actively?

ZT: I'll talk about macros, transducers, and core.async, among other things. Being conversant in each will help, but isn't necessary.

LC: Where can people follow you and your adventures?

ZT: I twit about software sometimes at https://twitter.com/ztellman. I write longer stuff less often at http://ideolalia.com/.

LC: Ok, last question: What is the average airspeed velocity of an unladed Clojure REPL?

ZT: JVM or V8?

LC: I don't know that!

Thanks for the interview. It was very informative.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Zach Oakes

Introduction

I had a conversation with Zach Oakes about game development in Clojure. He is giving a talk at Clojure/conj. Read the background to his talk.

Interview

LispCast: Briefly, how did you get into Clojure?

Zach Oakes: Being hosted on the JVM was my original draw to Clojure. I wanted to hack on an anonymous networking project called I2P, which is written in Java, but I wanted to use a more enjoyable language. Clojure's interop made it effortless, and later on I came to enjoy the other aspects of Clojure as well.

LC: How long have you been into games programming?

ZO: Games were how I originally got into programming as a teenager. However, I had a ten-year period where I did nothing but crypto software. I only got back into games last year, and since Clojure was my favorite language by then, it was inevitable that I would start using it for game development.

LC: What is the main concept of play-clj? Is it based on immutable game states, or something else?

ZO: The main technical concept is that most of your game should just be list transformations, so your code mostly consists of using normal Clojure functions to transform your entities. It definitely encourages immutability, but since it builds on top of a Java framework, there are times when you will need to mutate things.

LC: What is the advantage of using list transformations instead of mutating game objects? Is this a drastic departure from normal game architectures?

ZO: The advantage of functional programming in games is the same as in other kinds of software; the code becomes simpler to understand, and parallelism becomes trivial. I think this is very unusual for game development. Many games keep entities as class data members that the methods mutate directly, which quickly becomes a mess.

That said, I think the REPL is by far the biggest reason to use Clojure for game development. A disciplined programmer might be able to avoid the aforementioned mess, but it's hard to replicate the runtime modifiability of Clojure if your language wasn't built for it, and I think it has a big effect on how you make your game.

LC: What kind of response from game programmers have you had to using functional programming in games?

ZO: So far the response to functional game development has been tepid. They are understandably conservative about their technical choices due to the performance sensitivity of games, but nonetheless they are slowly adopting functional idioms in their non-functional languages. The next step is to get them to use a functional language. I think it's up for grabs, and the languages that are hosted and provide an excellent REPL will have the advantage.

LC: Can you tell us a little about play-clj? Is it based on a Java framework?

ZO: Yes, play-clj is essentially a big wrapper around libGDX, a popular Java framework. I used it to teach Java game development, and eventually made play-clj so I could use Clojure instead. It is a fairly thick wrapper and hopefully a nice teaching tool.

The core of the library is defscreen, a macro in which you provide functions that will be run during various events like "on-show" and "on-key-down". Making a game consists primarily of transforming your entities in response to these events.

There is always a design compromise when you build on top of a large object-oriented framework, but I think it is crucial to leverage existing work to make it practical to use a functional language to build games worth shipping.

LC: Yeah, I agree. Game frameworks have a lot of work put into them. Was it a challenge combining OO/mutable state with the Clojure functional approach?

ZO: It's definitely a challenge, and since this is the first library I've ever made I have been learning as I go along. I took some inspiration from neko, a Clojure wrapper for the Android API, since it is doing something similar. I don't try to hide mutations; most of the library is just a series of macros that allow you to use the underlying Java classes with a nicer syntax.

That said, you can at least keep your core game logic free of side-effects by deferring those mutations until later. In my play-clj games, I typically first map over my entities with a series of pure functions, then afterwards I do various side-effecting things like playing sounds and rendering. This allows me to trivially parallelize the pure portion of my code while also doing the necessary side-effecting things.

LC: That's pretty cool. I've always wanted to program games, but every time I start, I get lost in a forest of spaghetti code. Have you found that a functional approach makes that better?

ZO: I think it definitely imposes discipline on your game's code. It is easy in Java to make your entities and other objects data members in a class and mutate them from methods without even explicitly passing them as arguments, which is an easy path to spaghetti code. In play-clj, entities are records so transforming them requires explicitly passing them to functions and returning new ones at the end.

I would say that play-clj also imposes an additional degree of structure as well. It has its own built-in system for managing state, including its own entity system, which is exposed to you in the defscreen macro. I think this causes play-clj games to essentially have the same general design. The downside is that it makes a lot of design decisions for you, so more advanced Clojure programmers may desire a thinner wrapper.

LC: What resources could a beginner use beforehand to make the most of your talk?

ZO: I wrote a tutorial for play-clj that people could check out if they'd like to see how it works:

https://github.com/oakes/play-clj/blob/master/TUTORIAL.md

There are also some example games they could play to see it in action:

https://github.com/oakes/play-clj-examples

LC: Where can people follow your adventures online?

ZO: I don't do much social networking, unless Reddit/HN count. I suppose the best way to see what I'm up to is follow my Github!

LC: One last question: If Clojure were a Star Trek character from the original series, what character would it be?

ZO: I suppose Clojure would be all the redshirts collectively. It sounds like a bad thing, but when they died they were always replaced, so I guess that means Clojure will live forever.

LC: Thanks for the informative interview. It was a pleasure.

*include: templates/pre-conj.html

You might also like

Pre-Conj Interview: Colin Fleming

Introduction

The next interview is with Colin Fleming. He is the developer of Cursive Clojure. His talk at Clojure/conj will be about that. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Colin Fleming: I got into Clojure bit by bit, starting about four years ago. I was working in Java on highly concurrent transaction processing systems and the promise of a language that would make that easier was very appealing. I was also playing with Scheme at the time so it seemed like a good fit since I already knew the JVM really well.

I’ve used IntelliJ for Java for years now and I wanted to use it for Clojure too, so I installed La Clojure, which was the only option at the time. But I quickly got frustrated with its limitations and started fixing it up, porting the parts I worked on to Clojure as I went. That was really my first real Clojure project, which in retrospect wasn’t a great choice - it was a tough project to get started on. But it evolved into Cursive, which I’m very happy with, so it worked out well in the end.

LC: Developing an IDE is quite a big task. And the web site indicates that you will charge for it. What has the response been? How many users do you have in the Early Access Program?

CF: Right, it’s definitely a big task. Initially Cursive started as a hobby, just scratching my own itch. But I quickly realised that if I was going to develop all the functionality I wanted to add it was at least a full-time job, probably more. I had just taken some time off after leaving my previous job at Google and I thought I would see if there was appetite in the Clojure community for a professionally-supported development environment, and the initial response on the Clojure mailing list was pretty positive.

I released the first public beta version almost a year ago now, and the response has really been fantastic. Quite a few people have asked me when they’ll be able to pay for it so they can continue to support development, which is really nice to hear - I’m really happy people like Cursive so much. It’s hard to tell exactly how many people are using it, but each version tends to get around 10,000 downloads. There’s quite a long tail so it depends how long each version is out for, but generally there are around 2,000 downloads within 24 hours of making a release so I guess that’s roughly the number of people using it daily, or at least have it installed in the copy of IntelliJ that they use daily.

LC: I use Emacs but I can't really recommend it to someone who just wants to try Clojure. Does Cursive fit that bill and if so, how?

CF: Sure, I think Cursive is definitely easier to get started with than Emacs. In general IntelliJ’s interface is more like a standard application than Emacs, that familiarity helps a lot. And I try to make sure that Cursive is easy to get started with and to minimise the yak shaving required to open a Leiningen project and a REPL. There are still some outstanding things to do there, a big one has been that Cursive didn’t ship with default keybindings assigned - that’s fixed in the next release which should be out shortly. Cursive also has paredit turned on by default for Clojure which can confuse newbies, but it’s easy to toggle. And Cursive still doesn’t have a great story for a Clojurescript REPL, which is always a big pain point for people getting started - I’m hoping to fix this soon. Light Table really did a great job on that one.

Cursive has a user guide with a lot of screenshots that should hopefully make it easier for people to get started. It’s not 100% up to date with all the latest features, but it should help people to get going. I think having one definitive source of friendly documentation really helps at the beginning. I’d actually like to have an interactive tutorial in the IDE that people could run that would show them how to use the editor and possibly the basics of Clojure too, but that’s looking like a v2 feature at this point.

LC: I've always liked IDE's for their ability to smooth over the rough parts of a language. And tooling often trumps terseness in a language with less tooling. What are some of the trouble-spots you see in Clojure, from a beginner's perspective, that haven't been addressed with tools?

CF: The classic one that a lot of people seem to struggle with is the stack traces - I’ve never found them to be too difficult, but I worked with Java for years so I know what they look like. It’s easy to forget how difficult things like that are for people new to the JVM. Leiningen has done a great job of hiding things like the class path and dependency management as well, which can be really tough.

The other thing that I’d really love to see improved is the error messages. Since Clojure macro forms are essentially parsed by hand some of the error messages are really impenetrable. For example, something that might be a common newbie error like putting an (:import) clause ahead of the docstring in an ns form will give you a "java.lang.Character cannot be cast to clojure.lang.Named” error, and will also frequently attribute it to the wrong namespace - these sorts of errors can be really discouraging for someone trying to get started. Cursive internally uses something similar to the seqex library (from Chris Houser and Jonathan Clagget’s great Illuminated Macros talk) for its parsing, I’d love to see libraries like that being used in Clojure core.

Cursive doesn’t actually mark a lot of these errors in the editor yet, but I’d like to mark as many as possible. In particular, I’ll be able to mark things like function invocations with the wrong number of args, non-gensym’ed variables inside syntax-quote and many of the problems detected by tools like Kibit and Eastwood. I hope that’ll make Clojure less frustrating for newbies and more experienced folk alike.

LC: There's a bunch of stuff I'd like to ask you about. Let's talk about the practical side of developing an IDE. You say it's a plugin for IntelliJ. How much of the IntelliJ system are you able to leverage, how much is custom, and how much is a hack to fit the Clojure peg in the Java hole?

CF: Right, it’s currently only available as a plugin to IntelliJ. JetBrains actually put a lot of work into making their base platform reusable so I’ll be able to create a standalone IDE as well, as JetBrains have done with Redmine for Ruby, or PyCharm for Python. But it’s a reasonable amount of work and probably adds some support load with cross-platform issues so it hasn’t been a focus for me yet.

The IntelliJ infrastructure is really great - it’s certainly saved me a huge amount of work being able to build on top of it. It’s very impressive how well abstracted it is, I’ve encountered very few places where I’ve had serious roadblocks from IntelliJ itself due to the nature of Clojure. Most of the real difficulties are just intrinsic to how flexible the syntax of Clojure is, they’re not problems with IntelliJ. Obviously all the base IDE functionality comes for free (project management, VCS support, indexing infrastructure, debugger and so forth) and the Java support is very easy to hook into which is important for Clojure. The biggest part that’s totally custom is the symbol resolution, which generally has to be written for each language anyway.

Cursive also supports Clojurescript, and that’s been more challenging since the Javascript support in IntelliJ is only available in the paid Ultimate Edition and is closed source. Cursive currently implements fairly primitive Javascript support itself which is just enough for Clojurescript, but it would be nice to be able to integrate that better with the existing support - I’m hoping to in the future.

The part that is very different in Cursive from a more standard IntelliJ language plugin is that it all has to be extensible since everything in Clojure is based on macros, even core language features. People will eventually have to be able to add support for the libraries they use or for the forms they define in internal projects so Cursive is internally based around an extension API that I’ll open up at some point soon. More conventional languages generally have a fixed syntax, even if it’s very flexible like Ruby for example, so this isn’t an issue for them but it’s essential for Clojure.

LC: In the talk description, you mention Rename Refactoring. It's not something I can say I miss in Emacs, but perhaps I don't know what I'm missing. Do people make use of that and other refactorings?

CF: Oh yes, I’d definitely argue that you don’t know what you’re missing out on! The two operations I use a lot are rename and find usages - I’d struggle to live without them now, especially find usages. I also recently added refactorings to extract and inline let bindings, which are really useful - people definitely use them a lot. I’m planning to add support soon for extracting and inlining functions, as well as introducing parameters which I use a lot in Java. Refactoring in Clojure will never be as accurate as in Java, but it’s still very useful. Things like marking unused local variables is also something which sounds trivial but really catches a lot of bugs while you’re editing.

I think this is the main difference between something like IntelliJ and something like Emacs, actually. When I’m working on Java code in IntelliJ the refactorings and all the small operations they provide as intentions are so good that I really feel like I’m working directly on my program, not on text. We’ve all experienced that flow state when programming when you’re totally absorbed in what you’re doing and you lose track of time, whether you’ve eaten or need to go to the bathroom or whatever. IntelliJ makes it much easier for me to achieve that state because it provides a huge number of code manipulations that directly reflect what I want to do to my program, without me then having to muck around with text. I’ve seen comments online along the lines of “IntelliJ is great but I type so much faster in Emacs/vi/whatever”. I’d argue that approach is totally wrong - I don’t want to avoid typing because I’m lazy, I want to avoid typing because text manipulation is the wrong level of abstraction. Cursive is a long way from providing that experience, partly because of Clojure’s language characteristics and partly because it took teams of dozens of smart people at JetBrains years to achieve it for Java, but that’s always my ultimate aim whether or not I ever actually get there.

LC: What resources would you recommend people make use of before your talk in order to get the most out of it? Anything useful about refactorings or IntelliJ?

CF: I don’t think so, no - hopefully it’ll be pretty self-contained. I’ll be showing the features I’m talking about and explaining some of the implementation, so hopefully it’ll be understandable to everyone. And hopefully it’ll be interesting for everyone too!

LC: Where can people follow your adventures online?

CF: There’s a twitter feed for Cursive and there are also some mailing lists for general Cursive discussion. I’ve also been hanging out more in the #clojure channel on IRC as cfleming in case anyone has questions. I’ve been meaning to blog more about Cursive but so far that’s just one to-do item among many!

LC: Nice. One more question: what is Clojure's favorite breakfast?

CF: Cursive is most frequently powered by Tropical Mango and Nut Crunch, but the true breakfast of the gods is scrambled eggs with little chunks of Venison Pepperoni salami cooked through it.

LC: Cool.

Thanks for the great interview. It was fun.

CF: Thanks Eric, that was fun! I’m looking forward to seeing it, and to catching up at the conj too!


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Ashton Kemerling

Introduction

I asked Ashton Kemerling some questions about his talk at Clojure/conj about using generative testing in Clojure. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Ashton Kemerling: The path I took to Clojure is fairly roundabout. During my College years I was very interested in function programming, although ML & Haskell were my favorites back then. I think a large amount of my fascination with them stemmed from the fact that I had more experience than my classmates, my high school had a very rich set of programming courses, and I needed something interesting to play with while taking introduction to OOP courses.

Fast forward a few years and I ended up working at a job using Common Lisp professionally in a legacy web application. The application had a lot of problems due to the condition of CL libraries at the time and we were looking for something to switch to. Clojure was an obvious choice to look at. It's still a lisp, so a lot of our habits would still work, but it fixes the deployment & library issue that CL has while adding a much richer set of data structures. Unfortunately our application relied extensively on mutation (CL doesn't really encourage or discourage any coding style) and OOP, so a conversion was deemed too costly to attempt.

Over the course of the following years I continued to tinker with Clojure on the side. I was convinced it was a great way to do things, but I could never convince anyone to consider using it in their code base. Most places were either too invested emotionally in OOP, or had well tested and working codebases that made a rewrite unwise. So I mostly just hacked on side projects using Clojure over the intervening years without using it professionally. But in the last year or so I realized that Clojure makes a great secondary language for odd-tasks in non-Clojure codebases, which brings me to the current state of using Clojure to test a very large Ruby on Rails and Javascript application.

LC: Can you give a basic outline of how your Clojure program tested the web front end?

AK: We've basically tested in two ways. The first was to simply compile our JS file along with Clojurescript tests and run them in a PhantomJS process similarly to how you'd run Clojurescript or Javascript tests normally. We're still working on this approach, but it's slower to work on that I would have hoped.

The second way was to leverage the fact that Clojure's on the JVM. This involves mixing JDBC, HTTP libraries, and Selenium in novel and exciting ways. This has been the most fruitful way of testing, and what I'll be focusing the most on during my talk. We've used the code I've written as both a tool to hunt down API mistakes, and as a means of narrowing down the reproduction steps on complicated bugs reported by the user.

LC: Can you explain a little more about that? What do you mean by "narrowing down reproduction steps"?

AK: Sure. I'm testing Pivotal Tracker, a very large (~24kloc) Javascript "single page app". With large apps like this there are ways for the application to subtly fall out of sync with the server or process data in a degenerate way.

In particular a lot of times you'll get a report from a user that says something along the lines of "I reloaded and it crashed". You can dig up the logs from that session, but you'll get the last 20 or 30 steps they did before the crash, which doesn't really help you out a whole lot. So we occasionally turn the generative tests towards a type of action (in this case, do things and then refresh) in the hopes that it will help us narrow down to the minimal reproduction steps required to trigger a problem.

In the case that happened last week, we were able to find 2 distinct actions that would provoke the issue reliably after about 20 minutes of test modification and running. Obviously this took hours off of the process required to find the actual root of the bug, since 2 is orders of magnitude easier to work with than 7 or 10 when debugging.

LC: I see. So how is test.check able to reduce the reproduction steps?

AK: Test.Check provides "shrinking". Just as the generators used to create randomized data have the ability to produce more complicated data, they also have the ability to produce less complicated data. Test.Check records all of the failures it finds and attempts to simplify them and find the smallest failing case it can.

LC: A lot of the Clojure/conj participants will be new to Clojure. What resources would you recommend them to make the most of your talk?

AK: They can't go wrong with the test.check README or any of the blog posts mentioned therein. I recommend reading the source directly because it's heavily commented, in particular I recommend the generator source, because the generators represent 90%+ of the test.check API a user will interact with.

Beyond that I have blog posts on my work blog and on my personal blog. Also Reid Draper and I were both on the Cognicast, both of which are based around this subject.

LC: Are there any resources on Selenium and other methods for running frontend tests?

AK: The Clj-Webdriver docs are all I can recommend.

LC: Where can people reach you?

AK: Twitter: @ashton

My blog: ashtonkemerling.com

LC: If Clojure were stranded on a desert island, what one book would it bring?

AK: A boat building book, clearly.

LC: Awesome! Thanks for the interview, it was a pleasure.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-Conj Interview: Steve Miner

Introduction

I had a conversation about generating test.check generators with Steve Miner. He's giving a talk at Clojure/conj about that. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Steve Miner: I think it was around 2008 when Paul Graham announced his new programming language, Arc. I was reading about Arc, when I came across a comment about Clojure and decided to take a look. I was very impressed with Rich Hickey's intro video. The immutability and concurrency features really resonated with what I was hoping to find in a new language. The Java integration made Clojure a practical language with a huge eco-system of tools and libraries. I was using Java at work and I managed to do a bit of Clojure for a side project, but mostly I was just dabbling. A couple of years later, I decided to work on my own with Clojure full-time.

LC: Can you describe Herbert for those who have never used it? Why would someone be interested?

SM: Herbert is a schema language for edn. The goal is to have a convenient language for describing the shape of Clojure data. I started out with an informal notation that I used for my internal documentation. For example, describing a map with certain required keys and the corresponding value types. I think you can guess what {:name str :num int} means as a schema. It turned out that with a little work, that informal notation could be used as a pattern language with a simple API for testing conformance. A Herbert schema is itself just edn data, open to all your Clojure tools. More recently, I added the ability to generate test.check generators from Herbert schemas, which makes it easy to generate test data.

LC: That sounds nice. Would you mind explaining test.check generators a bit for those who don't know?

SM: Reid Draper ported QuickCheck from Haskell to Clojure, and it became a contrib library called test.check.

Test.check is a property-based testing tool. A property is basically an invariant that should hold true over a range of input values. The test.check library gives you combinators that allow you to define generator functions which create data of specified types with optional constraints. The idea is to think about the whole range of possible inputs that your system should handle. Test.check then automatically tests across a random sample of generated data, probably generating example data that you might not have considered in your typical unit testing. If it finds a failure case where the desired property does not hold, test.check is smart enough to regenerate test cases so as to shrink the failure example to a reasonable size. That helps you isolate the cause of the failure.

Clojure/West had two excellent talks about property-based testing last year. Reid Draper covered test.check, and John Hughes talked about QuickCheck.

LC: So you're able to automatically create generators from your schemas, which can also be used as contracts on your function arguments. Has having both improved your bug rate?

SM: I don't have any numbers, but subjectively I think it's helped. For me, Herbert schemas are primarily documentation tools, which help me to keep track of my data. That being said, I often test schema conformance in preconditions or asserts, especially with new code or when I'm trying to debug a problem.

Of course, I still make errors in specifying schemas and sometimes my properties aren't exactly correct the first time. Particularly with new property-based tests, I have to look carefully at failures in case the bug is actually in the test. My hope is that schema-based generators will make property-based testing easier to use.

Using test.check definitely improves my confidence that I'm finding bugs in testing and avoiding regression errors. It's been a great way to catch bugs in my own Herbert library.

LC: So you have runtime check to make sure the function arguments conform to certain schemas. And you have a generative test that exercises a large space of that schema. Sounds pretty good to me!

But it sounds like you're saying the primary benefit is more for you or other readers of your code. Can you elaborate on that?

SM: My approach started with a notation designed to help me keep Clojure data structures straight in my mind. I wanted something simple and terse, what I called a "whiteboard compatible" notation. My goal was that the notation should look something like the data it was supposed to represent as opposed to code or a type system.

So I began with documentation in mind, and I still think of that as the primary benefit. Once I got the idea of implementing conformance testing against formal schemas, the project became more about the code.

LC: In what other areas do you see schemas playing a part? The first thing that comes to mind is writing core.typed type annotations. Anything else?

SM: There's some conceptual overlap between schemas and type systems, but I see core.typed as a much more ambitious project. Herbert schemas only cover edn data and don't deal with function types, for example. The Datomic database naturally has a schema language, so it would be interesting to see if Herbert could useful for data modeling. In the near term, I plan to extend Herbert so that it supports the Transit datatypes.

LC: What resources would you recommend to a beginner who wanted to make the most of your talk?

SM:

LC: Where can people follow your adventures online?

SM:

LC: One last question: If Clojure were a food, what would it be?

SM: I'll say "pizza" because it's a food that is made by composing fancy toppings on a classic crust. And hackers like it while coding.

LC: Thanks, Steve, for a great interview.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like