Clojure is Imperative

Summary: Clojure is imperative as opposed to declarative. Its operations are clearly defined in terms of concrete actions, even when those operations are abstract. This is in contrast to many languages which are more declarative. However, Clojure being imperative is related to how easily it can be bootstrapped.

I was recently on the Cognicast and I mentioned something really important to me, but I did not go that deep into it.

Clojure, and Lisps in general, are imperative languages (as opposed to declarative). Yes, they are good for doing functional programming, but their main paradigm is executing lists of commands in order.

On the podcast I mentioned the first imperative example that came to mind, which was the do form, which executes each expression in the body and returns the value of the last expression. You would only want to execute an expression and throw away its value for its side effects.

Another deeper example is the way Clojure deals with top-level forms. By top-level forms, I mean all the defs and defns. These are executed in order. On the surface, you may think "I'm defining a function called foo". But defn is really doing a list of things, in order: it's checking if the var foo exists already in the namespace. If it doesn't, then it creates it, modifying the namespace. Then it sets the root binding of foo to the function.

This might seem obvious, but look at languages like C or Haskell. In C, top-level forms are declarative. You are not telling the compiler to perform certain concrete steps. Instead, the compiler reads that you want a function called foo. It's the compiler's job to know how to build that into the program and export it out of the module. Haskell is even more declarative. Order of definitions does not matter.

Clojure, in fact, projects a very thin illusion of declarative expressions. Within certain limitations (like that the order of your defns matters) you can treat defns as declarative. They're declarative if you squint.

Haskell is super-declarative, but that makes it somewhat magical. The Glasgow Haskell compiler, according to this talk, was over half-a-million lines of code in 2011. I'll just let that sink in for a bit. If anything can be called magic, it's so much code that you'll never understand it.

Here's the code for defn. It does a few things. It handles metadata and some other details. But it's readable. In the end, it converts a (defn foo ...) into a (def foo (fn ...)).

def is one of the few special forms, so it is implemented in the compiler. But there are so few special forms! Implementing all of those could not possibly run to 500,000 lines of code. Everything else can be written in the language itself. That's leverage. Write the core in machine code, and the rest in Lisp.

And this gets to the heart of it: you can write a Lisp yourself. Many people have. You could write a declarative language compiler and keep adding to the compiler over years and years. Or you can write an easy Lisp compiler in a weekend and build features on top of it, almost never having to change the original compiler.

This is the magic of bootstrapped languages like Lisps. They have a small core that you need to get right, then everything else can be written in that core. It's the ultimate minimal virtual machine.

What's the relationship between bootstrapping and imperative? A truly declarative language must push complexity into the compiler. The more declarative a language, the more the compiler must do to maintain the illusion. So a bootstrapped, declarative language is right out. But by letting the imperative underpinnings shine through the small core, you are continually able to work at the fine grain necessary to continue bootstrapping.

I like Lisps (and Clojure) because I feel that I can understand them. I don't actually understand everything, but I could if I tried. Declarative languages can be very powerful (and I like many of them). But somewhere along the way I developed a deep interest in bootstrapping. Bootstrapping is compounded leverage. You build small abstractions on top of the previous ones, and use those to build yet grander ones.

If you like this attitude toward programming languages, you should learn a Lisp. I suggest Clojure, and I recommend the LispCast Introduction to Clojure video series. You'll learn about building up powerful abstractions, one layer at a time, in a small amount of code.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

You might also like

Complex Syntax

Summary: Lisps are revered for their simple syntax, but parens are complex. They complect function calls and macro calls, which have drastically different semantics.

One of the problems that people have with Lisps is that they hate the parentheses. Clojure does a pretty good job of minimizing unnecessary parens and giving them a much clearer meaning. But there's a deeper problem that people express all the time when they're first learning. It's frustrating to watch people struggle with it, because it's not their fault. It's a problem with Lisps in general.

Parens in all Lisps I've seen, including Clojure, are complex. I'm not using the word lightly. Parens complect two similar but distinct ideas: macro application and function application.

Macros and functions are obviously different. Macros are expanded at a time just before compilation called "macro-expansion time". They typically cannot be accessed at runtime. Functions, on the other hand, are applied at runtime. And they are first-class, meaning they are runtime values. In addition, the calling semantics are different. Macros are call-by-name. The code of each gets passed unevaluated. Functions are call-by-value. Functions and macros are two distinct species.

However, despite being distinct semantics, the syntax for calling the two is identical. Parens complect applying macros with applying functions. Beginners trip up on this all the time. Their head is already spinning from the notion that some of the things they are learning are macros, called at compile time. Now add on top that the syntax of the language does not help one bit in distinguishing macro calls from function calls. You just have to memorize what's a macro and what's a function.

We learned in The Next 700 Programming Languages that our syntax should serve to elucidate the semantics. Lisp just fails at this pretty hard. The only consolation is that you actually can remember, with time and experience, what's a macro and what's a function. Every Lisp programmer is proof of that.

A simple solution would be to have a weird syntax for calling macros. You know, instead of parens, you use something else. Something that distinguishes the two to decomplect them. This would have broad and deep implications for the language that I cannot begin to fathom.

The takeaway for the beginner is that, sorry, Clojure won't help you much with this, but it's very important to know what's a macro and what's a function. You just have to keep track in your head. If you're not sure, you can call clojure.repl/doc1 on any symbol. If it names a macro, it will tell you.

So, there you have it. Lisps complect function calls and macro calls, which have drastically different semantics, using the same notation. Common Lisp and Scheme use parens for much more than that, making the syntax complex and context-dependent2. Clojure removes a lot of those parens, replacing them with square braces or removing them altogether. However, the complexity of macro and function calls remains.

Despite this, Clojure is still a great language! If you'd like to learn Clojure, I have to recommend the LispCast Introduction to Clojure video series.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

You might also like


  1. That's a macro.

  2. For instance, inside of a let, parens take on the meaning of grouping the bindings and also grouping the variable with its value.

Is core.async Against the Clojure Philosophy?

Summary: Clojure core.async is a way to manage mutable state. Isn't that against functional programming?

When core.async was first announced, there was a lot of fanfare. But among the celebration, there was some consternation about core.async. Isn't core.async against the functional principles of Clojure? Aren't channels just mutable state? Aren't the <! and >! operations mutation?

Well, it's true. core.async is about mutation. It's procedural code. Go blocks run their bodies one step at a time. It's imperative.

But that's what Clojure is all about. It makes functional programming easy (with fns, immutable data structures, and higher order functions). It also makes mutable state easy to reason about. It does not eliminate it. It simply gives you better abstractions. That's what Atoms, Refs, Vars, and Agents are: useful abstractions for dealing with state.

core.async is just another abstraction for dealing with state. But, following the Clojure philosophy, it was chosen to be easy to reason about. The hardest part about coordinating and communicating with independent threads normally is that neither of them know what the other is doing. You can make little signals using shared memory. But those signals get complicated fast once you scale past two threads.

And that's what a channel is: it's just a shared coordination point. But it has some cool properties that make it super easy to reason about:

  1. Carry semantics: the channel carries its own coordination semantics (buffering, unbuffered, etc).
  2. Simple interface: channels have put, take, and close. That's it.
  3. Very scalable: any number of processes can use a single channel with no additional cost.
  4. Decoupling: consumers don't need to know producers and vice versa.

Channels are awesome, but they're not the whole story. The other part of core.async is the go block. Go blocks are another abstraction. They allow you to write code in an imperative style that blocks on channels. You get to use loops and conditionals, as well as local let variables, global variables, and function calls -- everything you're already using, but augmented with the coordination power of channels.

All of these features add up to something you can reason about locally. That's the key: the code you're looking at now can be understood without looking at other code.

But there's a downside: you now have more choices. In theory, they're easier choices. But that requires you to understand the choices. You need to understand the abstractions, the idioms, and the tradeoffs. That's the goal of the LispCast Clojure core.async video course. If you'd like to use core.async but you don't know where to start, this is a good place.

You might also like

Reification

Summary: Reification means making an abstraction into a concrete value that can be manipulated at runtime. Reification is the core of what makes a language dynamic. Three types of reification in Clojure are discussed.

What made Object Oriented programming (in the Smalltalk sense) so powerful? So powerful, in fact, that the GUI, WYSIWYG editing, overlapping windows, MVC, and more, were invented using it, not to mention an entire programming paradigm.

What made Lisp so powerful? So powerful that we still see new Lisps popping up and its legend looms over every serious programmer's mind.

Now, I don't want to boil it down to one thing. But one thing that was important, that you see in both of these languages, was the idea of reification. In fact, reification is possibly the essence of what makes a language "dynamic".

Reification, as I define it here in this article, means to make an abstraction available at runtime. Many languages have a text-based syntax that is read in by a compiler and compiled to machine code. But Smalltalk made each line of code into an object that you could manipulate, either with the input devices or with other code. And it let programmers bootstrap an IDE that was unparalleled at the time.

But Smalltalk's coup de grace reification was the class, which is an object which represents the behavior of another object. Instead of some static switch statement that dispatched the methods, the methods were stored in a data structure that could be inspected and added to at runtime. Dynamic dispatch! You see this in multimethods in Clojure.

Clojure's namespaces are reified. Some languages have a linking step in the compiler where modules are brought together to form a binary. But in Clojure, the namespace is accessible at runtime. Does your code want to know what Vars are defined? Easy. How about add a new Var. Done. These serve to support interactive programming--a hallmark of dynamic languages.

Lisps have always, from the very beginning, supported homoiconicity, which is a silly way of saying that programs are reified into the language as data. This means you can write functions that write code--also known as macros. Macros serve a very useful bootstrapping function because you can gradually add to the language instead of having to design it up front. And sometimes you get a huge win, like core.async, which adds a totally new semantic.

The next level of reification in Lisps is the higher-order function. Functions are not just things to call, but things to pass as arguments, save in collections, etc. They are real values, just like numbers and strings. There was a time in the history of programming when you could not refer to a function except to call it. Now, we take it for granted. Being able to reify an abstraction into a thing to pass around is amazing, and we should all just take a moment to ponder just how awesome it is.

Lisps have traditionally gone to the next level, which is to reify a problem into a data-driven solution. Nowadays, people call this type of programming "DSL". But it's just a type of reification. Instead of writing code to solve the problem, let's encode the solution as data. The problem domain is encoded in the interpreter for that data. Now it's accessible at runtime in a way simple code never could be.

Prismatic's Schema is a great example of this. You define a validator for a piece of data using existing data types: maps, vectors, strings, classes, regexes, etc. Then the library can interpret that data structure and tell you if a piece of data is described by that validator. If schemas were merely a static construct, this would not be possible. You would have to wait for the language to "support" it, which is a terrible form of tyranny.

Here's the secret to compounding the power: that data structure can be interpreted in many ways. Take Prismatic's Schema again. You can generate schemas at runtime. You can print them out. You can use them to build test.check generators. When things are reified and use the same interface as everything else, you can see synergy between libraries. You use one reification to enhance another.

Data-driven solutions are superior to macro-driven ones or even higher-order function solutions. Data can be stored. It can go over the wire. It can be meaningful in different contexts. A macro is useful at compile time, which happens once. Functions are black boxes and can really only do one thing (apply to arguments). But data is just data, ready to be interpreted.

Many Clojure libraries are considered "language features" in other languages. You don't have to mess with the internals of the language. Dynamic languages can do this, but Clojure (and most Lisps) has it at enough levels that interesting things happen.

Here's another reification: Haskell reifies side effects into values that can be composed. That's cool.

If you're into this whole reification thing, where language features that require major releases in other languages are just libraries; if you think you should learn functional programming; if you are curious about what everyone is talking about, check out the LispCast Introduction to Clojure video series.

The series is 1.5 hours taking you from zero Clojure knowledge through data-driven programming, one of the coolest types of reification. You'll help a robot who always wanted to be a baker learn to make bread :) Help him learn the recipes and convert them from static code to dynamic data.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

core.async in Browsers

Summary: Javascript's concurrency model forces code to give up control of when a callback will be called. core.async gives you back that control and hence lets you code in a more natural style.

Well, there comes a time in every programmer's life when they take a look at the ThoughtWorks Technology Radar and they realize that core.async is in the Trial circle, meaning you should see if you might want to use it.

And if you're there, right there in that phase of your programming trajectory, eyeballing core.async for your next (or current) project, Welcome. This post is for you. Here it goes.

Why core.async? Well, the short answer is that it makes concurrency much, much, much, very much easier. I mean, let's face it: concurrency is so hard by itself, it has plenty of muches to spare. Now, I haven't used core.async a lot on the JVM. I wrote some, but it wasn't really the right thing for it. I plan on writing more later, I just haven't had the right project for it.

But I have used it a lot in ClojureScript in browsers. 1 And it is nice. It lets you do things that you could write yourself, given enough time. But you're more likely to solve the 16-ring Tower of Hanoi before you get all the kinks out. It's much better to let a machine do the hard work. That's what the 20th Century was all about: machines instead of muscle. And the 21st Century will be about computers instead of brains. Best get ahead of the curve.

I say you should let the machine do the work, but maybe that's too vague. Let's look at a concrete example. First, how do you do an ajax request then do something with the value? Easy:

(ajax "http://example.com/json-api"
      #(js/console.log %))

2

We're in Javascript, so we have to pass a callback which will get the result. That was easy. A little harder is making two API calls and doing something with both results.

(ajax "http://example.com/random-number"
      (fn [r1]
        (ajax "http://example.com/non-random-number"
              (fn [r2]
                (js/console.log (/ (:n r1) (:n r2)))))))

Alright, that wasn't too bad. A little indentation never hurt anyone. But, wait a second! We don't do the second request until the first request is already done. I've got a browser the size of a minivan and a 20 Megabit internet connection, and I'm doing one request at a time? That sucks!

We could start them both at the same time. But what order will they come back in? Welcome to the world of concurrency!!!! Things happening (maybe) at the same time, or at least you don't know what order they will happen in!

Well, let's try something. What if the first one to finish wrote its result down, then the second one to finish would know that it was second and it could do the final calculation? What would that look like?

(def r1 (atom nil))
(def r2 (atom nil))

(defn final-calculation []
  (js/console.log (/ @r1 @r2)))

(defn try-final-calculation []
  (when (and @r1 @r2)
    (final-calculation)))

(ajax "http://example.com/random-number"
  #(do
    (reset! r1 %)
    (try-final-calculation)))

(ajax "http://example.com/non-random-number"
  #(do
    (reset! r2 %)
    (try-final-calculation)))

Ok, well, that should work. What happens if you have to do 3 AJAX requests? Not so bad, either. What about 17? Oh, man, that sucks. We could do something like make a super-promise, where you can promise many values and only call a function at the end when they're all there. Yes, you can do that. It really wouldn't be hard, even.

(defn super-promise
  "Create a promise for many values. Use `deliver`
  to add values.

  keys: all of these keys must be present before calling f
  f: the function to call. Will be passed a map."
  [keys f]
  (let [r (atom {})]
    (add-watch r :promise
               (fn [_ _ _ s]
                 (when (every? #(contains? s %) keys)
                   (f s))))
    r))

(defn deliver [promise key value]
  (swap! promise assoc key value))

(def rs (super-promise [:r1 :r2]
                       (fn [{:keys [r1 r2]}]
                         (js/console.log (/ (:n r1) (:n r2))))))

(ajax "http://example.com/random-number"
  #(deliver rs :r1 %))

(ajax "http://example.com/non-random-number"
  #(deliver rs :r2 %))

Fhew! That's done. It works. It scales to many simultaneous AJAX calls. It's generic. Well, generic for this particular pattern. If we have a different pattern, we'd have to come up with a different solution.

We're looking through a small porthole into callback hell. The identifying characteristic of callback hell is that you give over control from your code, which was all nice and procedural and easy to follow, you give the control over to whatever demon is going to call that callback. You sell your virtual soul for a bit of asynchrony. But you can't cheat the Devil. When all is said and done, all of your work gets done but you need some savior angel to help you coordinate all of the pieces back together again. In this case, it's the super-promise, which works in the first circle of hell, but even Dante can't help you if you go further.3

Now that we've got a decent solution to this particular problem established pre-core.async, let's look at what it would be using core.async. We'll assume that our ajax-channel function returns a core.async channel.

(let [r1-channel (ajax-channel "http://example.com/random-number")
      r2-channel (ajax-channel "http://example.com/non-random-number")]
  (go
    (js/console.log (/ (:n (<! r1-channel)) (:n (<! r2-channel))))))

Let me just get it out of the way and never mention it again: it's shorter. It's shorter even than the naive solution using two atoms. And it's shorter than the super-promise solution even if you don't include the super-promise code. I'm done talking about the size, because it's only a little important.

Now that that's out there, on to the more significant stuff. First and foremost is that you never lose control. The code even reads procedurally. Start two ajax requests and remember the channels. Start a go block (which means run the code asynchronously) and log the result of dividing the first result by the second result.

Does it scale? You betcha! Imagine we need to make 192 imaginary AJAX calls before the Devil takes his due. The only way to do that is to do them all as fast as the browser fairies let you.

(let [numbers (range 192)
      urls (map #(str "http://example.com/choir?angelid=" %) numbers)
      channels (map ajax-channel urls)]
  (go
    (doseq [c channels]
      (js/console.log "Got: " (<! c)))))

The AJAX requests come back as fast as they can (meaning arbitrary order), and the results are logged in their original (numeric) order. You could do them in any order you want. That's because you're not giving up control.

How does this work? How can you have asynchrony and not give up control?

I alluded to it before: you're making the machine do the work. That go block up there is actually a powerful macro that transforms your procedural code into a mess of callbacks (like in our super-promise example) that you would never want to write yourself. I mean, maybe you want to, but maybe you're nuts. And you'll get it wrong.

The transformation in the go block is pretty easy, as things go. It's mechanical. It's easy like lifting a car with your hands. Put enough leverage (by using a jack) and you can do it. It converts an easy motion (pushing down on the lever or turning the screw) into a powerful force. The go macro converts your easy code into a bunch of callbacks and coordinates them with a powerful state machine which will angelically reassemble them without ever losing control.

It's all good-ol' callbacks and mutable state underground. But above ground, you've got code that's easy to reason about. No Devil's bargain. You've got an angel negotiating for you. That's the key thing! Channels are amazingly easy to reason about because each channel is so simple. But that's a story for another day!

I should just mention that, yes, core.async is about procedural programming. Channels are mutable state. core.async is made for the small part of your code that is procedural and side-effecting. Every program has got such a part. If you're doing concurrent things (and in Javascript, you always are), core.async might be able to help provide a first-class mechanism for communication and coordination.

That's what you might call the "core" of core.async in ClojureScript. It's about regaining control of your asynchronous calls and not smearing your logic across your code in little bits contained in callbacks. You keep your code's semantic integrity and you keep your sanity.

If staying out of callback hell is to your liking, you just might like the divine help of a LispCast video course dedicated to teaching core.async in a gentle, graceful way. Presented in a unique visual format and at just the right pace, LispCast Clojure core.async will guide you to a deep understanding of the fundamentals of core.async so you can clean up your code, get more concurrency, and get back control.

You might also like


  1. Don't say "the browser" because there are many and they are different.

  2. Let's imagine these functions exist and work as expected.

  3. And thank Clojure for the atom, which is like a cross or holy water when you find yourself down there.

Stop Refactoring and Start Factoring

Summary: Refactoring is focused on the quality of code, while factoring aims to uncover the underlying beauty of the problem domain, as expressed in code. Instead of cleaning up your code, try factoring.

You have some code. You notice that it's not too readable. Maybe it's a little messy. There are some obvious code smells: some repeated code and large functions.

You start refactoring. After a while, it's a clean, neat bit of code. It's very understandable and will be cheaper to modify next time.

But is it correct?

I don't mean in the "all-the-tests-pass" kind of way, because refactoring takes care of that. I mean: does the code do what it should? Refactoring only says that it does not modify the outward behavior of the code, not make it more correct. And although it's clear what the code does (thanks to all that cleanup), it's not clear that the code does what it should.

I am a big fan of the book Refactoring by Martin Fowler. It's an edifice of analytical thinking and presentation. Go read it now. It will make you a better thinker and programmer. However, I have a slight, semantic beef with refactoring. Here's Fowler's definition from the book:

noun: a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior

verb: to restructure software by applying a series of refactorings without changing its observable behavior

That's a great definition of refactoring. My beef is not with the definition. My beef is with its purpose, which is to "make it easier to understand and cheaper to modify". Again, it's a great thing to make your code easier to understand and cheaper to modify. But that's not what I'm after, most of the time.

What I'm after is code that models the problem. This is the only reliable way to make software that works. Code that inadequately models the problem is littered with nested conditionals for special cases, is unnecessarily bound in time and context, and is generally obtuse. You might be able to understand what the code is doing, but it's unclear whether it should be doing it.

The only known way to write code that models the problem is to factor. Let's get a definition:

verb: to decompose code to reveal the structure of the problem

Factoring is inherently about decomposition. It means splitting functions into smaller functions (along the structural lines of the problem). It means finding those functions which are fundamental to the problem (you can tell they are fundamental because they are used in multiple places). It means revealing symmetries. It means separating concerns. Factoring is about uncovering structural beauty in problem domains.1 Symmetry, proportion, and harmony.

The problem with factoring is that it takes a long time. And you actually have to understand the domain. You have to explore the problem a lot longer, perhaps trying different variations in the code, before you can be satisfied that the code models the problem. Time is not something we have in our "Just ship it!" modern world.

The feeling of refactoring is like bringing order to a room: you put things away, you label things clearly, you might even throw out some old junk. But the feeling of factoring is like rebuilding a room for a specific purpose. Refactoring is cleaning up the kitchen. Factoring is taking the kitchen apart and building a new kitchen better suited to the styles of the individual chef. It's not practical to rebuild your kitchen all the time, though it is practical to tidy up. But when you do it, it makes all the difference.

That metaphor gets at the other fundamental difference between factoring and refactoring: refactoring does not change the behavior of the code, while factoring might. It might because the code might turn out to be incorrect for the problem. Refactoring can reveal bugs. But if you're going to fix the bug, you've stopped refactoring and gone to something else. In factoring, changing the behavior is just part of the process. From the factoring perspective, you're not fixing a bug. You're correcting the expression of your problem.

Refactoring by design and definition is focused on the code itself. Factoring is more of a process. It's a journey the programmer takes into the heart of the problem. In its wake, the hills and valleys of the problem are mapped out in the code. And the programmer ends, like in most journeys, a different person.

For more inspiration, history, interviews, and trends of interest to Clojure programmers, get the free Clojure Gazette.

Learn More

Clojure pulls in ideas from many different languages and paradigms, and also from the broader world, including music and philosophy. The Clojure Gazette shares that vision and weaves a rich tapestry of ideas from the daily flow of library releases to the deep historical roots of computer science.


  1. I suggest you choose a good notation.

Ring 1.3

Summary: Ring is great because it closely models the HTTP message format using native Clojure data structures. It strictly defines a message format that any software can use and rely on. With Ring 1.3, the specification has gotten even closer to the HTTP spec.

A couple of months ago, Ring 1.3 was released without much fanfare. It included a few improvements and updates, but in general, not much had changed.

One change, though, is very significant: the specification is shorter. It's simpler. Three keys were deprecated in the Ring request map (:content-type, :content-length, and :character-encoding). These keys were unnecessary because their values were in the headers, which are also in the Ring request. Equivalent utility functions have been added for pulling the data out of the headers.

Why is this important? While many libraries get more complex and overburdened, it is refreshing to see a library going in the correct direction of shedding complexity. It does not significantly impact application development. Nor does it reduce the already low barrier to entry. Still, I welcome this kind of change.

Ring is the central specification that ties most of the Clojure web ecosystem together. The spec should be minimal. And a mark of good software is that it models the problem very closely without unnecessary abstraction. Ring merely defines a common format (using Clojure data structures) that mirrors the text-based HTTP message format. That's why Ring has worked so well thus far and why it is appreciated in Clojure.

Because I was so happy about the change, I decided to update my Ring Spec to Hang on the Wall PDF. The newly deprecated keys are gone. It used to be two pages long. The Ring Request took up an entire page, and the Response took up about half of one. But now, with three keys removed and a little tweaking of the font sizes, everything fits on one page.

One page in big, readable fonts, with just the information you need for quick reference. I like it. I'm printing one out right now to tack on the wall. You can get a free copy for yourself by getting on the PurelyFunctional.tv mailing list here.

You might also like

Church vs Curry Types

Summary: Static vs dynamic typing debates often flounder because the debators see from two different perspectives without knowing it. Learning to identify the two perspectives can calm the discussion. The tension between the two perspectives has led to Gradual Typing and other technologies.

Many discussions about type systems around the internet fail to be interesting because one or both parties are not versed in type theory. There's a less common (yet related) reason, which I have begun to notice more and more: people who are not familiar with the difference between Church Types and Curry Types. These are also known, respectively, as Intrinsic Types and Extrinsic types. Because the participants are not aware of the two perspectives, they blame the other one for ignorance, when in fact they just have a different perspective.

Church Types are what Haskell has. Church types are named for Alonzo Church, the inventor of the lambda calculus. In a Church-style system, types are an intrinsic part of the semantics of the language. The language would be different without the types--it may even be meaningless. With an intrinsic type system, the meaning of the program is different from the runtime behavior of the program. One way to think of this is that so much of the meaning of the program occurs at compile time that you can begin to think of the program having properties you can reason about even if you never run the program.

The other kind of types are Curry types (aka extrinsic types). They are named for Haskell Curry, the man the Haskell language is named after. Curry-style types is when a system of types is applied that is not part of the semantics of the language. This is what Clojure has in core.typed. The meaning of a Clojure program is not dependent on it passing the type checker--it can be run without it. The type checker simply alerts you to type errors in your code. Note that you could consider your type checker to be your own head, which, as flawed as it may be, is what most Clojure programmers use. The types could be anywhere outside of the language.

Each perspective is valuable and bears its own fruit. Intrinsic types are great because you are guaranteed to have a mathematically-sound1 safety net at all times. You always have something you can reason about. Such is not guaranteed for extrinsic type checkers. They may not be able to reason about your code at all.

Extrinsic types are useful because you can apply multiple type systems to your code2--or even write something that you don't know how to prove is sound. There are more benefits on both sides, but you get the idea.

We now have a new perspective which is slightly "higher" than either of them. We can now see that both perspectives exist and talk about them as such. What can we see/say now that we couldn't before?

A famous article by Robert Harper exemplifies the Church perspective very well. It argues that untyped programs are a subset of typed programs. They are programs that have a single type and all values are of that one type. So instead of being liberating, dynamic languages restrict you to one type. Notice the assumption that languages have a type system by default which is typical of the Church-style perspective. We can now say "This reasoning is correct given that perspective."

On the other side, you'll often see a dynamic typist say the exact opposite: that well-typed programs are a subset of dynamically typed programs. In other words, well-typed programs are just dynamic programs with fewer errors. Curry-style to the core: static type errors are something that is added onto the semantics of the language. We can now see that they are right, from their perspective.

Here's a diagram:

Church vs Curry Types Language Subset Diagrams

Church vs Curry Types Language Subset Diagrams

Notice how they're isomorphic? That means something, I just don't know what :)

My ambitious hope is that this perspective will quiet a lot of the fighting as people recognize that they are just perpetuating a rift in the field of mathematics that happened a long time ago. The perspectives are irreconcilable now, but that could change. A paper called Church and Curry: Combining Intrinsic and Extrinsic Typing builds a language with both kinds of types. And Gradual Typing and Blame Calculus are investigating the intersection of static and dynamic typing. Let's stop fighting, make some cool tools and use them well.

For more inspiration, history, interviews, and trends of interest to Clojure programmers, get the free Clojure Gazette.

Learn More

Clojure pulls in ideas from many different languages and paradigms, and also from the broader world, including music and philosophy. The Clojure Gazette shares that vision and weaves a rich tapestry of ideas from the daily flow of library releases to the deep historical roots of computer science.

You might also like


  1. As sound as the type system in the language.

  2. See Liquid Haskell for an example of applying an extrinsic type system on Haskell.

JSON Serialization for APIs in Clojure

Summary: Clojure is well-suited for processing JSON, but there are some decisions you have to make to suit your application. The major decisions are actually easy, though they took me a while to figure out.

I tend to use JSON instead of edn for an API serialization format, if only because JSON is more readily readable from other languages. I could do both, but it's good to eat your own dogfood and test those JSON code paths.1

edn is better, but JSON is generally pretty good. However, JSON's expressibility is decidedly a subset of the Clojure data structures, so there is considerable loss of information when going from Clojure to JSON. That information is not recovered in a round-trip, at least not automatically. There are lots of decisions that have to go into how to, at least partially, recover this.

One bit of information that is lost is the type of keys to a map. JSON only allows strings as keys. Clojure allows anything. But most of the time, I find myself using keywords for keys. I say most, but really, it's the vast majority. Maps are bundles of named values pretty much all the time.2 So the optimal decision, after trying lots of combinations, is to convert keywords to strings (the default in JSON libraries I've seen) when emitting JSON; and to convert map keys (always strings in JSON) to keywords (also known as keywordize-keys) when parsing JSON. That covers nearly all cases, and pinpointed special cases can cover the rest.

But that's not the end of the keyword/string story. What about namespaces? Surprisingly, the two major JSON libraries, clojure.data.json and cheshire handle things differently. How do you parse a JSON key that has a slash in it, indicating a namespace? If we're keywordizing (as I suggest above), they both give namespaced keywords (keyword will parse around the /). But when emitting JSON, they act differently. clojure.data.json will emit the name of the keyword (and ignore the namespace) while cheshire emits a string with "namespace/name".

I like to keep the namespace, or, put another way, I like to drop as little information as possible. So I prefer the namespace approach. I'm not sure how to get clojure.data.json to do that. I just use cheshire.3 The other gotcha for namespaces is that ClojureScript's clj->js and js->clj are similarly asymetrical.4

Keywords in other places besides the keys of maps will just get turned into strings, but they don't get converted back to keywords. That sucks, but it's minor. You'll just have to convert them back some other way. At work, we use Prismatic Schema's coercions. They do the trick nicely, in a declarative way.

So, back to other JSON issues. The other issue is other data types. Dates, URI's, and UUID's are in our data as well. Dates, well, it's up to you how to encode them. I've always been a fan of the Unix timestamp. It's not exactly human readable, but it's universally parseable. There's also the ISO datetime format, which is probably a better bet--it's human readable and agreed upon among API designers. You could emit that as a string and coerce back to a Date object later.

URI's and UUID's are by definition strings, so that's easy. How do you set up cheshire to handle the encoders? It's pretty simple, really.

(cheshire.generate/add-encoder java.net.URI cheshire.generate/encode-str)

That means add the encoder for the java.net.URI type to be encoded as a JSON string. str will be called on the value. You can figure out the other types you need. There are some JSON emission settings built-in, including Date (the ISO string format) and UUID. Weirdly URI is not in there, so you have to add it.

What's next? Oh, pretty-printing. Yeah, I pretty-print my JSON to go over the wire. It's nice for debugging. I mean, who wants to curl one long, 1000-character line of JSON? Put some whitespace, please! How to do that?

(cheshire.core/generate-string mp {:pretty true})

That's right, it's basically built in, but you have to specify it. But, oh man, that's long. I don't want to type that, especially because my lazy fingers are going to not do it one time, then I'm going to look at the JSON in my browser and see a one-line JSON mess. So, what do I do? I put all my JSON stuff for each project in json.clj. It's got all my add-encoder stuff, and it's got two extra functions, just for laziness5:

(defn parse [s]
  (cheshire.core/parse-string s true))

(defn gen [o]
  (cheshire.core/generate-string o {:pretty true}))

Or of course whatever options you want to pass those functions. This one is my choice--you make your choice. But these two functions are all I use most of the time. Parsing strings and generating strings. Woohoo! Much shorter and less to keep in my little head.

Well, that just about wraps up my JSON API story. There's the slight detail of outputting JSON from liberator, which is its own blog post. And there's a bit of generative testing I do to really challenge my assumptions about how I set up the round-tripping. But that, too, is another blog post for another day. Oh, and what about all that JSON middleware? Again, another post.

If you like peanut butter and you like jelly, you will probably like peanut butter and jelly sandwiches. If you like web and you like Clojure, you will most definitely like Web Development in Clojure, which is a gentle, soothing, visually rich video course ushering in the fundamentals of Clojure web development through your eyes and ears and down through your fingertips and into your very own Heroku-hosted web server. At least watch the preview!

You might also like


  1. It may be hubristic to think anyone else will use my API.

  2. There are exceptions, but these typically are not communicated to the outside. Those that are need special-casing. C'est la vie!

  3. :key-fn in clojure.data.json only works for keys, not all keywords. Emitting a keyword in any other place emits the str of it, which includes the : in the string. Ick.

  4. clj->js uses name for keywords and str for symbols, so keywords lose their namespace when emitting JSON, but retain their namespace when parsing key strings as keywords.

  5. also known as ease, peace of mind, DRY, and cleanliness

What is Functional Programming?

Summary: I prefer to define Functional Programming as making a distinction between pure and impure code. With this definition, you can program functionally in any language. What differentiates the functional languages is how much help they give you to make the distinction.

There are a lot of conflicting definitions of Functional Programming out there. I'd like to share mine, which serves me well. It explains why Haskell is more functional than Scheme, and also how you can program functionally in a non-functional language like Java.

Functional programming means programming with a distinction between pure code and impure code. Pure code has no side effects. It's referentially transparent. It means the same thing every time you run it. Impure code contains side effects, so running it twice is different from running it once.

The distinction between pure code and impure code uniquely identifies functional programming and distinguishes it from other paradigms such as procedural and Object Oriented. Procedural is about modeling your solution as sequential steps. Object Oriented is about modeling your solution as communicating objects. Functional programming is about modeling your solution as pure functions.

Now, this definition is very practical. Notice that it's not about choice of language. You can write functional code in any language, just as you can code up an object system in C and say you're doing OO. The question is how much the language helps you write functional code or OO code.

On one extreme, you've got Haskell. There is no doubt that Haskell is a functional language. How does it help you write functional code? It has no mutable values and side-effects are confined to a single type: IO. The language forces you to make the distinction between pure and impure.

On the other extreme, you've got machine code or assembly. At the lowest level, the language pushes you to avoid the distinction. All operations are about changing at least one location in memory. It could be a register or the top of the stack or something. But you are forced to change something. However, with a lot of super-human discipline, you could keep the distinction in your head. You might create a little heap and keep the discipline "a procedure can only write to memory it allocates directly". And this way, you make a bit of room for some functional programming. But that language is not giving you any help.

So why functional programming? Well, it turns out that knowing that running code twice will produce the same result makes it very easy to reason about it. And reasoning about code is basically our job as software engineers. What's more, the kind of reasoning you can do with functional programs can reach all the way up to the highest forms of reasoning, like math. That's where Haskell really shines. All of the category theory stuff (monads, functors, applicatives, etc) is an expression of that--mathematical concepts that are applicable in Haskell code.

That's it. That's my definition. The definition is inclusive yet gets at the essence. Functional Programming is a perspective that makes code easier to understand and maintain as it's being used in a system complex beyond your possible comprehension. And at its most sublime and abstract levels, Functional Programming approaches mathematical reasoning.

If you'd like to get started with Functional Programming in Clojure, you can do worse than using the LispCast Introduction to Clojure video course.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

You might also like