Pre-West Prep: Fumiko Hanreich

March 30, 2015

Talk: HoneySQL: SQL Queries as Clojure Data Structures

Fumiko Hanreich's talk at Clojure/West is about HoneySQL, a Clojure DSL for representing SQL queries.

Background

People in the Clojure community talk a lot about representing things as data. This is an approach inherited from its Lisp forebears. HoneySQL is a data-oriented DSL that represents SQL queries. You build up SQL statements in a structured form, not string concatenation.

HoneySQL is a good example of a case where the Clojure philosophy allows for a solution to an existing problem. The problem is that SQL is designed as a human-facing language. That's fine by itself, but it is not a good language for composing pieces of a query into a larger query--the kind of things a program would do. HoneySQL provides an intermediate representation that is way more amenable to manipulation. In a way, it does the same thing for SQL as Lisp s-expressions did for code.

About Fumiko Hanreich

Twitter


This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.

You might also like

Clojure Test Directory

March 30, 2015

Summary: Where to put your tests is a common question. You could put them anywhere, but you want to pick a place that makes it easy to find, easy to exclude from production, and work well with your tools. My recommendation is to follow what most projects do, which takes care of all of these requirements.

You want to write some tests in Clojure. Maybe they're unit tests. Maybe they're integration tests. The first question you must answer is where do you put your tests?

And you don't want them just anywhere. You actually have some important requirements dealing with where they are:

Here's my recommendation, which is the de facto standard of organizing your tests. It works with Leiningen, CIDER, and vim-fireplace.

First, you make a new namespace structure in a test/ directory. It should mirror the src/ directory.

If you have:

src/
  lispcast/
    core.clj
    init.clj
    util.clj

Then your test directory should look like:

test/
  lispcast/
    core_test.clj
    init_test.clj
    util_test.clj

But also notice the second point: that the structure is the same, but the names are slightly different, but it a systematic way. It's really easy: you just add -test to the namespace name, which becomes _test in the file name.1 Then, you put all the tests that test lispcast.core into lispcast.core-test. Now they're easy to find!

If you need to write a test that crosses two different namespaces (like an integration test might), then you can just make a new test namespace that doesn't correspond to one or the other.

Leiningen will load the test/ directory selectively, depending on if you're deploying to production (it won't load test/) or running the tests (it will load test/).

So, it's that easy. You are free, of course, to put the tests wherever you like. But this is my recommendation!

If you're getting into testing in Clojure, you should check out LispCast Intro to clojure.test. It's an interactive course. It has animations, screencasts, exercises, code samples, and text.

You might also like


  1. Clojure file names have to replace - (hyphens) with _ (underscores) to be compatible with Java.

Pre-conj Prep: John Hume

March 29, 2015

This summary was graciously written by Nola Stowe. She's a programmer, the co-founder of DevChix, and a prolific teacher. She recently ran ClojureBridge Austin. Please shout out to her and say thanks!

Talk: Life of a clojure expression: a quick tour of clojure internals

John Hume's talk at Clojure/West is about Clojure internals.

Background

You can get a lot done in Clojure without understanding much about its internals. But when your code doesn't do what you expect, whether in terms of behavior or performance, it can be extremely useful to dig into the underlying Java code.

John's presentation will walk the audience through some of that code by following a simple map-literal expression through Clojure's reader and analyzer, particularly focusing on code generation in clojure.lang.Compiler, and then onto runtime evaluation. The hope is that you'll come out of the talk better able to analyze what's really happening in your own Clojure code.

About John Hume

Github - Twitter - Website


This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Yoko Harada

March 29, 2015

This summary was graciously written by Nola Stowe. She's a programmer, the co-founder of DevChix, and a prolific teacher. She recently ran ClojureBridge Austin. Please shout out to her and say thanks!

Talk: Joys and Pains to Write a Clojure Curriculum for Beginners

Yoko Harada will be speaking on ClojureBridge and the challenges of writing a curriculum for beginner programmers.

Background

ClojureBridge aims to increase diversity within the Clojure community by offering free, beginner-friendly Clojure programming workshops for women. The curriculum is a work in progress which you can see here. They can use help and you can look at the issues on github to see what is needed!

Please watch the introductory talk on ClojureBridge by founder Bridget Hillyer.

About Yoko Harada

Github - Twitter


This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.

You might also like

LispCast Intro to clojure.test

March 28, 2015

Summary: LispCast Intro to clojure.test will launch this week.

LispCast Intro to clojure.test is ready for launch! It's been incubating in the PurelyFunctional.tv Early Access Program (PEAP) for a month. People who opted in to the PEAP got an early version of the course, have been asking questions on the forum, learning a ton, and having a direct impact on the quality of the course. I'm so glad I did it this way. It let me test out the new course format and work out the kinks with the help of my customers.

And now it is clear the new course format has performed really well. It's very interactive. It contains exercises, animations, screencasts, code samples, and narration. All directed at getting clojure.test installed in your brain, best practices, conventions, and all.

It's suitable for beginners to Clojure and for anyone who wants to learn how to properly use clojure.test. clojure.test is the main testing library that comes with Clojure and works well with Leiningen, CIDER, vim fireplace, midje, test.check, and more!

The course will launch this week. Sign up to learn when it comes out.

You might also like

Example-based Unit Testing in Clojure

March 26, 2015

Summary: Unit testing in Clojure is straightforward. Here are a few testing ideas as they apply to Clojure.

Most of the Unit Testing literature discusses how to unit test Object Oriented code. However, Unit Testing is very useful for functional code, including Clojure. I'd like to document a few unit testing ideas as they relate to Clojure.

The Unit you're testing is the function

Unit testing is about testing the smallest unit of code. In Object Oriented languages, the unit is the class. In functional languages, the unit is typically the function. This is true in Clojure. If you're testing individual functions, you're unit testing Clojure.

Example-based tests

The easiest kind of tests to do is example-based, which means you test that for a given argument, you get a known return value. Let's look at a simple example:

(deftest addition-tests
  (is (= 5 (+ 3 2)))
  (is (= 10 (+ 5 5))))

You're testing that + works on two different inputs! Notice: 2 lines and 2 inputs, it looks like we'll get linear growth in tests as our coverage increases.

Round-trip testing

Ok, it's not exactly unit testing, if you are strictly going by the definition of "unit", because you're actually testing two functions. But who's really being so strict? A really useful kind of test is the round-trip test.

In Clojure, pr-str prints a value readably, meaning if the value can be read back in, this will make a string that could be read in using clojure.edn/read-string. You can do a round-trip from value to string back to value, and the two values should be equal. You're testing the property that these two functions are inverses of each other.

Example:

(deftest pr-str-read-string-round-trip
  (is (= [1 2 3] (read-string (pr-str [1 2 3]))))
  (is (= {:a :b :c :d} (read-string (pr-str {:a :b :c :d})))))

Again, we're getting linear test growth.

Here's another example where I test that addition is the opposite of subtraction:

(deftest +---round-trip
  (is (= 5 (-> 5 (+ 10) (- 10))))
  (is (= 10 (-> 10 (+ 100) (- 100)))))

What examples to test

If you're writing example-based tests one-by-one, and you're getting linear benefit for your examples, you've really got to maximize what you test, because linear growth is actually quite bad. In that case, what do you test? The best thing to test are the corner cases. Corner cases are mostly domain-dependent, but there are some domain-independent ones.

Empty collections

It's good to test what happens when you call a function on an empty collection. It could be that you didn't handle that, or didn't handle it correctly. The biggest gotcha is stuff like dividing by the size of the collection. If it's empty, the size is zero, and that's undefined.

Empty strings

The bane of the web programmer's existence, empty strings are usually not valid input, but of course that doesn't stop someone from passing one in. Are you testing that it's valid?

Zero

Zero is actually a typical corner case. Try it out.

One

One is also a typical corner case.

Normal cases

You should have at least one normal case to test the expected behavior. A normal case is a list with 5 elements, or a small integer (7, 12, 34).

Bugs

Now, here's the thing that makes having a test system set up totally worth it: having a place to put test for known and fixed bugs. If someone reports a bug, it's really nice to reproduce it in code in a failing test before you fix it. If the bug happened once, it could happen again, so make it part of your anti-regression suite.

Multiple assertions on the return value

One last thing that happens in Clojure is you want to assert a few things about the same return value. Instead of calling the function several times, why not save the value and assert a few things about it?

(deftest map-test
  (let [return (map - [1 2 3 4 5 6 7])]
    (is (= 7 (count return)))
    (is (every? neg? return))))

One step further

Ok, I've mentioned a few times that example-based testing does not scale. Code coverage grows linearly as the number of examples grows. How do you get better than linear? One way is to use property-based testing (also known as generative testing). Instead of the programmer giving examples, the program generates examples! Here's a preview:

(defspec edn-roundtrips 50
  (prop/for-all [a gen/any]
    (= a (-> a prn-str edn/read-string))))

This tests that any value (gen/any) can be printed to a string and read back in, and you get an equivalent value. Three lines. You can run this with as many randomly generated values as you'd like. Thousands. Millions. With three lines. That's leverage.

Conclusion

Ok, those are just a few ideas that could get you started with example-based unit testing in Clojure. If you'd like to start automated testing in Clojure, I suggest you check out LispCast Intro to clojure.test. clojure.test is the built-in, standard Clojure testing library that most systems use (or are compatible with). The LispCast course is an interactive course with animation, examples, screencasts, text, code samples, and more!

Now, if you'd like to up your game at testing, and want to get more than linear bang for your buck, you've got to get into generative testing.

You might also like

The conj Mental Bump

March 23, 2015

Summary: conj can be confusing if you're used to other languages. It is not a commonly defined operation. Most languages define common positional adding operations. conj, however, has more useful semantics. How do you use conj usefully if you can't guarantee the position?

When I was at university, we were taught object-oriented programming as it exists in Java. We learned about interfaces, inheritance, and the Liskov Substitution Principle. It makes sense. If you're claiming that you've got a (sub)type of car, it still has to do everything a car can do. Otherwise it's not really a subtype of car.

The point of confusion

Whenever I'm teaching someone Clojure, there's a point in the journey where everyone gets at least curious, if not outright confused.

(conj '(1 2 3) 4) ;;=> '(4 1 2 3)

vs

(conj [1 2 3] 4) ;;=> [1 2 3 4]

What's the deal? Does conj add to the beginning or the end? What possible contract could allow both of these behaviors?

Then I show them that the confusion goes even deeper:

(conj #{1 2 3} 4) ;;=> #{1 4 2 3} ;; or some other random order

and

(conj {1 2 3 4} [5 6]) ;;=> {1 2 3 4 5 6}

What gives? How is this even possible? Do hashmaps and linked lists even share a common class ancestor?

The answer is, sure, if you want them to. The student must get over a tiny conceptual bump. And once that bump is surmounted, poof, a new way of seeing is discovered.

The bump

Let's get over that bump right here, right now.

Let's imagine a traditional collection interface, let's call it List. It has two methods, addFirst and addLast that add new elements. So you write an algorithm that adds a bunch of items to the end with addLast. It takes a List as argument, because that's the least subtype you need to perform the algorithm.

You call that algorithm with an ArrayList, which has the nice property that addLast is constant time. Woohoo! Your algorithm is fast and great.

A few months later, you get a phone call from another developer. He's complaining that he used your routine and can't figure out why it's so slow. It was working fine for a while, but as the users generated more records in the database, the routine was grinding to a hault.

You check out the code and immediately see the problem: the database query was returning not an ArrayList but a LinkedList. The implementation of addLast on LinkedLists is actually linear. Adding a bunch of stuff to the end was turning into a quadratic operation.

Let's say that again: even though the location semantics of the operation were the same, addLast on one had constant time and on the other had linear time. They both gave equivalent lists, but one of them was too slow. Does this satisfy the Liskov Substitution Principle? In practice, can you really substitute one for the other? Algorithmic complexity matters.

Clojure avoids that mess (while swapping it for another, which I'll get to shortly). It defines conj, which means not "put this at the beginning" or "put this at the end", but "hey, collection, you know yourself better than I ever can. Please add this wherever it makes sense for you as long as you do it in constant time. Thanks."1

Practically, that means that conj on LinkedList adds to the front, because that's constant time. And conj on ArrayList adds to the end. But, because the operation doesn't talk about order, like addFirst and addLast do, you can now extend conj to Set and even Map if you consider key/value pairs as single items. And that means that linear algorithms using conj will remain linear regardless of which collection you use.

The mess that Clojure chooses over the other mess

Does this satisfy the Liskov Substitution Principle? Well, that depends on how you look at it. You certainly don't guarantee that you get the same or even equivalent answers out. Consider this:

(def a [1 2 3])
(def b '(1 2 3))

(= a b) ;;=> true
(= (conj a 4) (conj b 4)) ;;=> false

So, here, performing the same operation on two equal values does not give equal results. That's kind of hard to reason about. But it's a similar tradeoff that you see with other operations that don't guarantee order. For instance, imagine two sets a and b.2

(= a b) ;;=> true

(= (seq a) (seq b)) ;;=> could be false!

The order of most sets is not guaranteed! This means that Clojure has some operations that do not maintain equality. conj just happens to be one of them.

What's the point?

So Clojure does not provide add operations that guarantee order regardless of collection type. Fine. What's the point?

The point is that, in practice, conj is more useful than addFirst and addLast combined. By defining a function using conj, it will work on a broader number of collections. It might give different answers for each, but it won't explode on one and do fine on the rest. And often the answers it gives are just fine. A basic version of into can be defined very easily. It works on all collections (for both to and from).

(defn into [to from]
  (reduce conj to from))

Common usage

One last thing before I wrap up: because the collection itself defines where the item will be added, I often find myself choosing the collection based on where I need it. A common idiom in Common Lisp was to make a new list by consing onto the front, then reversing it at the end because you really wanted them in the other order. In Clojure, there's no need, because you can just use a vector (and use conj). As long as the vector is local to the algorithm, it's not part of the contract, so it's your choice.

Conclusion

Java was wrong. addFirst and addLast cannot be substituted in LinkedList and ArrayList. They have different algorithmic complexities and at some point one's performance will be totally unacceptable. The operation that does allow for substitutibility in algorithm complexity is conj, which is always constant time. But then it doesn't maintain equality. However, I find that conj is way more natural and helps algorithmic reasoning more than guaranteeing where the item is placed.

If you'd like to learn Clojure, I recommend my video course LispCast Introduction to Clojure. It's a great introduction to the language using animations, exercises, and screencasts. It's designed to give a deep dive straight to what makes Clojure interesting. It begins with syntax, goes through functional programming, and ends with data-driven programming.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

You might also like


  1. It's even polite!

  2. (def a (set (range 100))) (def b (apply sorted-set (range 100)))

Pre-West Prep: Elango Cheran

March 21, 2015

Talk: Exploring Programming Clojure in Other Human Languages

Elango Cheran's talk at Clojure/West is about writing Clojure in other spoken languages.

Background

I once heard someone from Poland say that they didn't mind so much that programming languages are mostly English-based. They compared it to classical music, where most terms are adapted from Italian. It's just part of learning the craft.

That said, making basic English a pre-requisite is a huge burden for the would-be programmer. Being able to write code using terms from your own language and characters from your own language could go a long way to making programming more accessible. Elango claims that Clojure is well-suited for the task because it has macros. That's intriguing.

Read his blog post about using unicode characters in Clojure. It introduces clj-thamil, a library for wrapping built-in Clojure with Thamil equivalents.

About Elango Cheran

Homepage - GitHub - Twitter


This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.

You might also like

Pre-West Prep: Dan Lidral-Porter

March 21, 2015

Talk: Generating Art in Many Worlds

Dan Lidral-Porter's talk at Clojure/West is about computer-generated art in Clojure.

Background

Dan's talk is about his solution for creating interesting art while live coding. Since there's a huge space to explore, you don't want to make your audience watch all the boring parts of the exploration. His solution is to be able to see the art ahead of the audience so they don't have to see it unless you've discovered it's interesting.

Check out some of his generated art. He's also the creator of this very cool ClojureScript + Om + Zelkova app for exploring Magic: The Gathering decks.

About Dan Lidral-Porter

GitHub - Twitter


This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.

You might also like

Pre-West Prep: Colin Fleming

March 21, 2015

Talk: Debugging Clojure code with Cursive

Colin Fleming's talk at Clojure/West is about debugging Clojure in Cursive.

Background

Colin Fleming is the developer behind Cursive. Cursive is the #2 most popular Clojure IDE (behind Emacs and just ahead of vim). It is based on IntelliJ, and so has a lot of useful JVM features. This talk will focus on debugging and also how the JVM debuggers fare with functional code.

For a great introduction to Cursive, see Colin Fleming's talk at Clojure/conj last year.

About Colin Fleming

Twitter


This post is one of a series called Pre-West Prep, which is also published by email. It's all about getting ready for the upcoming Clojure/West, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

That rich context is what Pre-West Prep is about. I want to enhance everyone's experience at the conference by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

Clojure/West is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/West or Cognitect. It is simply me (and helpers) curating and organizing public information about the conference.

You might also like