Summary: If something is important, you should deal with it sooner and more often. That's the approach Clojure takes to data representations. It means serialization of internal data is a non-issue.
Netflix has a counterintuitive but brilliant system called Chaos Monkey. Chaos Monkey will randomly kill services. It is trying to simulate, in the microcosm of a business day, the craziness that happens in the production macrocosm. Services die, servers fail, latency spikes, and networks partition. It might not be soon, but eventually it will happen. Better deal with it now, when people are at work to deal with it. Chaos Monkey encourages you to write your code to work in such a chaos-filled environment so that unexpected failures are a breeze. The idea is an instance of the more general principle of "if something is important, deal with it sooner and more often".
There's something related to this idea of surfacing the errors early that I've been thinking about. It's about data. Have you ever written some classes and everything is well and good. And then after a while, you need to serialize that to disk or across a network. Now you have to come up with some format (maybe JSON or XML) to represent your class. Then, whenever the class changes, you have to remember to change the data format to keep them in sync. It's kind of painful!
Well, I'll tell you, that rarely happens in Clojure. Because in Clojure, you often start with data structures. Meaning, you're already thinking in data, modeling in data, and working in data. Serialization becomes a non-issue, because Clojure data structures come built in with readers and writers. Just like causing errors to happen early because they will happen eventually, Clojure programmers code in data in part because you're going to need data (serialization and deserialization) eventually.
If this idea intrigues you, you may want to check out Clojure. Its data structures are cutting edge! Check out LispCast Introduction to Clojure if you would like to get into functional programming using animations, exercises, and screencasts.
I think of functional programs as ones which do their jobs without stopping to take notes along the way.
That got me thinking of the metaphor of taking notes. It's an interesting metaphor, because as she mentions in the article, modern computers are based more on Turing Machines, with their tapes (notes), rather than Lambda Calculus. Lambda Calculus being abstract instead of mechanical (what instead of how), it doesn't show its work.
In the spirit of more perspectives being better than fewer, I'd like to give my metaphor, inspired by idea above. Functional programs are those that have a very simple rule: never write on the same paper twice. In Turing Machine terms, it means never write on the same piece of tape twice. This rule is simple enough that you can build it into the compiler and runtime to enforce it. In programming language theory, we call this immutability. Once a paper is written on, it is immutable.
Imperative programs, on the other hand, don't have such a rule. They can write on any piece of paper, even write over or erase what's written. They rely on their own code to enforce rules for how to organize these pieces of paper so that it still makes sense. They might use locks, which are notoriously hard to get right.
The nice thing about functional programs is you get a nice record of what's happened. You can step through all of the intermediate work because it was never overwritten. It's like an artist keeping their old sketches that led up to their masterpiece. Imperative programs are like finding out that it was all done on the same canvas, art painted over art, irrevocably lost. There is no xray to uncover it.
The other nice thing about functional programs is multiple programs can share the same stack of paper. Since they're following a rule, they will never overwrite someone else's work. But imperative programs need to have complex coordination rules to make sure that they aren't writing on the same page at the same time, or overwriting something that someone else will need.
If you'd like to learn Functional Programming, I can recommend LispCast Introduction to Clojure. It will teach you Functional Programming using Clojure, animations, and screencasts, along with many exercises.
Summary: LispCast Intro to clojure.test is on sale now for a limited time through the PurelyFunctional.tv Early Access Program.
Update:PEAP is now closed for Intro to clojure.test. Sign up below to find out when the finished product is published.
Folks, it's happening. I just opened up the PurelyFunctional.tv Early Access Program for LispCast Into to clojure.test. If you like to be on the cutting edge or get lots of access to the teacher, now's your chance. Open for 48 hours only!
Summary: The PurelyFunctional.tv Early Access Program is a way to make courses in a more iterative and interactive way. I'll be publishing Intro to clojure.test this week under PEAP.
Computers and the Internet have allowed people to produce better products than has ever been done. Photography is better because of Flickr. Handmade knives are stronger and sharper than ever. Stone arrowhead making is better than ever. People are able to experiment, share their results, and learn the results of others faster than ever before, because the Internet allows them to communicate faster.
It's simply a matter of speed and accessibility. I want to bring that speed and accessibility to LispCast courses. I have been working hard on building a platform for me to develop courses iteratively. That means I can make them in smaller steps. The other piece is the feedback, where I ask you, my awesome customers, to borrow your brain for a bit. Talk about what works for you and what doesn't. I'll listen, answer questions, discuss them with you, and incorporate what I learn into the course.
Everyone wins. You get to learn what you need to learn faster, with your questions answered. And I get to make the course better for everyone. Seems like a good deal.
I'm creating what I'll call the PurelyFunctional.tv Early Access Program. It's simple. As soon as a course has a bit of content and interest, I'll post it up for sale. You get to download the course in a first draft. Let me know what questions you have, where you're having trouble, or things that were too easy. I'll continuously work those things in, and let you know when to download the new one.
I'll throw in another small benefit: if you're part of the PEAP, meaning your buy the course before it's finished, I'll put your name in a Thank you page in the course and on the course information page. Everyone who goes there will be able to see it.
To kick off the Program, I'm going to be publishing LispCast Intro to clojure.test, in its current state, this week. Entry into the Program will be available for a limited time. So sign up below if you'd like to be a part of it. I'll send out an announcement with the instructions.
Summary: Hiccup is a Clojure DSL for generating HTML. If you're using it, you might like these tips.
Hiccup is a Clojure Domain Specific Language (DSL) for programmatically generating HTML. It's one of the three tools I recommend in my Clojure Web stack. It's a thin layer over HTML, but oh, how I welcome that layer! The biggest win is that since it's an internal DSL, you can begin abstracting with functions in a way that you will never be able to, even in templating engines.
Hiccup is an example of a great Clojure DSL. It uses literal data structures pretty well, it's as or more readable than what it translates into, and, as a layer of indirection, solves a few sticky problems of HTML. If you don't know it, go check out the Clojure Cookbook recipe for Hiccup. Hiccup might take some getting used to, but once you do, you'll appreciate it.
This article will assume you are familiar with the syntax and want to up your game.
Cross-site Scripting Vulnerability
Ok, this one is a pretty bad problem. Hiccup generates Strings, just plain old Java strings. If you put a String in the body of a Hiccup tag, it will just concatenate it right in there, no questions asked. That's just asking to be exploited, because that String is going to be shipped off to the browser and rendered, scripts and all.
Most templating libraries will default to escaping any String you pass them. The non-default, usually obviously marked alternative is to pass in a String unescaped. Hiccup got this backwards and it just sucks. It means you have to do extra work to be secure, and if you forget just once, your site is vulnerable.
The fix: This is the work you have to do every time you're getting a String that could be from the "outside" (a form submission, API request, etc.). Normally, you'd do this:
That will work but it's unsafe. You should do this:
[:div (h content-of-div)]
That little h (hiccup.core/h to be exact) there means escape the String. It sucks, but that's how you do it in Hiccup. One day I want to write a secure fork of Hiccup.
One downside to Hiccup and any DSL that overloads the meaning of vectors is that vectors are no longer useful as sequences within Hiccup. They now mean "start a new HTML tag". It's not a huge deal, but I've spent a lot of time debugging my code, only to realize that I was using a vector to represent a sequence. I use literal vectors everywhere else (because they're convenient and readable), but in Hiccup land they're wrong.
The fix: You can't use a literal vector, but you can call list to create a list. Not as beautiful, but it is correct. Sometimes I will call seq on a return value to ensure that it's never a vector.
I don't know why this still happens, but it's common, so I'll mention it. Sometimes I'll be looking at the HTML output in a browser and I just can't find an element. It's gone. Reload, still not there. When I look through the code, the hiccup to generate the tag is right there! Why won't it render?
Well, long story short, it's because in Clojure, only the last value of a let or fn is returned. Doh! My missing element was being rendered then discarded.
(defn list-with-header [header items]
[:h3 header] ;; this header is missing
(for [i items]
The fix: Wrap the two (or more) things in a list (not a vector!).
(defn list-with-header [header items]
(list ;; wrap in a list, not a vector
[:h3 header] ;; now it's there
(for [i items]
Hiccup plays nice with nil
This one is just a little design touch with some perks, not a problem that it's solving. In Hiccup, the empty list renders nothing. This is extended to nil as well. A common thing in HTML is that you want to render a bunch of children in a parent tag, but you don't want the parent tag if the list is empty.
Standard constructs will render the parent tag:
(for [i items]
When items is empty, you still get <ul></ul>. This is a problem with lots of templating libraries.
The fix: The fix in Hiccup is due to it playing nice with nil. Wrap the parent in a when:
(when (seq items)
(for [i items]
It's not beautiful, but then again, you could be using HTML with Moustache.
Use defn for snippet abstraction
HTML has no way to abstract. The crap you see is the crap you get. Many templating libraries have some kind of snippet concept, often referring to different files. Well, because Hiccup is just code inside of Clojure, you've got a better designed way of making reusable pieces of Hiccup.
You may know this, but Hiccup is a compiling macro. That means it takes your literal vectors, maps, etc, and at compile time, generates code that will generate your HTML. All this means is that Hiccup is really fast, about as fast as concatenating strings can be.
But, because the Hiccup compiler doesn't do a full examination of your code, it can't compile everything. It inserts run time fallbacks for stuff it can't handle at compile time which will interpret it at run time. So, for instance, if you're calling a function that returns some Hiccup, it can't compile that automatically. It has to wait till the function returns to know what it is. That is, unless . . .
**The fix: ** The way to get Hiccup to compile something is with the hiccup.core/html macro. That's the macro that does the compilation and it will do it anywhere. So if you've got code like this:
For this little example, it probably won't make a noticeable difference. But it can be significant for larger documents.
Just to note: the Hiccup compiler can understand if and for forms, so there's no need to wrap them in the compiler. No hard either.
Just a good thing to know about HTML.
Did you know that this is not legal HTML?
It's true. A script tag can't be self-closing.
There's all sort of silly rules in HTML like this. And then there's XML mode versus HTML mode. We are lucky: Hiccup does all of this for you, so you don't have to wade through the HTML spec(s!) looking for why something won't render in IE7.
The fix:hiccup.core/html takes a map of options as the first argument (it's optional). If you pass in a :mode option, it will set the correct HTML mode, which unfortunately are incompatible. There are three modes, :xml, :html, and :xhtml. The default is :xhtml.
Hiccup is a DSL. And it has its own sub DSL for HTML ids and classes. It's similar to CSS selectors.
Let's say you have a div like this:
:class "blue-background green-border"}
(h "Here's some content.")]
Well, Hiccup lets you make that shorter and easier to read.
The fix: Use the id/class DSL:
(h "Here's some content")]
Here's how it works. Every element has an optional id and zero or more classes. After the tag name (div here), you can put the id after a #. Then list your classes starting with a . each. Omit the # if there's no id. Ditto for the . if there's no class. Oh, and the id must come first. That will get converted to the id and class attributes you want. Also, these will conflict with attributes in the map, so choose one or the other, not both.
Generating Hiccup from HTML
Last thing, I promise!! Sometimes you have some HTML that someone gave you and you want to Hiccupify it so you can just stick it into your code. Manually doing that is kind of tedious. Luckily, there's a solution our there for you.
The fix:This page lists three options for outputting HTML from Hiccup. I have personally used Hiccup-bridge. It does a good job (and it even goes both ways). You call it like this:
lein hicv 2clj hello.html
That will output hicv/hello.clj in hiccup format. Pretty rocking when you need it.
Well, there you go. My Hiccup tips. Hiccup is pretty nice for templating. I recommend using it (or my future secure fork) in your web projects. If you'd like to learn more Hiccup and how to build web apps in Clojure, please check out [Lispcast Web Development in Clojure][webdev]. It's a video course teaching just that. All you need to know is Clojure and HTML.
Summary: The next course will be about clojure.test. Sign up to be notified when it is published.
Last week I mentioned my new course format, and I also teased at more announcements. Well, it's a course on clojure.test, the Clojure testing library that comes built in with Clojure.
I chose clojure.test as my next topic for a number of reasons. The main reason is that I think it's a good idea to begin with the fundamentals. I have a long list of topics, and testing was as good as any. It's also a video that opens doors to other topics, including testing using Midje, test.check, and simulation testing.
The course is called LispCast Intro to clojure.test. clojure.test is not a big library, but it's important to know. This course will teach you how to use the library and conventions around testing that might not be so obvious. For instance, how to name your testing namespace.
The course is built in the new format, and it's shorter than my previous courses. It's common for someone to tell me how much they like my courses. My first question is always "Did you finish them?". And it disturbs me how many people say "no". They like the course but they don't finish. So I'm doing smaller courses that are easier to fit into your day and finish. I expect to make more, smaller courses from now on. And Intro to clojure.test will be the first.
And now that that announcement is done, I must tease you with just one more announcement coming later this week. If you don't want to miss it, sign up for the mailing list below. You'll be the first one to know when clojure.test is coming out and what that cliffhanger announcement is.
Summary: LispCast video courses have a new, interactive format. It's easier for me to make courses (read: more, faster) and it's a better experience for learners.
I've released 3 Clojure LispCast video courses. They've been selling okay (many thanks to my beautiful buyers and other supporters!), but I need more videos to bring it to a full time wage. After it took 9 months to do the core.async videos1, I decided to take a step back and optimize my process.
The first thing I wanted to do was to be able to generate the videos programmatically. Editing video takes a long time, and I knew that most of what I did during editing was automatable. That would eliminate hours of work per hour of video.
I began coding up some routines to compose videos functionally (inspired by the functional animation work by Conal Elliott). It worked really well. I could code up simple animations, concatenate videos, and rerender a preview very quickly. This combined with the Emacs Lisp scripts I have for automatically recording the "typing" portions of the videos, I was ready to code up the video.
I'm working on the next LispCast course, in the new format. It's already way faster to make, and the viewer experience is amazing. It combines video, interactive exercises, animations, and more into one package. I can't wait to show the world. I've got a lot of great things to announce soon, so sign up below to get updates as they come out.
Summary: Ring, the Clojure Web library, defines three main concepts that you use to construct web applications.
Ring is the center of the Clojure web universe. It's not the only option in town, but the other options refer to Ring to say how they are different. Understanding Ring will help you learn any other web system in Clojure.
Ring has three main concepts that work together to build a web application.
I like to think of Ring Adapters as plug adapters. When you go to a different continent, you often have to adapt the holes in the electical outlet to fit your cords. Ring Adapters let you convert different web server implementations into a standard Ring Request/Response format. That way, all of your code is standardized to the Ring format. Your code can travel into any kind of server as long as an adapter exists.
There are Ring Adapters for many existing servers.
Handlers do the work of your application. They are like the computer. They are just Clojure functions. HTTP is basically a request/response protocol that maps well to functions, which are just a protocol from argument to return value. Handlers take a Ring Request and return a Ring Response. They should do whatever logic is necessary for your application.
Middleware are the voltage converters. Here in North America, wall sockets run at 120 volts, which is different from almost everywhere. In order to run an appliance from elsewhere, you not only need to adapt the socket, you also need to transform the current to a compatible voltage. Middleware are often used to convert the incoming request in some standard way. For instance, there is middleware to parse a JSON body into a Clojure map and store it away in the request.
The transformer also "cleans up" the current. Voltage spikes are evened out so they never get to the computer. Middleware can similarly protect a handler by making sure the browser is logged in.
The analogy kind of breaks down, because middleware can do work (like the computer). Middleware are the hardest part of the Ring idea. They're not hard because the concept is hard. They're hard because they require design decisions. If all you had were Adapters and Handlers, you wouldn't have to think about where to put your logic. It would all go in the Handlers.
But there would be a lot of duplicated logic in your handlers. Authentication, routing, content-type switching, all of these things are done the same way over and over. It's the perfect problem for a little higher order programming. That's essentially what Middleware is.
Ring Middleware are functions that take a Handler and return a new Handler. Since Handlers are functions, Middleware are higher-order functions. The transformer on your computer's power cord takes a machine that requires a certain current and turns it into a machine that takes a different current. Middleware are used to do all sorts of things.
So, for instance, there's a Middleware called Prone that captures exceptions in the Handler and displays them in a nice format. Prone is a function that takes a Handler and returns a new Handler that catches exceptions and returns a different Ring Response in that case. Or you have Middleware that handle session cookies. The Middleware take a Handler and return a new Handler that understands sessions.
My recommendation for what to put in Middleware versus what to put in Handlers is simplest to explain with a graph.
Along the x-axis, we have logic that ranges from HTTP logic (handling header values, query params, etc.) to business logic (which bank account to withdraw from). Along the y-axis, we have how unique the logic is, ranging from highly duplicated to custom. These are the two axes I use to figure out whether it should be in the Handler or the Middleware.
The clear cases are easy. In the upper right corner (red dot), where it's custom business logic, it's definitely in the Handler. In the lower left (blue dot), where it's duplicated HTTP logic, I prefer Middleware. The hard part is in the middle. Somewhere between those two, there's a fine line where a case-by-case decision is required.
Ring is great because it requires so few concepts to capture so much of HTTP. But it's not everything. Standard Ring does not support WebSockets, for instance. Small adaptations are necessary. In general, I think this is a great abstraction. And Ring is so central to Clojure on the Web, it's important to know.
If you want to learn more about Ring and how to construct web applications in Clojure, you should check out LispCast Web Development in Clojure. It's a video course designed to guide you through the major libraries used for web development. You'll learn how to build Clojure web applications from existing and custom parts. You build Middleware to make your application adapt to browser limitations. And if you sign up below, you'll get a handy Ring Spec reference sheet, which specifies the Request and Response formats.
Let's take a quick look at programming from an interesting perspective. Let's say that programs are divided into code and data. 1These are the two main axes: structuring code and structuring data.
Now, let's design a language. First decisions: which axis is primary, which is secondary?
In an OOP-style approach, we have a pointer to an object which is the data. And the data also tells you where the code is to access it (methods in the vtable). An FP-style approach will make functions (code) primary, with data referred to by the closures (the captured lexical environment).
I'm positive I'm not the first person to remark on this, but I think it's pretty cool that they are duals of each other. A simple transformation will switch one in the table with the other.
Perhaps the next obvious question is "which one is better?". Many arguments about OOP vs FP focus on this choice, about which axis should be primary. Because a compiler could easily transform between them, the decision is not quite so important as we might think at first. I think that a language should make both available and let the programmer choose. The programmer is then free to choose whichever best models the problem.
For instance, in Clojure, fns are closures, so they are really FP-style -- code meant to be executed which happens to refer to some data. But collections are data which refer to code which knows how to access the data. Both coexist quite cooperatively.
And Clojure gives you, the programmer, ways of expressing both styles. When you want to create some code to be executed, fn lets you define lexical closures. But sometimes you want to make a new data structure which acts like others. For instance, a new type of sequence. deftype lets you define data with interface and protocol implementations. I use both fn and deftype very often.
How does your language prioritize these two axes? Is there a way to choose which to use to best implement your design? If you'd like to learn Clojure, which does have both styles, you should check out LispCast Introduction to Clojure, a video course that teaches functional programming through Clojure with animations, exercises, and code screencasts.
Summary: I am looking for more sponsors for the Clojure Gazette and I need your help.
I have some big plans for the Clojure Gazette this year.
As far as I can tell, the Clojure Gazette is the only Clojure-focussed newsletter out there1. It's usually a collection of links to material I think would interest and educate people interested in Clojure. It's not always directly about Clojure. But Clojurists seem to be interested in a variety of things. Sometimes the issue is based around a strong theme, like Haskell or Object Oriented Programming. And sometimes it is an interview. This year, I'd like to do more themed issues are more interviews.
I think it plays an important part in the community. I have interviewed the Google Summer of Code participants for a couple of years now. The interviews let the community know what they were working on and what progress they had made. And some of the themed issues got the most positive commentary I've ever received. People seem to want to learn and to have existing information organized for them.
As you can imagine, the Clojure Gazette takes a lot of work to put out each week. There are over 3,200 subscribers now, and it's growing every day. In July 2014, I began inviting sponsors to purchase a placement in issues of the Gazette. I've had a few generous sponsors contact me, which got me a few months of sponsorship.
The sponsorships have been a success, in different ways. One job advertisement got hundreds of clicks and valuable exposure to a company that is just beginning to use Clojure. Others brought qualified visitors to developer-related services. I'm very greatful to Clojure/conj, which bought a placement. It is a big vote of confidence when the official Clojure conference asks to put their logo in your newsletter.
Now I am trying to be more active to find the sponsors. The goals for the Gazette are modest. None of them require any more money, just more time and work, and a little more organization. But the total time input is significant. With a small amount of funding, I'll be able to justify increasing the quality of the Gazette. I've always thought it would become a business one day. Putting money on the line always makes things more crisp and professional.
Now is where I ask for a favor. If you can spare a few moments, would you please think about whether you get value from the Gazette and what you can do to send some sponsors my way. It may be contacting the hiring department at your company and suggesting a paid job listing. Or it could be experimenting with selling your company's service to the thousands of smart Clojure programmers subscribed to the newsletter. Or maybe you just want to show your appreciation and get some good karma in the community. I appreciate all help. Get in touch by , twitter, or call me at 504-302-3742.
Clojure pulls in ideas from many different languages and paradigms, and also from the broader world, including music and philosophy. The Clojure Gazette shares that vision and weaves a rich tapestry of ideas from the daily flow of library releases to the deep historical roots of computer science.