Pre-conj Prep: David Pick

Talk: Building a Data Pipeline with Clojure and Kafka

Background

David Pick's talk at the conj is about Kafka, which is a distributed messaging system. It implements a publish/subscribe model and runs in a cluster. Its documentation claims that Kafka has very high throughput.

Why it matters

Braintree, where David Pick works, is a huge payment processor, now owned by PayPal. They must have very strict availability and consistency requirements in addition to the scale. The talk description hints that Clojure was helpful to their success. Some of the best talks I've seen are this kind of experience report.

About David Pick

Twitter - Github


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

Learn More

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Colin Fleming

Talk: Cursive: a different type of IDE

Background

Colin Fleming's talk at the conj is about Cursive Clojure, a Clojure IDE that runs as a plugin for IntelliJ IDEA, a very popular Java IDE.

Why it matters

Cursive Clojure uses static analysis to provide some nice features that are hard to come by in Clojure editors, including underlining errors. Also, since it is built into IntelliJ, it handles projects (including projects with some Java classes) seamlessly. I have heard more than one person mention that they prefer it to Emacs after using Emacs for years.

About Colin Fleming

Twitter

You can install Cursive Clojure by following this User Guide.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

Learn More

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Brian Goetz

Talk: Keynote

Background

Brian Goetz's talk at the conj is the keynote, which doesn't have a description (which is common with keynotes). I would only be guessing if I said what the talk was about. But here are the things that I wouldn't be surprised to hear in a keynote by the Java Language Architect at a Clojure conference:

  • Java Lambdas (their design and implementation)
  • Future features of Java/JVM (that they're working on now)
  • JVM and dynamic languages like Clojure
  • How Java will approach concurrency

Whatever he talks about, it will be important.

Why it matters

Rich Hickey made the decision to design Clojure as a hosted language. Clojure is tied to the JVM. Also, JVM languages are gaining popularity, and they obviously must play a part in Oracle's strategy. This talk has the potential to set the tone for Java's interaction with Clojure in the future.

About Brian Goetz

Twitter

Brian Goetz is quite an important figure in the Java world. He's the Java Language Architect at Oracle (the company that produces Java). He was instrumental in bringing lambdas to Java 8. The book he wrote, Java Concurrency in Practice is one of the books on Rich Hickey's reading list.

He's been giving a talk recently called Lambda: A Peek Under the Hood about the design decisions and how they are finally implemented in Java Bytecode.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

Learn More

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Bozhidar Batsov

Talk: The evolution of the Emacs tooling for Clojure

Background

Bozhidar Batsov's talk at the conj is about CIDER, which is the Clojure IDE and REPL built into Emacs. You should know that previously, the CIDER project was known as nrepl.el and it replaces SLIME, which is what we used to use in Emacs. CIDER connects to an nREPL server.

If you'd like to install and use CIDER, check out the Using Emacs with Clojure chapter from Clojure for the Brave and True.

Why it matters

Emacs with CIDER is the most popular IDE according to the 2013 State of Clojure Survey. Bozhidar Batsov took over the project one year ago and CIDER has added many new features. I will enjoy hearing about all the things I don't use because I don't know about them.

About Bozhidar Batsov

Twitter - Github - Blog

Bozhidar Batsov is the maintainer of several Clojure- and Emacs-related projects, including CIDER, Prelude, Projectile, and the Clojure Style Guide.

If you can believe it, Batsov only gets $30 per week on Gittip, even though half the Clojure world uses his stuff. Reach into those pockets and sponsor some development.


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

Learn More

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Ashton Kemerling

Talk: Generative Integration Tests

Background

Ashton Kemerling's talk at the conj is about Generative Testing (also known as Property-based Testing) as applied to integration tests. Generative Testing is a methodology for generating many tests randomly to find failing cases (as opposed to writing each test by hand). It often finds difficult bugs and can present a very small, reproducible failing case.

The best talk I have seen about Generative Testing is John Hughes' talk at Clojure/West. John Hughes wrote Quickcheck for Haskell and Erlang. Reid Draper has recently ported it to Clojure in the form of test.check. He also gave a talk at Clojure/West about test.check.

Why it matters

Generative testing is great for unit testing. It has already found bugs in Clojure itself. What's not obvious is how to apply the technique to integration tests to test the interaction of different systems. This talk appears to be an experience report from a company that has actually used Generative Integration Tests.

About Ashton Kemerling

Twitter - Github - Blog


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

Learn More

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep: Anna Pawlicka

Talk: Om nom nom nom

Background

Anna Pawlicka's talk at the conj is about Om, David Nolen's project, started in December of 2013. It's essentially a wrapper around Facebook's React. It's a way to define HTML components functionally and has been making quite a splash.

David Nolen's talk at Clojure/West relates Om and React to the history of the GUI. It's the best high-level introduction I can imagine and discusses the ideas that motivated its design. If you watch one video to prepare for this talk, this should be it.

For a good talk about React itself, check out Pete Hunt's talk Be Predictable, Not Correct. He's one of the main contributors to React and has been evangelizing it.

Why it matters

React is significant because it is very different from other Javascript MVC frameworks. It takes a very functional approach instead of object oriented. React makes much easier to reason about the UI. Om has also made waves because David Nolen has shown that using ClojureScript's immutable data structures with React can be faster than Backbone.js (a very popular library) even though immutable data is commonly considered slower than mutable data.

About Anna Pawlicka

Twitter - Github - Blog

Anna Pawlicka recently gave a talk at EuroClojure about Reactive data visualisations with Om. The slides are available (they're not visible in the video).


The Clojure/conj is going to be awesome. I've never been, but I have watched all of the videos from the prior conferences. I'm going this year, and you should, too! Go buy your tickets. There's some cool stuff happening there, including Opportunity Grants. Check out the Clojure/conj site and buy your tickets while they last. Clojure/conj is organized by Cognitect, the company behind Clojure.

This post is one of a series called Pre-conj Prep, which originally was published by email. It's all about getting ready for the upcoming Clojure/conj, organized by Cognitect. Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media.

Learn More

That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations, as well as the beautiful venue and city of Washington, DC.

Clojure/conj is a conference organized and hosted by Cognitect. This information is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

Pre-conj Prep 2014

Prepare for the conj

People have been asking me how they can get ready for all of the great talks lined up for Clojure/conj. There are many topics, and people are feeling overwhelmed. They might know about some of the ideas, but not all of them.

Wouldn't it be nice to do just a little homework before the conj so that you would be sure to maximize your understanding and be able to participate in the conversations afterward?

Unfortunately, googling any topic reveals lots of material, and it's not always the best. It would take hours to sift through the background material for just one topic.

That's why I am publishing the Pre-conj Prep. Each day, I will send you one or two things to read or watch (whichever is better) to get up to speed on the topics of the talks (which are listed on the Clojure/conj speakers page). I'm used to finding great content for the Clojure Gazette, so you can be sure it will be helpful.

Conferences are ongoing conversations and explorations. Speakers discuss trends, best practices, and the future by drawing on the rich context built up in past conferences and other media. That rich context is what Pre-conj Prep is about. I want to enhance everyone's experience at the conj by surfacing that context. With just a little homework, we can be better prepared to understand and enjoy the talks and the hallway conversations.

There are 18 talks listed. After I go through all of them (that will take 18 days), I will go further than the published topics into other helpful stuff to get ready for a great conj. I am trying to get interviews with the speakers, restaurant/event/city guides, and more, to make the conference a special one.

Sign up below

It's ok if you missed some of the emails. They're all archived. Sign up now and it will link you to all of the past emails.

The conj is a single-track conference, which means all talks happen in series in a single auditorium. The advantage of that is that everyone has a very similar experience. We all see the same talks (unless you skip one). The disadvantage is that you can't choose a different talk if you don't know about a topic. Preparation is key to having a great experience.

As you probably know, conferences are social events. The talks are mostly an excuse to get together, meet each other, and talk about Clojure. The conversations that happen after a talk can be very rewarding. But if you don't understand why the topic is important, you may feel left out. Don't let that happen! You're travelling far and paying (or expensing) lots for the ticket, hotel, and travel. Make the most of it!

Clojure/conj is a conference organized and hosted by Cognitect. This email list is in no way official. It is not sponsored by nor affiliated with Clojure/conj or Cognitect. It is simply me curating and organizing public information about the conference.

You might also like

4 Features Javascript can steal from Clojure(Script)

Summary: ClojureScript adds a lot of value on top of the Javascript platform. But some of that value can be had directly in Javascript without using a new language.

ClojureScript is a dialect of Clojure that compiles to Javascript. It lets you run Clojure in a browser or on Node. Just like Clojure, it has immutable data structures, literals for those data structures, macros, and core.async support.

These features are very powerful and they give ClojureScript a lot of power above Javascript. But a lot of that power can be had in Javascript through libraries, tools, and now some ES6 features. If you can't make the switch to ClojureScript, these tools might help you out. If you've heard of all the cool stuff that ClojureScript can do, well, this might be a good way to dip your toes in before you jump in.

1. Use ClojureScript's awesome data structures in Javascript

David Nolen, the maintainer of ClojureScript, has wrapped up the ClojureScript immutable data structures to be used as a Javascript library which he calls mori. He's put in a lot of work to make it convenient to use from Javascript. It's not just the datastructures. It also has all of the functional goodness that makes them powerful to use.

2. Write CSP like core.async without core.async

CSP stands for Communicating Sequential Processes. It's a model of concurrency that is used in Go and in Clojure core.async. It's a powerful tool, especially in Javascript, where you don't have threads. Once you use it, you'll never go back.

Well, thanks to Generators in EcmaScript 6, you can write CSP code in Javascript. There's a library called js-csp that does just that.

3. Better data formats than JSON

JSON is a great format for data exchange, but it has limitations. One of the biggest limitations is that it is not extensible. You cannot define new types. Wouldn't it be nice to have values that get parsed as Javascript Date objects? Another limitation is that map keys can only be strings.

The edn format is Clojure's literal syntax made into a data format. It includes many data structures, including symbols, keywords, sets, maps, lists, and vectors. It is also extensible with tagged literals, meaning a small tag can be placed in front of a value to indicate the semantic meaning of that value. There are a couple of built-in tags, such as #instant for Dates and #uuid for, yes, UUIDs. But the important thing is you can define your own and define how they get parsed.

There's a Javascript library for parsing and generating edn.

edn is a great format, but parsing it in Javascript is never going to be as fast as parsing JSON. The JSON parser is written in C and highly optimized. Because of that limitation, the folks who build Clojure wrote Transit, which is a data format that serializes to JSON. It's semantically equivalent to edn, so it's extensible, but it's also JSON, so it's fast to parse. There's a Javascript implementation.

4. Write your own syntax with macros

Macros are one of those legendary things Lisps are known for. They let you write code that writes code. Well, that's kind of a meaningless statement. Think of it more like extensible syntax. You can write a new type of syntax which will be translated to plain Javascript.

There's a macro system for Javascript called Sweet.js. The macro system is similar to the one found in Scheme systems. This isn't directly from Clojure, but you get the same power. Macros are a great way to extend a language. You no longer have to wait for a new language feature to be implemented. You can do it yourself.

Conclusion

The Javascript community is active and growing. There are plenty of smart people making cool libraries that can revolutionize your programs. One of Clojure's philosophies is to have a small core language and extend, to the extent possible, using libraries1. So, those libraries can often be leveraged outside of Clojure, which is true in the case of mori. Or they are simply standardized to work cross-platform, like edn and Transit. CSP and macros are simply powerful tools that can be added to many languages.

Go give these tools a try. If you like them, you might like Clojure/ClojureScript. They will be here when you're ready. And when you are ready, I must recommend LispCast Introduction to Clojure. It's a video course designed and produced to take you from zero to Clojure.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

You might also like


  1. Those libraries often contain macros which extend the base language.

Convince your boss to use Clojure

Summary: Clojure has been successfully adopted by many companies. There are many resources available by people who did the hard work of introducing Clojure to their team.

Do you want to get paid to write Clojure? Let's face it. Clojure is fun, productive, and more concise than many languages. And probably more concise than the one you're using at work, especially if you are working in a large company. You might code on Clojure at home. Or maybe you want to get started in Clojure but don't have time if it's not for work.

One way to get paid for doing Clojure is to introduce Clojure into your current job. I've compiled a bunch of resources for getting Clojure into your company.

Take these resources and do your homework. Bringing a new language into an existing company is not easy. I've summarized some of the points that stood out to me, but the resources are excellent so please have a look yourself.

The Strategy

Before you begin your quest to introduce Clojure, you're going to need a good strategy. By far the best presentation of a strategy is by Neal Ford. Neal Ford is a Director at ThoughtWorks and has a great strategy for introducing Clojure into an existing company. Watch this video.

  1. Spread Clojure outside of the company.
  2. Get a groundswell of people inside the company.
  3. Use Clojure for things it's great at.
  4. Get the Clojure jar file included.

If your company happens to be using Ruby, Joshua Ballanco has some great tips for How to Sneak Clojure Into Your Rails Shop.

Sean Corfield helped move a sizeable legacy application to Clojure. He's got some good, sobering advice.

  1. Be ready to explain the lack of a framework.
  2. OOP habits are ingrained.
  3. Don't underestimate the difficulty.

A lot of great advice from someone who's actually done it.

  1. Find allies.
  2. Answer the questions.
  3. Take responsibility.
  4. Get help.
  5. Be an advocate.

Some great advice from Logan Campbell, someone who convinced his coworkers to use Clojure at a Post Office.

  1. Be positive: if they say "We need static typing", say "Great! Clojure has that!" (which it does with Typed Clojure).
  2. Show them working code.
  3. Be ready for performance questions.

Material for other developers

Prismatic has been using Clojure to great success. They've written about how Clojure is used throughout their stack. This is a great introduction to answer the question "Why Clojure?". Spread this post whenever anyone asks why?.

Leo Polovets polled Clojurists at Factual and summarized their answers to Why Clojure?.

A while ago, this great post was trending on Hacker News. It explains Why Clojure?.

Material for the project manager

If you'd like a high-level overview of the business advantages to using Clojure, you can do a lot worse than asking Cognitect, the company that develops Clojure itself. They've published a case study, meant for non-tech folks, to understand the implications. It's focused mainly on Datomic, but it touches on Clojure.

Though a little hyperbolic, this post is a good one for the skeptical manager, the one who wonders whether their team can really learn a new technology quickly enough to justify the cost.

For those who like to follow industry trends and what others are recommending, look no further than the ThoughtWorks Technology Radar. It's a compendium of recommendations, published regularly, that takes a realistic view of a constantly changing landscape. Clojure has been rated at Adopt since October 2012. A lot of secondary Clojure technologies are also on the radar, including core.async, om, Datomic, and ClojureScript.

Documentation, Training, Support

I mainly want to show that there's plenty out there and plenty of new stuff coming out, not recommend anything specific.

Books

There's plenty going on with documentation. There are many books available on Amazon. These are all of the same quality as any enterprise Java book.

Videos

Videos are an up-and-coming type of training, but there's plenty out there.

Training

Besides these courses, there are often Clojure courses before or after the Clojure conferences.

Support

Besides the normal IRC (#clojure on freenode), Google Group, and Jira, Cognitect offers support.

Conclusion

Clojure is gaining traction. It's fun, it's productive. But it's still a little fringe in larger companies. Though it will still take a lot of work, these resources should help you make a case for Clojure. It's my mission to help people thrive with Clojure. If you'd like to keep up to date on what's happening in the Clojure world, you may be interested in getting the Clojure Gazette for free. Sign up here.

For more inspiration, history, interviews, and trends of interest to Clojure programmers, get the free Clojure Gazette.

Learn More

Clojure pulls in ideas from many different languages and paradigms, and also from the broader world, including music and philosophy. The Clojure Gazette shares that vision and weaves a rich tapestry of ideas from the daily flow of library releases to the deep historical roots of computer science.

You might also like

Just Hack Something Together

Summary: Lisp is viewed as difficult and academic but it is a great language for hacking a solution together.

I used to be in a carpool with other developers. We would drive an hour to work each way. And sometimes we'd talk about programming languages. Once I was talking about why Common Lisp wasn't more popular. Someone answered "people just want to hack something together".

That took a while to unpack and it felt like many conversations I had with that group. Without exploding all of the context, I'll just say that he meant that Common Lisp was less amenable to quickly building a solution than Java1. That notion caused my head to spin and I didn't know how to respond.

So I'll respond here, now (5 or 6 years later!). Lisp was made to explore solutions quickly. Lisp is often seen as a difficult, academic language. But Lisp has several things that make it a beautiful language for hacking a solution together.

1. The REPL

You can type code in and it will run. Right there. The entire language is available without having to open a text file. You can experiment on a solution, testing as you go. This is much faster than having to write a main method.

2. Incremental compilation

You can redefine a function. Any code that calls that function will now use the new definition. You no longer have the edit-compile-run cycle. You can make changes and fixes much faster.

3. Data structures

Even the humble Lisp list is better than most of what Java gives you. Lists can represent linear data, trees, or associative data. But more importantly, the interface is powerful. There are many operations built to use lists, so you're compounding effectiveness on top of everything built in. And if you bring in Clojure's data structures, there's no comparison.

In Java, you can use Lists and Maps. But most of the time, people will create a new class to represent their data. And making a new class means you have to write all of the methods. And that is definitely slower than using an existing data structure with existing methods.

4. Less code

Lisp is less verbose than Java. If you're trying to go quickly, one limiting factor is how fast you can type the code in.

I purposefully left out a lot of the features of Lisp that make it great because some of them are considered "advanced". I want to keep the list down to those basic things that you're going to have to use in order to use the language. The "advanced" features require expertise in order to be more efficient at them. You can be more effective in Lisp with a small cheatsheet of syntax and functions.

I think it's pretty clear that Lisp is superior to Java for "hacking". It was developed to experiment with programs interactively. And it serves that purpose very well. If you think you'd like to learn a modern Lisp, I suggest learning from LispCast Introduction to Clojure. It's a great video series that takes you from zero knowledge to a deep understanding of functional programming.

Learn Functional Programming using Clojure with screencasts, visual aids, and interactive exercises
Learn more

You might also like


  1. I don't mean to pick on Java, it was his language of choice. Having a comparison language merely helps the imagination