LispCast http://www.lispcast.com/ A blog about the simple joys of functional programming. en Separation of Presentation and Content http://www.lispcast.com/separation-of-presentation-and-content http://www.lispcast.com/separation-of-presentation-and-content Thu 17 Apr 2014 01:56:15 PM CDT <p>Summary: <em>One reason to separate style from content is to reuse HTML or CSS. Ultimately, we would like a solution where we can reuse both.</em></p> <h3 id="reusable-content">Reusable Content</h3> <p>There is an economic reason to separate presentation from content. Publishers have thousands of pages of HTML on their site, yet they want to enhance the style of their pages over time. It would <strong>cost a lot of money to change every single page to match their new style</strong>. So they invest a little more time writing each page so that the HTML markup does not refer to styles but to the semantics of the content (referred to as <em>semantic HTML</em>). Then they hire a designer to write CSS to make their existing content look new. The HTML is permanent and reusable, and the CSS is temporary and not-reusable. <strong>The separation is only one way: the HTML doesn't know the CSS, but the CSS does know the HTML.</strong></p> <p><strong>Examples:</strong> <a href="http://www.csszengarden.com/">CSS Zen Garden</a>, newspaper websites, blogs</p> <p><strong>Characteristics:</strong> Semantic markup, CSS tailored to classes/structure of HTML</p> <h3 id="reusable-styles">Reusable Styles</h3> <p>Yet another economic reason is a relatively newer phenomenon. It has become very easy to create a new web site/application. Writing (or generating) lots of HTML is cheap, and it changes often during iterative development. What is <strong>relatively expensive is to <em>design</em> each of those pages each time the pages change</strong>. CSS is not good at adapting to page structure changes. So people have built CSS frameworks where the CSS is (relatively) permanent and the HTML is temporary. In these cases, <strong>the HTML knows the CSS, but the CSS doesn't know the HTML</strong>. The separation is again one way--this time the other way.</p> <p><strong>Examples:</strong> <a href="https://github.com/stubbornella/oocss/tree/master/oocss">Open Source CSS</a>, <a href="http://getbootstrap.com/">Bootstrap</a>, <a href="http://foundation.zurb.com/">Foundation</a>, <a href="http://purecss.io/">Pure</a></p> <p><strong>Characteristics:</strong> HTML tailored to classes/structure of CSS, Reusable CSS</p> <h3 id="reusable-content-and-styles">Reusable Content and Styles</h3> <p>What if a newspaper site, with millions of existing HTML pages, could cheaply take advantage of the reusable styles of frameworks like Bootstrap? That is the Holy Grail of separation of concerns. What would be required to do that?</p> <p><strong>What we really want is a two-way separation.</strong> We want HTML written in total isolation and CSS written in total isolation. We want permanent HTML and permanent CSS. How can the style and content, each developed separately, finally be brought together? The answer is simple: <strong>a third document to relate the two</strong>.</p> <p>We have already seen that <a href="http://www.lispcast.com/css-abstraction-combination">CSS is not good at <em>abstraction</em></a>. CSS cannot name a style to use it later. However, <a href="http://www.lispcast.com/less-abstraction-combination">LESS does have powerful forms of abstraction</a>. LESS has the ability to <strong>define reusable styles and apply them to HTML</strong> that did not have those styles in mind. If you put the <em>definition of reusable styles</em> in one document and the <em>application of those styles</em> in another document, you achieve true separation. And it is already happening a little bit. <strong>You can do it in your own code.</strong></p> <p>It is a bit like a software library. We put the reusable bits in the library, and their specific use in the app.</p> <p><strong>Examples:</strong> <a href="http://compass-style.org/">Compass</a>, <a href="http://semantic.gs/">Semantic Grid System</a></p> <p><strong>Characteristics:</strong> Semantic markup, Reuseable Styles, Tie-in document to relate Style to Content</p> <h3 id="conclusion">Conclusion</h3> <p>CSS preprocessors, which began as convenience tools, is actually <strong>powerful enough to solve fundamental problems with HTML and CSS</strong>. While it is still early, LESS and other CSS preprocessors, if harnessed correctly, could dramatically transform how we build and design web sites. Typography, grids and layout, and other design concerns can be used as plugable libraries. And other languages that are specifically designed to do that may emerge. <strong>What would a systematic, analytical approach to such an approach look like?</strong></p> <p><center><a href="http://www.lispcast.com/separation-of-presentation-and-content"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> Clojure Web Security http://www.lispcast.com/clojure-web-security http://www.lispcast.com/clojure-web-security Sat 05 Apr 2014 04:35:56 PM CDT <p>Summary: <em>Use the OWASP Top Ten Project to minimize security vulnerabilities in your Clojure web application.</em></p> <p>Aaron Bedra gave a very damning talk about <a href="https://www.youtube.com/watch?v=CBL59w7fXw4">the security of Clojure web applications</a>. He went so far as to say that Clojure web apps are some of the worst he has seen. You should <a href="https://www.youtube.com/watch?v=CBL59w7fXw4">watch the talk</a>. He has some good recommendations.</p> <p>One of the jobs of web frameworks is to handle security concerns inherent in the web itself. Because most Clojure programmers build their own web stack, they often <strong>fail to look at the security implications of their application</strong>. They do not protect their site from even the easiest and most common forms of vulnerabilities. These vulnerabilities are problems with the way the web works, not with the particular server technology, yet it has become the server's responsibility to mitigate the vulnerabilities. Luckily, <strong>the vulnerabilities are well-studied and there are known fixes</strong>.</p> <p>The <a href="https://www.owasp.org/index.php/Main_Page">Open Web Application Security Project (OWASP)</a> does a very good job of documenting common web vulnerabilities and providing good fixes for them. They have a project called the <a href="https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project#tab=OWASP_Top_10_for_2013">Top Ten Project</a> which every web developer should refer to regularly and use to improve the security of their app. You should also run through the <a href="https://code.google.com/p/owasp-asvs/wiki/ASVS">Application Security Verification Standard</a> checklists to audit your code. <strong>But the Top Ten should get you to understand the basics.</strong></p> <p>Warning: <em>I am not a security expert. You should do your own research. The code I present here is my own interpretation of the OWASP recommendations. It has not been audited by experts. Do your own research!</em></p> <p>Also, security is an ongoing concern. <strong>If you have any comments, suggestions, or questions, please bring them up!</strong></p> <p>Here is the Top Ten 2013 with a small breakdown and a Clojure solution, if applicable.</p> <h3 id="a1.-injectiona1"><a href="https://www.owasp.org/index.php/Top_10_2013-A1-Injection">A1. Injection</a></h3> <p>If a server accepts input from the outside and then <strong>parses and interprets that input as a scripting or query language, it is open to attack</strong>. The most common form is SQL Injection, where an input form is posted to the server, the value of that form is concatenated into a string to make a SQL statement, and then the SQL statement is sent to the database to be executed. What happens if a malicious user types in <code>&quot;'; DELETE FROM USERS;&quot;</code>?</p> <p>My preferred solution to SQL Injection in Clojure is to <strong>always use parameterized SQL statements</strong>. <code>clojure.java.jdbc</code>, supports these directly. <strong>The parameters will be escaped, making injection impossible.</strong></p> <p>Another problem is if you want to read in some Clojure data from the client, and you call <code>clojure.core/read-string</code> on it. <strong><code>read-string</code> will execute arbitrary Java constructors.</strong> For instance:</p> <pre><code>#java.io.FileWriter[&quot;myfile.txt&quot;]</code></pre> <p>This will create the file <code>myfile.txt</code> or overwrite if it already exists. Also, there is a form (called read-eval form) to execute code at read-time:</p> <pre><code>#=(println &quot;Hello, vulnerability!&quot;)</code></pre> <p><strong>Read in that string, and it will print. Any code could be in there.</strong></p> <p>The solution is to <strong>never use <code>clojure.core/read-string</code></strong>. Use <code>clojure.edn/read-string</code>, which is a well-documented format. It does not run arbitrary constructors. It has no read-eval forms.</p> <p>Summary: <em>Always use parameterized SQL and use <code>clojure.edn/read-string</code> instead of <code>clojure.core/read-string</code> on edn input.</em></p> <h3 id="a2-broken-authentication-and-session-managementa2"><a href="https://www.owasp.org/index.php/Top_10_2013-A2-Broken_Authentication_and_Session_Management">A2 Broken Authentication and Session Management</a></h3> <h4 id="authentication">Authentication</h4> <p>This is a big topic and I can't address it all here. Clojure has the <a href="https://github.com/cemerick/friend">Friend library</a>, which is the closest thing we have to a de facto standard. My suggestion is simply to <strong>read the entire Friend README</strong> and evaluate whether you should use it. This is serious stuff. Read it.</p> <h4 id="session-management">Session Management</h4> <p><strong>Ring provides a session system which is fairly good.</strong> It meets many of the <a href="https://code.google.com/p/owasp-asvs/wiki/Verification_V3">OWASP Application Security Verification Standard V3</a> requirements. But it does not handle all of them automatically. <strong>You still need code audits.</strong> For instance, if you are logging requests, OWASP recommends against logging the session key. You must ensure that the session key is added after the request is logged.</p> <p>The ASVS also recommends expiring your sessions after inactivity and also after a fixed period, regardless of activity. Ring sessions do not do this automatically (the builtin mechanism has no notion of expiration) and the <strong>default implementations of session stores will store and accept sessions indefinitely</strong>. A simple middleware will do the trick of expiring them in both cases:</p> <pre><code>(defn wrap-expire-sessions [hdlr &amp; [{:keys [inactive-timeout hard-timeout] :or {:inactive-timeout (* 1000 60 15) :hard-timeout (* 1000 60 60 2)}}]] (fn [req] (let [now (System/currentTimeMillis) session (:session req) session-key (:session/key req)] (if session-key ;; there is a session (let [{:keys [last-activity session-created]} session] (if (and last-activity (&lt; (- now last-activity) inactive-timeout) session-created (&lt; (- now session-created) hard-timeout)) (let [resp (hdlr req)] (if (:session resp) (-&gt; resp (assoc-in [:session :last-activity] now) (assoc-in [:session :session-created] session-created)) resp)) ;; expired session ;; block request and delete session {:body &quot;Your session has expired.&quot; :status 401 :headers {} :session nil})) ;; no session, just call the handler ;; assume friend or other system will handle it (hdlr req)))))</code></pre> <p>Set the <a href="https://www.owasp.org/index.php/Session_Management_Cheat_Sheet#HttpOnly_Attribute">HttpOnly attribute</a> on the session cookie. Very important for preventing stealing of session ids from XSS attacks.</p> <p><em>Do not</em> set the <a href="https://www.owasp.org/index.php/Session_Management_Cheat_Sheet#Domain_and_Path_Attributes">Domain attribute</a>, and <em>do</em> set the Path if you want something more restrictive than <code>/</code> (the Ring session default).</p> <p><em>Do not</em> set the <a href="https://www.owasp.org/index.php/Session_Management_Cheat_Sheet#Expire_and_Max-Age_Attributes">Expire and Max-Age</a> attributes. Setting them makes the browser store the session id on disk, which simply expands the number of ways an attacker can get ahold of it.</p> <p><a href="https://www.owasp.org/index.php/Session_Management_Cheat_Sheet#Session_ID_Name_Fingerprinting">Change the session cookie name to something utterly generic</a>, like &quot;id&quot;. You don't want to leak more information than necessary about how your sessions work.</p> <p>Use HTTPS if you can and set the <a href="https://www.owasp.org/index.php/Session_Management_Cheat_Sheet#Secure_Attribute">Secure</a> attribute of the cookie.</p> <p>Do not use in-cookie sessions. In-memory are good but they can't scale past one machine. <a href="https://github.com/ptaoussanis/carmine"><code>carmine</code></a> has a redis-based session implementation.</p> <p>Summary: <em>Here's how I use Ring sessions (with <code>carmine</code>) based on these OWASP recommendations.</em></p> <pre><code>(session/wrap-session (wrap-expire-sessions handler {:inactive-timeout 500 :hard-timeout 3000}) {:cookie-name &quot;id&quot; :store (taoensso.carmine.ring/carmine-store redis-db {:expiration-secs (* 60 60 15) :key-prefix &quot;&quot;}) ;; leak nothing! :cookie-attrs {:secure true :httponly true}})</code></pre> <h3 id="a3-cross-site-scripting-xssa3"><a href="https://www.owasp.org/index.php/Top_10_2013-A3-Cross-Site_Scripting_(XSS)">A3 Cross-Site Scripting (XSS)</a></h3> <p>Whenever text from one user is shown to another user, there is the potential for injecting code (HTML, JS, or CSS) that is run in the victim's browser. Imagine if Facebook allowed any HTML in the post submission form. A malicious user could add a <code>&lt;script&gt;</code> tag with some keystroke logging code. <strong>Anybody who viewed that post in their feed would also get the key logger installed.</strong> That would be bad.</p> <p>XSS is common because of how easy it is to make an app that stores user input (from a form post) in a database, then constructs the page out of stuff from the database. If you're not extremely careful, <strong>you could create a place where people can exploit each other</strong>.</p> <p>The solution is to <strong>only use scrubbed or escaped values to build HTML pages</strong>. Because HTML pages can include different languages (HTML, CSS, JS), text needs to be scrubbed differently in each context. OWASP has a <a href="https://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheet">set of rules to follow which will guarantee XSS prevention</a>.</p> <p><code>hiccup.util/escape-html</code> (also aliased as <code>hiccup.core/h</code>) will escape all dangerous HTML characters into HTML entities. JS and CSS still need to be handled, and rules for HTML attributes need to be followed.</p> <p>If you want to allow some HTML elements, you will need to do a complex scrub. Luckily, <strong>Google has a <a href="https://code.google.com/p/owasp-java-html-sanitizer/wiki/GettingStarted">nice Java library that sanitizes HTML</a></strong>. Use it.</p> <p>Summary: <em>Validate and scrub input from the user and scrub/escape text on output.</em></p> <h3 id="a4-insecure-direct-object-referencesa4"><a href="https://www.owasp.org/index.php/Top_10_2013-A4-Insecure_Direct_Object_References">A4 Insecure Direct Object References</a></h3> <p>This one is a biggie: <strong>each handler has to do authentication</strong>. Does the particular logged in user have access to the resources requested? There's no way to automate this with a middleware. But having <em>some</em> system is better than doing it ad hoc each time. Remember: an attacker can construct any URL, including URLs with a database key in it. Don't assume that just because a request contains a key, the user must have the rights to it.</p> <p>Summary: <em>Always check the authority of the requesting session before performing an action.</em></p> <h3 id="a5-security-misconfigurationa5"><a href="https://www.owasp.org/index.php/Top_10_2013-A5-Security_Misconfiguration">A5 Security Misconfiguration</a></h3> <p>This is about keeping your software up to date and making sure the settings of all software makes sense.</p> <h3 id="a6-sensitive-data-exposurea6"><a href="https://www.owasp.org/index.php/Top_10_2013-A6-Sensitive_Data_Exposure">A6 Sensitive Data Exposure</a></h3> <p>Having data is risky. Don't let it leak out.</p> <h3 id="a7-missing-function-level-access-controla7"><a href="https://www.owasp.org/index.php/Top_10_2013-A7-Missing_Function_Level_Access_Control">A7 Missing Function Level Access Control</a></h3> <p>Use an authorization system (<a href="https://github.com/cemerick/friend">Friend</a>) and audit the roles used for access control.</p> <h3 id="a8-cross-site-request-forgery-csrfa8"><a href="https://www.owasp.org/index.php/Top_10_2013-A8-Cross-Site_Request_Forgery_(CSRF)">A8 Cross-Site Request Forgery (CSRF)</a></h3> <p>Let's imagine you have a bank account at Bank of Merica. You just checked your balance and didn't log out. Then you go to some public forum, where someone has posted a cool file. There's a big download button. You click it, and the next thing you know, you're on your bank page and all of your money has been transfered out of your account.</p> <p>What happened?</p> <p>The download button said &quot;Download&quot; but <strong>it was really a form submit button</strong>. The form had hidden fields &quot;to-account&quot;, and &quot;amount&quot;. The action of the form was &quot;http://www.bankofmerica.com/transfer-money&quot;. By clicking that button, <strong>the form was posted to the bank, and because you were just logged in, oops, it transfered all your money away</strong>.</p> <p>The solution is that you only want to <strong>accept form posts that come directly from your site</strong>, which you control. You don't want some random person to convince people to click on other sites to be able to transfer people's money like that.</p> <p>There are several possible solutions. One approach is to add a secret to the session and also insert that secret into every form. That is the approach taken by the <a href="https://github.com/weavejester/ring-anti-forgery">ring-anti-forgery</a> library.</p> <p>The solution that I like is to do a <a href="https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet#Double_Submit_Cookies">double-submit</a>. This means you submit a secret token in the cookie (sent with each web request) and in a hidden field in the form. The server confirms that the cookie and the hidden field match. But the hidden field in the form is added by a small Javascript script which reads it from the cookie. <strong>Browsers don't allow Javascript to read cookies from other sites, so you guarantee that they form was posted from your site.</strong></p> <p>There are three parts to the solution.</p> <ol style="list-style-type: decimal"> <li>Install a secret token as a cookie.</li> <li>Install a script to add the hidden field to all forms.</li> <li>Check that the field matches the cookie on POSTs.</li> </ol> <p>Here is some code to do 1 and 3.</p> <pre><code>(defn is-form-post? [req] (and (= :post (:request-method req)) (let [ct (get-in req [:headers &quot;content-type&quot;])] (or (= &quot;application/x-www-form-urlencoded&quot; ct) (= &quot;multipart/form-data&quot; ct))))) (defn csrf-tokens-match? [req] (let [cookie-token (get-in req [:cookies &quot;csrf&quot;]) post-token (get-in req [:form-params &quot;csrf&quot;])] (= cookie-token post-token))) (defn wrap-csrf-cookie [hdlr] (fn [req] (let [cookie (get-in req [:cookies &quot;csrf&quot;] (str (java.util.UUID/randomUUID)))] (assoc-in (hdlr req) [:cookies &quot;csrf&quot;] cookie)))) (defn wrap-check-csrf [hdlr] (fn [req] (if (is-form-post? req) (if (csrf-tokens-match? req) ;; we&#39;re safe (hdlr req) ;; possible attack {:body &quot;CSRF tokens don&#39;t match.&quot; :status 400 :headers {}}) ;; we don&#39;t check other requests (hdlr req))))</code></pre> <p>The Javascript should be something like this:</p> <pre><code>(def csrf-script &quot;(function() { var cookies = document.cookie; var matches = cookies.match(/csrf=([^;]*);/); var token = matches[1]; $(&#39;form&#39;).each(function(i, form) { if(form.attr(&#39;method&#39;).toLowerCase() === &#39;post&#39;) { var hidden = $(&#39;&lt;input /&gt;&#39;); hidden.attr(&#39;type&#39;, &#39;hidden&#39;); hidden.attr(&#39;name&#39;, &#39;csrf&#39;); hidden.attr(&#39;value&#39;, token); form.append(hidden); } }) }());&quot;)</code></pre> <p>You should add it to all HTML pages. Note that this example script requires jQuery. Put it right before the <code>&lt;/body&gt;</code>.</p> <pre><code>[:script csrf-script]</code></pre> <p>The nice thing about this solution is that it is <strong>strict by default</strong>. If you don't include the script, form posts won't work (assuming <code>wrap-check-csrf</code> is in your middleware stack).</p> <p>Summary: <em>CSRF attacks take advantage of properties of the browser (instead of properties of your server), so their defense can largely be automated.</em></p> <h3 id="a9-using-components-with-known-vulnerabilitiesa9"><a href="https://www.owasp.org/index.php/Top_10_2013-A9-Using_Components_with_Known_Vulnerabilities">A9 Using Components with Known Vulnerabilities</a></h3> <p>Software with known vulnerabilities is easily attacked using scripts. You should ensure that all of your software is up-to-date.</p> <h3 id="a10-unvalidated-redirects-and-forwardsa10"><a href="https://www.owasp.org/index.php/Top_10_2013-A10-Unvalidated_Redirects_and_Forwards">A10 Unvalidated Redirects and Forwards</a></h3> <p>One common pattern for login workflow is to have a query parameter that contains the url to redirect to. Since it's a user parameter, <strong>it's open to the world and could be a doorway for attackers</strong>.</p> <p>For example, let's say someone sends an email to someone asking them to log in to their bank account. In it, there's this link:</p> <pre><code>http://www.bankofmerica.com/login?redirect=http://attackersite.com</code></pre> <p>What happens when they click? They see the legitimate site of their bank, which they trust. But it redirects them to the attacker's site, which has been designed to look like the bank site. <strong>The user might miss this change of domains and unwittingly reveal private information.</strong></p> <p>What can you do?</p> <p>OWASP recommends never performing redirects, which is impractical. The next best thing is to never base the redirect on a user parameter. This would work, but puts a lot of trust in the developers and security auditors to check that the policy is enforced. <strong>My preferred solution allows redirects that conform to a whitelist of patterns.</strong></p> <pre><code>(def redirect-whitelist [#&quot;https://www.bankofmerica.com/&quot; ;; homepage #&quot;https://www.bankofmerica.com/account&quot; ;; account page ... ]) (defn wrap-authorized-redirects [hdlr] (fn [req] (let [resp (hdlr req) loc (get-in resp [:headers &quot;Location&quot;])] (if loc (if (some #(re-matches % loc) redirect-whitelist) ;; redirect on our whitelist, it&#39;s ok! resp ;; possible attack (do ;; log it (warning &quot;Possible redirect attack: &quot; loc) ;; change redirect back to home page (assoc-in resp [:headers &quot;Location&quot;] &quot;https://www.bankofmerica.com/&quot;))) resp))))</code></pre> <p>Summary: <em>Redirect attacks can largely be avoided by checking the redirect URL against a whitelist.</em></p> <h3 id="conclusion">Conclusion</h3> <p>Web security is hard. It takes education and vigilance to keep our servers secure. Luckily, the main security flaws of the web are well-understood and well-documented. However, this is only half of the work. These need to be translated into Clojure either as libraries and simply as &quot;best practices&quot;. Further, these libraries and practices need to be discussed and kept top-of-mind.</p> <p>If programming the web in Clojure interests you, you might be interested in my <a href="http://www.purelyfunctional.tv/web-dev-in-clojure">Web Development in Clojure</a> video series. It covers all of the basics of web development, building a foundation to understand the entire Clojure web stack.</p> <div class="homepage-offer-box"> <a class="homepage-offer-box-link" href="http://purelyfunctional.tv/web-dev-in-clojure"> <img class="homepage-offer-image" src="http://purelyfunctional.tv/img/web-dev-shot.png"> <div class="homepage-offer-box-title"> LispCast Web Development in Clojure </div> </a> </div> <p><center><a href="http://www.lispcast.com/clojure-web-security"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> A Ring Spec to Hang on the Wall http://www.lispcast.com/ring-spec-hang-on-wall http://www.lispcast.com/ring-spec-hang-on-wall Fri 28 Mar 2014 09:16:16 PM CDT <p>Summary: <em>The Ring SPEC is the core of the Clojure web ecosystem. The standard is small and <a href="http://www.lispcast.com/files/ring_spec_1_1.pdf">a reference</a> is handy.</em></p> <p>If you program the web in Clojure, you probably use Ring. Even if you don't, your server is likely <a href="http://http-kit.org/">Ring compatible</a>.</p> <p>Ring has a small SPEC. It's centered around defining the keys one can expect in the request and response maps. And the exact names for keywords are easy to forget.</p> <p>I don't want to forget. I use Ring often enough that I want a quick reference. A while ago, I printed out a quick summary of the keys for the request and response maps and hung it on the wall behind my monitor. I refer to it frequently.</p> <p>If you program the web in Clojure, you might appreciate this printout. If you're learning, it could be an invaluable reference. <a href="http://www.lispcast.com/files/ring_spec_1_1.pdf">Please download the PDF.</a></p> <p>And if you like it, you might also like my <a href="http://www.purelyfunctional.tv/web-dev-in-clojure">Web Development in Clojure video series</a>.</p> <p><center><a href="http://www.lispcast.com/ring-spec-hang-on-wall"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> What Web Framework Should I Use in Clojure? http://www.lispcast.com/what-web-framework-should-i-use http://www.lispcast.com/what-web-framework-should-i-use Sun 23 Mar 2014 05:03:04 PM PDT <p>Summary: <em>There are a number of web frameworks in Clojure, but beginners should roll their own server stack themselves to tap into the Ring ecosystem.</em></p> <p>One question that I am asked a lot by beginners in Clojure is <strong>&quot;What web framework should I use?&quot;</strong> This is a good question. In Python, there's Django. In PHP, Drupal. And of course in Ruby, there's the king of all web frameworks, Ruby on Rails.</p> <p>What framework should you use in Clojure? The question is actually kind of hard to answer. There are a number of web frameworks out there. Some people call <a href="https://github.com/weavejester/compojure">Compojure</a> a framework, though it is really a library. <a href="https://github.com/noir-clojure/lib-noir">lib-noir</a> does a lot of work for you. Then there's your true frameworks, like <a href="https://github.com/pedestal/pedestal">Pedestal</a> or <a href="http://hoplon.io/">Hoplon</a>, which provide infrastructure and abstractions for tackling web development. All of these projects are great, but for a beginner, <strong>I have to recommend building your own web stack starting with Ring</strong>.</p> <p>Compojure is really just a routing library, not a framework. You can use it for your routing needs, though there are alternatives, such as <a href="https://github.com/ericnormand/playnice">playnice</a>, <a href="https://github.com/juxt/bidi">bidi</a>, <a href="https://github.com/clojurewerkz/route-one">Route One</a>, and <a href="https://github.com/thatismatt/gudu">gudu</a>. <strong>If you don't want to decide, use Compojure. It's widely used and works great.</strong> If you want to go in depth, read the docs for the others. They are each good for different cases.</p> <p><a href="https://github.com/noir-clojure/lib-noir">lib-noir</a> comes from <a href="http://www.webnoir.org/">Noir</a>, which was a web framework (now deprecated). It was easy and provided a bit of plumbing already built for you, so you could just start a project with a lot of the infrastructure built in. lib-noir is that infrastructure in library form. I haven't used it, but a lot of people like it. However, when I look at it, I see that <strong>most of what it provides I either won't use or it is trivial to add myself</strong>. That would normally be ok if there was huge adoption for it (like with Rails) so you get an ecosystem effect, but there isn't. <strong>lib-noir is used but certainly not dominant.</strong></p> <p><a href="https://github.com/pedestal/pedestal">Pedestal</a> has a lot of backing. It aims to tackle single-page apps by providing a sane front-end environment using ClojureScript in the form of a message queue. If you're into &quot;real-time apps&quot;, this may be for you. Though, I would caution you that it's not for a Clojure beginner. <strong>Pedestal introduces a lot of new concepts that even experienced Clojure programmers have to learn.</strong> The <a href="https://github.com/pedestal/app-tutorial/wiki">tutorial</a> is long and arduous. You will have problems learning Pedestal without knowing Clojure.</p> <p><strong>Update:</strong> <em><a href="https://groups.google.com/forum/#!topic/pedestal-users/jODwmJUIUcg">Pedestal has changed dramatically</a> since I last looked at it. It is no longer a frontend framework. It is now a high-performance backend server for high performance asynchronous processing. It's worth looking at if you need that. Otherwise, I stick with my recommendation to use basic Ring.</em></p> <p><a href="http://hoplon.io/">Hoplon</a> is also designed for web apps. It gives you a DOM written in ClojureScript (including custom components), dataflow programming (like a spreadsheet), and client-server communication. It's a bold step, but again, <strong>requires you to buy into programming models that will take a long time to understand</strong>. If you are not already familiar with Clojure, you are asking for trouble.</p> <p>There are other frameworks out there. But I recommend rolling your own stack. If you're learning Clojure, <strong>the best way to grasp how web apps work is to get a <a href="https://github.com/ring-clojure/ring">Ring</a> Jetty adapter set up with some basic handlers</strong>. Add existing middleware as you need it. Write some middleware of your own. Use Compojure to route. Use <a href="https://github.com/weavejester/hiccup">Hiccup</a> to generate HTML. <strong>That setup will get you a long way.</strong></p> <p>Ring is just functions. With a few basic concepts and a copy of the Ring SPEC handy, you can build a web server very quickly that does exactly what you want and <strong>you can understand every aspect of it</strong>. The experience of building one yourself can teach you a lot about how the other frameworks are put together.</p> <p>What's more, Ring <em>is</em> dominant. Most people write functionality (in the form of middleware and handlers) assuming Ring and no more. So by staying close to the metal, you are <strong>tapping into a huge resevoir of pre-written libraries that are all compatible with each other</strong>. Ring is the locus for the Clojure web ecosystem.</p> <p>Wiring up your own middleware stack is not that daunting. If you want guidance, <strong>my <a href="http://www.purelyfunctional.tv/web-dev-in-clojure">Web Development in Clojure video series</a> is now for sale</strong>. It starts with a brand new Clojure project and ends with a fully functional app, backed by a database, and hosted on Heroku (for free!). In one hour, <strong>it explains all the concepts and shows you lots of examples</strong>. There's lots of exercises to get your brain whirring.</p> <p>Recommended stack:</p> <ul> <li><a href="https://github.com/ring-clojure/ring">Ring</a> (with Jetty adapter) <ul> <li><a href="http://ring-clojure.github.io/ring/ring.middleware.params.html#var-wrap-params">wrap-params</a> (parsing params)</li> <li><a href="https://github.com/ring-clojure/ring/blob/master/ring-devel/src/ring/middleware/reload.clj">wrap-reload</a> (for development)</li> <li><a href="http://ring-clojure.github.io/ring/ring.middleware.resource.html#var-wrap-resource">wrap-resource</a> and <a href="http://ring-clojure.github.io/ring/ring.middleware.file-info.html#var-wrap-file-info">wrap-file-info</a> (static files)</li> </ul></li> <li><a href="https://github.com/weavejester/compojure">Compojure</a></li> <li><a href="https://github.com/weavejester/hiccup">Hiccup</a></li> </ul> <p>Then keep adding and customizing.</p> <p><center><a href="http://www.lispcast.com/what-web-framework-should-i-use"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> When To Use a Macro in Clojure http://www.lispcast.com/when-to-use-a-macro http://www.lispcast.com/when-to-use-a-macro Fri 07 Mar 2014 03:45:07 PM CST <p>Summary: <em>Macros should be avoided to the extent possible. There are three circumstances where they are required.</em></p> <p>There's a common theme in Lisp that you should only use macros when you need them. <strong>It is very common to see a new lisper overuse macros.</strong> I did it myself when I first learned Lisp. They are very powerful and make you the king of syntax.</p> <p>Clojure macros do have their uses, but why should you avoid them if possible? The principle reason is that <strong>macros are not first-class in Clojure</strong>. You cannot access them at runtime. You cannot pass them as an argument to a function, nor do any of the other powerful stuff you've come to love from functional programming. In short, <strong>macros are not functional programming</strong> (though they can make use of it).</p> <p>A function, on the other hand, is a first-class value, and so is available for awesome functional programming constructs. <strong>You should prefer functions to macros.</strong></p> <p>That said, macros are still useful because there are things macros can do that functions cannot. <strong>What are the powers of a macro that are unavailable to any other construct in Clojure?</strong> If you <em>need</em> any of these abilities, write a macro.</p> <h3 id="the-code-has-to-run-at-compile-time">1. The code has to run at compile time</h3> <p>There are just some things that need to happen at compile time. I recently wrote a macro that returns the hash of the current git commit so that the hash can be embedded in the ClojureScript compilation. This needs to be done at compile time because the script will be run somewhere else, where it cannot get the commit hash. Another example is <strong>performing expensive calculations at compile time as an optimization</strong>.</p> <p>Example:</p> <pre><code>(defmacro build-time [] (str (java.util.Date.)))</code></pre> <p>The <code>build-time</code> macro returns a String representation of the time it is run.</p> <p><strong>Running code at compile time is not possible in anything other than macros.</strong></p> <h3 id="you-need-access-to-unevaled-arguments">2. You need access to unevaled arguments</h3> <p>Macros are useful for writing <strong>new, convenient syntactic constructs</strong>. And when we talk about syntax, we are typically talking about raw, <strong>unevaluated sexpressions</strong>.</p> <p>Example:</p> <pre><code>(defmacro when &quot;Evaluates test. If logical true, evaluates body in an implicit do.&quot; {:added &quot;1.0&quot;} [test &amp; body] (list &#39;if test (cons &#39;do body)))</code></pre> <p><code>clojure.core/when</code> is a syntactic sugar macro which transforms into an <code>if</code> with a <code>do</code> for a <em>then</em> and no <em>else</em>. The <code>body</code> should not be evaled before the <code>test</code> is checked.</p> <p>Getting access to the unevaluated arguments is available by quoting (<code>'</code> or <code>(quote ...)</code>), but that is <strong>often unacceptable for syntactic constructs</strong>. Macros are the only way to do that.</p> <h3 id="you-need-to-emit-inline-code">3. You need to emit inline code</h3> <p>Sometimes calling a function is unacceptable. That call is either too expensive or is otherwise not the behavior you want.</p> <p>For instance, in Javascript in the browser, you can call <code>console.log('msg')</code> to print out a message and the line number to the console. In ClojureScript, this becomes something like this: <code>(.log js/console &quot;msg&quot;)</code>. Not convenient at all. <strong>My first thought was to create a function.</strong><sup><a href="#fn1" class="footnoteRef" id="fnref1">1</a></sup></p> <pre><code>(defn log [msg] (.log js/console msg))</code></pre> <p>This worked alright for printing the message, but the line numbers were all pointing to the same line: the body of the function! <code>console.log</code> records the line exactly where it is called, so <strong>it needs to be inline</strong>. I replaced it with a macro, which highlights its purpose as syntactic sugar.</p> <p>Example:</p> <pre><code>(defmacro log [msg] `(.log js/console ~msg))</code></pre> <p>The body replaces the call to log, so <strong>it is located where it is needed for the proper behavior.</strong></p> <p>If you need <strong>inline code</strong>, a macro is the only way.</p> <h3 id="other-considerations">Other considerations</h3> <p>Of course, <strong>any combination of these is also acceptable</strong>. And don't forget that although you might need a macro, macros are only available at compile time. So you should consider <strong>providing a function that does the same thing</strong> and then wrap it with a macro.</p> <h3 id="conclusion">Conclusion</h3> <p>Macros are very powerful. Their power comes with a price: they are only available at compile time. Because of that, functions should be preferred to macros. The use of macros should be reserved for those special occasions when their power is needed.</p> <div class="footnotes"> <hr /> <ol> <li id="fn1"><p>As it should be :)<a href="#fnref1">↩</a></p></li> </ol> </div> <p><center><a href="http://www.lispcast.com/when-to-use-a-macro"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> LESS has Better Forms of Abstraction than CSS http://www.lispcast.com/less-abstraction-combination http://www.lispcast.com/less-abstraction-combination Wed 12 Feb 2014 10:16:42 PM CST <p>Summary: <em>LESS has obviously better forms of abstraction and combination than CSS. It has recursive style definitions, which is enough to consider it a &quot;powerful language&quot;.</em></p> <p>Ok, it's obvious that <a href="http://www.lispcast.com/css-abstraction-combination">CSS has weak forms of combination and abstraction</a>. But now we have a good framework for understanding <em>why</em>. &quot;CSS Preprocessors&quot;, as they are called, are getting really popular now. We would be smart to <strong>analyze LESS in the same way that we analyzed CSS</strong>, if only to temper the glamor of trendiness that surrounds it. <a href="http://twitter.com/ericnormand">Comments are welcome.</a></p> <p>Because LESS aims to be a superset of CSS, it has all of <a href="http://www.lispcast.com/css-abstraction-combination">the primitive expressions, means of combination, and means of abstraction that come baked into CSS</a>. I already went over those last time, so I will not go over them again. <strong>So what things are added by LESS?</strong></p> <h3 id="primitive-expressions">Primitive Expressions</h3> <p>Besides existing CSS properties, <strong>LESS adds two new primitive expressions.</strong> <em>Mixin application</em> (<code>.rounded-corners(10px);</code>) in a rule <strong>recursively applies the primitive expressions defined in the body of the mixin to the current rule</strong>. Mixin applications can be parameterized with value expressions, or they can have no parameters. <em>Extension</em> (<code>&amp;:extend(.blue-button);</code>) is similar, but instead of applying the primitive expressions to a rule body, it adds the selector of the rule to the rule selector of the extension. <strong>Extension is recursive as well.</strong></p> <p>Variables and mathematical expressions change the way primitive properties work. In CSS, primitive properties were comprised of a property name and a literal property value. In LESS, variables and math expressions, as well as literal values, can be in the value place (right hand side) of a property.</p> <p>Variables can also be used in selectors.</p> <h3 id="means-of-combination">Means of Combination</h3> <p><strong>The principle means of combination are still the rule</strong>, but add to it the <strong>ability to nest rules</strong>, and things are more interesting. Nesting two rules is shorthand for writing out two rules (unnested) with a nested selector. While in the simple case it is simply a shorthand, <strong>when nested rules are applied as mixins, you gain a lot more than better syntax</strong>. Mixins with nested subrules allows you to <em>name</em> a nesting and refer to it later.</p> <h3 id="means-of-abstraction">Means of Abstraction</h3> <p>CSS did not contain much in the way of abstraction. <strong>LESS focuses primarily in the realm of abstraction</strong>, probably to appease the will to power of front-end designers. <em>Variables</em> allow property values to be named, and naming is a form of abstraction. <strong>Variables are a good way to name values that all have the same meaning and would therefore change at the same time.</strong> For instance, a shade of green that is used throughout the styles is a perfect use for variables. Variables can be used in a similar way to name selectors.</p> <p>A more powerful form of abstraction comes from the ability to define mixins, apply mixins, and use of <code>:extend()</code>. In LESS, any rule using a single class or id selector can be used as a mixin. This is essentially <strong>a way to name a rule</strong>--our principle form of combination. In addition, if you put empty parentheses after the class selector in the rule, the rule is not outputted into the generated CSS, which can save bytes. Mixins can also have parameters (scoped variables), so they can be <strong>abstracted over a variety of values</strong>. Extend allows a similar kind of abstraction which promises to be more efficient.</p> <p>Mixins are very powerful. In fact, <strong>this is the kind of abstraction that is needed for LESS to be powerful</strong>, as defined by Abelson and Sussman. Because you can now name a group of styles (mixin) and then use that name in another group of styles (mixin application), <strong>LESS has full-on recursive style definitions</strong>. With extension, it also has recursive selector definitions. In LESS, we can talk of &quot;levels of abstraction&quot; whereas in CSS there was only one.</p> <h3 id="conclusion">Conclusion</h3> <p><strong>LESS has recursion</strong>. It lets you define and name groups of properties, then refer to those groups by name in other groups of properties. We can consider LESS powerful enough to express useful abstractions. Yet though it is more powerful than CSS, it still has many of the problems of CSS (especially complex rules governing the combination of multiple rules to a single element). How can LESS be leveraged to gain its power but tame its weakness? <strong>Is there a subset of LESS that can gerrymander the good parts away from the bad parts?</strong></p> <p><center><a href="http://www.lispcast.com/less-abstraction-combination"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> CSS has Weak Forms of Abstraction and Combination http://www.lispcast.com/css-abstraction-combination http://www.lispcast.com/css-abstraction-combination Sat 08 Feb 2014 01:44:44 PM CST <p>Summary: <em>According to the requirements proposed by Abelson and Sussman, CSS does not provide adequate means of combination and abstraction to be considered a powerful language.</em></p> <p>I am trying to improve the maintainability and reusability of my CSS over the longterm. I've written about <a href="http://www.lispcast.com/cascading-separation-abstraction">how to organize CSS</a> before. I've learned a lot since I wrote that. I've tried lots of things and talked to lots of people, I finally seem to have found a conceptual framework to capture my new understanding. I'm trying to explore it here. <a href="http://twitter.com/ericnormand">Comments are welcome.</a></p> <p>I'm going to take a cue from the first page of <a href="http://mitpress.mit.edu/sicp/">SICP</a> and analyze CSS as a language.</p> <p>Abelson and Sussman in <a href="http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-10.html#%_sec_1.1">SICP 1.1</a> (<em>italics</em> mine):</p> <blockquote> <p>A powerful programming language is more than just a means for instructing a computer to perform tasks. The language also serves as <em>a framework within which we organize our ideas</em> about processes. Thus, when we describe a language, we should pay particular attention to <em>the means that the language provides for combining simple ideas to form more complex ideas</em>. <em>Every powerful language has three mechanisms</em> for accomplishing this:</p> <p><strong>primitive expressions</strong>, which represent the simplest entities the language is concerned with,</p> <p><strong>means of combination</strong>, by which compound elements are built from simpler ones, and</p> <p><strong>means of abstraction</strong>, by which compound elements can be named and manipulated as units.</p> </blockquote> <p>Let's analyze CSS in terms of these three mechanisms.</p> <h3 id="primitive-expressions">Primitive Expressions</h3> <p>The <em>simplest entities the language is concerned with</em> are <em>properties</em> and <em>primitive selectors</em>. CSS properties, though they have a property name and property value part, are meaningless if split up. Primitive selectors include element name selectors (<code>body</code>, <code>a</code>, <code>div</code>), class name selectors (<code>.main-wrapper</code>), id selectors (<code>#login-form</code>), and pseudo-class selectors (<code>:hover</code>), among others. Properties appear inside the rule body (<code>{}</code>), while selectors appear before the rule body. <strong>The two are semantically and syntactically separated.</strong></p> <h3 id="means-of-combination">Means of Combination</h3> <p><em>Properties</em> can be combined in two ways. First, <strong>multiple properties can be put inside the same rule body</strong>. This is the most obvious and most readable form of property combination. The second form is harder to reason about. It occurs automatically within the browser during rendering. That form of combination, involving the application of multiple rule bodies to the same HTML element, uses a <strong>complex ordering of properties from all bits of CSS and element styles on the page</strong>.</p> <p>Tomes have been written about how difficult it is to reason about this automatic form of combination. Usually, the answer is <strong>limiting it (or avoiding it altogether) through programmer discipline</strong>, with varying degrees of success.</p> <p><em>Primitive selectors</em> can be combined in several ways. Without spaces between them, multiple selectors will <em>intersect</em>, meaning they target elements more specifically. <code>div.main-container</code> will target <code>div</code> elements that ALSO have the class <code>main-container</code>.</p> <p>With spaces, multiple selectors indicate <em>nesting</em>. <code>div .main-container</code> matches any element of class <code>main-container</code> within any <code>div</code>. There are several operators which combine them in different ways (<code>&gt;</code> indicates direct nesting, etc.). Nested selectors are associated with CSS that is <strong>strongly coupled with the structure of the HTML</strong> it is styling and therefore <strong>less reusable</strong>.</p> <p>Selectors that are combined with commas create a <em>group</em>. These compound selectors will match any element that matches at least one of the component selectors. <code>header, .header</code> will match all <code>header</code> elements and all elements with class <code>header</code>.</p> <p>There are more types of selector combintation operators, but they are more specialized and less frequently used.</p> <p>The locus of combination, for both properties and selectors, is the <em>rule</em>. The rule has one compound selector and zero or more properties. Rules with zero properties have no effect.</p> <h3 id="means-of-abstraction">Means of Abstraction</h3> <p>The means of abstraction in CSS are quite limited. There is no way to name <em>anything</em>. People lament the lack of named values (often refered to as <em>variables</em>) or named styles (sometimes called <em>mixins</em>). <strong>Naming is out in CSS.</strong></p> <p>The only means of abstraction is the class and id, which are labels that can be applied to HTML elements. With an id or class (or combinations), you can target precisely the elements you need to and achieve some reuse. For instance, I can &quot;reuse&quot; the <code>#login-form</code> id selector in two different rules. I can also <strong>add the class <code>rounded-corner</code> to two different HTML elements, effectively &quot;reusing&quot; the same rule twice</strong>. By a <em>very disciplined</em> use of class selectors by combining them with commas, one can apply &quot;rule bodies&quot; as a unit in a <em>very limited way</em>, though it is <strong>impracticable in practice</strong>.</p> <p>The disadvantage to this technique of using id and class selectors is that the HTML must be modified when styles change, defeating the purpose of using CSS for content/style separation. There is a lot of discussion about using semantically named classes. For instance, call the button <code>login-button</code> instead of <code>green-shiny-button</code>. This is thought to be more robust in the face of style changes, but <strong>requires existing CSS to be thrown away</strong> in order for the page to be redesigned. <strong>CSS offers no good way to modify HTML and CSS independently.</strong></p> <h3 id="conclusion">Conclusion</h3> <p><strong>CSS does not meet the criteria for a &quot;powerful language&quot; as used in SICP.</strong> This is no surprise. The reasonable means of combination are limited to the rule. The means of abstraction are almost non-existent. There is no way to name anything. And the other form of abstraction (ids and classes) provides no way of reusing both the HTML and the CSS. <strong>It is obvious why CSS is typically unmaintainable.</strong> With the current crop of compile-to-CSS languages (commonly known as &quot;CSS Preprocessors&quot;), there is hope that better means of abstraction are possible. <strong><a href="http://www.lispcast.com/less-abstraction-combination">How will compile-to-CSS languages fare in this same analysis?</a></strong></p> <p><center><a href="http://www.lispcast.com/css-abstraction-combination"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> Hindley-Milner in Clojure http://www.lispcast.com/Hindley-Milner-in-Clojure http://www.lispcast.com/Hindley-Milner-in-Clojure Mon 13 Jan 2014 08:48:37 AM CST <p> <center> <a href="http://ro-che.info/ccc/17" title="Why static vs dynamic typing battles are rarely interesting."><img src="/img/typing.png" alt="Typing Comic from Cartesian Closed Comics" /></a> </center> </p> <p>All sarcasm aside, the above diagram has a kernel of truth. The important thing to note is that the intersection between &quot;Proponents of dynamic typing&quot; and &quot;People familiar with type theory&quot; is very small.</p> <p>In an effort to increase the size of that intersection, I decided to familiarize myself with a little more type theory. I developed an implementation of Hindley-Milner which operates on a simple Lisp-like λ-calculus with let polymorphism. Everything you need for Hindley-Milner Algorithm W.</p> <h3 id="background">Background</h3> <p>Hindley-Milner is a type system that is used by ML and Haskell. Algorithm W is a fast algorithm for inferencing Hindley-Milner which is syntax-directed (meaning it is based on the type of expression) and recursively defined. In this way, it is similar to <a href="http://www.paulgraham.com/rootsoflisp.html">Lisp's <code>eval</code></a>.</p> <h3 id="implementation">Implementation</h3> <p>I based my implementation on a paper called <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.7733&amp;rep=rep1&amp;type=pdf"><em>Algorithm W Step by Step</em></a> which implemented it in Haskell. My implementation started very similar to that implementation and diverged as I refactored it into a more Clojure-esque style.</p> <p>This implementation uses no in-place mutation, instead using a &quot;substitution style&quot; which is slightly harder to read. It is, however, easier to write and prove correct.</p> <p>In addition to the type inferencer, I wrote an interpreter for the same language, just because. My original intent was to expose (to myself) the similarities between syntax-driven type inference and <code>eval</code>. There might be some, but the full clarity I desire is yet many refactorings away. Note that the interpreter and type inferencer are completely independent, except that both apply to the same set of expressions.</p> <p>I added a couple of minor luxuries to the language having to do with currying. Writing fully parentheisized function applications for curried functions is a pain, as is writing a hand-curried function with multiple arguments. I added two syntax transformations which transform them into a more Lispy style. For example:</p> <pre><code>(fn [a b c d e f] 1) =&gt; (fn a (fn b (fn c (fn d (fn e (fn f 1))))))</code></pre> <p>and</p> <pre><code>(+ 1 2) =&gt; ((+ 1) 2)</code></pre> <p>I'm pretty sure the syntactic transformation is completely safe. All of my tests still type check.</p> <p>The final luxury is that it is a lazily-evaluated language. That's not strictly necessary, but it is strictly cool. It builds up thunks (Clojure delays and promises) and a trampoline is used to get the values out. This lets me define <code>if</code> as a function. The only special forms are <code>let</code> and <code>fn</code>.</p> <h3 id="where-to-find-it">Where to find it</h3> <p>You can find the code in the <a href="https://github.com/ericnormand/hindley-milner">ericnormand/hindley-milner</a> Github repo. I don't promise that it has no bugs. But it does have a small and growing test suite. Pull requests are welcome and I will continue to expand and refactor it.</p> <h3 id="what-i-learned">What I learned</h3> <p>Type unification is why the error messages of most type inferencers are so bad. Unification by default only has local knowledge and is commutative (<code>unify a b == unify b a</code>). No preference is given to either argument. A lot of work must have gone into making the error messages of Haskell as good as they are.</p> <p>Let polymorphism is useful and I'm glad that Haskell has it.</p> <p>Hindley-Milner is powerful, but it does not by itself work magic on a languageq. A language still requires a lot of good design and a well-chosen set of types.</p> <h3 id="your-turn">Your turn</h3> <p>I think you should implement Hindley-Milner in the language of your choice for a small toy λ-calculus. There is a lot to learn from it, even if you never program using a Hindley-Milner language. At the very least, you'll know what the fuss is about.</p> <p>If you think it would be helpful, have a look at <a href="https://github.com/ericnormand/hindley-milner">my implementation</a>. Like I said, pull requests are welcome.</p> <p><center><a href="http://www.lispcast.com/Hindley-Milner-in-Clojure"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> React: Another Level of Indirection http://www.lispcast.com/react-another-level-of-indirection http://www.lispcast.com/react-another-level-of-indirection Tue 31 Dec 2013 01:01:25 PM CST <blockquote> <p>Any problem in computer science can be solved with another level of indirection. -- David Wheeler</p> </blockquote> <p>The <a href="http://facebook.github.io/react/">React</a> library from Facebook makes DOM programming functional by using a Virtual DOM. The Virtual DOM is an <em>indirection mechanism</em> that solves the difficult problem of DOM programming: <strong>how to deal with incremental changes to a stateful tree structure</strong>. By abstracting away the statefulness, the Virtual DOM turns the real DOM into an immediate mode GUI, which is perfect for functional programming. Further, the Virtual DOM provides <strong>the last piece of the Web Frontend Puzzle for ClojureScript</strong>.</p> <p>David Nolen has written up a <a href="http://swannodette.github.io/2013/12/17/the-future-of-javascript-mvcs/">short explanation</a> of how the Virtual DOM works, as well as some amazing performance benchmarking of React in a ClojureScript context. His work is important, so you should read it. I'd like to focus a bit more on the expressivity and why <strong>I would use React even if it were not so fast</strong>.</p> <p>One can view MVC frameworks as an attempt to impose some structure on the code that has to interface with the DOM. The bargain they propose is this: <strong>wrap the DOM in View objects</strong>, so that subtrees of DOM nodes are managed by a View object. <strong>Wrap your state in Model objects</strong>, which will notify the View objects of changes. You thereby keep a <em>layer of indirection between Models and Views</em>, which inherently need different structure and need to change independently.</p> <p>The layer of indirection (usually an event or observer system) solves a <em>coupling problem</em>. It decouples your state from the DOM, while <strong>leaving you to deal with all of the difficulties of the DOM</strong>. You are essentially making a new type of DOM that is better suited to your GUI domain than plain HTML, but is still a PITA. It is little more than a coat of paint. <em>The DOM is still stateful.</em></p> <p>React takes a different approach. It provides a level of indirection which solves the actual problem with the DOM--statefulness. The DOM has become a smart canvas. <strong>Paint the whole picture again, but only the different parts get wet.</strong></p> <p>Instead of wrapping the DOM in View objects, you create a Virtual DOM (completely managed by React) which mirrors the real DOM. When the model changes, you generate a new Virtual DOM. The differences are calculated and converted into batch operations to the real DOM. In essence, <strong>React is a function which takes two DOMs and generates a list of DOM operations</strong>, <em>i.e.</em>, <strong>it's referentially transparent</strong>.</p> <p>It's easy to imagine how this changes the game. <strong>You no longer need an initializer to set up the DOM and observers to modify it.</strong> The first Virtual DOM rendering is like the second one, in fact like any other! Your &quot;View&quot;, if you want to call it that, is simply a function from state to Virtual DOM nodes. <strong>If the state and DOM nodes are immutable, all the better.</strong> There's less work to know if it has changed.</p> <p>This also means that your Views can be composed functionally. Define a component (as a function) and your subcomponents, and build them up functionally. All of the functional abstraction and refactoring that you're used to is available. If you're doing it right, <strong>your code should get shorter, easier to read, and more fun to maintain</strong>.</p> <p>My (short) experience rewriting a program to use React converted me. It was the only library I have used that actually made DOM programming fun and functional--and dare I say my code now works!</p> <p>Which gets me to my last point, which is that React is the final puzzle piece for ClojureScript web frontend development.</p> <ul> <li>Problem: <strong>Global state management</strong></li> <li><p>Solution: Atoms and persistent data structures</p></li> <li>Problem: <strong>Client-server communication</strong></li> <li><p>Solution: EDN (also solved pretty well by JSON)</p></li> <li>Problem: <strong>Callback hell</strong></li> <li><p>Solution: core.async</p></li> <li>Problem: <strong>Stateful DOM</strong></li> <li><p>Solution: React</p></li> </ul> <p>Any other problems left? I can't think of any. That's something to <a href="http://www.twitter.com/ericnormand"><em>discuss on Twitter</em></a>.</p> <p>I suggest you try React. <a href="https://github.com/swannodette/om"><em>Om</em></a> by David Nolen is the most mature React ClojureScript library I know of. It does a bit more than I've described here (Om manages your state tree for you, among other things) and is evolving quickly. I have some code that generates React Virtual DOM using <code>hiccup</code> style with macros (so the work is done at compile time), but other than that, contains only half-baked implementations of what's in Om. If you're interested in the <code>hiccup</code> macros, <em>let me know what you'd use it for and I'll put it on github</em>.</p> <p><center><a href="http://www.lispcast.com/react-another-level-of-indirection"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p> Deconstructing Functional Programming http://www.lispcast.com/deconstructing-functional-programming http://www.infoq.com/presentations/functional-pros-cons Sun 22 Dec 2013 04:33:32 PM CST <blockquote> <p>Because if you think about it, the stack itself is just an optimization. Right? There are these frames which contain information about each invocation. Each stack frame. Each activation record. And that's what they are--they're activation records. They're sort of objects. If you really have objects on the brain, like I do, then you realize that they're all just objects.</p> </blockquote> <blockquote> <p>And they should be treated uniformly. You can even build a language that works this way--as I have. If that's the case, this is really a garbage collection problem. Right? Your stack traces might go away if you don't need them. You do need them when it's not a tail call because you need to go back there and use that information. But if it's a tail call, you don't need them, you don't need a tail call back to that frame. You need a pointer back to the last frame that wasn't a tail call. And they might get collected.</p> </blockquote> <blockquote> <p>Which doesn't mean you actually have to implement it that way. Erlang sort of does. And there are many implementations that do that. They are not noted for their speed. If you can have real stacks, what happens when you run out of space? If you don't run out of space, it didn't really matter if you optimized the tail calls or not.</p> </blockquote> <blockquote> <p>When you run out of space, you should GC the damned stack. You shouldn't just throw up your hands and say you're dead. And for all the normal programs that people write where it didn't matter, they won't care. And for your tail recursive programs, well, they might be a bit slower, but they will work. And then it's a matter of flags to the garbage collector if in production you don't want to debug it if you're sure it's all going to be fine--then go ahead and tell it to don't bother and just slam it and overwrite those frames directly.</p> </blockquote> <p>Why is this so hard for implementers to do? Optimization is an optimization and should be optional.</p> <blockquote> <p>It's a cool algorithm, but there's nothing nice that I'm prepared to say about Hindley-Milner.</p> </blockquote> <p>He nails a lot of things I didn't like about Haskell.</p> <p>All in all, this talk gets a lot of things right.</p> <p><center><a href="http://www.lispcast.com/deconstructing-functional-programming"><span class="brackets">&lt;</span><font face="Times New Roman">&lambda;</font><span class="brackets">&gt;</span></a></center></p> <hr /> <p> You may also be interested in the <a href="http://www.clojuregazette.com/">Clojure Gazette</a>, a free, weekly email newsletter to inspire Clojure programmers. </p>