If you're interested in getting started with web development in Clojure there's something wrong with you. It's an elegant, fun, infuriating, and enlightening experience. I've spent some of my spare time in the past few weeks building an abstract rock garden in HTML/Canvas backed by Clojure's Noir framework at rocksandsand.com (source).

I'm really excited by Noir as a framework as it's the first Clojure framework that has a strong opinion about a lot of the things that really matter to a productive web framework.

Let's dive in.

Noir's hello world is what you'd expect in a post-sinatra world:

Where noir really gets interesting is in the tools, convenience, and opinions it brings.

Here's what you get when you first run `lein noir new`. Right here is Noir's first distinction, there's a lein (clojure's main build tool) plugin that will generate a bare-bones Noir app from scratch for you, much like `rails s`.

Noir actively tries to make a bunch of simple web development tasks easy. It's easily the Clojure framework that has the most 'batteries' included. You get things like automatic code-reloading on page refresh, built in lein commands to start up a server in both production and dev mode, a preset folder structure, and a few other such nice opinionated ways of doing things.

Moving on from there, you'll notice your `src` folder contains two folders 'models' and 'views'. Here's where you'll first notice one of Noir's greatest departures from most other web frameworks. There are no controllers by default (you can read the mailing list thread on why if interested). You wind up with combinations of controllers, views, and routes that--while initially scary--are actually quite elegant. An example of a more complex Noir view:

For a small app like Rocks and Sand, this sort of thing works well, it's more reasonable than splitting such stuff across 3 separate files. Whether it scales up to a large app however, is something I'm still on the fence about.

Views in Noir are not mandated to use any particular language, but Hiccup is by far the most popular solution, and is one of my all time favorite template languages. And it's nothing but clojure vectors and maps! Here's a quick example:

Models in noir are simply straightforward Clojure namespaces, if you want to, noir ships with a validations library you can use with it.

While my experience with Noir is obviously limited I have a list of praises and recommendations for it:

• Coupling of Routes -> Controllers -> Views is nice for small stuff, I'd probably decouple them more in a larger app. Luckily, if you really want, there's no reason you can't just do that by your own convention. Noir won't get in your way.
• Models are really bare bones, but have a nice little validation library. Not as complete as Rails', but quite easily extended
• WebSockets are more and more important these days. Tighter integration with Aleph (defsocket anyone?) would be great. A clojure framework that supports building combination synchronous / asynchronous apps would be awesome. While this is possible, the sugar that Noir's brought to Compojure could be extended to Aleph.
• The integrated util packages (S3, App Engine, Crypt, Etc.) are awkward and should be in separate jars.
• The ecosystem is still immature.I can still get more functionality out of Rails per hour due to its great community. However, for side-projects I'll be using noir. Additionally, for data-heavy web-services (like a JSON only backend) something like Noir is great!
• I wound up writing my own JSON based configuration package. Config files are part of every real web-app, and I think a good config package (preferably with deployment environment (prod/staging/dev) built in.

All in all Noir is exactly what Clojure's been needing to move to the next level in the world of web-apps. It's a beautiful framework, with a few odd opinions, but in generally it's a great way to get a small web-app off the ground. I'd recommend getting started by checking out the Noir homepage, and maybe even diving into the source my rock garden app.

I hack on code, I don't hack on photography.

Engineers tend to pick photography as a hobby over other art forms because it's so technical, and for this terrible reason it generally it winds up a confused train wreck.

This was motivated to write this by today's post on hacker news where Peter Norvig gives an intro to photography. Turning photography into a technical hobby is something I find agonizingly painful to watch.

On a technical level, Peter's photo tutorial's concise, accurate, and extremely clear. I couldn't have written a better one myself, it's a great resource. What it's missing however is perspective.

Engineers who become photographers generally spend an outsized amount of time making their images sharp, have true color, be adequately lit, etc. It's like audiophiles who listen to their stereo system but not the music.

When it comes to photography, engineers spend a disproportionate amount of time following compositional rules, shooting photos with straightforward narratives, and thinking about their photos as bullet points of technique before, during, and after shooting. It's as if the aim is for an acceptable, homogeous pastiche with the consistency of oatmeal.

What makes photos interesting? Well, that's the eternal question, but I'll say this, a perfectly clear, crisp, balanced photo is just a tool. The magic is up to you.

I'm not going to lay out the details like the engineer in me would, because the engineer in me has no idea what he's talking about. He might describe a formula for success in nine steps mentioning fibonacci, the golden ratio, the rule of thirds, etc., but that'd be a lie.

I'm just going to say, go out there and find photos that move you (and don't make yourself be moved, wait for it), and take as many photos as you can till you find a message in your own photos that means something to you.

My own photos aren't that great, but I understand my photos as a reflection of myself, they're deeply personal in terms of representing my point of view, and they give me a tremendous amount of personal satisfaction. That's what motivates me, the effect of my photos, not the SNR of my CCD.

I would say that it's richest to find photography as an experience and a method of communication.

Addendum:

Lastly, i'd like to apologize to Peter for being a prick about this. Peter's a brilliant, and inspirational developer, and personally I could only hope to have a fraction of his contribution to humanity throughout my life, but super-technical internet photo tutorials tend to have this effect on me, and this one just happened to catch me when I was sick, at home, with not much else to do.

And to be fair, if your aim is simply photos of your kids to look at as time passes, there's nothing wrong there, but it's just not in my DNA to truly understand investing time in an art like photography with such a cold eye toward the medium itself.

Clockwork's an awesome gem, but using it via its funky wrapper I'm not so nuts about. Here's how to use it from Rake.

I've been bits of my free time working on parbench over the last month, a visual version of the venerable 'ab' (apache-bench) HTTP load testing tool as a way to get up to speed on clojure. It's been a fun project as far as getting up to speed, and I thuoght I'd share what I've learned about clojure and its ecosystem.

Let's walk through the tool first, basically, you run it against an URL like 'ab', and it shows, in real time, your requests being executed. Each horizontal line is a 'user', and each square is a request, it fills from left to right as each user goes through its requests.

Structuring the App

I'm a ruby programmer by day, and the structure here was a bit different than I'd structure a ruby app, as classes / objects aren't generally used in Clojure code. Rather, the organizing principle is the namespace, and the philosophy behind Clojure is functions operating on core datastructures, or things like defrecord, which, while POJOs in implementation, mimic the core map type.

Clojure has very strong opinions about concurrency, it hasimmutable (persistent) datastructures, software transactional memory, actors, and much more in its bag of tricks. Nothing is mutable in clojure without a fair amount of explicitness. Rather than being encumbering, its actually refreshing, as you spend less time worrying about contention and race conditions.

To get a better feel for how a Clojure program might be built, lets break down parbench. The overall flow of parbench is split into two parts, the http request engine in benchmark.clj and the console and GUI output functions in displays.clj. A brief overview follows:

benchmark.clj:

• Initialize a 2d array (vectors actually) of refs, each representing a request. A 'ref' being a wrapper around a persisent datastructure allowing you to change its state in a transaction. Transactions here work almost exactly like database transactions. This is referred to as the 'grid' in parbench.
• Each row in the grid is a simulated user-agent, and should execute one request at a time. Each row gets its own backing thread via a Clojure future, which is basically a fancy Java thread.  Since I'm using an asynchronous Netty based http-client from Sonatype it is possible, even optimal, to do this all in a single thread rather than thread per-user, but this was more straightforward.
• As requests change state, the hashes representing their current state, nested in the grid, are mutated inside a transaction demarcated using dosync.
• There's a weird function, block-till-done, that polls the grid and checks to see if all requests are completed, if they are it stops blocking. While this isn't optimal with a bunch of threads, each of which could simply signal when it complete, I went with this method with a previous concurrency model, and its very flexible letting me try out different concurrency models without worrying about housekeeping.

displays.clj:

• The GUI uses processing for display. The UI is initialized before the benchmarks, this is important, as it blocks until its first full render, letting the JVM and UI get pre-warmed so they don't impact results too much.
• Due to clojure's refs, the UI always sees the grid in a consistent state, since all mutations to the request refs are transactional inside those dosyncs we spoke about earlier.
• The console output uses a java TaskTimer to regularly print output to the terminal, iterating over the datastructures.

Impressions of Clojure and the its ecosystem

• It's an active community with a vibrant, supportive, and fun group of members, which is good, because there's a lot of work to do.
• There's a dearth of documentation for libraries. A lot of the code you'll need is on github, and a lot of it requires diving into the source to figure out the full API. It's somewhat expected in a community this young, but it's a bit of a drag on productivity.
• There aren't enough blog posts about Clojure, problems just aren't that googleable in clojure at the moment. A lot of the common ones are, but a lot of roadblocks require you to roll up your sleeves and experiment.
• Sequences, and the functions around them are elegant and powerful, though there is a bit of a learning curve, it isn't too bad. There are many powerful tools in Clojure to work with sequences, and they are one of the things that makes clojure an absolute joy to work in.
• It's all about the datastructures, while clojure now has more integrated java objects with defrecord and friends, the core data-structures are extremely powerful and easy to use.
• Clojure is concise, though it can become impenetrable at times. This makes breaking up functions extremely important. The same conciseness that gives it its beauty can be its undoing.
• You need to know Java. I'd hoped to learn clojure without learning any Java, it didn't take long to figure out that was a terrible plan, and I had to spend a while learning some basic java first. Clojure leans heavily on java for its library support, and some libraries don't yet have clojure wrappers. Luckily, java interop is so good in Clojure, it isn't an issue to go back and forth most of the time.
• Get the Joy of Clojure, it's a great book. The online resources for learning Clojure are scattered and hard to piece together into a consistent whole.

Overall, Clojure's becoming my free time language of choice because it's concise, expressive, and elegant. It's also a good departure for those coming from more OO languages, and will definitely work different muscles in your brain.

Lastly, I'm pretty new to Clojure, I'm pretty sure parbench's code sucks in myriad ways. Got some ideas as far as improving it? Let me know!

JSON is a fantastic format, anywhere people in your organization want to reach for XML, it's always a good thing to ask the question "Why not JSON?". The question I'd like you to ask the next time you're reaching for your JSON hammer, is "Why not MessagePack?". 

MessagePack has the following things going for it when compared to JSON.

1. JSON Compatible: Anything that works with JSON will work with MessagePack.
2. More space efficient: MessagePack uses an extremely efficient binary serialization format, for things like numbers and binary data MessagePack can be hugely more efficient. For use in persisting datastructures in something like Redis, where you want to be careful with memory usage, this is quite useful
3. Faster: MessagePack is usually much faster to encode / decode than JSON.
4. Supports Your Language: Ruby, Python, Perl, Javascript, PHP, Java, C++, C#,Go, Erlang, Haskell, OCaml, Scala, Clojure, and more all support MessagePack.
5. RPC: There's a separate MessagePack RPC project that maintains a high performance MessagePack based RPC server and client available in most of the languages above.

For the record, I'd probably still use JSON for a public facing API in most cases, but for internal ones, MessagePack generally wins.

The following example, written in Ruby, illustrates the advantages of MessagePack in action:

For more info, check out the Message Pack Website.

Loving byobu terminal for dev at the moment. It's like a super-nice, and sanely configured screen.

We recently moved to a new office here at Vokle, and that means brand new, blank, chalk walls!

I've got a giant chalk wall next to my desk now just waiting to be filled. So, since this wall's in the engineering office, I figured we needed something a little geeky, hence Unicode Chalkpoints!

(download)

Click here to download:

unicode-chalkpoints-ceIncnqyGjnnpzynguJt.zip (6.08 MB)

So, let's see what we have here:

• ⚝ \u269D, the Coat of Arms for Morocco, well at least according to wikipedia's page on Miscellaneous Unicode Symbols.
• ⚸ \u26B8, the Black Moon Lilith, the astrological sign for earth's second moon that turned out to not actually exist.
• ⚆ \u2686, White Circle With Dot Right. I just like the way this one looks
• ♾ \u267E, Permanent Paper Sign. This means paper is acid-free. Which means it can last for up to 1000 years.

 curl -I http://www.example.net | grep X-GitSHA

 git rev-parse HEAD

 GIT_SHA = `git rev-parse HEAD`.chomp

 headers['X-GitSHA'] = MyApp::Application::GIT_SHA 

PROD_SHA=`curl -I http://www.example.net | grep X-GitSHA | cut -d':' -f2` git diff $PROD_SHA...master

Or, you can see what the difference is in terms of commits messages rather than actual lines changed with `git show-branch`

PROD_SHA=`curl -I http://www.example.net | grep X-GitSHA | cut -d':' -f2` git show-branch $PROD_SHA master

Lastly, if you need the cache on a resource busted between deploys be it in either HTTP or Memcached, having a GIT_SHA constant available in your app makes it pretty easy to do. Note that this isn't great for a lot of situations where a file may NOT have changed between deploys.

Bruce Momjian is best known as a member of the PostgreSQL core team, what most people don't know is that in the early '90s, before he was working on Postgres, he wrote another SQL database.

And he wrote it in shell.

The result, SHQL is a pretty damn cool piece of software. I had an informal chat about it with Bruce a couple weeks ago at SCALE, and just had to write about it. The source is definitely a fun browse because not only is it a cool idea, it's the kind of software that belongs to a specific time. Asking Bruce how he picked shell, he mentioned that it was a language he knew, and that Perl just seemed like it was too heavy at the time.

What's immediately striking about SHQL is its completeness given its mere 760 SLOC. It supports, CREATE, DELETE, DROP, INSERT, SELECT, UPDATE, WHERE, PRINT, EDIT as commands, but has surprising completeness, including UNION and DISTINCT. It even supports a rudimentary form of views, through which you can do a basic sort of joining!

SHQL is not pure shell, it makes heavy use of both awk and grep. For instance, updating a table involves filtering the entire table through awk, directing the output to a tmp file, then replacing the current data file with the new one.

The source is definitely a fun browse, I'd recommend you download the file, and check out the README and demo as well!

One install note, on my ubuntu system /bin/sh has a few issues with it, I'd definitely use bash proper to run it. Also, be sure to mkdir -p ~/shql/MYDBNAME.