Andrew Cholakian's Blog

Why Not MessagePack?

JSON is a fantastic format, anywhere people in your organization want to reach for XML, it's always a good thing to ask the question "Why not JSON?". The question I'd like you to ask the next time you're reaching for your JSON hammer, is "Why not MessagePack?". 

MessagePack has the following things going for it when compared to JSON.

  1. JSON Compatible: Anything that works with JSON will work with MessagePack.
  2. More space efficient: MessagePack uses an extremely efficient binary serialization format, for things like numbers and binary data MessagePack can be hugely more efficient. For use in persisting datastructures in something like Redis, where you want to be careful with memory usage, this is quite useful
  3. Faster: MessagePack is usually much faster to encode / decode than JSON.
  4. Supports Your Language: Ruby, Python, Perl, Javascript, PHP, Java, C++, C#,Go, Erlang, Haskell, OCaml, Scala, Clojure, and more all support MessagePack.
  5. RPC: There's a separate MessagePack RPC project that maintains a high performance MessagePack based RPC server and client available in most of the languages above.

For the record, I'd probably still use JSON for a public facing API in most cases, but for internal ones, MessagePack generally wins.

The following example, written in Ruby, illustrates the advantages of MessagePack in action:

For more info, check out the Message Pack Website.

Unicode Chalkpoints


We recently moved to a new office here at Vokle, and that means brand new, blank, chalk walls!

I've got a giant chalk wall next to my desk now just waiting to be filled. So, since this wall's in the engineering office, I figured we needed something a little geeky, hence Unicode Chalkpoints!

(download)

So, let's see what we have here:

  • ⚝ \u269D, the Coat of Arms for Morocco, well at least according to wikipedia's page on Miscellaneous Unicode Symbols.
  • ⚸ \u26B8, the Black Moon Lilith, the astrological sign for earth's second moon that turned out to not actually exist.
  • ⚆ \u2686, White Circle With Dot Right. I just like the way this one looks
  • ♾ \u267E, Permanent Paper Sign. This means paper is acid-free. Which means it can last for up to 1000 years.

Making life easier with GIT SHAs in your HTTP Headers

Do you know what code is running on your servers? Perhaps someone deployed something in a weird way and circumvented your normal deploy-logging process. Maybe you suspect an application server didn't actually restart. Maybe you just want to know quickly because you're lazy.

My answer to this question has been adding the Git SHA of the currently running code to the HTTP headers of my app. By doing this, figuring out what's actually running is as simple as a:

 curl -I http://www.example.net | grep X-GitSHA

Accomplishing this is pretty simple, first, you'll want to run and store the output of:

 git rev-parse HEAD

in whatever directory your deployed code runs in. This is the fastest, and easiest way to determine the current SHA. In Rails this is easily done in an initializer or application.rb using something like:

 GIT_SHA = `git rev-parse HEAD`.chomp

Then, just make sure the constant gets inserted as a header. Again, in Rails you could do this with a before_filter in controllers/application_controller.rb

 headers['X-GitSHA'] = MyApp::Application::GIT_SHA

And you're done! It should just work. Of course there's a million ways to actually do this.

Now that you have this info you can see what code is different between what's deployed and what's on say your master branch with a `git diff`:

PROD_SHA=`curl -I http://www.example.net | grep X-GitSHA | cut -d':' -f2` git diff $PROD_SHA...master

Or, you can see what the difference is in terms of commits messages rather than actual lines changed with `git show-branch`

 

PROD_SHA=`curl -I http://www.example.net | grep X-GitSHA | cut -d':' -f2` git show-branch $PROD_SHA master

Lastly, if you need the cache on a resource busted between deploys be it in either HTTP or Memcached, having a GIT_SHA constant available in your app makes it pretty easy to do. Note that this isn't great for a lot of situations where a file may NOT have changed between deploys.

Before PostgreSQL, Bruce Momjian wrote an SQL in Shell

Bruce Momjian is best known as a member of the PostgreSQL core team, what most people don't know is that in the early '90s, before he was working on Postgres, he wrote another SQL database.

And he wrote it in shell.

The result, SHQL is a pretty damn cool piece of software. I had an informal chat about it with Bruce a couple weeks ago at SCALE, and just had to write about it. The source is definitely a fun browse because not only is it a cool idea, it's the kind of software that belongs to a specific time. Asking Bruce how he picked shell, he mentioned that it was a language he knew, and that Perl just seemed like it was too heavy at the time.

What's immediately striking about SHQL is its completeness given its mere 760 SLOC. It supports, CREATE, DELETE, DROP, INSERT, SELECT, UPDATE, WHERE, PRINT, EDIT as commands, but has surprising completeness, including UNION and DISTINCT. It even supports a rudimentary form of views, through which you can do a basic sort of joining!

SHQL is not pure shell, it makes heavy use of both awk and grep. For instance, updating a table involves filtering the entire table through awk, directing the output to a tmp file, then replacing the current data file with the new one.

The source is definitely a fun browse, I'd recommend you download the file, and check out the README and demo as well!

One install note, on my ubuntu system /bin/sh has a few issues with it, I'd definitely use bash proper to run it. Also, be sure to mkdir -p ~/shql/MYDBNAME.

Fiction in Fiction

In the third act of Hamlet, Shakespeare erects a stage on the stage; the fact that the play enacted there—the poisoning of a king—in some way mirrors the primary play suffices to suggest the posibility of infinite involutions. (In an 1840 article, De Quincey observes that the stolid, heavy-handed style of this minor play makes the overall drama that includes it appear, by contrast, more lifelike. I would add that its essential aim is the opposite: to make reality appear unreal to us.)

-- Jorge Luis Borges, From his essay "When Fiction Lives in Fiction"

My New Job: Developer at Online Greetings Company Cocodot

So, it's been a month, and I still haven't put it up here, I'm now working as a Rails developer for online invitations and greetings company cocodot.

If you haven't see cocodot, it's a pretty interesting concept that I'm quite excited about. We occupy the the greetings space, but have better execution than the competition, we probably have the best designed greetings and cards of anyone out there, thanks to an amazing creative staff. The wedding invitations system is pretty slick as well, it's probably the best way to do online wedding invitations at the moment.

Screen_shot_2010-08-06_at_11

Esoteric and Archaic Words in The Adventures of Sherlock Holmes

I've recently finished reading the Project Gutenberg edition of The Adventures of Sherlock Holmes. Since, I was reading it on my oh so Convenient Kindle, annotation was quite easy. I ended up making a list words and phrases I needed to look up. Some of these words are still in common use in the British Isles, so be forewarned there's an American bias here.

1. Pips - Small, hard seed in a fruit.
2. Vesta - a short match with a shank of wax-coated threads; also : a short wooden match
3. Mousseline de soie - A fine crisp fabric made of silk
4. Gladstone Bag - a small portmanteau suitcase built over a rigid frame which could separate into two equal sections. They are typically made of stiff leather and often belted with lanyards. The bags are named after William Ewart Gladstone (1809–1898)
5. Frogged Jacket - Best described in this image
6. Billycock - A felt hat with a low, rounded crown, similar to a derby.
7. Assizes - The Courts of Assize, or Assizes, were periodic criminal  courts held around England and Wales until 1972.
8. Disjecta Membra - Scattered parts or fragments, as of an author's writings ( or in the context of this Holmes story, the remnants of a recently eaten chicken).
9. Carbuncle - An abscess larger than a boil, usually with one or more openings draining pus onto the skin. ( in the context of Holmes story, The Adventure of the Blue Carbuncle, the Blue Carbuncle is in fact a Jewel).
10. Gaol - Jail
11. Wilton Carpet - A Wilton  carpet is a woven wool carpet noted for having up to, but never more than, five colors per pattern.
12. Hasp - A clasp secured with a lock
13. Tout - One who solicits customers brazenly or persistently
14. Monomaniac - A type of paranoia in which the patient has only one idea or type of ideas.
15. Fuller's Earth - Any non-plastic clay  or claylike earthy material that can be used to decolorize, filter, and purify animal, mineral, and vegetable oils and greases.
16. Jezail - An Afghan matchlock  or flintlock  musket  fired from a forked rest.
17. Epistle - A writing directed or set to a group of people.
18. Tide Waiter - A customs officer who boards incoming ships at a harbor.
19. Caltrops - An antipersonnel weapon  made up of two or more sharp nails or spines arranged in such a manner that one of them always points upward from a stable base
20. Fess - In heraldry, a fess or fesse (from Middle English fesse, from Old French, from Latin fascia, "band")[1]  is a charge on a coat of arms that takes the form of a band running horizontally across the centre of the shield.
21. Sable - In heraldry, sable is the tincture black, and belongs to the class of dark tinctures, called "colours". In engravings  and line drawings, it is sometimes depicted as a region of crossed horizontal and vertical lines or else marked with sa. as an abbreviation.
22. Distaff - As an adjective  the term distaff is used to describe the female side of a family.
23. Danseuse - A woman who is a ballet dancer.
24. London Season - The social season or Season has historically referred to the annual period when it is customary for members of the a social elite of society to hold debutante balls, dinner parties and large charity events. It was also the appropriate time to be resident in the city rather than in the country, in order to attend such events.
25. Vestry - A storage room in or attached to a church or synagogue.
26. Ulster - A long, loose overcoat made of rough material
27. Coronet - A small crown consisting of ornaments fixed on a metal ring. Unlike a crown, a coronet never has arches.
28. Will-o'-the-wisp - A delusive or misleading hope.
29. Dun - Marked by dullness and drabness
30. Compositor - One that sets written material into type; a typesetter.
31. Dog-cart - A dogcart is a light horse-drawn vehicle.
32. Personate - Impersonate
33. Locus Standi - The ability of a party to bring a lawsuit or participate  in a particular  case