Andrew Cholakian's Blog

Tonights hacking: DripDrop now supports routing

DripDrop now supports routing, a feature sorely needed for more complex apps. It was pretty easy to implement by using ruby's singleton class. Here's the entire implementation:

I've also written a full example showing how to use the new routing syntax.

A new minor point release of the gem will be out within the week, once I've been able to test the routing out a bit more to make sure I like the style.

DripDrop bump to 0.3.0: Subclassing,Filtering, Consistency, and Specs

Just released DripDrop v 0.3.0. The big new features in this release are message subclassing and pub/sub topic filtering, both courtesy of John W. Higgins (wishdev), who did a phenomenal job engineering these features cleanly. The other major change is a revamped argument order for xrep and http servers and clients. These are more consistent with the web socket interface (message as first arg) and really clean up the API.

Message subclassing is especially exciting because now you're able to send messages with much richer behavior. The javascript API doesn't yet support subclasses explicitly (they just come through as plain DD.Message objects), but support should be forthcoming in the next release.

On top of all this, we now have more spec coverage! I'll admit the test coverage hasn't been great, since it wasn't baked in, but we're adding it in as we go, and when I have time, I'm trying to fill it in as well. Questions? Comments? I'd love to hear them, either here, on github, or on twitter @andrewvc .

PS.

My plans for distributed workers in DripDrop are on hold till I feel the reactor code is well spec'd. Additionally, I think a rails routes style DSL for managing all your socket names in larger apps might be more important. Or maybe it's a terrible idea, it's not quite a fully formed idea yet.

-- Andrew

Cleaned up DripDrop Syntax

I finally found the time to make some important small changes to DripDrop. I cleaned up the syntax quite a bit using instance_eval. The main style differences are as follows:

I'm finally adding specs to it (Bad... I know). Writing specs for ZeroMQ isn't necessarily easy due to the fact that some sockets like pub/sub, while working fine in practice, don't exhibit the 'perfect' behavior your test framework wants. Shouldn't be too bad for the other socket types though.

Next Step for DripDrop: ZeroMQ Distributed Workers

I've been thinking about DripDrop's design goals lately. One of the core goals of DripDrop is to make writing highly concurrent apps that run on both single and multiple machines as easy as possible. The idea is that anywhere there's a ZeroMQ socket, you can just break that part of the app off and run it over your network (so long as latency won't be prohibitive). This composability isn't something that DripDrop provides, that fully comes from the magic of ZeroMQ and BERT. The role of DripDrop is simply to make using those sockets so easy that you'll use them before you even need them (at the cost of low level control and performance in some situations).

As it currently stands, DripDrop is a framework for making message based apps easier to build in a reactor pattern by leveraging ZeroMQ. Since DripDrop is built on top of ZMQMachine and EventMachine, DripDrop apps inherit all the strengths and weaknesses of the reactor pattern. Really, DripDrop is just a thin wrapper around these libraries, normalizing and simplifying the API to deal exclusively with its message format. This lets us semi-seamlessly stich together HTTP, WebSockets, and ZeroMQ sockets in very few lines of code. However, Reactors are not the be-all end-all of concurrency design.

While reactors are useful for expressing some problems, often times a thread pool fits the problem better. EventMachine actually handles this reasonably well with EM.defer, which simply hands over a block of code to be executed in a thread pool. I'd like to add a worker pool to DripDrop than can be broken out to run over any number of machines on your network, and I'd like to make this as easy to use as EM.defer.

This should be dead simple, but foresee some possible pitfalls. ZeroMQ sockets do apply backpressure, which is useful in a situation like this, but I'll have to see for myself how well it works for this scenario (as they say, trust but verify). My initial plan is to build a new version of the ZeroMQ HTTP Load Tester with this extension to DripDrop, to give it some basis in reality. 

Additionally, I've had thoughts of monitoring the state of all the sockets in a DripDrop app. I'm probably going to add a control backchannel to DripDrop sockets that'll give application writers a high level view of traffic on their DripDrop apps, perhaps reporting back to a Sub socket that keeps counters in Redis. 

Additionally, it'd be nice to know when distributed processes go down, so a long term goal would be to implement a set of deployment scripts that only start up certain services based on role, and establishing a central control server that monitors this backchannel for problems.

If anyone finds these goals useful (or not) let me know. Sadly, I only have so much free time, so they may be a while in coming... If anyone out there thinks these would be interesting things to work on, shoot me a line, I'll be glad to offer as much help as possible. 

Learn ZeroMQ by Building an HTTP Load Tester

I've just added a new, fully functional, example to my "Learn Ruby Zeromq" project: an HTTP load tester, in the vein of tools like the venerable `ab` from apache.

By leveraging ZeroMQ, we have a tool that's easily scalable across multiple machines, all reporting back to a control server. Check out http_load.rb on github to see it in action.

It's short at about 180 lines (including whitespace!), and contains a reasonable amount of error handling code, so it's not so simple as to be practically useless.

For more examples, (including beginners ones, if this one has left you a little lost), check out Learn Ruby ZeroMQ on github.

DripDrop now supports more socket types.

My DripDrop framework for zmqmachine/EventMachine just saw a whole bunch of ZMQ improvements, it now supports PUSH/PULL and XREQ/XREP in addition to PUB/SUB sockets. Also, the ZMQ handlers have been completely refactored to be much cleaner, making adding additional socket types much easier.

Checkout the examples folder for usage.

Learn ZeroMQ in Ruby With These Examples...

I've started releasing a series of learn by example ZeroMQ lessons on github, you'll be able to receive updates with a simple git pull. There wasn't much in the way of docs / manuals / guides for ZeroMQ in the Ruby world, hopefully this will help fill that void.

Most of the current docs are either for the C api, or in the man pages. While fairly comprehensive, they're daunting, dense, and take a lot of time to go through.

These examples are a work in progress, I'd appreciate feedback and contributions as I move forward. If you finish these examples, I highly recommend reading the ZeroMQ man pages and the official site, even if they're a bit dense, for a more in-depth look at ZeroMQ.

Tailing MongoDB Capped Collections in Ruby

This wasn't documented anywhere, so I figured others might find it useful. In MongoDB you can tail a capped collection, much like using the tail -f command. This works because Mongo uses cursors. Tailing's a nifty trick, especially if you're using Mongo for logging. 

I'm using Phil Burrows mongo_db_logger to log all Rails reqs to a capped collection at the moment, I'd recommend dropping this into your rails app, after modifying it to read your mongo config so you always have it ready.

ZeroMQ: What You Need to Know Braindump

In a recent email conversation, I was asked about getting started with ZeroMQ by another Rubyist (the awesome Ilya Grigorik)

This was my caffeine addled response, which others may find useful as well:

I'm going to warn you, some things you hear when you first read about ZMQ sound crazy. If there's one thing to be said for ZeroMQ, it's this sentence, from the Mongrel2 book "[ZeroMQ is] sockets the way programmers think sockets work". 

The more time I spend with ZeroMQ, the less I can think of a reason I'd ever have to open up a raw TCP or UDP socket, except in extraordinary circumstances, again. I think of ZMQ as common IPC and network communication patterns abstracted into messages and sockets that don't require a broker infrastructure. The whole message queue aspect of it is great, but ZMQ is really designed for a whole range of situations you'd never use AMQP for, and IMHO, that's the truly interesting thing about it. You don't really run ZMQ brokers (mostly), you communicate socket to socket using queue semantics.

The mongrel2 book has the most concise rundown of ZMQ, and IMHO best frames how to use it properly. It's not very in depth, maybe 15 minutes of reading, but is probably the single best introduction to ZeroMQ.

http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-590005.2

After that, I'd read the rbzmq rdocs for ZMQ::Socket. It's actually a fantastic rdoc, and a better intro to ZMQ than a lot of the docs on the official ZMQ site. It's kind of sad that someone wrote that fantastic doc and it doesn't get any publicity.

Another good post on ZMQ is this blog post:

The mongrel2 project would probably be the best example of a project that really is aggressively using ZMQ in an interesting way. You could check out the m2r ruby adapter to get an idea about ZMQ use in an actual app: http://github.com/perplexes/m2r . You may notice a lot of people using ffi-rzmq rather than the zmq gem. They're almost completely compatible at the API level, except ffi-rzmq works on non MRI/YARV rubies. I'm personally using it exclusively. 

The project I'm working on, dripdrop. is just a simple serialization format and reactor api on top of ZMQ, to make building an app out of a lot of ZMQ building blocks easy, and as much of the tedium out possible. It initially started because I wanted a simple async interface to events on multiple servers running Rails, but I think it made me realize ZMQ needs a proper super-simple API, preferably one that integrates non-ZMQ exit points, like HTTP,  Websockets, XMPP, etc.

DripDrop is still definitely still at the playing around with different ideas stage, and isn't really stable yet. I'm happy with the serialization format though (which is pretty much just BERT with a header for PUB/SUB filtering at the ZMQ level, similar in approach to mongrel2's format, but using BERT). I already have plans to redo about most of it, and need to start adding in support for socket types other than pub/sub. I really do like pub/sub though, it may not have the right semantics as far as making sure messages get delivered, but it enables semi-aspect-oriented architectures where random other processes can hook into a message stream (am I abusing the term aspect-oriented?)

What I plan on working on is redoing it style-wise into a simple DSL like this (this is a simple forwarder that uppercases and copies everything to a websocket and a ZMQ pub socket as an example):


I like the conciseness of this, I can see integrating ZMQ, HTTP, Websockets and other protocols in an async fashion, using this style of API, as worthwhile. What do you think? Biting off too much?

Anyway, I've clearly had too much coffee :)