Patterns of method missing


One of the more dynamic features of Ruby is method_missing, a way of intercepting method calls that would cause a NoMethodError if method_missing isn’t there. This feature is by no means unique to Ruby. It exists in Smalltalk, Python, Groovy, some JavaScripts, and even most CLOS extensions have it. But Ruby being what it is, for some reason this feature seem to have more heavily used in Ruby than anywhere else. It’s also a feature most Ruby developers seem to know about. Is this because Ruby people are power hungy, crazy monkey patchers? Maybe, but method_missing is also potentially very useful, if used correctly. But of course, it’s exceedingly easy to misuse. In almost all cases you think you need method_missing, you actually don’t.

The purposes of this post is to take a look at a few ways people are using method_missing in the wild, what the consequences are and what you can do to mitigate them. I’m bound to have missed a few use cases here, so please feel free to add more in the comments.

Adding better debug information on failure

One of the most simple but still very powerful ways of using method_missing is to allow it to include more information in the error message than you would usually have got. A simple example of that could look like this:

class MyFoo
  def method_missing(method, *args, &block)
    raise NoMethodError, <<ERRORINFO
method: #{method}
args: #{args.inspect}
on: #{self.to_yaml}
ERRORINFO
  end
end

This usage is pretty common - and is in my opinion a very valid use of the functionality. The only thing you have to be careful about is to not introduce any recursive calls to method_missing. Say if you forget to require YAML in the above example - the error would be a stack overflow.

One of the places where you’ve almost certainly seen this used is in Rails, where the feature is called whiny nils. The idea is that nil will have a method missing that gives some extra information. It can guess based on the method name what object you were expecting. This could be a typical message from Rails whiny nil:

Loading development environment (Rails 2.2.2)
>> nil.last
NoMethodError: You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.last
	from (irb):2

This functionality is exceedingly simple to implement, but gives you lots of leverage to find and debug your problem quicker and easier.

Encode parameters in method name

Another common pattern is to use the name of the method to encode parameters, instead of sending them in as explicit parameters. In some cases this can be used to good effect, but if possible it would be better to encode the possible names beforehand, or send in the parameters as actual parameters instead. Contrast a Rails-style find expression:

Person.find_by_name_and_age("Ola", 28)

With another way of creating the same API:

Person.find_by(:name => "Ola", :age => 28)

The difference here isn’t that large, and in the case of Rails I do think they are harmless - but creating these kinds of API’s make it much harder to debug and maintain an application, so care should be taken.

Builders

Creating XML, HTML, graphical UIs and other hierarchical data structures lend themselves very well to the builder pattern. The idea of a builder is that you use Ruby’s blocks and method_missing to make it easy to create any kind of output structure. The canonical example in Ruby is Jim Weirich’s Builder, that can be used to easily create complicated XML structures. A small example:

builder = Builder::XmlMarkup.new
xml = builder.books { |b|
  b.book :isbn => "124" do
    b.title "The Prefect"
    b.author "Alastair Reynolds"
  end

  b.book :isbn => "65565" do
    b.title "Against a Dark Background"
    b.author "Iain M Banks"
  end
}

The result of this code will be a properly formatted and escaped XML document. Most notable, all the finicky details of closing tags and escaping rules are taken care of for us.

In general, this approach is very pleasant to work with. It’s easy to test (since you don’t even have to generate the real XML to make sure it’s correct), and it works well with your existing Ruby tools. It’s also quite easy to implement a basic version of. For the fully general case you need to use a blank slate object, though.

Accessors

The inversion of the builder pattern is to use a parser that slurps in an XML document (or a YAML, database or anything else really), and then allow you to access the elements of it by using regular Ruby method calls - intercepting these calls with method missing and looking them up. A usage could look something like this:

slurper = Slurp <<XML
<books>
  <book isbn="14134">
    <title>Revelation Space</title>
    <author>Alastair Reynolds</author>
  </book>
  <book isbn="53534">
    <title>Accelerando</title>
    <author>Charles Stross</author>
  </book>
</books>
XML

puts slurper.books.book[1].author

I’m not much of a fan of this approach. In almost all cases there are better ways of doing it than using method_missing. The only valid use case for something like this would be for a throwaway really hacky oneoff thing. But in general, Ruby allows you to define methods dynamically anyway, so you can do that instead for this case.

Proxy/delegation

When you want to insert a proxy that resends method calls somewhere else, method_missing can be an easy way to get that to work. You can resend method calls to another object, you can resend to several objects, you can send method calls over the wire, to implement a crude RMI system. You can also record method calls and write them to disk. All of these can be achieved with just a few lines of code. But in many cases there are better options - especially if you want to do delegation. One of the dangers (and the power also, of course) of method_missing is that it can take any kind of method call. So if you misspell something, method_missing will happily treat it the same way.

But when delegating, you generally want to be explicit about what you delegate, to avoid this problem. There are several classes in the standard library that allow you to explicitly say what methods to delegate and where to delegate them - and if you can, try using this instead. Proxying and delegation should be explicit if possible.

Making parts of an API extensible and optional

In some cases you might want to create a base class for an API, but allow the subclasses to add additional API methods. In some cases it can make sense to ignore calls to these subclass API methods if called on something that doesn’t support it. By definition, the super class can’t actually know which API methods the subclasses might add, so it makes sense to use method_missing to open up the API and make it more convenient. This is not very common - and in most cases should probably not be done, but sometimes it can be a useful technique.

Test helpers

All kinds of test helpers can be created using method_missing. They can be used to implement factories, delegate and do all kinds of things. If you take a look at any open source Ruby project, the tests is the place where you are most likely to find implementations of method_missing. I can’t say that these implementations actually follow any specific patterns either.

Summary

Finally, remember. Method missing is a powerful powerful feature - it should not be used in almost all the cases. But if you do want to use it, don’t forget to implement responds_to? correctly. And if you’re designing your class for subclassing, it’s also important to design your method_missing usage for inheritance. Liskovs Substitution Principle applies here.



ThoughtWorks signs EU petition against software patents


I am strongly against software patents, and as such I’m very happy that ThoughtWorks has decided to sign the EU petition against software patents.

I recommend any individual with the same views to sign - and to convince their company to do the same. You can do it here: http://www.stopsoftwarepatents.eu.



RubyConf India was a great success!


This weekend ThoughtWorks in collaboration with RubyCentral and several sponsors arranged the first ever RubyConf India. I was there as a keynote speaker and wanted to take a few minutes to tell you about the experience, since I think this was an event that showed how much pent up interest there really is for Ruby in India.

When planning the conference we aimed for about 150 delegates - but that sold out in a few days so we rearranged the event to accomodate about 400 people, and we managed to fill that. At the end of the day, there was about 420 people there, including both delegates and speakers.

Unfortunately there had been some problems with visas for Chad Fowler and Ivan Porto Carrero. We solved that by having Ivan present over Skype, and rearrange some of the talks and give Brian Guthrie a last minute spot.

The conference started out with Roy Singham, the founder and chairman of ThoughtWorks, setting the tone for the rest of the conference by getting people to think about how India can start innovate for real using technologies such as Ruby.

After that I did a keynote about programming languages, quite similar to what I did at RailsWayCon in Berlin last year. Hopefully people thought it was interesting. I tried to first discuss why we need different languages, where some Ruby language features comes from, a small taxonomy of languages, and some ideas about what might happen in the future.

Obie Fernandez was next up with a controversial keynote called Blood, Sweat and Rails. The basics of the keynote was a description of different lessons Obie has learned from running HashRocket. The controversial bit was based around two different factors, the first being Obie’s heavy use of profanity. (Heavy enough that one of the organizers went up on stage halfway through the presentation and asked him to tone it down). The other controversial part was that Obie used his keynote spot to spend quite a lot of time promoting HashRocket. Later in the day, a representative from ThoughtWorks, and one representative from Castle Rock (another sponsor) went on stage, mentioning that the point of sponsorship was not to push their respective companies but push Ruby in India.

After Obie’s keynote it was lunch time. I was tired and jetlagged so I walked around in a bit of a daze after my keynote was done. I did catch some of Aman’s talk about Ruby OO with objects instead of classes. What I saw sounded intruiging, but I didn’t get the full picture since I walked in and out of the presentation.

The highlight of the day was definitely Matz keynote, where he called in using Skype and did a presentation and some Q&A. Matz talked a bit about the history of Ruby and then started mentioning some things about the future of Ruby. The most interesting concrete information was that 1.9.2 will come this summer, and after that they will start work on 2.0. This version will also make some heavier changes, some of them which I’m not sure I like (such as requiring parenthesis for invocation).

The second day started out with Nick Sieger from EngineYard talking about the next version of Rails. Lots of useful information about what we can expect from the next major revision. Compared to mine and Obie’s keynotes, this was shock full with technical information instead of high level concepts. Good stuff.

After that, Pradeep Elankumaran from Intridea doing a very interesting session about startups. His talk ended up being a long discussion with most of the audience about startups. This discussion kept most people in the audience interested enough to stay long into lunch time. Very good session.

After lunch I had a major conflict of interest. Both colleagues of mine had interesting sessions going. Sarah Taraporewalla talking about Ruby view technology and Sidu Ponnappa and Niranjan Paranjape talking about entropy in long running Ruby projects. I ended up choosing Sarah’s talk - which was brilliant. She did a great job explaining why the current view technologies are generally too permissive and make it harder to test the behavior of your view correctly.

Of course, Sidu and Niranjan got good reviews - and I heard lots of things that sounded like it was a session full of controversial ideas. Sounds like fun - wish I’d been there too.

The next session was about building a Ruby Application Server. Sadly, I’d kinda misunderstood what this session was about, since I was assuming that “Application Server” was meant in the Java environment meaning. This was not the case - instead it was about implementing a Ruby web server. I kinda lost interest quite quickly and ended up doing some work instead.

Brian Guthrie’s replacement session was called “Advanced Ruby Idoms so clean you can eat off of them”, and was both hilarious and very on point. The room was standing room full and Brian’s presentation sparked lots of debate. The gist of it was that basically there is no such thing as magic in programming. Everything is a function of your understanding of what’s going on in the language. You can have good code or bad code. Clean code and dirty code. But magic code just means you don’t understand the language, and is not really something you should use as an argument for or against an implementation. Brian expanded on this by giving loads of tips and tricks on what to do and what not to do.

I didn’t catch the final session, since I was very tired at that point. After the final session, Roy Singham came back with a keynote about a number of different things. He talked about the current state of agile in the world, the Ruby and Rails culture in the US, and how that should inform the Ruby culture in India, what things Unicef has been doing lately with Rails and other technology, and how we can use new technology to start being more socially responsible. The keynote sparked a lot of debate, and as usual Roy made quite a large amount of controversial statements in soundbite form. Take a look at my twitter stream for this weekend to get some quotes.

All in all, I think this was a total success. Lots of interesting talks, fantastic networking opportunities and a great vibe in the air. Looking at the tweets from the conference, RubyConf India seems to have been very appreciated by a large majority of all attendees. Here’s hoping for an even better next year!



ThoughtWorks JRuby Geek Night in Pune


I should really have blogged this earlier, but the last few weeks have been hectic. Anyway, better late then never, eh?

Tonight - that is Wednesday, March 24th - ThoughtWorks Pune will host a geek night where I will talk about JRuby. I will talk a bit about what’s coming in the upcoming 1.5 release, and other fun things happening in JRuby land.

If you’d like to come, you can find more information and registration here.



ÜberConf


I will speak at ÜberConf in Denver in June. Should be lots of fun! I will talk about JRuby and building languages. I might also possibly cover Ioke - we’ll see what happens.



Destructuring extravaganza


A few months back I added support for destructuring assignment and tuples to Ioke. Since Ioke’s assignment is just a regular method call, this was actually fairly easy to do. The end result is that you can do things like (x, y) = (13, 14). You can also do more interesting things, such as ((x, y), (x2, y2)) = [[1,2],[3,4]]. Notice that the right hand side is not a tuple anymore, but a list. Anything that can be turned into a tuple using the asTuple method can be on the right hand side, or an item in a recursive destructuring.

All this functionality makes code slightly more readable. But last week I decided to add support for eachCons and eachSlice, and suddenly I realized that destructuring would be very nice to have not only in the explicit assignment case, but also in cases where you want to pick apart the arguments to an enumerable or sequence method. So I added those, which means that suddenly lots of code becomes much more simple.

Short story, in all Sequence and Enumerable methods, at every place where you could put an argument name, you can now put a destructuring statement instead. Let’s take a look at an example:

Point = Origin with(asTuple: method((x, y, z)))

points = [
  Point with(x: 42, y: 14, z: -1),
  Point with(x: 20, y: 0, z: 444),
  Point with(x: 31, y: 646, z: 3),
  Point with(x: 456, y: 14, z: 12)
  ]

distances1 = points consed map(obj,
  ((obj[0] x) * (obj[1] x) +
    (obj[0] y) * (obj[1] y) +
    (obj[0] z) * (obj[1] z)) sqrt)

distances2 = points consed map(
  ((x1,y1,z1), (x2,y2,z2)),
  (x1*x2 + y1*y2 + z1*z2) sqrt)

distances1 inspect println
distances2 inspect println

This code first creates a Point that can be coerced into a tuple of x, y and z coordinates. We then create a list of Points with different coordinates. We then want to calculate the three distances between the four points. We do this in two ways, using the old method and then using destructuring. The method consed is a sequence version of eachCons. The default cons length is 2, so this will yield three entries with two points in each. We then call map on the sequence. We will get a List of two entries, where each entry is a point. Finally we use Pythagoras to calculate the distance.

The second version is very similar - the only difference is that instead of using the square brackets to index into the lists, we instead give a pattern. This pattern contains two patterns, and the variable names inside of it will be bound to the right parts of each point.

At least in my mind, the destructured syntax is much more readable than the original one. And remember, this works for anything that can be turned into a tuple, which means you can use it on any Enumerable - you can use it on a Pair (such as what a Dict will yield) or any thing you would want to add asTuple to on your own.



RubyConf India


I am part of a team at ThoughtWorks helping out organizing the very first RubyConf in India. I’m very excited about this. So if you have the possibility to come to Bangalore, the event will be March 20 and 21.

We already have some solid speakers lined up. Chad Fowler will keynote, and so will I, and we have a number of other people coming in. A few of my colleagues from ThoughtWorks, such as Sarah Taraporewalla, Sidu Ponnappa and Aman King. Other speakers include Hemant Kumar, Pradeep Elankumaran, Arun Gupta and others. Finally, Nick Sieger will also come to Bangalore for this event!

So as you can see, this is gearing up to be a great event! Hope to see you there.



Ioke P released


I am very happy to announce that Ioke P has finally been released!

Ioke is a language that is designed to be as expressive as possible. It is a dynamic language targeted at the Java Virtual Machine. There also exists a version for the CLR. It’s been designed from scratch to be a highly flexible general purpose language. It is a prototype-based programming language that is inspired by Io, Smalltalk, Lisp and Ruby.

Homepage: http://ioke.org
Download: http://ioke.org/download.html
Programming guide: http://ioke.org/wiki/index.php/Guide
Wiki: http://ioke.org/wiki

The two specific releases that encompass Ioke P are ikj 0.4.0 and ikc 0.4.0.

Ioke P is the fourth release of Ioke. It includes many new features compared to Ioke E:

  • Number Infinity
  • eval
  • Reflector
  • Hooks
  • First class Runtime
  • New parser
  • Tuples
  • Structs
  • Destructuring assignment
  • Message rewriting
  • Functional composition
  • Sequences
  • Dictionary and Set versions of Enumerable methods
  • Enumerable group, Enumerable groupBy
  • Set operations for union, intersection, membership, subset and superset testing
  • ISpec stubbing and mocking
  • IIk history
  • DokGen on separate projects

Ioke P also includes a large amount of bug fixes.

Features:

  • Expressiveness first
  • Strong, dynamic typing
  • Prototype based object orientation
  • Homoiconic language
  • Simple syntax
  • Powerful macro facilities
  • Condition system
  • Aspects
  • Java integration
  • Developed using TDD
  • Documentation system that combines documentation with specs
  • Runs on both the JVM and the CLR

The many things added in Ioke P could not have been done without the support of all the Ioke contributors. Thank you!

Regards
Ola Bini    - ola.bini@gmail.com



Path problem with Emacs on Mac OS X


One of the problems you might encounter using a native Emacs on top of Mac OS X is that your custom PATH variable doesn’t really translate into your running Emacs instance. This might show up as making it impossible to run things like Ruby or Ioke from inside of Emacs - complaining that it can’t find the executable in question.

Another thing you might notice is that your Ruby environment is messed up - all the gems you so carefully installed isn’t around, and overall things just seem weird. One thing that could cause this is when you have a custom built Ruby installation in for example /opt/local or /usr/local instead of /usr. But Emacs just seems to pick up the original versino that shipped with OS X.

The reason for all these weird behaviors is that if Emacs is started from the Dock, or using the application directly, it doesn’t actually pick up the carefully prepared PATH variable you have in your terminal. Instead it will go ahead and use the default system PATH, which is very restrictive.

The way I prefer to solve this is to make sure that OS X plist version of the PATH variable is the same as my terminal one. I do that by resetting this plist on every login. Of course, OS X will not actually reload this without a reboot. If you just want to fix this once - and you’re reasonably sure you won’t change the path again, you can execute this command from the terminal and restart. Everything should be fine after that. The magic incantation looks like this:

defaults write $HOME/.MacOSX/environment PATH “$PATH”

(Note, I’ve had to solve this problem twice, since the first time I encountered it I didn’t blog about it. So now I’m blogging so it might also stick in my long term memory. Maybe it’s enough if it’s in my blog term memory…)



Conference Hat Trick - QCon, RubyConf, JRubyConf


I’ve just come back from several different conferences. It’s been tiring but also very rewarding. The conferences I attended and presented at was QCon San Francisco, RubyConf and JRubyConf. I thought I’d just mention some of the highlights from these three events.

First QCon - after JAOO, QCon is my favorite conference. They always manage to put together an interesting week with great speakers and lots of things to learn. This year, me and Martin Fowler did a full-day tutorial about domain specific languages.

During the Wednesday I spent most of my time hanging out and chatting with people. I did attend Josh Blochs and Bob Lee’s Java Puzzler presentation. This is always an entertaining hour. I also enjoyed Douglas Crockfords keynote about the history and future of JavaScript. Hearing how this all happened is always enlightening.

On the Thursday I had my track about languages. I think it went very well, my speakers did a great job. Eishay Smith talk about Scala, Stu Halloway about Clojure, Martin Fowler about Ruby, Jonathan Felch about Groovy and Amanda Laucher and Josh Graham about F#. I’m very happy with how it went, actually.

During Friday I mostly sat in on Neal Fords DSL track. My colleague Brian Guthrie started out with a strong hour about internal DSLs in various languages. Ioke got a few code examples, which was fun. After that Neal and Nate Schutta talked about MPS. I haven’t seen this much detail about MPS before so it was helpful.

After lunch Don Box and Amanda Laucher did a talk about the technology formerly known as Oslo. I didn’t think this tech was anything cool at all until I saw this presentation. In retrospect this was probably my favorite presentation of the conference. What came together was how you can use M as a fully typed language with some interesting characteristics, and also the extremely powerful debug features. It’s nice indeed.

Glenn Vanderburg put forward some arguments against language workbenches. This made for an interesting hour but I’m not entirely sure I buy his arguments. And after that Magnus Christerson from Intentional showcased what they’ve been working on lately. Very impressive stuff as usual.

I only spent one day at RubyConf, but it was still enough to get a feeling for what was going on, spend some time with several people I haven’t met before and so on. Good times. Charles Nutter did a very good presentation about his Ruby mutants (Duby and Surinx). After that Ryan Davis and Aaron Patterson did a hilarous presentation about weird software.

JRubyConf was a total success. All of the presentations were very interesting, and provided insight into what people liked about JRuby and what they wanted from it. It was fantastic to see so many people come together just for JRuby. It’s great to be part of that. I did a presentation about testing with JRuby, and then I was part of the closing panel. Both went well.

All in all a great week of conferences.