I’ve been spending the last few days in San Francisco, attending CommunityOne and JavaOne. We are right now up to the second day of JavaOne, so I felt it would be a good idea to take a look at what’s been going on during the first two days.

I will not talk about the general sessions here since I as a rule avoid going to them. So, I started out CommmunityOne seeing Guilloume talk about what is new in Groovy 1.6. Pretty interesting stuff, and many useful things. Although, one of the things I noted was that many of the default usages of AST transformations actually just make up for the lack of class body code. Things like “@Singleton” that neeeds an AST transformation in Groovy, is a very simple thing to do by executing code in the class body in Ruby.

After that I saw John Rose talk about the Da Vinci machine project. Pretty nice stuff going on there, really. The JVM will really improve with this technology.

Charles Nutter did a reprise of his Beyond Impossible JRuby talk. It’s a really good talk that focuses on the things that you really wouldn’t think possible to do on the JVM, that we’ve had to do to get JRuby working well.

Guido talked about Python 3000 – much of that was really a look at the history of Python, and as such was really interesting. Unfortunately, my jetlag started to get the better of me at that point, so my focus could have been better.

For me, the first day of JavaOne started out with the Script Bowl. This year the languages represented was Jython, Groovy, Clojure, Scala and JRuby. I think they all did a pretty good job of showcasing the languages, although it’s very hard to do that in such a small timeframe. I think I sympathized the most with Rich Hickey (creator of Clojure) – the reason being that the Clojure model is the most dissimilar from the rest of the languages. But this dissimilarity is actually the key to understanding why Clojure is so powerful, so if you don’t understand it, you’re just going to be turned of by Clojure’s weird surface semantics. (Hint: they are not weird, they are necessary and powerful and really cool). Rich did a valiant effort to conveying this by talking a lot about the data structures that is Clojure, but I’m unsure how much of it actually penetrated.

Tom did a great job with the JRuby demos – he had a good flash 3d game running using a JRuby DSL, and then some slides showcasing how much benefit JRuby gets from the Ruby community. Good stuff.

After that I went to Rich’s Clojure talk. I’ve seen him give similar talks several times, but I don’t get tired of seeing this. As usual, Rich did a good job of giving a whirlwind tour of the language.

After lunch I went to the talk by Konstantin about JetBrains MPS. I was curious about MPS since I’ve been spending time with Intentional lately. I came away from the talk with a pretty different view of MPS compared to going in, actually. My initial reaction is that MPS seems to be pretty limited to what you can do with Intentional.

Then it was time to see Yehuda Katz talk about Ruby – this was a great intro to Ruby and I think the audience learned a lot there.

The first evening of JavaOne was really crazy, actually. I ended up first going to Brian Goetz and John Rose’s talk about building a Renaissance VM. This was a bit of an expansion of John’s CommunityOne talk, and gave a good overview of the different pieces we’re looking at in JSR 292, and also other things that should be in the JDK in some way to make a multi-language future possible.

Tobias Ivarsson gave a BOF about language interoperability on the JVM. This ended up being more about the interface injection feature that Tobias has been hacking on. We had some pretty good discussion, and I think we ended up with a feeling that we need to discuss this a bit more – especially if the API should be push or pull based. Good session by Tobias, though.

And then it was finally time for my BOF, called Hacking JRuby. This was actually a pretty mixed things, containing lots of funny small pieces of JRuby knowledge that can be useful if you want to do some weired things with JRuby. The slides can be found here: http://dist.codehaus.org/jruby/talks/HackingJRuby.pdf. I think the talk went pretty well, although it was in a late slot so not many people showed up.

The final session of the day was a BOF called JRuby Experiences in the Real World. This ended up being a conversation between about 10-12 people about their JRuby experiences. Very interesting.

After that I was totally beat, and ended up going home and crashing. So that was my first day at JavaOne.

1 Comment | By Ola Bini | In: blogging | tags: clojure, communityone, javaone, jruby, presentations. | #

Currently sitting in a session on day two of the Google I/O conference. The morning opened up with the keynote and announcement of Google Wave, which is something that seems very cool and has a lot of potential. Very cool start of the day.

After that I watched Ben and Dion talk about Bespin. I hadn’t seen Bespin before – it was definitely interesting, although I will be hard pressed to give up Emacs any day soon.

During lunch I came up with a fun idea, but it required something extra. I talked to Jon Tirsen, a Swedish friend from his ThoughtWorks days, who is on the Google Wave team – and he managed to get me an early access account for Google Wave. So I spent the next few hours hacking – and was able to unveil an Ioke Wave Robot during my talk. It is basically only a hello world thing, but it is almost certainly the first third-party Google Wave code… You can find it at http://github.com/olabini/iokebot. It is deployed as iokebot@appspot.com so when you have your Wave account you can add it to any waves. Very cool. I do believe there is a real potential for scripting languages to handle these tasks. Since most of it is about gluing services together, dynamic languages should be perfectly suited for it.

Finally I did my talk about JRuby and Ioke – that went quite well too. The video should be up on Google sooner or later.

And that was basically my Google I/O experience. Very nice conference and lots of interesting people.

3 Comments | By Ola Bini | In: blogging, ioke | tags: conferences, google io, google wave, ioke, jruby, presentations, san francisco. | #

I have just released version 0.4 of JtestR. This version doesn’t really provide any new features – instead it contains updates to all included libraries, upgrading JRuby to 1.2 being the most important one.

More information here: http://jtestr.codehaus.org.

No Comments | By Ola Bini | In: blogging | tags: jruby, jtestr, release, rspec, testing. | #

It is less than a month to RailsWayCon in Berlin, so I thought I’d mention it here. This look like it will be a very nice conference. The dates are May 25 to May 27, in Berlin, Germany.

I will do two presentations and one keynote there. The presentations will be “JRuby Internals” and “Ioke for Ruby developers”. The keynote is called “Present and future of programming languages” and will feature my typical kind of ranting about programming languages.

Anyway. Hope to see you in Berlin! You can find more information here: http://it-republik.de/conferences/railswaycon.

1 Comment | By Ola Bini | In: blogging, ioke | tags: berlin, conference, conferences, ioke, jruby, keynote, presentations, railswaycon. | #

I will be doing two geek nights, organized by JAOO, in Aarhus, Denmark, and Copenhagen, Denmark. I will be talking about JRuby.

If you’re interested in the Aarhus event, it will be held on May 5th, and you can find more information and register here: https://secure.trifork.com/aarhus-2009/freeevent/register.m?eventOID=1845.

If you’re interested in the Copenhagen event, it will be held on May 6th, and registration and information can be found here: https://secure.trifork.com/aarhus-2009/freeevent/register.m?eventOID=1846.

Hope to see many of you there!

No Comments | By Ola Bini | In: blogging | tags: aarhus, copenhagen, geek night, jaoo, jruby. | #

This is the third post in a series detailing information about the newly announced Google App Engine support for Java. In this post I thought I’d go through the steps you need to take to get a JRuby on Rails application working on GAE/J, and also what kind of characteristics you should expect from your application.

You need a fairly new copy of JRuby. Most of the changes needed to JRuby was added to JRuby trunk right after the JRuby 1.2 release, so check out and build something after that. The newest Rails version works fine too.

Once you have the basic Rails app set up, there are few things you need to do. First of them is to install Warble and pluginize it, and finally generate the Warble configuration file. You do that by doing “jruby -S gem install warble”, “jruby -S warble pluginize” and then “jruby -S warble config”. The last two should be done in the root of the Rails application.

You should freeze the Rails gems too. Once you have done that, you need to go through all the files there and remove anything that isn’t necessary. As it turns out, GAE/J has a hard limit on a 1000 files, and a typical Rails application will end up with much more files then that. You can remove all of ActiveRecord, all the test directories and so on.

Since you’re on GAE/J, you won’t need ActiveRecord, so you should not load it in config/environment.rb. The next step is to modify your warble.rb file. These are the things you need to do:

First, make sure that the needed GAE/J files are included, by doing:

config.includes = FileList[“appengine-web.xml”, “datastore-indexes.xml”]

You should also set the parameters for how many runtimes will be started:

config.webxml.jruby.min.runtimes = 1
config.webxml.jruby.max.runtimes = 1
config.webxml.jruby.init.serial = true

The last option is available in trunk version of JRuby-rack. If you don’t have min=1 and max=1 then you need this option set, because otherwise JRuby-rack will actually start several threads to initialize the runtimes.

Finally, to be able to use newer versions of the libraries, you need to set what Java libraries are used to the empty array:

config.java_libs = []

You will add all of the jar-files later, in the lib directory.

The last configuration option that I added is something to allow Rails to use DataStore as a session store. You can see how this is done in YARBL.

I have set several options in my appengine-web.xml file. The most important ones are to turn off JMX and to set os.arch to empty:

      <property name="jruby.management.enabled" value="false" />
      <property name="os.arch" value="" />

This is all pretty self explanatory.

One thing that I still haven’t gotten to work correctly is “protect_from_forgery”, so you need to comment this out in app/controllers/application.rb.

You need to put several jar-files in the lib-directory, and you actually need to split the jruby-complete jar, since it is too large for GAE/J in itself. The first jar-file is the appengine-api.jar file. You also need a late build of jruby-rack, and finally you need the different slices of the jruby-complete jar. I use a script like this to create several different jar-files:

#!/bin/sh

rm -rf jruby-core.jar
rm -rf ruby-stdlib.jar
rm -rf tmp_unpack
mkdir tmp_unpack
cd tmp_unpack
jar xf ../jruby-complete.jar
cd ..
mkdir jruby-core
mv tmp_unpack/org jruby-core/
mv tmp_unpack/com jruby-core/
mv tmp_unpack/jline jruby-core/
mv tmp_unpack/jay jruby-core/
mv tmp_unpack/jruby jruby-core/
cd jruby-core
jar cf ../jruby-core.jar .
cd ../tmp_unpack
jar cf ../ruby-stdlib.jar .
cd ..
rm -rf jruby-core
rm -rf tmp_unpack
rm -rf jruby-complete.jar

This creates two jar-files, jruby-core.jar and ruby-stdlib.jar.

These things should more or less put everything in order for you to be able to deploy your application to App Engine.

YARBL

As part of my evaluation of the infrastructure, I created a small application called YARBL. It allows you to have blogs, and post posts in them. No support for comments or anything fancy at all really. But it can be expanded into something real. I use both BeeU and Bumble in YARBL. BeeU allow me to make sure that only logged in users that are administrators can actually post things or change the blog. This support was extremely easy to add through the Google UserService.

You can see a (hopefully) running version at http://yarubyblog.appspot.com. You can find the source code in my GitHub repository: http://github.com/olabini/yarbl.

Bumble

Bumble is a very small wrapper around DataStore, that allow you to create data models backed by Google’s DataStore. It was developed to back YARBL, so it really only supports the things needed for that application.

This is what the data model for YARBL looks like. This should give you a feeling for how you define models with Bumble. One thing to remember is that the DataStore actually allows any properties/attributes on entitites, so it fits a language like Ruby very well.

class Person
  include Bumble

  ds :given_name, :sur_name, :email
  has_many :blogs, Blog, :owner_id
end

class Blog
  include Bumble

  ds :name, :owner_id, :created_at
  belongs_to :owner, Person
  has_many :posts, :Post, :blog_id, :iorder => :created_at
end

class Post
  include Bumble

  ds :title, :content, :created_at, :blog_id
  belongs_to :blog, Blog
end

To actually use the model for something, you can do things like these:

Blog.all

Post.all({}, :limit => 15, :iorder => :created_at)

blog = Blog.get(params[:id])
posts = blog.posts

Blog.create :name => name, :owner => @person, :created_at => Time.now

Post.all.each do |p|
  p.delete!
end

Here are most of the supported methods. The implementation is incredibly small and you really can’t go wrong with it. Of course, it is not tuned at all, so it does lots of fetches it could avoid. I’m happily accepting patches! The code can be found at http://github.com/olabini/bumble.

BeeU

When working with Google’s user service, you can use BeeU – a very small framework for helping with some things. You basically get a few different helper methods. There are three different filter methods that can be used. These are assign_user, assign_admin_status and verify_admin_user. The first two will create instance variables called @user and @admin respectively. The @user variable will contain the UserService User object, and @admin will be either true or false if the user is logged in and is an administrator or not. The last one will check that the current user is an administrator. If not logged in, it will redirect to a login page, and if logged in but not administrator, it will respond with a Not Authorized. These three methods should all be used as before filters.

There is a high level method called require_admin that you can use to point out what methods should be protected with admin access. This is really all you need.

Finally, there are two methods that generate a login-URL and a logout-URL, both of these will redirect back to where you were when the URL’s were generated.

BeeU can be found in my GitHub repository: http://github.com/olabini/beeu.

Summary

Overall, JRuby on Rails works very well on the App Engine, except for some smaller details. The major ones are the startup cost and testing. As it happens, you can’t actually get GAE/J to precreate things. Instead you’ll have to let the first release take the hit of this. Now, GAE/J does a let of preverifying of bytecodes and so on, so startup is a bit more heavy than on other JDKs. One runtime takes about 20 seconds wall time to startup, so the first hit takes some time. The good news is that this used to be worse. The last few weeks, the infrastructure has gotten a lot faster, and I’m confident this will continue to improve. It is still a problematic thing though, since you can’t precreate runtimes, which means that some request will end up taking quite a bit longer than expected.

It’s interesting to note that performance is actually pretty good once it gets running. I’ve seen between 120ms to 500ms for a request, depending on how much calls to DataStore is involved on the page – these times are not bad, considering what the infrastructure needs to do. It also seems mostly limited to the data access. If I’d had time to integrate memcaching, I could probably improve these times substantially.

The one remaining stickler for me is still testing. It’s not at all obvious how to do it, and as I noted in my earlier post there are some ways around it – but they don’t really fit in the way most Rails applications are built. In fact, I have done mostly manual testing on this application, since the cost of automating it seemed to be costly.

In all, Google App Engine with JRuby on Rails, is a really compelling combination of technology. I’m looking forward to the first ThoughtWorks project with these pieces.

55 Comments | By Ola Bini | In: blogging | tags: cloud, gae, google app engine, java, jruby, jruby on rails. | #

I am currently sitting with Ribs, trying to decide how the definitions for associations should look like. I’ve also decided that I don’t want to use the current scheme for properties either. I also have some interesting constraints. First, I do not want to evaluate the block using instance_eval, so introducing nice method_missing hooks in the block context is out (so I can’t do anything like “has n” as DataMapper). Also, all definitions need to be able to take additional parameters in a hash. So something like “rib.has * :Posts” won’t work either.

I have come up with something that I’m reasonably happy with. It’s quite terse but definitely readable. It doesn’t pollute any namespaces. And hopefully it will only be used when the model need to differ from the conventions. I’m thinking about taking conventions a step further with Ribs and automatically try to figure out associations based on naming and foreign keys – but I haven’t decided about that yet.

All of these examples are supposed to be executed inside the Ribs! block, with r being the Rib parameter. I’ve decided to allow the table name as a parameter to the Ribs! method call instead of being something you set inside.

First of all, setting which property/properties are a primary key, I’m thinking about one or more of these alternatives – where track_id is the property name:

r.track_id :primary_key
r.track_id :primary_key => true
r.track_id = :primary_key
r.track_id.primary_key!

I’m skittish about the track_id= alternative, but the others I think can coexist quite well.

The next thing is to handle the mapping from one property name into a different column name (if no column name is specified, the property name will be used, of course):

r.title :column => :track_title
r.title.column = :track_title

Nothing very extraordinary here. I would probably like to have both alternatives here.

The next one is more interesting. I haven’t yet decided how to handle specifying primitive types. This code is supposed to handle the case that in ActiveRecord are covered with has_one and belongs_to. I don’t really see why they have to be different, really, so I’m thinking about doing something like this:

r.blog :is_owner
r.blog :is => Blog

I’m skittish about :is_owner, but it also feels slightly icky to have to specify the type of all single associations. This is definitely an area I would like to have some good ideas about.

Finally we come to the one-to-many and many-to-many associations. What makes this different is that I can have both a Set and a List (and also Map) since Hibernate handles these for me:

r.comments :list_of => Comment
r.comments :set_of => Comment
r.comments :is_set
r.comments :is_list

Once again I’m not sure how to handle it in such a way that you don’t have to write the name of the other entity in all definitions.

Anyway, this is what I currently have. I would appreciate innovative ideas here – especially if they are innovative within the constraints I’ve set for myself.

3 Comments | By Ola Bini | In: Uncategorized | tags: associations, hibernate, jruby, ribs, ruby. | #

Ribs is now available from Gems. You can install it using this command:

jruby -S gem install ribs

Much pleasure, and do give your comments and ideas to me.

2 Comments | By Ola Bini | In: Uncategorized | tags: gems, hibernate, jruby, ribs. | #

I am extremely pleased to announce the first release of Ribs.

Ribs is a library for JRuby, that allows you to persist Ruby objects using Hibernate. Some time ago I wrote about ActiveHibernate. I have now decided to implement this myself, and the result is the Ribs project.

The first release is quite minimal in scope. You can define and work with models that have primitive values only – there is no support for associations. You can find, create, update and delete model objects. All of this uses Hibernate and JDBC.

To get started, you just define that an object is to be a Ribs model:

class Artist
  Ribs!
end

Once that’s done, you can start working with it.

Of course, this is just the beginning. I have a quite long list of things I’d like to have in the project, but I felt the need to release quickly and often to be more important than to implement everything first.

This release is not really for production usage, but I would appreciate if people tried it out and came with suggestions. The current planned features can be found in the PLAN file, in the git repository.

More documentation can be found here: http://olabini.com/projects/ribs/doc.
You can download the gem at: http://olabini.com/projects/ribs/downloads/ribs-0.0.1.gem.
The git repository is at: git://github.com/olabini/ribs.git.

Ribs will soon be available in the regular gem repositories – as soon as my Rubyforge project has been approved.

The project is released under the MIT license.

5 Comments | By Ola Bini | In: Uncategorized | tags: activehibernate, database, hibernate, jruby, ribs. | #

Had an interesting bug using Hibernate from JRuby today. Totally unexpected actually. Interestingly, it actually exposed a problem with dynamic dispatch when going into a static language. To a degree I guess it’s about getting our overload resolution more correct, but it seems like a general rule will be hard.

Basically, my problem was this. I was calling update() on org.hibernate.Session. Now, I used the version that take a String with the entity name, and the actually entity as the other parameter. So the signature update(String, Object) was the one I was aiming for. Sadly, things failed, and kept on failing. And I really couldn’t figure out why. I got this lovely error message: org.hibernate.MappingException: Unknown entity: java.lang.String. This problem can show up from several different reasons, so Google didn’t help.

And then, after tracing the calls for a bit, I finally understood. It just so happens that the default implementation of Session (called org.hibernate.impl.SessionImpl), have a few public update methods that are not part of the Session interface. One of them has the signature update(Object, Serializable). The first parameter is the object to update, and the serializable parameter is the id. JRuby was very helpful in choosing to call that method instead of the update(String, Object) one, since my entity happened to be serializable. Of course, this meant that Hibernate tried to persist a String, instead of my real object, and this fails. The workaround was simple in this case: just use the single argument version of update, since the entity name can be figured out from the object.

But in general these kind of problems will show up sometimes – it’s the price you pay for having an extremely flexible dynamic programming language, interfacing with a statically typed language. But we can improve the overload resolution, and also make it possible to control it more explicitly. I’m currently thinking that it might be a good plan to have some debug flags that will give you some output about overload resolution and things like that too. What do you think? How would you solve this in JRuby?

7 Comments | By Ola Bini | In: Uncategorized | tags: hibernate, jruby, overloading. | #