• BlogNav:
• Archive of all posts

Rutie and Magnus, Two Good Ways to Build Ruby Extensions in Rust

Posted: Tue, 18 April 2023 | permalink | No comments

I wrote the Ruby bindings for the Enquo Project, my attempt to bring queryable encryption to all databases, using the Rutie library. Recently, I’ve rewritten the bindings to use Magnus instead, and I thought I’d put down my thoughts about the whole situation.

The Story So Far

The Enquo Project core cryptography is all written in Rust, as seems to be the vogue these days. Rust is fast, safe, and easily interoperable with most of the rest of the modern software development ecosystem, making it a good choice as a language to implement the cryptographic primitives that Enquo needs, like Order-Revealing Encryption.

Of course, since not everyone writes their applications in Rust, we need to provide the functionality of the Enquo client in the languages that people do use, such as Ruby, Python, and so on. Since re-writing all that cryptographic code in a myriad of languages would be tedious and error-prone, we instead provide bindings to the “core” Rust code. These are just thin shims of code that translate the data types and function calls between Rust and the target language.

As I’m most familiar with Ruby and its development ecosystem (particularly Ruby on Rails), it was natural that I’d make Ruby bindings for Enquo as my first target. Rummaging around, it seemed that Rutie was a good library to use, so off I went.

What are Rutie and Magnus, Anyway?

Both libraries share the same goal: provide the ability to write some Rust code, run that through a compiler, and produce something that can be loaded by the Ruby interpreter and used just like any other Ruby class. They’re both fairly “high level” interfaces, trying to abstract away much of the gory details, and do a lot of the common “heavy lifting” that can make writing bindings fiddly and annoying. Things like mapping data types (like strings and integers) between Rust data types and the closest equivalents in Ruby.

This mapping never goes perfectly smoothly. For example, Ruby integers don’t have a fixed range of values they can represent – you can store a huge number like 2²⁵⁶ more-or-less as easily as you can the number 12. But Rust, being a lower-level language, only has a set of integer types that have fixed boundaries, like the u32 type, which can only store integers between zero and about four billion (2³² - 1, to be precise).

There’s also lots of little things that need to be just right, also, like translating the different memory management approaches of the languages, and dealing with a myriad of fiddly little issues like passing arguments and return values in and out of method calls, helpers for defining classes and methods (and pointing to the correct Rust functions), and so on.

All in all, these libraries are fairly significant pieces of work, and I’m mighty glad that someone else has taken on the job of building (and maintaining!) them.

So Why the Change?

Good question.

It’s important to say at the outset that there’s nothing particularly wrong with Rutie. I found using Rutie to be very straightforward, and the Ruby bindings came together very quickly and easily. If someone chose to use Rutie for their project, I’m sure they’d have a good experience.

What made me take the time to rewrite using Magnus was a set of a few tiny things, which together gave me enough of a shove to do the work.

Firstly, I’d had a hiccup with Rutie’s support of newer versions of Ruby, particularly 3.2 (PR). Also, I’d hit a couple of segfault issues, which were ultimately caused by Ruby garbage-collecting data out from underneath me. These were ultimately my fault, of course, but Rutie wasn’t helping me out in avoiding the problems in the first place.

Finally, while Rutie helped translate data types, there was still a bit of boilerplate and ugliness that needed to be included. This wasn’t a showstopper, but I’m appreciating the extra smoothness that Magnus provides here.

As an example, here’s what’s required in Rutie to get “native” Rust data types from Ruby method parameters (and the self reference to the current object):

fn enquo_field_decrypt_text(ciphertext_obj: RString, context_obj: RString) -> RString {
    let ciphertext = ciphertext_obj.to_str_unchecked();
    let context = context_obj.to_vec_u8_unchecked();

    let field = rbself.get_data(&*FIELD_WRAPPER);
    // etc etc etc

The equivalent in Magnus is just the function signature:

fn decrypt_text(&self, ciphertext: String, context: String) -> Result<String, magnus::Error> {

You can also see there that Magnus signals an exception via the Result return value, while Rutie’s approach to raising an exception involves poking the Ruby VM directly, which always struck me as a bit ugly.

There are several other minor things in Magnus (like its cleaner approach to wrapping structs so they can be stored in Ruby objects) that I’m appreciating, too. Never discount the power of ergonomics for making a happy developer.

The End Result

I spent a bit over half of last weekend doing the rewrite – maybe ten hours of so. Since Magnus did more type checking and data validation, and its approach to error handling was smoother, I took the opportunity to rewrite a bunch of Ruby “wrapper” code I’d written (which just existed to check things like ranges of values and string encodings) into Rust, as well.

To make sure that the conversion was accurate, I added a heap more unit tests to the bindings. I also took the opportunity to restructure the codebase to split the code for the different Ruby classes into separate files, which I hadn’t done initially as the code had originally accreted, rather than being purposefully written.

All up, though, my rewrite ended up removing over 60 lines (excluding the extra specs I added):

$ git diff --stat -- lib ext/enquo/src
 ruby/ext/enquo/src/field.rs       | 342 ++++++++++++++++++++++++++++++++++++++
 ruby/ext/enquo/src/lib.rs         | 338 ++++---------------------------------
 ruby/ext/enquo/src/root.rs        |  39 +++++
 ruby/ext/enquo/src/root_key.rs    |  67 ++++++++
 ruby/lib/enquo.rb                 |   6 +-
 ruby/lib/enquo/field.rb           | 173 -------------------
 ruby/lib/enquo/root.rb            |  28 ----
 ruby/lib/enquo/root_key.rb        |   1 -
 ruby/lib/enquo/root_key/static.rb |  27 ---
 9 files changed, 479 insertions(+), 542 deletions(-)

Considering that I was translating from a “higher level” language into a “lower level” one, the removal of so much code is quite remarkable. Magnus was able to automagically replace rather a lot of raise ArgumentError if something.isnt_right code in those .rb files.

So, in conclusion, if you, too, are building Ruby extensions in Rust, while Rutie is a solid choice (and you probably should stick with it if you’re already using it), I highly recommend giving Magnus a look for your next extension.

---