RSA parameters in OpenSSL, Ruby and Java


I would just like to publish this information somewhere, so that Google can help people find it easier than I did.¬† If you have ever wondered how the internal OpenSSL RSA parameters map to the Java parameters on RSAPrivateCrtKey, this little table will probably help you a bit. There are three different names in motion here. The first one is the internal field names in OpenSSL. These are also used as method names in Ruby. The second name is what gets presented when you use something like to_text on an RSA key. The third name is what it’s called in Java.

  • n == modulus == modulus
  • e == public exponent == publicExponent
  • d == private exponent == privateExponent
  • p == prime1 == primeP
  • q == prime2 == primeQ
  • dmp1 == exponent1 == primeExponentP
  • dmq1 == exponent2 == primeExponentQ
  • iqmp == coefficient == crtCoefficient


Ruby Security quick guide


I’ve looked around a bit and it seems that there is no real good guide to security programming in Ruby. Neither is there any book available (although Maik Schmidts book Enterprise Recipes with Ruby and Rails will be the best reference once it arrives). The aim for this blog entry will be to note a few things you often would like to do, and how you can do it with Ruby. The focus will be mostly on the cryptographic APIs for Ruby, which doesn’t have much documentation either. In fact, the best documentation for this is probably the OpenSSL documentation, once you learn how to map it to the Ruby libraries.

You want to avoid clear text passwords in your database

One of the very good properties of handling passwords, is that you usually don’t need to actually know what they are. The only thing you need to be able to do is to reset them if someone forgets their password, and verify the correct combination of username and password. In fact, many practitioners feel better if they don’t need to have the responsibility of knowing the passwords of every user on their system, and conversely I feel much better knowing that my password is secure from even the administrators of the system. Not that I ever use the same password to two different systems, but someone else might… =)

So how do you solve this easily? Well, the easiest way – and also the most common way – is to use a digest. A digest is a mathematical function that takes as input a series of numbers of any length and returns a large number that represents the input text. There are three properties of digests that makes them useful: the first is that a small change in the input text generate a large effect in the outcome data, meaning that olabini and olbbini will have quite different digests.

The second is digests generally have a good distribution and small risk of collisions – meaning that it’s extremely unlikely that two different texts have the same output for a specific digest algorithm. The third property is that they are fundamentally one way mathematical operations – it’s very easy to go from plain text to digest, but extremely hard to go in the other direction.

There are currently several algorithms used for digests. MD5 and SHA-1 is by far the most common ones. If you can avoid MD5, do so, because there have recently been some successful attacks against it. The same might happen with SHA-1 at some point, but right now it’s the best algorithm for these purposes. Oh, and avoid doing a double digest – digesting the output of a digest algorithm – since this generally makes the plain text easier to crack, not harder.

Let’s see some code:

require 'openssl'

digest = OpenSSL::Digest::SHA1.hexdigest("My super secret pass")
p digest # => "dd5f30310682e5b41e122c637e8542b1b39466cf"

digest = OpenSSL::Digest::SHA1.hexdigest("my super secret pass")
p digest # => "f923786cc72ed61ae31325b6e8e285e6c35e6519"

d = OpenSSL::Digest::SHA1.new
d << "first part of pass phrase"
d << "second part of pass phrase"
p d.hexdigest # => "f13d7bdee0634c017babb8c72dcebe18f9e0598e"

I have used two different variations here. The first one, calling a method directly on the SHA1 class, is useful when you have a small string that you want to digest immediately. It’s also useful when you won’t need different algorithms or send the digest object around. The second method allows you to update the string data to digest several times and the finally generate the end digest from that. I have taken the liberty of using the methods called hexdigest for both cases – this is because they are more readable. If you replace hexdigest with digest, you will get back a Ruby string that contains characters from 0 to 255 instead, which means they doesn’t print well. But if you were to compare the result of hexdigest and digest, you will see that they return the same data, just in different formats.

So as you can see, this is extremely easy to incorporate in your code. To verify a password you just make a digest of the password the person trying to authenticate sent in to you, and compare that to what’s in your database. Of course, the approach is generalizable to other cases when you want to protect data in such a way that you can’t recover the data itself.

Finally, be careful with this approach. You should generally combine the password with something else, to get you good security. There are attacks based on something called rainbow tables that makes it easy to find the password from a digested password. This can be avoided using a salt or other kind of secret data added to the password.

You want to communicate with someone securely
When you want to so send messages back and forth between actors, you generally need a way to turn it unreadable and then turn it back into something readable again. The operation you need for this is called a cipher, but there are many kinds of ciphers, and not all of them are right. For example, rot-13 be considered a cipher, but there is no real security inherent in it.

In general, what you want for communication between two parties is a symmetric cipher using a secret key. A symmetric cipher means that you can use the same key to “lock” and “unlock” a message – or encrypt and decrypt it. There are other kinds of ciphers that are very useful, which we’ll see in the next example, but symmetric ciphers are the most commonly used ciphers since they are quite easy to use and also very efficient. For a symmetric cipher to be completely safe, the key should always be as long as the data that should be encrypted. Generally this doesn’t happen, since key distribution becomes a nightmare, but the current approaches are reasonably sure against most kinds of cracking. Coupled with asymmetric ciphers, they become extremely useful.

There are three things you need to use a symmetric cipher. The first one is the algorithm. There are loads of different kinds of symmetric ciphers around. Which kind you will use doesn’t matter that much as long as you choose something that is reasonably sure. One of the more widely used algorithms is called DES. In it’s original form DES should definitely not be used (since the key length is only 56 bits, it’s actually not that hard to crack it.). There is another form of DES called Triple DES, or DES3, which effectively gives you more security. DES3 might work in some circumstances, but I would recommend AES in almost all cases. AES come in three varieties called AES-128, AES-192 and AES-256. The difference between them is the key length. As you might guess, AES-128 needs a 128 bit long key, AES-192 a 192 bit long and AES-256 a 256 bit long. These key lengths all give reasonable security. The more security you need the longer key algorithm you can choose. The tradeoff is that the longer the key is, the more time it will take to encrypt and decrypt messages.

Once you have an algorithm, you need a key. This can come in two varieties – either you get a key from a humans, where that key is generally a password. Otherwise you might want to automatically generate a key. This should be done with a secure random number generator – NOT with rand().

Depending on which algorithm you choose, you might also need something called an IV (initialization vector). The algorithms that require an IV is called cyclic block ciphers (CBC) and will work on a small amount of bytes at the same time. In the case of AES-128, a block of 16 bytes are generated on every cycle of the algorithm. These 16 bytes will be based on the algorithm, the key, and the 16 bytes generated the last time. The problem is that the first time there were no bytes generated, which means these will have to be initialized another way and this is where the IV comes in. It’s just the first block that will be used for generating the first real block of data. The IV does not need to be secured and can be sent in the clear. In Ruby, if you don’t provide an IV when initializing your cipher, the default will be a part of the string “OpenSSL for Ruby rulez!”. Depending what length the IV should have, a substring will be used.

So, let’s take a look at some code. This code will just encrypt a message with a password and then decrypt it again:

require 'openssl'

cipher = OpenSSL::Cipher::AES128.new("CBC")
cipher.encrypt
cipher.key = "A key that is 16"
cipher.iv =  "An IV that is 16"

output = ""

output << cipher.update("One")
output << cipher.update(" and two")
output << cipher.final

p output # => "\023D\aL\375\314\277\264\256\245\225\a\360|\372+"

cipher = OpenSSL::Cipher::AES128.new("CBC")
cipher.decrypt
cipher.key = "A key that is 16"
cipher.iv =  "An IV that is 16"

output2 = ""

output2 << cipher.update(output)
output2 << cipher.final

p output2 # => "One and two"

We first create a cipher instance, giving it CBC as the type. This is the default but will generate a warning if you don’t supply. We tell the cipher to encrypt, then initialize it with key= and iv=. The key is 16 bytes because it’s a 128 bit cipher, and the IV is 16 bytes because that’s the block size of AES. Finally we call update on the data we would like to encrypt. We need to save the return value from the update call, since that’s part of the generated cipher text. This means that even encrypting really large texts can be very efficient, since you can do it in smaller pieces. Finally you need to call final to get the last generated cipher text. The only difference when decrypting is that we call decrypt on the object instead of encrypt. After that the same update and final calls are made. (Am I the only one who thinks that encrypt and decrypt should be called encrypt! and decrypt!)?

That’s how easy it is to work with ciphers in Ruby. There are some complications with regards to padding, but you generally don’t need to concern yourself with that in most applications.

You want nodes to be able to communicate with each other securely without the headache of managing loads of secret keys

OK, cool, symmetric ciphers are nice. But they have one weakness, which is the secret key. The problem shows up if you want to have a network of computers talking to each other securely. Of course, you can have a secret key for all of them, which means that all nodes in the network can read messages to other parties. Or you can have a different secure key for each combination of nodes. That’s ok if you have 3 nodes (when you will just need 3 secret keys). But if you have a network with a 100 nodes that all need to communicate with each other you will need 4950 keys. It will be hard to distribute these keys since they need to be kept secure, and generally just managing it all will be painful.

The solution to this problem is called asymmetric ciphers. The most common form of this is public key cryptography. The idea is that you need one key to encrypt something, and another key to decrypt it. If you have key1 and encrypt something with it, you can’t decrypt that something with key1. This curious property is extremely useful. In the version of public key cryptography you generate two keys, and then you publish one of the keys widely around. Since the public key can’t be used to decrypt content there is no security risk in not keeping it secret. The private key should obviously be kept private, since you can generate the public key from it. What does this mean in practice? Well, that in your 100 node network, each node can have a key pair. When node3 wants to send a message to node42, node3 will ask around for the public key of node42, encrypt his message with that and send it to node42. Finally, node42 will decrypt the message using his private key.

There is one really large downside with these ciphers. Namely, it is quite expensive operations, even compared to other ciphers. So the way you generally solve this is to generate a random key for a symmetric cipher, encrypt the message with that key and then encrypt the actual key using the asymmetric cipher. This is how https works, it’s how SSH works, it’s how SMIME and PGP works. It’s generally a good way of matching the strengths and weaknesses of ciphers with each other.

So how do you use an asymmetric cipher? First you need to generate the keys, and then you can use them. To generate the keys you can use something like this:

require 'openssl'

key = OpenSSL::PKey::RSA.generate(1024)
puts key.to_pem
puts key.public_key.to_pem

This will give you the output in form of one private key and one public key in PEM format. Once you have this saved away somewhere, you can start distributing the public key. As the name says, it’s public so there is no problem with distributing it. Say that someone has the public key and wants to send you a message. If the PEM-encoded public key is in a file called public_key.pem this code will generate a message that can only be decrypted with the private key and then read the private key and decrypted the message again.

require 'openssl'

key = OpenSSL::PKey::RSA.new(File.read("public_key.pem"))
res = key.public_encrypt("This is a secret message. WOHO")
p res

key2 = OpenSSL::PKey::RSA.new(File.read("private_key.pem"))
res2 = key2.private_decrypt(res)
p res2

Note that the methods we use are called public_encrypt and private_decrypt. Since the public and private prefix is there, there has to be a reason for it. And there is, as you’ll see soon.

You want to prove that it was you and only you who wrote a message

Being able to send things in private is all good and well, but if you distribute you public key wide you may never know who you get messages from. Or rather, you know that you get a message from their addresses (if you’re using mail) – but you have no way of ensuring that the other person is actually who they say they are. There is a way around this problem too, using asymmetric ciphers. Interestingly, if you encrypt something using your private key, anyone with your public key can decrypt it. THat doesn’t sound very smart from a security perspective, but it’s actually quite useful. Since you are the only one with your private key, if someone can use your public key to decrypt it, then you have to be the one who wrote the message. This have two important consequences. First, you can always trust that the person who wrote you a message was actually the one writing it. And second, that person can never retract a message after it’s been written, since it’s been signed.

So how do you do this in practice? It’s called cryptographic signatures, and OpenSSL supports it quite easily. If you have the keys we created earlier, you can do something like this using the low level operations available on the keys:

require 'openssl'

pub_key = OpenSSL::PKey::RSA.new(File.read("public_key.pem"))
priv_key = OpenSSL::PKey::RSA.new(File.read("private_key.pem"))

text = "This is the text I want to send"
signature = priv_key.private_encrypt(text)

if pub_key.public_decrypt(signature) == text
  puts "Signature match"
else
  puts "Signature didn't match!"
end

There happens to be a slight problem with this code. It works, but if you want to send the signature along the message will always be twice the size of the original text. Also, the larger the text to encrypt the longer it takes. The way this is solved in basically all cases is to first create a digest of the text and then sign that. The code to do that would look like this:

require 'openssl'

pub_key = OpenSSL::PKey::RSA.new(File.read("public_key.pem"))
priv_key = OpenSSL::PKey::RSA.new(File.read("private_key.pem"))

text = "This is the text I want to send"*200

signature = priv_key.sign(OpenSSL::Digest::SHA1.new,text)

if pub_key.verify(OpenSSL::Digest::SHA1.new, signature, text)
  puts "Signature verified"
else
  puts "Signature NOT verified"
end

As you can see we use the sign and verify methods on the keys. We also have to send in the digest to use.

You want to ensure that a message doesn’t get modified in transit

Another usage of signatures, that we actually get for free together with them, is that there is no way to tamper with the message in transit. So even if you send everything in clear text, if the signature verification succeeds you know that the message can’t have been tampered with. The reason is simply this: if the text was tampered with, the digest that gets generated would be different, meaning that the signature would not be equal to what’s expected. And if someone tampered with the signature, it wouldn’t match either. Theoritically you can tamper with both the text and the signature, but then you’d first have to crack the private key used, since otherwise you would never be able to generate a correct signature. All of this comes for free with the earlier code example.

You want to ensure that you can trust someones public key
Another thing that can get troublesome with public and private keys is the distribution problem. When you have lots of agents you need to make sure that the public key you get from someone is actually their public key and that they are the ones they say. In real life this is a social problem as much as a technological. But the way you generally make sure that you can trust that someones public key is actually theirs is that you use somethong called a certificate. A certificate more or less is a signature, where the signer is someone you trust, and the text that is signed is the actual public key under question. If you trust someone that issues certificates, and that someone has issued a certificate saying that a public key belongs to a specific entity, you can trust that public key and use it. What’s more interesting is that certificates can also be signed, which means that you can trust someone, that someone can sign another certificate authority, which signs another certificate authority, which finally signs a public key. If you trust the root CA (certificate authority) you should also be able to trust all the certificates and public keys in the chain. This is how the infrastructure surrounding https works. You have a list of implicitly trusted certificates in your browser, and if an https site can’t be verified with this trust store, you will get one of those popup boxes asking if you really want to trust this site or not.

I will not show any code for this, since X509 certificates are actually quite well documented for Ruby. They are also a solution that takes some more knowledge to handle correctly, so a real book would be recommended.

You want to use https correctly

The protocol https is very useful, since it allows you to apply the whole certificate and asymmetric cipher business to your http communication. There is a library called net/https which works really well if you use it right. The problem is that I see lots and lots of example code that really doesn’t use it correctly. The main problem is the setting of verification level. If you have the constant VERIFY_NONE anywhere in your code, you probably aren’t as secure as you’d like to think. The real problem is that I’ve never been able to get Ruby to verify a certificate with VERIFY_PEER. No matter what I do in the manner of adding stores and setting ca_file and ca_path, I haven’t ever been able to make this work. It’s quite strange and I can’t seem to find much mention of it online, since everyone just uses VERIFY_NONE. That’s not acceptable to me, since if no verification happens, there is no way to know if the other party is the right web site.

This is one area where JRuby’s OpenSSL seems to work better, in fact. It verifies those sites that should be verified and not the others.

If anyone feels like illuminating this for me, I would be grateful – the state of it right now is unacceptable if you need to do really secure https connections.

Keep security in mind

My final advice is twofold and have nothing to do with Ruby. First, always keep security in mind. Always think about how someone externally can exploit what you’re doing, intercept your connections, or just listen to them. Programmers generally need to be much more aware of this.

The last thing I’ll say is this: start reading Cryptogram by Bruce Schneier. While not completely technical anymore, it talks about lots of things surrounding security and also gives a very good feeling about how you should think and approach security. Subscribe here.



Where is the Net::SSH bug


Yesterday I spent several hours trying to find the problem with our implementation of OpenSSL Cipher, that caused the Net::SSH gem to fail miserable during negotiation and password verification. After various false leads I finally found the reason for the strange behavior. But I really can’t decide if it’s a bug, and if it’s a bug where the bug is. Is it in Ruby’s interface to OpenSSL, or is it in Net::SSH?

No matter what cipher suite you use for SSH, you generally end up using a block cipher, mostly something like CBC. That means an IV (initialization vector) is needed, together with a key. The relevant parts of OpenSSL used is the EVP_CipherInit, EVP_CipherUpdate and EVP_CipherFinal family of methods. Nothing really strange there. The Ruby interface matches these methods quite closely; every time you set a key, or an IV, or some other parameter, the CipherInit method is called with the relevant data. When CipherUpdate is called, the actual enciphering or deciphering starts happening, and CipherFinal takes care of the final block.

At the point EVP_CipherFinal is called, nothing more should be done using the specific Cipher context. Specifically, no more Update operations should be used. The man page has this to say about the Final-methods:

After this function is called the encryption operation is finished and no further calls to EVP_EncryptUpdate() should be made.

Now, what I found was that same documentation is not part of the Ruby interface. And Net::SSH is actually reusing the same Cipher object after final has been called on it. Specifically, it continues the conversation, calling update a few times and then final. The general flow for a specific Cipher object in Net::SSH is basically init->update->update->final->update->update->final.

So what is so bad about this then? Well, the question is really this: what IV will the operations after the first final call be using? The assumption I made is that obviously it will use the original IV set on the object. Something else would seem absurd. But indeed, the IV used is actually the last IV-length bytes of encrypted data returned. Is this an obvious or intended effect at some level? Probably not, since the OpenSSL documentation says you shouldn’t do it. The reason it works that way is because the temporary buffer used in the Cipher context isn’t cleared out at the end of the call to final.

In contrast, the Java Cipher object will call reset() as part of the call to doFinal(). Where reset() will actually reset the internal buffers to use the original IV. So the solution is simple for encryption. Just save away 8 or 16 bytes of the last generated crypto text and set that manually as the IV after the call to doFinal. And what about decryption? Well, here the IV needs to be the last crypto text sent in for deciphering, not the result of the last operation.

So Net::SSH seems to work fine with JRuby now. I’m about to release a new version of JRuby-OpenSSL including these and many other things.

But the question remains. Is it a bug? If it is, is it in the Ruby OpenSSL integration, or in the Net::SSH usages of Ciphers? If it’s in the Net::SSH code, why does it actually work correctly when communicating with an SSH server? Or is this behavior of using the last crypto text as IV something documented in the SSH spec?

Enlightenment would be welcome.



The FINAL OpenSSL post?


Possibly.

I’ve checked in all functionality I will add to OpenSSL support in JRuby at this point. Of course, there will be more, but not concentrated in a spurt like this. Tomorrow I will modify the build process and then merge everything I’ve done into trunk.

Let’s back up a little. What have I accomplished? This: All OpenSSL tests from MRI run (except PKCS#7). That includes tests of SSL server and SSL client. Simple https-request also works. This is sweet. Everything else there is tests for in Ruby works. But… this is also the problem. Roughly half of Ruby’s OpenSSL library is not tested at all. And since the current OpenSSL initiative from my part is based on tests, I haven’t done anything that isn’t tested for.

So, some things won’t work. There is no support for Diffie-Hellman keys right now, for example. Will be easy to add when the time comes, but there isn’t any testing so I haven’t felt the need.

The only thing not there, as I said, is PKCS#7. That was just too involved. I’ll take care of that some other time, when someone says they want it… Or someone else can do it? =)

So, what this boils down too is that JRuby trunk will have OpenSSL support sometime tomorrow. Hopefully it will be useful and I can get on to other JRuby things. I have a few hundred bugs I would like to fix, for example…

Oh yeah, that’s true. Tomorrow will also be YAML day. I’ll probably fix some bugs and cut a new release of JvYAML. It’s that time, the bug count is bigger than it was, and JRuby needs some fixes. So that’s the order of day for tomorrow. First OpenSSL and then YAML. Any comments on this, please mail or comment directly here.

G’night.



Further OpenSSL progress


Sometimes it feels like the progress I’ve done with the OpenSSL library is almost swallowed up by all the new functions I’ve found that needs to be implemented. It’s an uphill battle.

implementation doesn’t handle the But, I’m happy to say that the latest digression is finished. I have successfully implemented the X509_STORE-family of functions plus all dependencies. In the end I had to create a wrapper for X509Certificate and my own PEM-reader and PEM-writer, since BouncyCastle’sOpenSSL specific aux trust information that can be appended to these certificates.

But anyway, that means there is a Java implementation of X509_STORE, X509_STORE_CTX, X509_VERIFY_PARAM, X509_LOOKUP, X509_LOOKUP_METHOD, X509_TRUST, X509_PURPOSE and a whole slew of others too. About the only thing I didn’t implement was X509_POLICY_TREE. That was just too much. I’ll tackle that hurdle if it seems Ruby needs it.

So, what is the status then? Coming in at almost 10 000 lines of code there is only three real parts left to do. Or, three parts left that have MRI test cases, I should say. Since there are quite a few files not tested in the MRI implementation. Like DH keys. Wow. But I ignore that for now. The current goal is the OpenSSL::X509::Store-family, the OpenSSL::PKCS7-family, and the SSL-stuff, the rest of the way. So, wish me luck.



Another OpenSSL woe.


My interesting OpenSSL implementation exercise continues. I am now very close. Very, very close. I’m actually so close that SSLSocket and SSLServer actually works, provided that you use Anonymous Diffie-Hellman (which is suicidal, but that’s another story). All of this have been submitted to my openssl-branch. What’s missing is the X509-store and PKCS#7. And the X509-store doesn’t really look good. Not good at all. It’s needed for full SSL support. But the bad thing is this: there isn’t any Java libraries that duplicate the functionality. Nada. At least not that I can find. The functionality needed is to read and write X509_store-formatted files and directories, to be able to add certificates and CRL’s and to verify against these a certificate, based on various interesting OpenSSL rules.

I wouldn’t say that I mislike OpenSSL. I wouldn’t say that I hate it either. It’s very impressive in many ways. But boy. It seems I have to port a substantial part of it to Java, and I’m not looking forward to it. I need to to do both a port, and add support for KeyStore and CertStore so the Java SSLEngine also can use the information. Will this be an interesting exercise? Oh yes.

So, without further ado, this is the plea of this blog post: If you know of any easier way to do this, please tell me. Now! (where “this” is the X509_STORE-family of functions.)



OpenSSL status report


I just checked in a few updates to my openssl branch for JRuby. Boy is it tricky getting everything right. It seems like every DER format Java crypto emits differs from the OpenSSL DER output. And it’s really incompatible. As an example I have been forced to reimplement the DER dumping for X509 certificates myself, and that’s not the only place.

But the work is actually going forward; as fast as I can make it when I’m only doing this in my spare time and my regular work takes lots of time right now. I can’t say for sure when it will be finished or usable, but I know for a fact that most of the MRI tests run now. What’s missing is PKCS#7, X509 CRL’s and X509 cert-stores, plus the regular SSL socket support. Not much, compared to what actually works.

But that leads to me to two issues. We have recently agreed that OpenSSL support will require BouncyCastle and Java 5. There is really no other way to get this working. 1.4.2 is fine for basic Digest support and some of the RSA/DSA support, but Java is sorely lacking in the ASN.1 and X509 department. Nothing whatsoever. Which is why we need BouncyCastle, which is fairly complete. I have only been forced to reimplement one or two central classes. Quite good. But SSL support is another story. As you may know, 1.4.2 has SSLSocket and SSLServerSocket. The problem is this: they aren’t any good. As a first, they are blocking, and there isn’t any support in 1.4.2 for NIO SSL sockets. Whoopsie. Which explains the requirement on Java 5. Tiger adds the SSLEngine class which can be used to implement NIO SSL, with the caveat that it heightens complexity. I have only taken a cursory look at this yet. Right now I want the other stuff working first, since there are so many dependencies on them.

But it’s really going forward. Now, if I only had this as my day job, this would be finished in a few days… Alas, that’s not the way it is. Expect further updates in a week or two.



OpenSSL in JRuby


This weekend I started work on OpenSSL in JRuby. It’s a pretty big undertaking, and it’s actually worse than I suspected from the beginning, so I’m going to tell a bit about my endeavours and have far I’ve gotten right now. First of all, if someone is interested in the work, I have set up a branch for this, since it will take time and be very big. The branch is called svn://svn.codehaus.org/jruby/branches/openssl.

The approach
I have investigated several variants of implementing OpenSSL in JRuby. One that seemed easy was to find a JNI-library that wraps OpenSSL in Java. But it seems there are no full scale versions implemented anywhere. I also checked around other script-on-JVM-languages to see how they had solved it, but I didn’t find anything close to OpenSSL functionality. Which left me with the approach I’ve decided to try out: implement as OpenSSL compatible Java code as possible with JCE and JSSE. I’m convinced this is doable, but right now it feels quite hard.

The progress
So, how far have I gotten these last days? Pretty far, but not far enough. I’m basing the work on MRI’s test suite for OpenSSL. From those I have test_digest.rb and test_cipher.rb passing all tests. This doesn’t sound like much, but especially test_cipher was a pain to get running.

The plan from hereon is to get the utils.rb-file to load, which means implementing OpenSSL::PKey::RSA and OpenSSL::PKey::DSA, getting the basics of OpenSSL::X509 in place and also find a way to fix OpenSSL::ASN1. Oh well, I’ve got loads of time for this. Or not. =)

The problem
The real problem when implementing this, is the fact that Ruby’s OpenSSL support is… Well, how shall I put it? Thin, you might say. It’s basically a a wrapper around the C-library, which means that the disconnect when implementing this functionality with JCE is quite large. Just translating OpenSSL cipher names to the JCE equivalent is a challenge. But the big problem with the ciphers was initiating the key and IV (initialization vector). I have tried all the PBE solutions available, including the versions in BouncyCastle ending with “-OPENSSL”. No luck.

The problem is that Ruby uses the function called EVP_BytesToKey, which, according to the documentation implements PKCS#5 1.5 with a few tricks up its sleeve. Not very nice. In the end I had to implement my own version of this to generate keys. And since I had to look like mad for this information, I will here give you the implementation to this function in Java. Just use the return value to initialize your own SecretKey-implementation and instantiate an IvParameterSpec and you should be set to go: (note, I release this into the public domain. And note, this is just quick, ported code to show the concept.)

    public byte[][] EVP_BytesToKey(int key_len, int iv_len, MessageDigest md, byte[] salt, byte[] data, int count) {
byte[][] both = new byte[2][];
byte[] key = new byte[key_len];
int key_ix = 0;
byte[] iv = new byte[iv_len];
int iv_ix = 0;
both[0] = key;
both[1] = iv;
byte[] md_buf = null;
int nkey = key_len;
int niv = iv_len;
int i = 0;
if(data == null) {
return both;
}
int addmd = 0;
for(;;) {
md.reset();
if(addmd++ > 0) {
md.update(md_buf);
}
md.update(data);
if(null != salt) {
md.update(salt,0,8);
}
md_buf = md.digest();
for(i=1;i<count;i++) {
md.reset();
md.update(md_buf);
md_buf = md.digest();
}
i=0;
if(nkey > 0) {
for(;;) {
if(nkey == 0) break;
if(i == md_buf.length) break;
key[key_ix++] = md_buf[i];
nkey--;
i++;
}
}
if(niv > 0 && i != md_buf.length) {
for(;;) {
if(niv == 0) break;
if(i == md_buf.length) break;
iv[iv_ix++] = md_buf[i];
niv--;
i++;
}
}
if(nkey == 0 && niv == 0) {
break;
}
}
for(i=0;i<md_buf.length;i++) {
md_buf[i] = 0;
}
return both;
}