December 2008

Perl SSL Modules

"You're in a twisty little maze of passages, all different"
    -- Will Crowther and Don Woods, "Colossal Cave Adventure"

If you want to implement SSL in perl, there's what seems like a dizzying maze of available options. Some better and some worse. Some more suited to one thing, and some to another. This article will help find your way through some of that maze (but not all of it). A thorough review would be nice, but this is not that.

I set out to write an IMAP mail client in perl, and with that in mind, I won't have much to say about SSL modules that are only or primarily concerned with the secure web (https) protocol. My goal was to come up with an implementation of a program that would work on the most different platforms, without too much trouble on my part, and without assuming that end-users can install perl modules.

I had to learn an unncessary amount about too many implementations none of which work extremely well. It's all a good argument for having some sort of perl-poobah that would oversee module development and bless certain recommended interfaces.

At any rate, for IMAP, I wanted the ability to establish a secure connection, and I wanted to do non-blocking reads from the socket descriptor from within a polling (select()) loop. I also wanted the ability to take an existing connection, and start up the SSL stuff in the middle, to support the IMAP STARTTLS command on an insecure connection.

I will briefly say that for the web in most cases, just use LWP. It will work as long as you have an appropriate SSL module installed. I can't give a concrete answer on what "appropriate" means, but I'm sure that Net::SSL (which is in turn part of Crypt::SSLeay) works. I've seen it reported that Net::SSLeay (and maybe also IO::Socket::SSL) will work, but that doesn't match with my testing. Then again, my error message wasn't the expected one of needing a module, so maybe it's my mistake, or maybe my LWP isn't up-to-date.

Overview

There are two main modules that have everything needed to set up an SSL connection. These are Crypt::SSLeay, and Net::SSLeay. Of the two, Crypt::SSLeay seems to be a bit more stable for my purposes than Net::SSLeay. It is hopefully possible to write anything you need with either of these modules. However, they don't provide a lot of convenience or quality documentation for this. Because of this, there's other included or add-on modules that attempt to simplify things.

The following modules are the things that I'm aware of that can do what we need:
Crypt::SSLeay
    Net::SSL
Net::SSLeay
    Net::SSLeay::Handle
    IO::Socket::SSL
        Net::Server::Proto::SSL

For my purposes, IO::Socket::SSL provides the most features I wanted, and Net::SSL close behind. They'll reverse positions if I can switch a live connection to SSL with Net::SSL, becauase Net::SSL looks to be better maintained and I think it deals with file handles better.. IO::Socket::SSL lives on top of Net::SSLeay. On the outside Net::SSLeay is primarily oriented towards https, but it supposedly has lots of good stuff hidden inside (and hidden from documentation too). You can use Net::SSLeay::Handle for non web things, but dealing with it's file descriptors is at least a pain, and possibly completely broken. Net::SSLeay itself can supposedly do non-web things, but it isn't well documented. Net::Server::Proto::SSL is so far "experimental". It's somehow related to IO::Socket::SSL.

Crypt::SSLeay and Net::SSL

This is where I started. Net::SSL is part of Crypt::SSLeay. Starting up an SSL connection is pretty straightforward:
use Net::SSL;
...
$sock = Net::SSL->new(PeerAddr => $host, PeerPort => $port,
			Proto => 'tcp', Timeout => 200) || die "sslsocket";
...
print $sock->getchunk;

Net::SSL includes a getchunk() function, which just grabs all the data available on the network (up to 32K) and gives it to you. It blocks until something is read, or a timeout expires. This worked great for my purposes.

Since I was using an IMAP server that was only running the secure port, this was also fine for testing. But if there's a way in Net::SSL to convert an existing connection to SSL, it's not obvious from the documentation.

Net::SSLeay::Handle

But, alas, I wanted to test things on a Mac, and Leopard does not have Net::SSL. But it does have Net::SSLeay (go figure). So my first attempt to find something that would work on that platform was Net::SSLeay::Handle. It looked a little less straight-forward, using the semi-obscure tie command, but still pretty simple. And it looks like it should support converting a regular socket connection into an SSL connection midstream.
use Net::SSLeay::Handle;
...
tie(*S2,"Net::SSLeay::Handle", $host, $port);
my $sock = \*S2;
So that seemed to work. At least, it made a connection, and reading from <$sock> would do just what you'd hope. But things went downhill from there. I didn't want to use <$sock>, I wanted to use read(), to read a chunk of data from the socket, just the way getchunk() in Net::SSL works. In other words something like "read($sock,$buf,8192);". Now this should be fine - with normal socket reading semantics, a read will block until it gets something, but then it will return even if it doesn't get everything. Perl does get in the way here a bit, but with normal perl sockets, using sysread() skips all that rot, and sysread() from a socket works just like I'd expect.

But this didn't do that. Using either read or sysread, the call blocked until the buffer size I gave was satisfied. Not good. Fine, I thought I can still work with this, I'll just make the socket completely non-blocking with "$sock->blocking(0);" (hint: don't forget to add in "use IO::Handle;").

That sort-of worked. But when I tried to put things inside of a select(), I found that $sock was getting select()ed, but then a subsequent sysread would find nothing there to read. At this point I punted. Granted, the error may have been mine, and I could have been minutes away from solving it, but I was getting annoyed.

IO::Socket::SSL

So, I tried IO::Socket::SSL, also available in a default Leopard installation.
use IO::Socket::SSL;
...
$sock = IO::Socket::SSL->new("$host:$port");
...
sysread($sock,$buf,8192);
Happy happy, joy joy. It worked fine. sysread() does exactly what I expect it to, as does select(). I don't care for the "$host:$port" syntax (too web-specific looking), but that's a minor quibble.

But that's still not a perfect solution.

A merged approach

The thing that doesn't work in IO::Socket::SSL is the $sock->read(). This has the same problem as the read command does in Net::SSLeay::Handle, where it blocks until it reads the full amount. The reason this would make me happy is that even though I started with getchunk() in Net::SSL, $sock->read() also does pretty much what I want. So I have $sock->read() available in both modules, but unfortunately, they don't behave the same, and I haven't found any easy way to make them.

Nevertheless, I needed a solution that would let one program run on platforms which had different SSL options. I can at least cover what seem to be the two most popular approaches. So here's what I've got at this point. If there's a better way, I'd be happy to hear it.

eval {
  require Net::SSL;
  Net::SSL->import();
};

if ($@) {
  require IO::Socket::SSL;
  IO::Socket::SSL->import();

  if ($@) {
    print STDERR "Can't find SSL module!\n";
    exit(-1);
  }
  $SSL="Net";
} else {
  $SSL="Crypt";
}

. . .

if ($SSL eq "Crypt") {
  $sock = Net::SSL->new(PeerAddr => $host, PeerPort => $port,
                        Proto => 'tcp', Timeout => 200) || die "new Net::SSL";
} else {
  $sock = IO::Socket::SSL->new("$host:$port") || die "new IO::Socket::SSL";
}

. . .

sub GetChunk {
  my $s=shift(@_);

  if ($SSL eq "Crypt") {
    return($s->getchunk);
  } else {
    my $buf="";
    sysread($s,$buf,8192);
    return($buf);
  }
}

And, I'm still working on implementing STARTTLS, which requires jump-starting SSL on an already in-use socket. I'm pretty sure I can do it with IO::Socket::SSL. I'll see if I can make Net::SSL do it too.


Reader Comments (Experimental. Moderated, expect delays. Posts may be edited or ignored. I reserve the right to remove any or all comments, at any time.)

6 comments:

At 2008/12/20 13:23
Scott Gifford wrote:

Have you tried making the sockets nonblocking, like this:

$sock->blocking(0);

For me at least, that seems to give the effect you're looking for.

At 2008/12/20 13:34
wrote:

Yes, I've tried that. It sort of works, but the behaviour is subtly different. Standard socket reading behaviour is to block until it reads something. The bad behavior I was getting was to block until it reads everything that was requested. With your non-blocking suggestion, it's possible to return after reading nothing.

A select() would in theory eliminate this issue, but when I tried non-blocking, I wasn't getting correct behavior from the select() -- it would come up in the select() when nothing was there to be read; though admittedly I didn't try to thoroughly debug that issue.

tom

At 2008/12/20 13:59
Scott Gifford wrote:

Ah, I see. Select and SSL don't always mix well, but I have had pretty good luck using nonblocking sockets with select() and sysread(), just ignoring errors EAGAIN and EINTR returned by sysread().

At 2010/01/09 20:04
Leif Pedersen wrote:

I'm navigating this chaos, and it seems that select() with SSL is supported (you would think Lighttpd proves that!). Anyway, you have to avoid some assumptions. Do not assume a read call only reads or a write call only writes. Do not assume reading zero bytes means EOF. A read may return zero if it only processed the beginning of a header but no application data, for example. A read call may need to write, and a write call may need to read because either side may initiate a new handshake midstream. So be sure to set O_NONBLOCK so that you don't accidentally block for write when you call read. Also pay attention to the error codes, which will indicate whether it needs to read or write next, and update select()'s fdset to watch for what SSL needs. Many iterations may be necessary before the application's data stream progresses.

I found this in docs about the underlying C libraries: http://www.openssl.org/support/faq.html#PROG10 and http://www.openssl.org/docs/ssl/SSL_get_error.html

That said, I haven't actually gotten anything cobbled together yet. I'm writing something that is a single thread and is absolutely never allowed to block unless there's actually nothing to do. I'm trying to stick with the lowest-level libraries I can because the more wrappers there are the more likely it is that I'll trip over some bug or limitation. Often those wrappers are far more complicated than necessary, but sometimes they really do a lot of necessary work for you.

Have you had any more recent findings?

At 2010/01/10 19:03
Leif Pedersen wrote:

I finally finished putting the pieces together, and wrote up my solution in a longer article. Thanks for sharing your findings, they saved me some time. Here are my findings: http://devpit.org/wiki/OpenSSL_with_nonblocking_sockets_(in_Perl)

At 2012/02/14 10:52
wrote:

How do you use IO::Socket::SSL in a client software to connect to a server through a proxy? Searching through the web, I read somewhere that NET::SSL provides the support for proxy, but not IO::Socket::SSL. Do you know or had any experience on this? Thanks,

End Comments

Add a comment


Tom Fine's Home Send Me Email