OE IPsec part 4, It Works!

Here's a short status update about my racoon hacking (project pages). I've made some progress and found a few silly mistakes.

First, the IPSECKEY RRs I added to the hack.org zone had a mistake. I accidentally marked the keys as DSA keys (algorithm 1) instead of RSA (algorithm 2). On the other hand, my code didn't even look at the algorithm to see that we actually had an RSA key. When I added this, it was the first time I noticed the mistake.

I've also changed the way keys are loaded. Instead of loading the public key to the internal list of keys, I just set it to be the peer's public key (the struct ph1handler element called rsa_p) after querying DNS.

I was stuck for a while here because when loading the binary key with binbuf_pubkey2rsa() I got an even modulus! It took a while to find out why. The reason was I accidentally allocated too large a buffer for the key.

Anyway, here's the debug output for node ipsec1 documenting the first successful security association setup between two nodes with transport mode opportunistic encryption using DNS keys:

Querying for IPSECKEY for ipsec1.hack.org.
Precedence: 10
GW type: 0
Algorithm: 2
About to load binary RSA key.
rdlength = 514
RSA key exp: 3
RSA key mod:
694913600036185438993921168228021937466057130393896470658117457642787491739628277347739214111009475566180366417075914489628611802311555976477394376215746650261879176034066424255318465503312147134539969215567953271646717204598342561705430835742785081980058810243747062929669386297941389266784069911908790958669624829252049368204263159776885004920853508395529997309049004264686570844655837509348340996362563661222290110998461210029247714608874709448708852854486035445398182880541292532261465658595945459794731603692369323935808537825251587769487509831287328187917490553609545412965997565515176372090553125521899596249141830135227228738309881444637223802982958463985902185081179057608461585384189650076205267084415494806805358202249321673305590906361755185424944833600130678011130562219609244020874378039663815076008003580667266737166092938960118742823445186464541147935210725546258002614112386720194987823671990667030489122370399617127538186975887826632841701328093084393632959251063222976988423463690745266575203616531356014967825161718241064781805911511758034131272770962425058529622542274935036119622969141568437886118555468855661770674419607044935142805396752764250140058372653682999058293922316647030399183884626344870920539322537
2012-01-26 12:37:26: WARNING: CERT validation disabled by
configuration
Checking signature.
Signature OK!!!!!
2012-01-26 12:37:26: INFO: ISAKMP-SA established
2001:16d8:ffff:1::4[500]-2001:16d8:ffff:1::3[500]
spi:4dbf33cd73600138:1f003e83f65af8e8
2012-01-26 12:37:26: [2001:16d8:ffff:1::3] INFO: received
INITIAL-CONTACT
2012-01-26 12:37:27: INFO: initiate new phase 2 negotiation:
2001:16d8:ffff:1::4[500]<=>2001:16d8:ffff:1::3[500]
2012-01-26 12:37:27: INFO: respond new phase 2 negotiation:
2001:16d8:ffff:1::4[500]<=>2001:16d8:ffff:1::3[500]
2012-01-26 12:37:27: INFO: Update the generated policy :
2001:16d8:ffff:1::3/128[0] 2001:16d8:ffff:1::4/128[0] proto=any dir=in
2012-01-26 12:37:27: INFO: IPsec-SA established: ESP/Transport
2001:16d8:ffff:1::4[500]->2001:16d8:ffff:1::3[500]
spi=58166992(0x3778ed0)
2012-01-26 12:37:27: INFO: IPsec-SA established: ESP/Transport
2001:16d8:ffff:1::4[500]->2001:16d8:ffff:1::3[500]
spi=160491056(0x990e630)
2012-01-26 12:37:27: INFO: IPsec-SA established: ESP/Transport
2001:16d8:ffff:1::4[500]->2001:16d8:ffff:1::3[500]
spi=215844934(0xcdd8846)
2012-01-26 12:37:27: INFO: IPsec-SA established: ESP/Transport
2001:16d8:ffff:1::4[500]->2001:16d8:ffff:1::3[500]
spi=25812946(0x189dfd2)

In case you're worried the line about the CERT validation being disabled is of no concern. It's just racoon's way of saying that we're not trying to compare the name in the CN in an X.509 cert with the peer's ID. It would be quite silly to check that in this scenario when there's no CN and no cert.

I need to polish the code at least a bit before publishing. Stay tuned.