Please note that the "notify: AUTHENTICATION-FAILED" message is received by ISA from your Vista machine: "1-16: 12:24:36:719:1a0 Receive: (get) SA = 0x0011ab98 from 172.16.5.1.500".
You need for Vista a certificate on ISA that contains the Server authentication(220.127.116.11.18.104.22.168.1) EKU, if you want to use the Verify the Name and Usage attributes of the server's certificate checkbox. Please read this.
What you described above, I've already explained here. (not with Vista, but with Mac OS X, anyway is pretty much the same thing).
The problem is that, since you need to support both Vista and Mac OS X L2TP/IPsec VPN clients, you have a situation:
- it appears the VPN server's certificate checks on Macs are hard-coded or so(I don't know how to disable them), and that you need a certificate on ISA with the SAN corresponding to the FQDN of the VPN server to which the Mac OS X VPN client connects and also this certificate must incorporate no EKU fields or the ikeIntermediate (22.214.171.124.126.96.36.199.2) EKU.
- with Vista, if you want to use the Verify the Name and Usage attributes of the server's certificate checkbox., you need a certificate on ISA with the CN(or SAN)corresponding to the FQDN of the VPN server to which the Vista VPN client connects and also this certificate must incorporate the Server authentication(188.8.131.52.184.108.40.206.1) EKU field.
So the EKU fields are the issue here from the clients perspective. I did not try to issue a certificate containing the both Server authentication and ikeIntermediate EKUs, with the appropriate SAN entry, to see if this certificate is accepted by both VPN clients.
The other issue is on the VPN server, as it appears that the Windows API is unable to be manually configured to select what certificate to be used for IKE authentication. So, this would imply, in case of ISA, when due to SSL bridging web publishing rules or so multiple certificates may exist on ISA, it's hard to know which certificate will be used by the Windows API for IKE authentication of the L2TP/IPsec VPN clients, that is, you don't know which FQDN to configure as the VPN server's address on you VPN clients performing certificates checks.
What you see, is a poor implementation of good/recommended security from both Windows and Mac OS X.
If they would want to perform such checks during IKE authentication to avoid MITM, they should have given the flexibility to configure which checks over which fields from the VPN server's certificate the VPN clients performe.
And, on the Windows server, we should be able at least to select which certificate uses the VPN server for IKE authentication.
What is really annoying, is that both Mac OS X and Vista check the EKU fields too, which may add exactly 0 security, as the VPN clients' certificates may contain no EKU, or contain too the ikeIntermediate or Server Authentication EKUs(for example if you use the autoenrollment for Windows domain member machines, you will get a machine certificate witht the Computer Template or so, containing the Server Authentication EKU). And, with IKE, unlike in case of TLS, it is not commonly known or so that the server's certificate contains the Server Authentication EKU and the client's certificate contains the Client Authentication EKU, so that everybody to follow these rules. Due to that, we have a jam here.
So if you have multiple certificates on ISA, and want to support both Vista and Mac OS X L2TP/IPsec clients, you may:
- disable the check Verify the Name and Usage attributes of the server's on Vista, and use your Windows Enterprise CA for issuing certificates for IKE authentication.
- create a separate PKI(a different CA) for Mac OS X L2TP/IPsec VPN clients, maybe like this.
For IKE authentication, by default, due to the Windows default L2TP IPsec policy, the VPN server will request certificates from the VPN clients from both CAs(as it was configured with certificates from both CAs, certificates that can be used for IKE authentication), and the VPN clients will select the appropiate certificate from their store, which will make the VPN server to use a certificate from the same CA too.
< Message edited by adimcev -- 16.Jan.2009 10:03:02 AM >
Get Our ISA 2006 Book!: http://tinyurl.com/2gpoo8