Showing posts with label cert mode in oam 11g. Show all posts
Showing posts with label cert mode in oam 11g. Show all posts

Tuesday, 20 October 2015

OAM R2PS3 - no need to add cacert.pem or aaa_chain.pem in cwallet.sso anymore

With the recent release of Webgate R2PS3, now a new feature is added which saves manual intervention of adding cert in cwallet.sso for SIMPLE & CERT Mode.


Previously for SIMPLE/CERT Mode one needs to add the cacert.pem or aaa_chain.pem cert in the wallet. If not than handshake between webgate & oam is not done successfully.

But now webgate itself picks the cert from the desired location like;
  • for simple mode -> cacert.pem is added in wallet which is placed at <webgate_install_dir>/tools/openssl/simpleCA directory location.
  • for CERT Mode -> aaa_chain.pem needs to be added in wallet which is to be placed in webgate instance directory location.
So now when we start the webserver in which webgate is integrated, during SSL Handshake first time no cert is found in cwallet.sso hence "TLS Handshake failure" message is gets logged. After detecting this error webgate itself adds the cert in cwallet.sso & retry the handshake once again.

Hence this time all goes good & NAP Channel is initialized in secure mode.


Enjoy :-)

Monday, 19 October 2015

Understanding SSL Handshake Mechanism Between Webgate & OAM Server

Currently Webgate supports 3 modes of communication with OAM Server:

  1. Open
  2. Simple
  3. Cert
With the SIMPLE & CERT Mode SSL comes in picture, but why at all we require it;
  • The answer lies in the question it self i.e. security. OAM combined with Webgate is a security product that provides SSO & other features thus itself requires to be secure in talking terms.


From webgate-oam perspective we will be talking about NAP over SSL.

With SSL we ensure that the data transamitted b/w 2 parties is safe & sound as;
  • Both the parties knows one another, but how because they do handshake before they start communication.
  •  Once they start communicating, the packets or the data been transferred b/w them is secure as it is encrypted.
The 2 SSL modes of communication described above performs handshake process before they are sending data to each other; The handshake mechanism involves following steps;

Remember: In webgate & oam conversation, it is always the webgate who sends the request & server responds to it. So the request-response model is not vice-versa in OAM/Webgate case.   

1. In SSL Handshake first it is require for both client & server to ensure that they are talking to right or intended party only. So how webgate & oam do this;
    1. It is the first INITIALIZE NAP message that is sent between both the parties;
    2. This init_nap msg comprises of set of messages that they both exchange. And all this is done to ensure that the listener & receiver are not evil.
    3. During this exchange webgate ask the server for the communication mode, if it is not open mode than webgate starts the SSL Handshake process. Describe in step 2.
2. At this step it is required to establish a secure connection between the webgate & oam. For this webgate prepare a CLIENT HELLO Message, which contains the following;
    • Supported set of Cipher-suite like
      SSL_RSA_WITH_AES_256_CBC_SHA , SSL_RSA_WITH_3DES_EDE_CBC_SHA
    • Which TLS Version to be supported; TLSv1.0, TLSv1.1, TLSv1.2; 
      • Now webgate with r2ps3 also supports TLSv1.2 as well.
    • Random Number - this number is of 32 bytes out of which 4 bytes contains date & time and rest are randomly generated. This random number will be used to prepare the master key (which is combination of client generated random number & server generated random number). This master key will be used for encrypting the data transferred b/w client & server once handshake is done.
    • Session ID: a null session id is sent. If this is not the first time, than a valid value is been sent here as a session already exist b/w them.
 Cipher-suite: it is the algo that need to be used for encrypting the data.
 TLS version - it is the SSL supported version, latest one is TLSv1.2

3. Based on the Client Hello, server responds with SERVER HELLO, & in this server responds with;
  • Supported cipher suite, that will be used in b/w webgate & oam for data encryption;
  • Supported TLS version;
  • And server sends its own certificate to webgate; And this is required to authenticate the server. Webgate does so by checking the cert in webgate truststore i.e. cwallet.sso. If webgate is able to verify the process is proceeded to next step, else it "TLS HANDSHAKE ERROR" is thrown.
    • Possible reason for this:
      • There is no trusted cert loaded in cwallet.sso, so webgate is unable to verify the server cert.
      • Else the server provided cert is not a valid cert, hence server need to provide a valid cert.
  • Random Number: server generated number same as client did while sending hello message.
  • Session ID: server sends this newly generated id, that will be used in further communications. By this server can detect whether a session exist b/w webgate & server or not.
4. Key Exchange Phase: 
  •  This phase is divided into 2 messages exchanged b/w client & server:
    • SERVER KEY EXCHANGE
    • CLIENT KEY EXCHANGE
Server key exchange - During this message exchange you will notice that while sending server hello, you see one more message along with it i.e. SERVER KEY EXCHANGE. This message is encrypted with server's private key and client decrypts it with the server's public key received with server's certificate.
  • The SERVER KEY EXCHANGE Message is not mandatory to be sent by server, as it depends on the cipher suite selected for communication.
    • DH_ANNON - for this cipher it will be sent
    • RSA/MD5 - it will not be sent
  • After this server key exchange, server hello is done. immediately after this from webgate side it is required to send CLIENT KEY EXCHANGE message. The data in this message is encrypted with server's public key.
Client key exchange -
  • The purpose of sending this message is to share the master key, which client generates using the (client random number + server random number) encrypts it using the cipher suite selected & encrypts the whole message with public key shared by server in its certificate.
Note
  • This above key exchange step is required so that to generate a master key that will be used as symmetric key b/w server & client for further data exchange.
  • And to achieve this step we use asymmetric key, as we saw client used server public key for sending CLIENT KEY EXCHANGE Message while server used its private key to send SERVER KEY EXCHANGE Message.
5. FINISHED: at this point handshake process is done & now onwards client & server exchange the messages encrypted using the master key.


Note:
  • It is possible that server will respond with error, if the list of cipher suite provided by webgate is not all supported not any one of it is supported or;
  • The tls version provided by webgate is not supported by server.
6. Once client & server handshake is done successfully, after remaining INIT_NAP messages are excahnged b/w webgate & oam server.

7) Once above step is done successfully, than NAP channel is initialized b/w webgate & server. Basically this NAP channel in simple terms means a socket connection b/w webgate & server is established that will transmit the data over SSL.



Note: Once a NAP channel is initialized than for the requests been sent via this channel need not to do the SSL handshake again and again. It is a one time process per connection until unless that connections is to be refreshed or gets expired.


Hence the WEBGATE-OAM SSL Handshake finishes here.....



More Info: Related to SSL (nice article)
http://robertheaton.com/2014/03/27/how-does-https-actually-work/


Enjoy :-)