Understanding Updates Rollout in Continuous Deployment

As we discussed in the << Purposed Model of continuous integration & deployment >> post about the process that how a developer performs code change/fix, how it gets propagated to the pipeline, how this become part of delivery & deployment.
In continuation to that there arises a point of how an update is rolled out, what are the possible ways to do that & how that can be benefited?

Let's start the journey with a possible deployment architecture explaining that how an update is rolled out:

Note: Here in this deployment example we will consider a replica set of 3 identical pods having same image i.e. "hello1".

• An update of new image is available from the container registry, this needs to be rolled out in the deployment.

• Now we have a new updated image say "hello2". In this case we will tell our kubernetes master to create a second replica set that will have containers with image "hello2".

• You will notice that with creation of 2nd replica set the service pointing to replica set (1) will gradually start pointing to replica set (2) pods.

• First replica set pods will start decreasing & second replica set pods will increase.

Note: At most in this deployment we will have 4 pods at a time & at least 3 pods.

• Finally you will observe all 3 pods of replica set (2) are created & you are left with last POD of replica set (1) that too will be vanished soon.

• Finally the new image version is rolled out

References:

• https://www.coursera.org/learn/google-kubernetes-engine/

Enjoy :-)

Details about Canary Deployment

About Canary Deployment:

• Definition: Canary deployments are a pattern for rolling out releases to a subset of users or servers. The idea is to first deploy the change to a small subset of servers, test it, and then roll the change out to the rest of the servers
• This means when i have to deploy an update in production, i can do this by canary way. This will allow me to deploy the change in production but only to a subset of servers. So by this way we subset of users can test the new update & report if any issue occurs. If all goes well than the update can be rolled out completely.

How to switch the traffic to from old version to new version?

This can be made possible with blue-green deployment. More info Understanding blue-green deployment.

References:

• https://www.coursera.org/learn/google-kubernetes-engine/

Enjoy :-)

Error: The authentication scheme protecting the resource sets 'Secure' OAMAuthnCookie/ObSSOCookie, but the resource is not being accessed via secure http

Error Statement:

If the authentication scheme is configured to set "Secure" OAMAuthnCookie/ObSSOCookie and the user is accessing an insecure resource, the browser may enter an authentication browser loop. Show an error i.e.:

"The authentication scheme protecting the resource sets 'Secure' OAMAuthnCookie/ObSSOCookie, but the resource is not being accessed via secure http."

Workaround:

In authentication scheme, remove the following parameter & save the changes;

Syntax for 11g Webgate and OAMAuthnCookie	Syntax for 10g Webgate and ObSSOCookie

ssoCookie=Secure

ssoCookie:Secure

Make sure changes are applied properly, as in the policy sync-up at OAM server happens successfully. 

You may restart the server instance (ohs/apache/iis etc) or you can wait for webgate cache clean. Try accessing the protected resource once again, you should be prompted for login.

Resolution:

Recheck you SSL settings at WebServer end.

References:

https://docs.oracle.com/cd/E25054_01/doc.1111/e15478/shared.htm

Enjoy :-)

Increase docker pool size by changing storage driver

Configure Docker with the devicemapper storage driver

NOTE: This is for Docker CE & Docker EE

Issue:

By default you will notice that docker storage device is "brtfs" i.e. default storage, which is limited to 20GB of data storage.

Usually you will find this storage as too limited to use. As most of the times we have to install multiple images that too of high storage like 4/8/10GB. And with this default storage you will be end up getting frustrated.

Solution:

Configure Docker with "devicemapper" storage driver.

How to do this:

1) Stop Docker

$ sudo systemctl stop docker 

2) Edit /etc/docker/daemon.json. If it does not yet exist, create it. Assuming that the file was empty, add the following contents.

{
  "storage-driver": "devicemapper"
}

Note: Docker will not start if the daemon.json file contains badly-formed JSON

3) Start Docker

$ sudo systemctl start docker

4) Verify that the daemon is using the devicemapper storage driver. Use the docker info command and look for Storage Driver

Note: This host is running in loop-lvm node, which is not supported on production systems. This is indicated by the fact that the Data loop file and a Metadata loop file are on files under /var/lib/docker/devicemapper/devicemapper.

Useful link: Use device mapper storage driver

Hope this helps :-)
Enjoy :-)

Enable SSL in between OHS & Outbound Applications

Enabling SSL in between OHS & OutBound Applications

Prerequisites:

1. OHS SSL is enabled.
2. Outbound App SSL is enabled like OAM, Weblogic, OIM etc.

What we are aiming is to setup SSL in b/w OHS & outbound apps

Eg: Consider you want to proxy your OAM server via OHS as a load balancer/proxy call it any. This is a very normal usecase where you have your OAM servers sitting in your data-center & you don't want it's hostname/IP to be exposed. So what you usually do is proxy OAM via OHS.

• Consider your OHS server name is https://abc.com. So if admin needs to access the oamconsole. Admin will fire the url as https://abc.com/oamconsole
• To enable this usecase, /oamconsole is to be added in ssl.conf/mod_wl_ohs.conf file(usual way).
• But the catch is that our OHS & OAM are in SSL mode.
• This means that they will do handshake before starting to talk to each other.
• As we all know that while doing handshake, server sends its user certificate, now this cert is verified by client i.e. here mod_wl_proxy of OHS. So the wallet used by it has to have the trusted certificate entry in it.

Steps you need to follow for this are as;

• Import the certificate used by Outbound app such as Oracle WebLogic Server into the Oracle HTTP Server wallet as a trusted certificate.
• To add trusted certificate you can use orapki utility or any of your choice.
• <MW_HOME>/oracle_common/bin/orapki wallet add -wallet ./ -trusted_cert -cert cacert.pem -auto_login_only 
• Note: './' is used as we consider that you are running this command from the directory where your cwallet.sso is present. You can substitute it with directory path of cwallet.sso as well.
•  Now you need to add 2 tags in ssl.conf or mod_wl_proxy.conf:

• SecureProxy On

  WlSSLWallet "<wallet location>" 

Complete Eg:

<Location /console>
SetHandler weblogic-handler
WebLogicHost xyz.us.domain.com
WebLogicPort 7001

SecureProxy ON
WlSSLWallet "/MW_HOME/keystores/newwallet"
</Location>

Now start your OHS server, and try to access the proxied url, you should be able to make a successful connection. You can also confirm the same by capturing wireshark traces. 

Hope this helps... :-)

Useful links:

https://docs.oracle.com/cd/E23943_01/core.1111/e10105/sslconfig.htm#ASADM10286

http://middlewaretimes.blogspot.sg/2014/03/configuring-modwlohs-to-use-ssl-between.html

Enjoy :-)

Configuring a DCC WebGate for X509 Authentication

DCC WebGate for X509 Authentication

Steps to configure this is mentioned very well in Oracle Documentation:

https://docs.oracle.com/cd/E52734_01/oam/AIAAG/GUID-DD4121B3-B8D1-4846-A9C4-2F0DCEA46E5D.htm#AIAAG90100


Enjoy:-)

Enabling OCSP Validation using X.509 Authentication Scheme

Assumptions:

• You have already configured & tested the X.509 authentication use case. And now we are enabling certificate validation using OCSP Server.
• To setup OCSP we will be using 'openssl'. So before proceeding check if you installed openssl or not. If not go get it first.
• You have imported the user certificate in your browser as part of X.509 Authentication.

Brief about OCSP Server:

The Online Certificate Status Protocol (OCSP) is an Internet protocol used for obtaining the revocation status of an X.509 digital certificate. The "request/response" nature of these messages leads to OCSP servers being termed OCSP responders.
An OCSP responder (a server typically run by the certificate issuer) may return a signed response signifying that the certificate specified in the request is 'good', 'revoked', or 'unknown'. If it cannot process the request, it may return an error code.

Steps to perform:

1) To setup OCSP Server: Steps to setup OCSP
2) Once you have setup OCSP, you will be having the following items with you;

• CA Authority certificate (self signed) - ca.pem
• CA private key - ca-key.pem
• OCSP Server URL -> i.e. host and port details http://abc.us.oracle.com:6060
• index.txt -> basically this is the file having the user certificates details which will be checked by the OCSP server for the received cert request. Based on this OCSP will respond 'good', 'revoked' or 'unknown'. It returns 'unknown' in case the cert entry is missing from the file.
• This file name could be any name of your choice.

a) OCSP server files;

 b) Staring OCSP server

2) Goto Access Manager Configuration Settings: Select Certificate Validation

3) Select OCSP/CDP Settings:

•  Enter OCSP Url

•  Provide the OCSP CA Authority Certificate Subject
• Apply the changes

4)  Now goto Authentication Scheme

5) Select X509Scheme

 6) Provide the details as required for enabling OCSP Validation

7) Now goto Authn Module

8) Select X509 Module

9) Remember to enable Cert Validation & OCSP. Provide the required details.

10) Now we need to attach the X509Scheme in our webgate profile Protected  Resource Policy.

11) Now when you fire a request from your browser, you will be asked for the user cert to be used for authentication. And if this user is not present in OCSP database i.e. the index.txt file or it is been revoked than you should get 'Authentication Failed' message.

12) To implement a use case to check certificate revocation:
Execute this command:
./openssl ca -revoke <user certificate name>
Eg: 
bash-3.2$ ./openssl ca -revoke user1\@oracle.com.crt
Using configuration from /scratch/ckukreja/openssl/openssl.cnf
Enter pass phrase for ./demoCA/private/cakey.pem:
Revoking Certificate BE.
Data Base Updated

•  Here i have copied the cakey.pem, ca.pem & index.txt to 'demoCA' folder. Basically openssl refer to this folder as it is configured in my openssl.cnf file.
• After this your index.txt gets updated and you can actually see the difference as well. Now when you try to access the protected resource with the user cert who has been revoked in OCSP databse, so this time OCSP will say 'revoked' to OAM server. And hence you will get authentication failed message.

Note: Important points to help you debug;

a) Always check if the log.txt file provided with OCSP server startup is getting populated with request or not.
b) If request is landing on OCSP and its a good cert request but still it is failing to authenticate than check OAM server diagnostic logs. You must be getting an exception there.

Important Informative Links:

Managing Common Services and Certificate Validation
X.509 Certificate Revocation Checking Using OCSP protocol with Oracle WebLogic Server 12c
How to set up OCSP using OpenSSL - I-Space Research Labs



That's All Guys......
Enjoy :-)