Encrypted connections are configured by using two environment variables:

  • SM_OUTGOING_PROTOCOL— Controls the connections that an program acting as a client is allowed to initiate.

  • SM_INCOMING_PROTOCOL— Controls the connections that an program acting as a server is allowed to accept.

    Each of these can contain a list of security levels. If you specify more than one, separate them with commas. If you do not set the variable, it is the same as specifying “0”, or CLEARTEXT.

    When two or more security levels are supported in common, the largest level number is used. For example, if a client has SM_OUTGOING_PROTOCOL set to "0,1" and the server has SM_INCOMING_PROTOCOL set to "0,1,2", the connection uses the highest common level, level 1.

    The order in which the different levels are specified for the environment variables has no significance—for example, "0,1" and "1,0" behave exactly the same.

    Applications that do not support encryption are treated as only having a cleartext level of encryption.

    When the two ends of the connection do not match, such as when SM_OUTGOING_PROTOCOL is 3 at server A and SM_INCOMING_PROTOCOL is 2 at server B, a connection cannot be formed. Both client and server will report errors.

    To set the environment variables so that they can be used by the programs of an product suite, edit the runcmd_env.sh file, which is located in the BASEDIR/smarts/local/conf directory of that product suite.

    1. Use sm_edit to open the runcmd_env.sh file. Invoke sm_edit from the BASEDIR/smarts/bin directory:

       sm_edit conf/runcmd_env.sh
 C:\>sm_edit conf\runcmd_env.sh

    2. Review the contents of the file. The default values for the SM_INCOMING_PROTOCOL and SM_OUTGOING PROTOCOL are 1 and 0.

       SM_INCOMING_PROTOCOL=1,0
 SM_OUTGOING_PROTOCOL=1,0

    3. Update the values for the variables to meet the needs of your system.

      For example, to raise security to the next highest level, change 1 to 2:

       SM_INCOMING_PROTOCOL=2,0
 SM_OUTGOING_PROTOCOL=2,0

    4. Save the runcmd_env.sh file and close it.

    5. Any program within the product suite started after this point will use the environment variables specified in the runcmd_env.sh file. programs that are already running need to be restarted for any new environment variable to take effect.