[Advanced] Database Security and SoftHSM How To (DECPRECATED v4)

NOTE: This is being DEPRECATED as of Senzing v4 as it is incompatible with G2Snapshot, the PostgreSQL governor, and any other tools/utilities/examples that access the database directly.  Since the Senzing API configuration (including passwords) is provided through parameters to API calls, a user (an API developer) can retrieve/secure credentials in any way desired (e.g., AWS Secrets Manager, LDAP, a secure store) rather than rely on a proprietary mechanism.

This article outlines using softHSM as a secure store for database credentials for the Senzing database connection URI. The example herein demonstrates using G2Loader.py to connect to the database leveraging the secure store using and the connection URI in G2Module.ini; the pattern is the same for using the APIs directly.

Prerequisites 

Senzing API version newer than 1.10.19210

Install SoftHSM

Red Hat / CentOS

• sudo yum install softhsm
• sudo usermod -aG ods <your_userid>
• Logout and login

Debian

• sudo apt install softhsm2
• sudo usermod -aG softhsm <your_userid>
• sudo mkdir -p /var/lib/softhsm/tokens/
• sudo chgrp -R softhsm /var/lib/softhsm/tokens/
• sudo chmod -R 770 /var/lib/softhsm/tokens/
• Logout and login
• export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/softhsm
• You can add this to ~/.bachrc (or similar) or modify senzing/setupEnv

Setup the Secure Store

• Add the bold lines to your G2Module.ini under the [PIPELINE] section

[PIPELINE]
SUPPORTPATH=<project_path>/data
SECURE_STORE_URL=pkcs11://token/?slotID=0
SECURE_STORE_LIB=softhsm2
SECURE_STORE_PIN=
ENABLE_SECURE_STORE=Y

• Modify the CONNECTION entry for your database in G2Module.ini under the [SQL] section and change the password field to SEC-STORE-G2USERPWD

[SQL]
# CONNECTION=mysql://g2:paswordhere@localhost:12345/?schema=G2
CONNECTION=mysql://g2:SEC-STORE-G2USERPWD@localhost:12345/?schema=G2

• Initialize and configure the secure store, change /opt/senzing/g2/bin to reflect your deployment if not using the default path. During these commands you will be asked to set up 2 different pins - do not lose these and securely store them!
  • cd <project_path>
  • ./bin/g2ssadm -tokinit -label G2_STORE -c etc/G2Module.ini
  • Comment out the previous SECURE_STORE_URL in your G2Module.ini and add a new one specifying the tokenLabel:
    

    [PIPELINE]
    SUPPORTPATH=<project_path>/data
    # SECURE_STORE_URL=pkcs11://token/?slotID=0
    SECURE_STORE_URL=pkcs11://token/?tokenLabel=G2_STORE
    SECURE_STORE_LIB=softhsm2
    SECURE_STORE_PIN=
    ENABLE_SECURE_STORE=Y

  • ./bin/g2ssadm -ssinit -c etc/G2Module.ini
  • ./bin/g2ssadm -ssinit -newpin -c etc/G2Module.ini
  • ./bin/g2ssadm -ssput -label G2USERPWD -c etc/G2Module.ini
  • When prompted, enter the password for the database user previously in your connection string in G2Module.ini

Test the Setup

A simple method to test if the secure store credentials are now working is to use G2Loader in test mode. This will not load any data in to the database; it will perform the database performance tests.

To run G2Loader in test mode: 

• python3 G2Loader.py -T -f demo/sample/sample_person.csv/?data_source=TEST

If the secure store credentials or configuration has not been completed correctly, G2Loader will fail upon attempting to establish a connection to the database to perform the performance tests. If the connection to the database cannot be established you will see error(s) similar to (MySQL example):

  [g2@g2centos python]$ python3 G2Loader.py -T -f demo/sample/sample_person.csv/?data_source=TEST

  ...
  Validating demo/sample/sample_person.csv...
  
  Testing demo/sample/sample/sample_person.csv, CTRL-C to end test at any time...
  ...
  
  UNHANDLED DATABASE ERROR: ((1045:Access denied for user 'g2'@'localhost' (using
  password: YES)28000 Access denied for user 'g2'@'localhost' (using password:
  YES) 28000 ))
  
  2019-07-30 20:02:42.771 [sql:139927931930432] CRIT: Exception: Access denied
  for user 'g2'@'localhost' (using password: YES)28000 Access denied for user 'g2'@'localhost'
  (using password: YES) 28000