System Installation Without Internet on Ubuntu

Note

There is an example of installing the system on the Ubuntu 22.04 minimal installation server

Warning

Instruction is relevant for Universe MDM installations not older than version 6.9, since migration from Elasticsearch to Opensearch has been performed

Before you start:

  • The archive with the distribution kit is provided to the client when purchasing the product through the manager of the company "Universe Data".

  • Unpack the distribution archive, which contains the installation scripts, to any location. The contents will be stored in the MDM_6.X_Ubuntu_22.04_offline_install directory. Next, this directory will be named as <OFFLINE_REF>.

  • Copy the contents of <OFFLINE_REP> to the target server.

Installing Using Bash Script

At the root of the directory MDM_6.X_Ubuntu_22.04_offline_install there is a script install.sh, which installs the system automatically with standard settings. The script contains 2 variables:

  • POSTGRES_PASSWORD - takes a value from a file setenv.sh

  • TOMCAT_HOME - the standard directory for installing Universe is specified.

The script also consists of functions, the sequence of which is described at the end of the file:

  • checkdirs - checks if all the directories needed for installation are in place,

  • check_files - checks for the necessary configuration files in the conf_files folder,

  • install_mc - installs Midnight Commander,

  • install_vim - installs Vim,

  • install_Opensearch - installs and configures Opensearch,

  • install_postgres - installs and configures postgresql,

  • install_java - installs and configures OpenJDK,

  • install_universe - performs all the necessary actions to launch Universe.

Before Running Script

The configuration files are located in the conf_files folder. Make sure that you are satisfied with the standard configuration.

  1. setenv.sh file contains the basic Universe settings:

    • export POSTGRES_ADDRESS="localhost:5432" - Postgres server address,

    • export POSTGRES_USERNAME="postgres" - DB user for Postgres,

    • export POSTGRES_PASSWORD="notpostgres" - Postgres password,

    • export DATABASE_NAME="universe" - Data base name,

    • export SEARCH_CLUSTER_NAME="universe-es-cluster" - the name of the Opensearch cluster that must match the value of the parameter cluster.name in opensearch.yml,

    • export SEARCH_CLUSTER_ADDRESS="localhost:9300" - address of the Opensearch cluster.

  2. postgresql.conf file contains the main PostgreSQL settings From the list of settings you need to change the following:

    • max_prepared_transactions = 300

    • max_connections = 1000

    • port = 5432

    • listen_addresses = '*'

  3. pg_hba.conf file contains PostgreSQL security settings. You can change this section according to the required security settings:

    # Database administrative login by Unix domain socket
local all postgres peer
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all peer
host replication all 127.0.0.1/32 scram-sha-256
host replication all ::1/128 scram-sha-256

  4. opensearch.yml file contains Opensearch configurations. You can change: cluster.name: universe-es-cluster, path.data: /var/lib/opensearch, path.logs: /var/log/opensearch and other parameters according to the system settings requirements.

  5. tomcat.service file is required in creating the tomcat daemon. It does not need editing, it will be automatically copied by the script.

Running Script

  1. Grant the script execution rights:

    sudo chmod +x ./install.sh

  2. Run the script on behalf of the super-user:

    sudo ./install.sh

  3. At the end of the script execution, Universe app will be available at: http://ваш_IP:8080/universe-frontend (login: admin, password: admin)

After entering the login and password, the system will request the license file and password change.

Manual Installation

Installing Midnight Commander (Optional)

  • All packages and dependencies for Midnight Commander are located along the path: ./<OFFLINE_REP>/mc

  • To install Midnight Commander, go to the specified directory and run the command:

    sudo dpkg -i *.deb

Installing Vim (Optional)

  • All packages and dependencies for vim are located along the path: ./<OFFLINE_REP>/vim

  • To install Vim, go to the specified directory and run the command:

    sudo dpkg -i *.deb

Installing Opensearch

Use on of 2 installing options by the commands:

wget https://artifacts.opensearch.org/releases/bundle/opensearch/2.7.0/opensearch-2.7.0-linux-x64.deb
sudo dpkg -i opensearch-2.7.0-linux-x64.deb

Or:

curl -o- https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo gpg --dearmor --batch --yes -o /usr/share/keyrings/opensearch-keyring
echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list
sudo apt-get update
sudo apt-get install opensearch=2.7.0

Opensearch Launch:

sudo systemctl enable opensearch
sudo systemctl start opensearch

Disabling security demo-configuration (Required)

Security demo-configuration is enabling by default in distribution, but it has to be disabled using commands:

export DISABLE_SECURITY_PLUGIN=true
sudo systemctl restart opensearch

Or add plugins.security.disabled: true to /usr/share/opensearch/config/opensearch.yml

Installing Analysis-ICU Plugin (Required)

  • The Analysis-ICU plugin for Opensearch is located along the path: ./<OFFLINE_REP>/Opensearch/analysis-icu-7.14.0.zip

  • To install the plugin, use the command (it will work if the <OFFLINE_REP> folder is unpacked to the home directory):

    sudo /usr/share/opensearch/bin/opensearch-plugin install analysis-icu

Installing Hunspell Dictionaries

  1. Hunspell dictionaries are located along the path: ./<OFFLINE_REP>/Opensearch/hunspell

  2. To install, copy the directory /<OFFLINE_REP>/Opensearch/hunspell to /etc/opensearch/:

    sudo cp -rv /path/to/hunspell /etc/opensearch

  3. Grant permissions for the new directory:

    sudo chown -R root:opensearch /etc/opensearch/hunspell/

  4. If there is no directory with the dictionaries, then create a new one:

    cd /etc/opensearch/
sudo mkdir hunspell
cd hunspell/
sudo mkdir ru_RU
sudo mkdir en_US
cd /etc/opensearch/hunspell/ru_RU
sudo wget https://cgit.freedesktop.org/libreoffice/dictionaries/plain/ru_RU/ru_RU.dic
sudo wget https://cgit.freedesktop.org/libreoffice/dictionaries/plain/ru_RU/ru_RU.aff
cd /etc/opensearch/hunspell/en_US
sudo wget https://cgit.freedesktop.org/libreoffice/dictionaries/plain/en/en_US.dic
sudo wget https://cgit.freedesktop.org/libreoffice/dictionaries/plain/en/en_US.aff
sudo chown -R root:opensearch /etc/opensearch/hunspell/

Cluster.name Configuration

  1. Open the configuration file (also available in mc):

    sudo vi /etc/opensearch/opensearch.yml

  2. Replace the parameter cluster.name: opensearch to another, for example: cluster.name: universe-os-cluster

  3. After the changes are made, restart Opensearch:

    sudo systemctl restart opensearch

Startup check is available with the command:

curl -X GET '127.0.0.1:9200'

Installing PostgreSQL 12

  1. All packages necessary for the correct installation of postgres are located along the path: /<OFFLINE_REP>/postgres

  2. To install postgresql, go to the specified directory and run the command:

    sudo dpkg -i *.deb

  3. The main postgresql configuration files are located along the paths: /etc/postgresql/12/main/postgresql.conf и /etc/postgresql/12/main/pg_hba.conf

  4. In the /etc/postgresql/12/main/postgresql.conf file, uncomment and change the following parameters: #listen_addresses = 'localhost' max_connections = 100 #max_prepared_transactions = 0 replace with listen_addresses = '*' max_connections = 1000 max_prepared_transactions = 300; port=5433 replace with port=5432.

  5. Sections of the /etc/postgresql/12/main/pg_hba.conf file should take the following form (it is allowed to change the parameters for individual needs):

    # Database administrative login by Unix domain socket
local   all             postgres                                peer

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     md5
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     peer
host    replication     all             127.0.0.1/32            scram-sha-256
host    replication     all             ::1/128                 scram-sha-256

  6. After installation and configuration, change the password to the postgres user ('notpostgres' is the password of the postgres user):

    sudo su
su postgres
psql
alter user postgres with password 'notpostgres'

  7. Create a database using the command:

    CREATE DATABASE universe

Installing OpenJDK

  • All packages necessary for the correct installation of OpenJDK are located along the path: ./<OFFLINE_REP>/java

  • To install openjdk, go to this directory and run the command:

    sudo dpkg -i *.deb

Installing Tomcat

  1. Create a user:

    sudo useradd -m -U -d /opt/tomcat-9 -s /bin/false tomcat

  2. The directory with tomcat is located in /<OFFLINE_REP>/tomcat-9.

  3. After executing the command, a tomcat user will be created with the home directory /opt/tomcat-9. Copy this directory to /opt/:

    sudo cp -rv /<OFFLINE_REP>/tomcat-9/* /opt/tomcat-9

  4. To create a daemon, copy the file ./<OFFLINE_REP>/conf_files/tomcat.service in /etc/systemd/system/:

    cp -v MDM_6.X_Ubuntu_22.04_offline_install/conf_files/tomcat.service /etc/systemd/system/

  5. Reload daemons:

    systemctl daemon-reload

Installing Universe App

  1. The Universe distribution is located at the path: /<OFFLINE_REP>/universe-6.x (6.x is the version number of the Universe system)

  2. To install, you need to go to this directory:

    cd /<OFFLINE_REP>/universe-6.x/Tomcat/

  3. Copy the configurations:

    sudo cp -v bin/setenv.sh /opt/tomcat-9/bin/ && \
sudo cp -rv conf/universe /opt/tomcat-9/conf/ && \
sudo cp -v libs/* /opt/tomcat-9/lib/ &&\
sudo cp -v webapps/* /opt/tomcat-9/webapps/

  4. Grant the rights to the directory tomcat-9:

    chown -R tomcat:tomcat /opt/tomcat-9

  5. Grant startup rights to executable files:

    chmod +x /opt/tomcat-9/bin/*.sh

System Setup:

To start the system correctly, you need to change the following files:

  • /opt/tomcat-9/bin/setenv.sh - bring it to the following form. The value of the settings is described in the section Before running the script:

    export JAVA_OPTS="${JAVA_OPTS} -Duniverse.conf=${CATALINA_HOME}/conf/universe -Dlogback.configurationFile=${CATALINA_HOME}/conf/universe/logback.xml"
#export JRE_HOME=/usr/lib/jvm/java-8-oracle
#export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export POSTGRES_ADDRESS="localhost:5432"
export POSTGRES_USERNAME="postgres"
export POSTGRES_PASSWORD="notpostgres"
export DATABASE_NAME="universe"
export SEARCH_CLUSTER_NAME="universe-os-cluster"
export SEARCH_CLUSTER_ADDRESS="localhost:9200"

  • /opt/tomcat-9/conf/universe/backend.properties - добавьте настройки пути хранения лога:

    org.unidata.mdm.system.bitronix.tm.journal.disk.logPart1Filename=${CATALINA_HOME}/logs/btm1.tlog
org.unidata.mdm.system.bitronix.tm.journal.disk.logPart2Filename=${CATALINA_HOME}/logs/btm2.tlog

System Launch:

  1. Reload tomcat:

    sudo systemctl restart tomcat

  2. After the installation is complete, go to: http://YOUR-IP:8080/universe-frontend (login: admin, password: admin)

  3. Next, proceed with logging into the system.

Security Configuration

Security configuration is required for:

  • Generating certificates or retrieving them from an authoritative source.

  • Deploying certificates in Opensearch and Universe MDM.

  • Setting up credentials MDM client.

Note

Below is an example of how to generate self-signed certificates

Generating Certificates

# Root CA.
openssl genrsa -out root-ca-key.pem 2048
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=RU/ST=SAINT-PETERSBURG/L=SAINT-PETERSBURG/O=UNIVERSE/OU=MDM/CN=root.dns.a-record" -out root-ca.pem -days 730
# Admin cert
openssl genrsa -out admin-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
openssl req -new -key admin-key.pem -subj "/C=RU/ST=SAINT-PETERSBURG/L=SAINT-PETERSBURG/O=UNIVERSE/OU=MDM/CN=A" -out admin.csr
openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730
# Node cert
openssl genrsa -out node-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in node-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node-key.pem
openssl req -new -key node-key.pem -subj "/C=RU/ST=SAINT-PETERSBURG/L=SAINT-PETERSBURG/O=UNIVERSE/OU=MDM/CN=universe-os-cluster" -out node.csr
echo 'subjectAltName=DNS:universe-os-cluster' > node.ext
openssl x509 -req -in node.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node.pem -days 730 -extfile node.ext
# Cleanup
rm admin-key-temp.pem
rm admin.csr
rm node-key-temp.pem
rm node.csr
rm node.ext

The -subj flag allows you to set the x509 certificate attributes in the script (CN: CommonName, OU: OrganizationalUnit, O: Organization, L: Locality, S: StateOrProvinceName, C: CountryName). If you skip it, the job will run interactively.

The CN and DNS of the host certificates must match the value of SEARCH_CLUSTER_NAME

Opensearch Configuration

Opensearch configuration can be located at the path: /usr/share/opensearch/config/opensearch.yml or etc/opensearch/opensearch.yml.

Below is an example of the most basic security configuration. For an overview of all the features and settings, see the official documentation.

Example of /usr/share/opensearch/config/opensearch.yml configuration file:

cluster.name: universe-os-cluster
network.host: 0.0.0.0
discovery.type: single-node
plugins.security.ssl.transport.pemcert_filepath: node.pem
plugins.security.ssl.transport.pemkey_filepath: node-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: node.pem
plugins.security.ssl.http.pemkey_filepath: node-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - 'emailAddress=my@email.com,CN=A,OU=MDM,O=UNIVERSE,L=SAINT-PETERSBURG,ST=SAINT-PETERSBURG,C=RU'
plugins.security.nodes_dn:
  - 'CN=universe-os-cluster,OU=MDM,O=UNIVERSE,L=SAINT-PETERSBURG,ST=SAINT-PETERSBURG,C=RU'
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
cluster.routing.allocation.disk.threshold_enabled: false
plugins.security.audit.config.disabled_rest_categories: NONE
plugins.security.audit.config.disabled_transport_categories: NONE

The formatted nodes_dn (admin_dn - similarly) can be obtained with the command:

openssl x509 -subject -nameopt RFC2253 -noout -in node.pem

Universe MDM Configuration

  1. Create a truststore by putting the node's public key certificate in there:

    keytool -import -file node.pem -keystore my-truststore.jks

  2. Create a keystore and combine client public and private key certificates, putting the resulting .p12 certificate in there:

    openssl pkcs12 -export -in admin.pem -inkey admin-key.pem -out volumes/admin.p12 -CAfile root-ca.srl -caname root
keytool -importkeystore -deststorepass my-keystore-pass -destkeypass my-keystore-pass -srckeystore admin.p12 -srcstoretype PKCS12 -srcstorepass my-p12-pass

  3. Configure the backend.properties file:

    # If any security measures are enabled (credentials, encryption)
org.unidata.mdm.search.security.enabled=true

# Credentials used by MDM to access search cluster
org.unidata.mdm.search.admin.login=admin
org.unidata.mdm.search.admin.password=admin

# Truststore of type JKS, stores public certificates of search nodes for MDM validation
org.unidata.mdm.search.truststore.path=/path/to/my-truststore.jks
org.unidata.mdm.search.truststore.password=my-truststore-pass

# JKS-type key store, stores the public+private .p12 MDM certificate to establish an encrypted connection.
org.unidata.mdm.search.keystore.path=/path/to/my-keystore.jks
org.unidata.mdm.search.keystore.password=my-keystore-pass

  4. The last step can also be done with the corresponding environment variables:

    SEARCH_SECURITY_ENABLED=true

SEARCH_ADMIN_LOGIN=admin
SEARCH_ADMIN_PASSWORD=admin

SEARCH_TRUSTSTORE_PATH=/path/to/my-truststore.jks
SEARCH_TRUSTSTORE_PASSWORD=my-truststore-pass

SEARCH_KEYSTORE_PATH=/path/to/my-keystore.jks
SEARCH_KEYSTORE_PASSWORD=my-keystore-pass