SDP documentation covers the deployment steps in detail, but as a quick reference, the fundamental steps are covered in this section. SDP uses a Kubespray installation, customized for SDP, named of sdpspray. This DVD uses a single-node SDP installation with optional external NFS-based storage.

SDP deployment involves the following steps:

Configure the Ubuntu VM using two disks (one for the operating system and another for SDP local storage) with some specific SDP prerequisites.
Update the SDP installation configuration files with disk layout, cluster nodes, and other parameters.
Install SDP with the desired size (small, medium, or large).
Install Kubeapps to set up InfluxDB and TimescaleDB sinks.
Create the SDP project and stream, then enable features for the stream.
Set up MQTT broker.
Deploy InfluxDB and TimescaleDB sinks with the desired configuration and retention period.

Configure a virtual machine hosting SDP

Configure an Ubuntu 20.04 virtual machine with the desired configuration to prepare for SDP installation. The subsequent sections provide the high-level steps for virtual machine configuration for SDP. See the Streaming Data Platform Installation and Administration Guide on the Documentation page of the Support Site for more details.

Create an Ubuntu virtual machine based on the OS support matrix of SDP containing two disks – a 400 GB OS disk and a 1 TB (or more) disk for SDP long-term storage.
Create an SDP user account and enable password-less SSH connectivity by creating appropriate SSH keys and copying the public key.
Install some prerequisites such as OpenSSL, Python, and PIP. Refer to SDP documentation for the complete list of prerequisites.
If an external NFS share/PowerScale is to be used for long-term persistent store, identify the shared folder for SDP deployment, and ensure that NFS share is accessible from SDP. The NFS server must support NFS v4.0. If PowerScale is being used, it must have OneFS 8.2.x or 9.x software, and NFS v4.0 must be enabled in the configuration.
Under the home directory of the SDP account, create a folder such as desdp, then download and extract the SDP installation files to that folder.

Update the SDP installation configuration files

Update the SDP installation-specific configuration files, such as env.yaml and inventory.ini file, for a single-node sdpspray-based SDP installation.

Update the configuration files based on the following example:

`# Installation node install_node: <SDP installation node> # upstream dns CoreDNS upstream_dns: - <DNS IP address> # MetalLB configuration metallb: true metallb_primary_range: # 10.10.10.1-10.10.10.10 # - "Range of IP addresses Start-End" local_nfs: true # when using NFS Set and use these parameters: local_nfs: false #nfs_server: 10.10.10.10 #nfs_path: '/nfs/path'`	`disk_settings: single_node_single_disk_mode: true single_node_shared_disk: 'sdb' bookeeper_ledger_size: 100 bookeeper_journal_size: 100 bookeeper_index_size: 10 dedicated_bookkeeper_disks: '' dedicated_lts_disk: '' dedicated_deault_sc_disk: '' powerflex_nodes: false share_os_disk_with_kubespray_sdp: false enable_ceph_cluster: false ceph_disks: 'sde,sdf,sdg'`

# Installation node
install_node: <SDP installation node>
# upstream dns CoreDNS
upstream_dns:
-  <DNS IP address>
# MetalLB configuration
metallb: true
metallb_primary_range:
# 10.10.10.1-10.10.10.10
#  - "Range of IP addresses Start-End"

local_nfs: true
# when using NFS Set and use these parameters: local_nfs: false
#nfs_server: 10.10.10.10
#nfs_path: '/nfs/path'

disk_settings:
  single_node_single_disk_mode: true
  single_node_shared_disk: 'sdb'
  bookeeper_ledger_size: 100
  bookeeper_journal_size: 100
  bookeeper_index_size: 10
  dedicated_bookkeeper_disks: ''
  dedicated_lts_disk: ''
  dedicated_deault_sc_disk: ''
  powerflex_nodes: false
  share_os_disk_with_kubespray_sdp: false
  enable_ceph_cluster: false
  ceph_disks: 'sde,sdf,sdg'

Edit the env.yaml file and update the parameters, specifying the name of the installation node, Metallb load balancer IP addresses, single node, single disk mode install set to “true”, and local NFS set to the correct value depending on which local NFS or external NFS is used for the configuration. Refer to the SDP installation guide for details on various configuration parameters. Here is a sample env.yaml file with some relevant updates.
```
[kubespray]
<node name> ansible_ssh_host=<node IP> ansible_python_interpreter=<Python installation path>
```
Edit the inventory.ini file with the SDP node information.

Install SDP

To install SDP using sdpspray auto-installer scripts, perform the following steps.

Change the directory where the SDP installer is extracted, look for ansible playbooks, and the run_sdp_auto_installer.sh script.
Execute run_sdp_auto_installer.sh, which runs through several SDP installation steps such as prepare node, OS packages verification and installation, disk configuration and partitioning, Kubespray installation, creating SDP values YAML file, core DNS setup, and SDP installation after all the prior steps are successful.
At the end of the installation, accept EULA and complete the installation steps. At this point, the following commands can help verify the successful status of the installation. Make sure that all the Pods show up in running state and that there are no errors reported for any Pods.
```
# kubectl get po -A | grep -i running 
# kubectl get po -A | grep -i pending 
# kubectl get po -A | grep -i error
```
The SDP UI can be launched by retrieving Keycloak and SDP UI credentials using the following commands. Copy the output of the ingress command to hosts file on the system to be used for launching the SDP UI. Launch the Keycloak UI first, then launch the SDP UI by logging in with the credentials listed.
```
# kubectl get ing -A | awk ‘{} {print $5, $4}’
# Change the directory where SDP is installed – default is /desdp/decks-installer/scripts and run the script
# ./fetch-admin-creds.sh
```

If successful, the SDP UI is enabled.

Configure SDP project/scope and prepare MQTT data ingest

Create SDP project/scope and configure Pravega MQTT broker for data ingest into SDP from convergence platforms, such as Telit deviceWISE Edge and Litmus.

Select the Analytics tab and create SDP project. Add the Pravega_MQTT_Broker feature to configure MQTT-based ingest.
After entering the desired values, click Save to save the project and MQTT broker configuration.
Gather the Pravega MQTT broker endpoint.
Capture the SDP certificate from the SDP installation home directory, in the CA certificate folder, and also copy the MQTT secret key and IP address. These values are used to set up the MQTT publisher configuration.
Once the MQTT broker is configured, the rest of the configuration must be done on convergence platforms such as Litmus and Telit deviceWISE Workbench.

Create a Pravega stream on an existing project

To configure Pravega stream on an existing project/scope, perform the following steps.

Start with the Pravega project/scope to add the stream to.
Click on Streams to add the stream.
Select Create Stream to see the stream configuration dialog and specify the configuration options.
Click Save to finalize the stream creation.

Deployment of database sinks

This section covers the configuration of database sinks, InfluxDB and TimescaleDB, using the Kubeapps application. The steps include:

Deploy KubeApps on an existing SDP cluster.
Deploy InfluxDB sink.
Deploy TimescaleDB sink.

Deploy KubeApps on an existing SDP cluster

To deploy KubeApps, perform the following steps.

Build the KubeApps Installer: To build KubeApps Installer, users must perform the following steps:
1. Download the installer from git.
2. Copy the sdp-kubeapps folder that you downloaded.
3. Execute the following commands:
```
# cd /sdp-kubeapps/scripts$
# ./build_installer.sh
```
  The Build folder is created.
4. Execute the following command:
```
# cd sdp-kubeapps/build/installer$ Setup kubeapps
```

Set up Kubeapps by running the following commands:

# tar xzvf kubeapps-deployment-1.0.3.tgz
# cd kubeapps-deployment Install kubeapps
# export LOCAL_DOCKER_REGISRY=sdp-registry:31001/desdp
# sudo -E /desdp/decks-installer/decks-install-linux-amd64 config set registry $LOCAL_DOCKER_REGISRY
# sudo -E /desdp/decks-installer/decks-install-linux-amd64 push --input kubeapp.tar.gz

Install Kubeapps by running the following commands:

add kubeapps url kubeapps.<base SDP URL>
# export KUBEAPPS_FQN=kubeapps.sdp.obs-demo.org
# ./install_kubeapps.sh

Create a project and get a token for Kubeapps by running the following commands:

# ./create_user.sh <project-namespace> <kubeapps-namespace> <kubeapps-user-name>
# ./create_user.sh sdp-demo kubeapps demo-user

Insert the KubeApps URL into the host file by running the following commands:

Retrieve the Kubeapps service IP address information using this command and add it to the host files where the browser is being run.
# kubectl get ing -A | awk ‘{} {print $5, $4}’

For the configured project and user name, get the token for the Kubeapps service by running the following commands:

Retrieve the service account name for the Pravgega project.
# kubectl get -n <Pravega Project Name> serviceaccount <user name created> -o jsonpath='{.secrets[].name}'

Using the service account generated, gather the Kubeapps token for the Pravega project.
# kubectl get -n <Pravega Project Name> secret <Service account name> -o go-template='{{.data.token | base64decode}}'

# Copy the token and log in to Kubeapps URL for the project.

Deploy the InfluxDB sink

To configure the InfluxDB sink for a specific project and stream, perform the following steps.

Select the SDPToInfluxDB app from the catalog and click DEPLOY.

Figure 52. Select InfluxDB sink app deployment from Kubeapps catalog
Specify the name of the integrator, and in the manifest file, update the input-stream parameter for the name of the steam (ignore the sampleexamples/examples and just use stream name as the argument value). The sink will be deployed for this project and stream.

Figure 53. SDP InfluxDB sink deployment for specified project and stream

Note: Users can specify other configuration details in the manifest file as well, such as the InfluxDB username and password.
Wait for the Pod to become Ready, review the Access URLs, port, and Application Secrets for connectivity to this InfluxDB.

Figure 54. SDP InfluxDB sink ready for use
Access the InfluxDB sink from Grafana, as shown in the following figure.

Figure 55. InfluxDB access from Grafana
Specify InfluxDB database details for Grafana, as shown in the following figure.

Figure 56. InfluxDB database details for Grafana

Configure the TimescaleDB sink

To configure the TimescaleDB sink, perform the following steps.

Note: These steps are similar to InfluxDB sink steps in the previous task.

Select the SDPToTimescaleDB app from Kubeapps catalog. Deploy the application and wait until the Pod becomes Ready to use.

Figure 57. SDPToTimescaleDB configured

Note: Users can specify other configuration details in the manifest file as well, such as the TimescaleDB username and password.
Use Grafana to access the TimescaleDB sink, as shown in the following figure.

Note: Do not use the prefix “https://” when entering Host under PostgreSQL Connection, as it is implied. Adding the prefix generates an error.

Figure 58. TimescaleDB access from Grafana

Your Browser is Out of Date

Deployment of SDP