Pravega client can be used to ingest metadata and images into SDP from VisionPro or any other application. Pravega client is written in Rust, but it has Python binding so it can be used for Python code. Refer to https://github.com/pravega/pravega-client-rust for more details. The following sections describe using Pravega client with Python.

Environment variables and SDP parameters

Pravega client requires certain OS environment variables to be set. This can be done through Shell, or they can be defined in the Python script as shown in the following examples:

os.environ['pravega_client_auth_keycloak'] = <Name of the keycloak Json file for the project>

Note: To retrieve Keycloak JSON for a specific project/scope, run the following command on SDP account using project/scope name as the command line argument:

# kubectl get secret <project/scope name>-ext-pravega -n <project/scope name> -o jsonpath=”{.data.keycloak\.json}” | base64 -d  >  <Project Name Keycloak secret>.json

os.environ['pravega_client_tls_cert_path'] = <Name of the SDP cert file>

os.environ['pravega_client_auth_method'] = “Bearer”  (SDP supports only the bearer method.)

Other parameter values that are needed by Pravega client from SDP are:

PRAVEGA_CONTROLLER_URI—Fully qualified name for Pravega Controller, for example: “pravega-controller.sdp.faredge1.lab:443”
PRAVEGA_STREAM—Name of the Pravega stream to write to
PRAVEGA_SCOPE—Scope (Project) of the stream
ROUTING_KEY (optional)—<Name of the routing key>: any identifying text to tag messages

Use Pravega client to ingest data

If an image needs to be stored in SDP, it must be converted into a Base64 string, as described in the following section.

Ingesting data into SDP is a three-step process which can be automated using Python script. This is accomplished using the Pravega client module.

Create Stream Manager using Pravega Controller URI:

stream_manager = pravega_client.StreamManager(PRAVEGA_CONTROLLER_URI, True, True)

Create writer using Pravega Scope and Pravega Stream settings:

writer = stream_manager.create_writer(f'{PRAVEGA_SCOPE}', f'{PRAVEGA_STREAM}')

Write JSON formatted data to the stream using writer:
```
writer.write_event(out_msg, ROUTING_KEY)
```

Converting an image to Base64 format

If an image is to be ingested along with text data, it must be converted to a Base64 string and then to the utf-8 format before inserting it in JSON structure along with rest of the data. In Python, the base64 module can be used. The following is a sample of Python code for conversion.

#Read and encode the image file
fhi = open(image_file, "rb")
b64_string = base64.b64encode(fhi.read())

# Convert to UTF-8 and add image to Json structure 
msg["image"] = b64_string.decode('utf-8')

#Convert to string for writing to Pravega
 out_msg = json.dumps(msg)

Your Browser is Out of Date

Pravega Python-based image ingest

Pravega Python-based image ingest

Environment variables and SDP parameters

Use Pravega client to ingest data

Converting an image to Base64 format