Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Modern Data Management with Boomi Accelerates Business Outcomes
Executive summary Use case
Introduction
Overview
Starting point
Boomi capabilities used
Data sources and data stores
Users and groups
Exploring data sets
Use case workloads
Schedule a job
RESTful API example
Integrating Boomi DCP with Hadoop Solution architecture Conclusions References

Use case workloads

Use case workloads


  • This document presents an example workflow in Boomi DCP. This example includes a fictitious loan department that has multiple data sources. Information about customers and loans that have been made resides in a PostgreSQL database. Information about customer communications resides in a file on a server. This workflow performs the following sequence:

    1. Combines the information from these sources
    2. Performs a data query
    3. Filters the data
    4. Transforms the data
    5. Generates a result
    6. Places the result into an HDFS data store

    This example uses two data sources with multiple tables and a file to create an output. The output lists the number of customer communications for every successful loan.

    Start the use case sequence

    The user begins the use case sequence as follows:

    1. Click the  Prepare menu icon (a check mark). 
    2.  Select Jobs, which displays a new page. 
    3. Click the Add icon (+) to start the jobs wizard. 
      The wizard contains seven stages for creating jobs. This example goes through each of the stages to show a sample job, and show the benefits for Boomi DCP users.

    Select a data set

    In this stage of the wizard, users can select any of the data sources that have been previously defined. The user selects a data set as follows:

    1. Click Select Data Set. See Select data set. 
      Figure 7. Select data set
      Select Data Set
    2. Select loans-pg9,3, which is the PostgreSQL 9.3 database. 

      This database was previously created by adding a Data Store, and was not shown for brevity.

    3. View the available data tables. 
    4. Select stg_cust, which is the customer table, as the primary data set. 
    5. At the bottom, click Sample data to display a preview of the data with which the user is working. See Data preview. 
      Figure 8. Data preview
      Data preview
      At the top of the data preview, the user can see sample statistics about any column of selected data. These statistics are based on a subset of the data. The amount of data to be summarized can be configured. The viewing of sample rows of data and seeing a visualization of data are helpful, and are powerful tools.

    Join data sets

    To join data sets:

    1. Click Join Data Sets. See Join data sets. 
      Figure 9. Join data sets
      Join data sets

      Boomi has multiple operators to allow for complex join operations. The joins themselves follow standard SQL functionality, such as which parts of the data to join. Boomi has additional functionality in this area to assist users. You can preview the data to be joined using the Sample Data pull-up menu.

      Boomi also provides OneClick Joins™, which leverages the underlying graph database that is loaded when DCP introspects data sets with foreign keys in databases or relationships between objects in Sales Force, for example.

    2. Select additional data sets to be used in the query. 
    3.  Define how the data sets are joined, following standard SQL set definitions. 

      Multiple such joins can be performed in this step.

    Enrich the data

    This step enables users to perform data transformations. This example merges the first and last names from the customer data table, and makes a single name. Boomi DCP comes with a rich set of predefined operators. Users can use these operators to create their own functions. This use case creates a data-enriching function as follows:

    1. Click Enrich Data. See Enrich data. 
      Figure 10. Enrich data
      Enrich data
    2. Click  Build Your Own to define a function from scratch. 
    3.  Provide a function name, merge_columns, and a description. See Enrich data function. 
      Figure 11. Enrich data function
      Enrich data function
    4. Click the Column selection icon at the upper right. 

      Here you select one or more columns of data to use as inputs to your function.

    5. At the top, click the fx symbol to display a list of integrated primitive functions. 
      These functions are standard for such data transformations.
    6.  On the right side, select String > concat_ws. 
      That selection concatenates elements from the two table columns, and combines them with a white space.

      This wizard provides powerful tools for users. The documentation for all the functions is attached to each function in the UI. You can pull up sample data and see what your transformation will do, and much more.

    Choose columns

    This step enables users to select what columns, or pieces of data, will be part of the output. Choosing from the listed data columns and types creates the selection. For this use case:

    1. Click Choose Columns. See Choose columns. 
      Figure 12. Choose columns
      Choose columns
    2. Select the output of the function created in Enrich the data. 
    3. Select other columns of interest. 

    Add a filter

    This step allows users to apply standard SQL filters to their job. Users can use all the standard grouping and selecting tools normally available. This use case is only interested in people with approved loans.

    1. At the top, select Add Filter. See Add a filter. 
      Figure 13. Add a filter
      Add a filter
    2. Click Build Your Own. 
    3. Click the Add icon (+), and then add the function text, [stg_loan].[status] = 'Approved'. 

      Boomi once again provides sample data and sample results that can be viewed by pulling up the Sample Data page from the bottom of the window.

      Boomi has OneClick Filters™, that also leverage the underlying graph database. When the data allows, it can be as simple as choosing one of the predefined filters created by Boomi using the graph data to get a valid and usable data output.

    Aggregations

    This step enables users to perform output functions on their selected data columns.

    1. At the top, select Aggregations. See Aggregations. 
      Figure 14. Aggregations
      Aggregations
    2. Click Select to select from that column or data source. 

      Each checked box results in the item being displayed in the output.

    3.  Click Count to provide a count of the selected column, and send it to the output. 

      Multiple aggregations are possible in this window. This use case example adds (counts) the amount of loans for each person.

    Output

    This step enables users to select any valid output data sinks that have been defined in Boomi DCP, or to create one or more data sinks to use. This use case example outputs the results to a file on an HDFS data store.

    1. At the top, select Output. See Output. 
      Figure 15. Output
      Output
    2. Note the following: 
      • The human-readable name of each column
      • The original data source column
      • Where the data is written
    3. Select the output data sink. In this use case example, select the tmp/sum directory on the HDFS data store. 

      The output enables users to select multiple types of output and locations.