6.2    SAP BW/4HANA DataSources

DataSources are SAP BW/4HANA objects that contain a collection of rules, parameters, and fields that allow you to access or extract data from source systems and make them available in SAP BW/4HANA. This set of fields can be organized either in a flat structure or in multiple flat structures, for example, the extraction of hierarchies.

DataSources are defined according the type of data they process. These DataSources types are as follows:

• Transaction data
• Master data (attributes, texts, hierarchies)
• Segmented data

These DataSources are grouped in application component hierarchies under each source system the DataSource is linked to. New application component nodes can be created locally in SAP BW/4HANA via the source system’s context menu or via any application component context menu by selecting New… • Application Component.

Note

File-type DataSources still use the SAP GUI DataSource editor embedded in an Eclipse session. Other source system connection types, such as ODP, big data, and SAP HANA, use the native Eclipse wizard for creating DataSources.

The following subsections describe the creation of DataSources using the Eclipse-based native wizard and discuss the special case of file-based DataSources, which use Transaction RSDS.

6.2.1    Creating a DataSource with the Native Eclipse-Based Wizard

You can create a DataSource in different ways:

• Right-click the BW project and select New… • DataSource….
• From the main menu, select File • New… • DataSource….
• Right-click the source system and select New… • DataSource….

After accessing the DataSource creation menu option, the New DataSource screen is displayed where you can enter the main properties for the DataSource.

The first part of the main properties, as shown in Figure 6.16, includes the following items:

• BW Project
  Project name that identifies the SAP BW/4HANA system, client, username, and language for the DataSource.
• Package
  Defines the transport strategy for the DataSource. Local developments use the package $TMP.
• Source System
  The technical name of the source system to be used as a source of data for this DataSource.
• Source System Type
  A display-only property showing the selected source system type.
• Application Component
  The hierarchical group node to assign the DataSource being created. This helps group DataSources that share the same functional areas.
• Add to Favorites
  A flag to add the DataSource shortcut to your favorites.

Figure 6.16    SAP HANA Database DataSource: General Parameters 1

The second part of the main properties, as shown in Figure 6.17, includes the following items:

• Name
  The technical name of the DataSource.
• Description
  Optional meaningful description of the DataSource.
• DataSource Type
  The DataSource type according to the data being sourced. It may be master data, hierarchies, or transactional data.
• Copy From
  Indicates the reference DataSource that will be used as a template for the one being created.
  

  Figure 6.17    SAP HANA Database DataSource: General Parameters 2

The last part of the DataSource main parameters, as shown in Figure 6.18, creates the association of the DataSource with the source system object used as the data provider. The available options depend on the selected source system type.

Figure 6.18    SAP HANA Database DataSource: General Parameters 3

After completing the first part of the wizard, the DataSource configuration is shown as in Figure 6.19. It has three main sections, each with its own tab: Overview, Extraction, and Fields.

Figure 6.19    SAP HANA Database DataSource Overview Tab

The Overview tab includes the following main properties:

• Description
  Optional meaningful description of the DataSource.
• DS for Data Reconciliation
  Allows you to have a DataSource with direct access to the data provider in the source system and make the data available in SAP BW/4HANA via a Virtual Provider. This enables you to easily perform a consistency check on the data that was transferred by the official DataSource.
• Opening Balance
  This option is used to perform an initial load into noncumulative key figures in the InfoProviders.
• Data Is Language-Dependent
  This option is set to avoid data duplication or data loss when the data being transferred to SAP BW/4HANA has a language dependency. DataSources in this case must have a field to identify the language of the record. In SAP systems, standard fields for language are LANGU and SPRAS.
• Data Is Time-Dependent
  This option is set to avoid data duplication or data loss when data being transferred to SAP BW/4HANA has a time dependency, such as validity periods. DataSources in this case must have at least one field to define either the lower limit or the upper limit of the validity period of the record. In SAP systems, standard SAP fields for validity period limits are DATEFROM and DATETO.
• Delivery of Duplicate Data Recs
  This indicator sets the behavior of the DataSource when processing duplicate records during the data load. The options to here are Undefined, None, and Allowed. Except for the Undefined option, you must set the fields in the DataSource that will be used as the primary key for the validation of duplicate records.

The Extraction tab, shown in Figure 6.20, provides a set of properties to define how data will be extracted or accessed from the source system:

• Extraction Properties (general DataSource properties):
  • Delta Process
    Defines the processing mode according to the data being loaded. For this DataSource type, we have the following options:
    • No Delta, Only Full
      Records are transferred as-is to the targets and you are also allowed to add filters to the data being extracted.
    • Overwrite Delta without Deletions
      Records are transferred with new values/status for all key figures and characteristics. The data selection is based on the delta pointers and conditions set in this DataSource and are replicated to the source system.
    • Pure Additive Deltas
      Records are transferred with only the modification to the key figure values and not their final value. The final value may be calculated by aggregation on the target object during the load or during reporting.
  • Direct Access
    Defines if the DataSource can establish direct access to the source provider and additional operations that can be performed on the data during the extraction.
  • Streaming
    Defines if the DataSource can stream data in real time.
  • Adapter
    Method used to access and load data from the source system to the SAP BW/4HANA.
• Extractor-Specific Properties
  This group contains information about the source system and the object that sources the data to the DataSource. You can modify the source object if necessary.

Figure 6.20    SAP HANA Database DataSource Extraction Tab

The Fields tab, shown in Figure 6.21, provides the available fields for the source in the left panel; and you can set the group of fields to be used to transfer data from the source to SAP BW/4HANA by selecting the Transfer checkbox for each field you want to use.

Figure 6.21    DataSource Fields Tab

Each field has a set of properties displayed in the right panel. You can further customize the properties of each field as follows:

• Field Name
  Technical name of the field in the DataSource structure. It can be different from the field name in the source table/view/object. You can see the field name in the source linked to the field in the DataSource in the Source Properties panel, as shown at the bottom right of Figure 6.21.
• Field Description
  A meaningful description for the field.
• Data Type
  Defines the technical information for the field, type of data, and length. For numeric values, you can also define the number of decimal places.
• Technical Properties
  Associate a field for unit of measure or currency for amount or quantity fields. For alphanumeric fields, you can define if lowercase characters are acceptable. Also, when you use duplicate records verification in the DataSource, you can set the Key Field flag to consider the field as part of the primary key for this validation.
• Conversion Properties
  Defines a conversion routine to be used in the transformation of the field and indicates if the values at the source are using the external format or the internal format of the conversion routine—for example, adding leading zeros for customer numbers.
• Selection Properties
  Defines if the field can be restricted during the data load with filters and indicates which filter operators can be used in this field.
• SDA-Specific Properties
  Used to specify which fields are used for creating packages or partitions of data during the data transfer and which key figures can be summarized during real-time replication when all delta records (including new records, changes or deletions) are processed internally using INSERT command.

You can use the DataSource in the system after clicking Activate (). As shown in Figure 6.22, this makes this DataSource available for use, and you can find it in under the Project Explorer panel.

Figure 6.22    DataSources in Project Explorer Panel

6.2.2    Creating a DataSource for File-Type Source Systems

File-type DataSources have been widely used in SAP BW since the very first version of the product; they’re also available in SAP BW/4HANA.

You can create a file-type DataSource via the same menu paths as described in Section 6.2.1, which are:

• Right-click the BW project and select New… • DataSource….
• From the main menu, select File • New… • DataSource….
• Right-click the source system and select New… • DataSource….

The first two options will open the Eclipse wizard, because the source system type isn’t yet selected. Once a file-type source system is selected and you proceed, the Eclipse wizard will open Transaction RSDS to complete the creation of the file type DataSource in an embedded session in Eclipse. The last option opens Transaction RSDS directly, because the source system is already preselected.

Figure 6.23 shows the first screen of a file-type DataSource creation. Here, you specify which source system will be used to establish a connection to your file system, the name of the DataSource, and the type of data being transferred. You can also create the DataSource using another object as a template. For example, we can use a target Advanced DSO as the basis for the field structure of this DataSource.

Figure 6.23    File-Type DataSource Creation: Initial Screen

The General Info tab shown in Figure 6.24 allows you to enter a Description to identify the DataSource, enter the Application Component, identify if the DataSource has unique records or if duplicates are allowed, and set if it’s an Opening Balance DataSource.

Figure 6.24    File-Type DataSource: General Info Tab

The Extraction tab shown in Figure 6.25 allows you to define the source of the data being transferred to SAP BW/4HANA and the extraction method.

Figure 6.25    File-Type DataSource: Extraction Tab

The source of the data is identified by the Adapter and Data Format as follows:

• File Adapter
  The type of file and location to be used to load the data. In flat files, you can write an ABAP routine to make the file names dynamic and determined during the load process, then access the file in the local machine from the application server file system. For Microsoft Excel types (as shown in Figure 6.25), you can see the Adapter Load Microsoft Excel File (XLS) from Local Workstation selected and you can specify the file name and the tab of the workbook to be used.
• Data Format
  Applicable to flat file types. You can define the structure of the file, such as the columns separator, text delimiter, or fixed length.

Note

Local machine adapters and MS Excel can’t have data processed in the background, including load via process chains. For background data loads, use the flat file adapter type with files stored on a file-system directory accessible by SAP BW/4HANA and the files visible in Transaction AL11.

The Proposal tab shown in Figure 6.26 allows you to automatically receive a proposal for the DataSource field names, data types, and field lengths based on a sample of the existing records in the source file. The number of records to be used for this sampling process is defined in the No. of Data Records field.

Figure 6.26    File-Type DataSource: Proposal Tab

The Fields tab shown in Figure 6.27 allows you to define the field structure of the DataSource. You can manually define the data types, length, case-sensitivity, primary key, conversion routines and the data format present in the source in case of using the conversion routines, which data selection operators are allowed, and time or language links for each field present in the DataSource structure.

You can also associate an InfoObject to the DataSource field as a template to retrieve the definitions of the InfoObject to the DataSource field. You can adjust those definitions according to the source of the data you’re loading into SAP BW/4HANA.

Figure 6.27    File-Type DataSource: Fields Tab

The Preview tab allows you to preview the data with all conversions and field characteristics in the DataSource structure to help perform the DataSource’s unit test.

Once the DataSource is created, you must activate it by clicking Activate (). The DataSource will be in the Project Explorer panel under the source system and application component associated with it and will be ready to be used in SAP BW/4HANA data flows.

6.2.3    ODP DataSources

ODP DataSources are usually created in SAP source systems and their metadata replicated to SAP BW/4HANA. Besides metadata replication, you can create manually ODP DataSources in SAP BW/4HANA and during their creation establish the link between the available ODP provider in the SAP source system.

For DataSources replication, navigate to the desired source system node in the Data Sources tree of the Project Explorer panel; for faster selection of available DataSources, you can navigate to the Application Component node to which the desired DataSources are linked.

Once you select the node for which you want DataSources to be replicated, right-click it and choose Replicate; a list of available DataSources for replication will appear, as shown in Figure 6.28.

Figure 6.28    List of Available ODP DataSources to Replicate

From the list of available DataSources for replication under the selected application component, you can select either all DataSources or just specific ones to replicate the metadata from the source system to SAP BW/4HANA. This way, you can avoid replicating unchanged DataSources currently in use, which may also be time consuming.

Proceeding to the next step, as shown in Figure 6.29, you can confirm the number of DataSources being replicated and can activate them after the replication by setting the Activate New DataSources flag.

Figure 6.29    Confirmation of ODP DataSources Replication

The replication of the DataSources will start in the background, and SAP BW/4HANA will open Transaction SM37 (Job Monitor) to help you monitor the progress of the replication. Once the job is finished, the selected DataSources are replicated and activated if the corresponding option was selected.

Once you refresh the respective Data Sources tree, you can see the DataSources active and ready for use in SAP HANA flows.