Configuring Scheduled File Imports

Before importing data files, the administrator must perform the following configurations:

1. Review the Mappings in the Import Templates

2. Import Template Mappings From CSV File

3. Create Azure File Storage folders

4. Create Import Trigger Flow

5. Configure the 'Overwrite with Blank' Field

6. Enable Statistics

7. Turn On the Trigger Flow and Test Import

Review the Mappings in the Import Templates

Prerequisite

CSV files with field mappings of the data source to be imported.

The predefined template mappings in the CSV file are mapped to out-of-the-box fields where available, and in many cases it is based on the vendor provided choice list values, which need to be reviewed and updated to suit a customer's configured environment.

In the template mapping CSV file, review the following types of mappings and add or change the mappings accordingly:

• Field mappings

• Virtual Field mappings

• Dataset mappings

• Default Field Value mappings

• Advanced settings (Import Batch Size, Import Row Batch Size, Enable Statistics)

Import Template Mappings From CSV File

After reviewing and updating the template mappings, import the excel files into the target environment.

1. In the Anthology Reach environment, navigate to Settings > Integration > Integration Mappings.

2. Click the Import from CSV option.

   

3. In the Import from CSV page, click Choose File to select the required file, and click Next.

4. Click Review Mapping and verify that fields are mapped.

5. Click Finish Import.

After the import has completed, check that your template mappings are all imported successfully.

Create Azure File Storage folders

In Microsoft Azure, following sample folder structure is available for reference in the Common\inbound folder:

• Sample-drop

• Sample-processing

• Sample-processed

• Sample-error

For each data source, create separate folders for the import.

Steps to create Azure storage folders for Import

1. Log into Azure Storage Explorer

2. Navigate to common\inbound\

3. Create new folders using the following pattern. Replace "sample" with your import template name, or desired name.

   • Sample-drop

   • Sample-processing

   • Sample-processed

   • Sample-error

4. Click New Folder on the toolbar, and in the New Folder dialog box add the required folder name.

Create Import Trigger Flow

The trigger flow contains the scheduling (recurrence) settings as well as template name, Azure folder names, and other parameter values used for the import.

An example "Sample - FileImport" flow is included in the Anthology Import Export solution to copy from.

Note

It is recommended to not edit default logic in the default flow, Sample - FileImport. Instead:

1. Copy the flows and edit logic in the copied flows.
2. Save and publish the copied flows in your institution’s implementation.
This will ensure, the original flow with its default functionality is available if the copied flow needs to be discarded. Additionally, as concurrency in the default flow is set to 1, it is recommended to retain this value in the copied flows because performance of the operation may be compromised if it is changed.

Steps to Create the Trigger Flow for Import

1. Log on to make.powerapps.com, and select the required environment and click the Solutions tab.

2. In the Anthology Import Export solution, locate the Sample - FileImport flow.

3. Create a copy of the Sample - FileImport flow and give it a desired name.

   Note: You cannot do a Save As on this flow as it includes a step which calls a child flow. Instead, create a new Cloud flow in your unmanaged solution and use the clipboard option in Edit mode to copy the steps in order from the Sample – FileImport flow to your new flow.

   Follow a similar naming convention for better organization and clarity in your target environment. E.g. MyNewImport - FileImport

4. Edit your new copy of the trigger flow, and review and update the following steps:

   1. Recurrence

      

   2. Get Integration Mapping

      Change the names for the highlighted items to match the import template mappings.

      Note: If you are not using CustomFlow as part of your import you will not need to specify a value for this.

      

   3. Check Configuration > Call Template Processing Flow

      Update the following parameters to match the import template settings:

      • TemplateName

      • Directory

      • DropFolder

      • ProcessingFolder

      • ProcessedFolder

      • ErrorFolder

      

      Edit the expression values for the following parameters and set the names to match your import template.

      • BatchSize

      • RowBatchSize

      • EnableStatistics

      Example
      Click on the BatchSize property, the Expression tab will open.
      
      Update this expression value by replacing the proper Import Batch Size name.
      Example
      int(if(and(contains(variables('VarIntegrationMappingData'),'Sample Import Batch Size'),not(empty(variables('VarIntegrationMappingData')['Sample Import Batch Size']))),variables('VarIntegrationMappingData')['Sample Import Batch Size'],100))

5. Save the trigger flow.

Configure the 'Overwrite with Blank' Field

The Overwrite with Blank field in the Integration mapping record of the field must be configured to determine whether the existing value in Anthology Reach must be retained or overwritten with blank, when no value is available for the field in the imported data:

• Yes - Indicates the field value will be overwritten to blank.
• No - Indicates the field value will not be overwritten to blank. It is the default value.

If Overwrite with Blank value is not specified for a field mapping, the import process will default to No and will not overwrite existing data if the field value in the data being imported is blank.

Enable Statistics

Summary counts of records created during an import can be optionally enabled to display in the integration log for the import job. The import statistics include the number of records created, updated, and unchanged for the imported entities, and is displayed in the Additional Details field of the Integration log record created for each file import.

• To enable import statistics for an import template:

  • Include a mapping record in the template with name = Import Statistics

  • Set the Internal Option Value field to 1 in the file import template of a file import to enable the counts to display.

    Note: By default, predefined templates have enable statistics turned off (i.e., the Internal Option Value field is set to 0 to turn off the counts).

Turn On the Trigger Flow and Test Import

1. Turn ON the new trigger flow created for the import in the Create Import Trigger Flow.

2. Drop your test file in the drop folder and then run your trigger flow and review results.