Import Best Practices

The Anthology Reach data import process (framework) is designed as a scheduled batch process best suited to import incremental transactional data to an Excel or CSV file output. The process works best for data added or changed within 24-48 hours. The import process is not suitable for large volume data migration activities.

The following topics provide information to help you evaluate your import process requirements during design of your import, and to optimize your import run time settings to best suit your business needs.

Import Template Design

Designing an efficient import process involves considering various factors. You can optimize your import runs by adjusting settings based on your import template design. When planning and designing your import, keep the following points in mind:

  1. Batch Size/row Batch Size

    Use an optimal batch size for imports/exports based on your template. Larger batch sizes/row Batch size can reduce server load and improve performance, but be mindful of limits. Adjust the batch size to suit your specific template.

  2. Flow Triggers

    Avoid triggering all flows simultaneously to reduce server load. Use scheduled runs to stagger processes.

  3. Delta Process

    Utilize the delta process, as the Import/Export framework is designed for handling delta records. This framework is not recommended for large-scale migration activities.

  4. Dataset Mapping

    Ensure dataset mapping for all option set fields during both import and export. When possible, use datasets for lookups to streamline operations.

  5. Duplicate Check

    For imports, leverage the external ID column if your template has a unique field. This can bypass Dynamics’ duplicate-checking rules, reducing load and enhancing performance.

    Note: The out-of-the-box(OOTB) Import/Export framework is intended as a general-purpose tool and should be implemented with careful validation of customer use cases, infrastructure capabilities, and load. It is not designed for migration purposes and is primarily meant to handle daily transactional data for a few key entities.