Reach Database Refresh Process

The scope of the Database (DB) Refresh process involves copying the latest production database into a sandbox or non-production environment to ensure data consistency for testing, validation, and troubleshooting activities. This includes all production data, configurations, schema, and customizations as of the refresh date. The process aims to provide an exact replica of the production environment, enabling teams to perform realistic testing, validation of new changes, performance checks, and issue reproduction using up-to-date data.

As part of the refresh, environment-specific settings, integrations, and user access controls will be reviewed and updated to prevent unintended connections to live systems or data exposure. The activity also includes validation steps post-refresh to ensure environment stability and readiness for testing purposes.

Prerequisites

Note: Customers must perform the following steps on the Anthology Student environment.

  1. On the Student database, run the Student Integration Workflows. To do so:

    1. Download the files from the StudentIntegrationWorklows.zip file.

    2. Run the EnableWorkFlowsFromCsv.ps1 and ExportandDisableStudentWF.ps1 scripts in windows powershell.

  2. Remove the Student SIS Registry Value

    Remove the CnsCneIntegrationUrl value from the SyRegistry table in the Anthology Student non-production database.

Steps for Performing DB refresh for a Reach Environment

Note: For DB refresh, the production environment is always the source system, and the non-production environment is the target system.
Important: The following steps must be performed by an Anthology Reach Professional Services / DevOps professional.

In this step, the "DB refresh pipeline pre-script" is run to export the non-production environment configurations.

Note: Before running the script, verify that the selected environment is non-production and not production to avoid accidental execution on the production environment.

The "DB refresh pipeline pre script" performs the following actions:

  1. Exports data of the following entities.

    • Configuration (cmc_configuration)

    • Portal Configuration (pav_portal)

    • SMS configuration

      • TeleSign SMS Short/Long Code (ts_shortlongcode)

      • TeleSign SMS Settings (ts_smssettings)

      • TeleSign SMS API Keys (ts_apikey)

    • Payment Gateway Configuration (cmc_paymentgatewayconfiguration)

  2. Exports all connection references.

    This action exports all connection references to retain both custom and OOTB (Out- of-the-Box) connection details, including FTP connection references (if configured).

    Notes:

    • Only if a connection reference is created for a connection, the connection details are retained after DB refresh.

    • Before performing DB refresh, save and take backup of all the configurations that are not pushed to the production environment.

    • Connections configured exclusively in the non-production environment, but not in the production environment, must be reconfigured after a DB refresh.

The Copy option in the Microsoft Power Platform admin center is used to copy the production environment to a non-production environment. For more information, see Microsoft Documentation.

Note: Use the Copy over everything option to create a copy of the Reach environment.

The "DB refresh pipeline post script" is run on the copied database in the non-production environment to perform the following cleanup actions:

  • Deactivates the following OOTB (Out- of-the-Box) Application Users:

    • EngageFunction

    • EngageFlows

    • Anthology Reach Portal App

    • ASBIntegration

    • Anthology Deployment Service

  • Updates TPC Portal URLs to null.

  • Updates System User Google and Outlook subscription fields to null.

  • Deletes all records of the following entities:

    • Real-Time Marketing

    • Occupation Insight details

    • MSCRM Add-on

    • Live Assist

    • Document Management

    • WhatsApp Configuration

    • Resets all environment variables to null.

    • Deletes all integration mappings containing webhook URLs.

  • Updates Connection References

    Updates all connection references (both Connection ID and Connector ID) using the data exported in Step 1 to ensure all integrations function correctly after DB refresh.

    Note: After the DB Refresh process is complete, validate that the connection is pointing to the correct environment.

  • Imports Configuration Data

    Imports all configuration entities that were exported in Step 1 to restore environment-specific configurations.

For privacy and compliance, data masking is applied. By default, all fields of type Email in the Contact entity are masked.

Based on customer requirements for masking additional fields (e.g., phone, SSN, or other entity fields), customers can request for additional masking scripts.

Masking Types

You can mask a list of entities, and can also configure masking types for the following fields of an entity:

  • All fields of type Email. For example, all Email type of fields in the Inbound Interest entity.

  • All fields of type Phone number. For example, all Phone number type of fields of the Contact entity.

  • Any specific field. For example, SSN field of the Contact entity.

You can mask the field values either with a masking format or a default value.

Field masking types with examples

  • Email Masking Type - It can be used for email fields or any other field where the field value is not null or empty, and “X” must be added at the end of the value.

    For example, abc@anthology.com will become abc@anthology.comX.

  • SSN Masking Type - It can be used for SSN field or any number field where the field value should be updated to an auto incremented 9-digit SSN number format prefixed with zeros.

    For example, 000-000-001, 000-000-002, 000-000-003

  • Null Masking Type - It can be used to replace all values with blanks.

  • Default Value - You can provide a default value to replace the existing value for a field:

    • Example 1 – To replace the email field values with a default value, you can use:

      MaskingType = "email";

      DefaultValue = ”test@anthology.com”

    • Example 2 – To replace the SSN field values with a default value, you can use:

      MaskingType = "SSN";

      DefaultValue = ”000-000-0000”

Customers can use the following template to add their masking requirements along with the DB Refresh request:

DB Refresh Request Form

Perform all steps required for a fresh installation of Anthology Reach on the non-production environment.

The following 3rd party servers and configurations are reconfigured:

  • Configurations performed by the DevOps team

    • MSCRM Addon

  • Configurations performed by the Professional Services team

    • Tele Sign SMS

    • Student Integration

      Note: The Anthology Reach DevOps team runs the student integration pipeline to update the environment variables, enable the flow, and update the connector.

    • Live Assist

    • Learn Integration

Note: Additionally, any other custom configurations on the Reach environment will be handled by Professional Services / Customer after the DB Refresh.

 

Note: DB Refresh excludes the following:

  • Portal database

  • Raise or any other 3rd party data (considered as custom configurations)