Skip to main content

Snapshot Flat File

Important

It is important to note that for the purposes of this document the use of the term 'snapshot' pertains to the general process of taking point-in-time data from the SIS that is stored in a flat file that is then consumed by Blackboard - not the snapshot controller.

Snapshot provides the ability to integrate using point-in-time data as provisioned by the SIS with two supported integration types (read: data formats) within the SIS Framework for Snapshot - Snapshot XML and Snapshot Flat File (or delimited data). The latter, Snapshot Flat File, is the 'go-forward' supported format and will receive updating to support new data objects. Snapshot XML will only receive maintenance updates.

Processing for both snapshot integration types is via an HTTP post of the data file to the appropriate endpoint.

Data supported

Data Object

Snapshot Flat File Support

Snapshot XML Support

Users

Yes

Yes

Courses

Yes

Yes

Memberships

(Enrollments and Staff Assignments)

Yes

Yes

Catalog Categories

Yes

Yes

Organizations

Yes

Yes

Observers

Yes

Yes

Institution Roles

Yes

Yes

Nodes

Yes

Cross Listed Courses

Yes

Terms

Yes

Standards (as a Course Attribute)

Yes

Requirements

Ability of the SIS to generate data formatted for the chosen integration type - Snapshot XML or Snapshot Flat File with appropriate delimiter.

A point worth reiterating - the Snapshot Controller and the Snapshot XML data format should not be the integration option of first choice as they are only receiving maintenance updates not new object support or feature development. It should be expected that at some time in the future an announcement for support removal will be made with the provision of ample time for client adjustment.

See the Enterprise 1.1 examples section for data samples, noting that the data format is slightly different between the two supported flavors of Enterprise 1.1 (standard and vista).

Setup: Snapshot integration types

In addition to an understanding of the data involved in any SIS integration, setup of the Snapshot Integration types requires knowledge of the SIS capabilities and endpoints with an eye on whether the data objects supported in the selected Snapshot type meet the client's integration goals. Additionally an understanding of the specific file format is necessary. You may create a Snapshot integration via the following steps in the Blackboard System Administrator SIS Integration UI: on the Administrator Panel select Data Integration, then Student Information System Integrations.

The following steps are the same for both Snapshot Integration types - the key difference in selection of the integration type is whether the data is XML (choose Snapshot XML) or delimited (Snapshot Flat File).

  1. Select Create Integration.

  2. Select Snapshot XML or Snapshot Flat File. On the next screen, select the configuration options.

    Note

    To perform advanced configuration (Data mapping for example), an integration must be set to Testing or Active status

  3. Integration Name: enter Snapshot XML or Snapshot Flat File.

  4. Description (optional): enter whatever you want to describe the integration; for example: Test Snapshot XML or Test Snapshot Flat File.

  5. Shared Username and Shared Password are used for data post operations.

  6. Set the Integration Status:

    1. Inactive — Integration performs no processing.

    2. Active — Integration performs processing and updates Blackboard data.

    3. Testing — Integration performs processing and does not update Blackboard.

  7. Set the log verbosity levels — for Testing, we recommend you select All Diagnostic and Debug Messages. For Active, we recommend a lower setting be used, such as Errors Only or Errors and Warnings.

  8. Select the Data Source Key (DSK) to be used for this integration. You may select to use a specific existing DSK, create a new key, or to use the data source provided in the feed.

    Note

    In all cases a DSK must be created before the data feeds may be processed.

  9. Only Snapshot Flat File supports the management of parent nodes. Select appropriately to meet your goals.

  10. Select the objects handled via this integration and how to handle parity. Generally, the defaults are fine. NOTE: You may have multiple integration types each handling a unique subset of available integration objects.

  11. Select Submit and your new Snapshot integration appears in the list of integrations.

One more step must be completed to support Snapshot integration and that is either configuring the SIS system to post data to Blackboard or to configure an app server side cron or batch job to post the data to Blackboard. Note that Snapshot endpoints are specific to the required task to be performed.

Application server side posting to Blackboard is accomplished by using CURL or similar application triggered by a cron or batch job:

curl -k -w %{http_code} -H "Content-Type:text/plain" -u SHARED_USERNAME:SHARED_PASSWORD --data-binary @PATH_TO_TEST_FILE ENDPOINT_URL

Replacing:

SHARED_USERNAME with the shared username

SHARED_PASSWORD with the shared password

PATH_TO_TEXT_FILE with the path the the xml data file

Replace ENDPOINT_URL with the endpoint url.

You can see a list of the endpoints on the server by going to the HTTP Information page off of the context menu of your flat file integration. On the Administrator Panel select Data Integration then Student Information System Integrations. From your integration's menu, choose Http information.

If it is supported by a middle tier application or by the SIS, data may be posted directly to the endpoint following the above protocol.

Once the above has been completed and the SIS has been enabled to generate data and post or save to file for posting to Blackboard you should begin to see logs on the integration.

Assess the data and determine if you need to make adjustments using the Advanced Configuration/Field Mapping on the available data objects. See section on Field Mapping at end of examples section for description of this functionality.

To learn more about processing data files, see Snapshot Flat File Automation.

Snapshot integrations

Snapshot integrations are based on sets of data that meet specific purposes. For example, to manage Users or Courses in the system. Of the two supported types - snapshot XML and Snapshot Flat File - only Snapshot Flat File should be considered most current with supported Blackboard data objects.

As with all the "file based" integration types (Snapshot and Enterprise 1.1 types) there are two processes for consuming data in those files - the use of the GUI Upload Feed File or the use of an http data transfer (for example: cURL). In the case of http data transfer a cron or batch job may be used to trigger the operation. Additionally, provided it is supported by the SIS or middle-tier, a data stream may be posted directly.

Each operation is achieved via a specific end point. These end points for the application server are available via the integrations HTTP Information link from the menu.

Create a data file

  1. In the Administrator Panel, under Integrations, select Data Integration.

  2. Select Student Information System Integrations.

  3. From the dropdown menu Create Integration, select Snapshot Flat File if you are creating a file manually.

  4. Enter an Integration Name (required). You have the option of entering a description.

  5. You can add security by entering a Shared Username and Shared Password. This is more often used for publishing to the endpoint.

  6. In Feed File Delimiter, select the character that separates fields in your data file.

  7. In Integration Status, select inactive, active, or testing.

  8. In Log Verbosity, select how detailed you want your logs to be.

The Data Support area allows you to do logical data segmentation. For example, if you have two different student information systems for two different campuses, you can segment for those two separate campuses. Or if you want to segment data for different academic terms.

  1. In Learn Data Source, you can select whether the data source is to be defined in the inbound data or use the same Blackboard Data Source for all new inbound data.

  2. You have the option to enter a Batch Uid Prefix. This is used to prefix identifiers for objects created by this integration to avoid id conflicts. It is not recommended that this be modified after data has been created for this integration.

  3. Select node placement in Parent Hierarchy Node.

In Advanced Configuration, you have an array of options to, for example, handle situations when specific records are not present in a batch. For this use case, we’ll leave the default selections in place.

When you’ve entered your data, select Submit. This will create your file for integration.

Upload a data file for integration:

  1. In the Administrator Panel, under Integrations, select Data Integration.

  2. Select Student Information System Integrations.

  3. Select the menu for the file you want to upload. Select Upload Feed File.

  4. In the Data Type menu, select your data type. The most common for integrations are Hierarchy Node (if your file represents your hierarchy), Course Association, Organization Association, and User Association.

  5. For Operation Type, the most common option is Store.

    • Use Complete Refresh when doing full data comparisons, and you want to delete something that’s not in the file. Use caution with this option.

    • Use Delete to delete the records you’re uploading.

  6. Select Submit.

You can check the SIS Log to confirm it is added. For larger files, it may take a couple of minutes to upload. Select Logs under Tools and Utilities, then select SIS Logs.

Snapshot endpoints

Snapshot operation endpoints for Flat File

Use this information to set up your SIS to POST Snapshot Flat File feed files to your integration. Stored objects will continue to exist, and retain their data. Deleted objects are removed. Complete Refresh will remove any existing objects that are not in the feed file. Posting a file of a specific object type to a non-matching endpoint will fail. For example: posting a Course data file to a Person endpoint will fail.

Important

Disabling a course will disable memberships in that course. This is new behavior.

You can find an operation's endpoint URL in the Blackboard user interface as well. Navigate to Administrator Panel and select Data Integration then Student Information Systems Integrations. Locate the integration, open its menu, and select HTTP Information. The integration's endpoint URLs are listed.

Operation

Endpoint URL

Course - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/course/store

Course - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/course/refresh

Course - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/course/refreshlegacy

Course - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/course/delete

Course Association - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/courseassociation/store

Course Association - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/courseassociation/refresh

Course Association - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/courseassociation/refreshlegacy

Course Association - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/courseassociation/delete

Course Category - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategory/store

Course Category - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategory/refresh

Course Category - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategory/refreshlegacy

Course Category - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategory/delete

Course Category Membership - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategorymembership/store

Course Category Membership - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategorymembership/refresh

Course Category Membership - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategorymembership/refreshlegacy

Course Category Membership - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/coursecategorymembership/delete

Course Membership - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/membership/store

Course Membership - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/membership/refresh

Course Membership - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/membership/refreshlegacy

Course Membership - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/membership/delete

Course Standard Association - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/standardsassociation/store

Course Standard Association - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/standardsassociation/refresh

Course Standard Association - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/standardsassociation/refreshlegacy

Course Standard Association - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/standardsassociation/delete

Hierarchy Node - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/node/store

Hierarchy Node - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/node/refresh

Hierarchy Node - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/node/refreshlegacy

Hierarchy Node - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/node/delete

Observer Association - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/associateobserver/store

Observer Association - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/associateobserver/refresh

Observer Association - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/associateobserver/refreshlegacy

Observer Association - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/associateobserver/delete

Organization - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organization/store

Organization - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organization/refresh

Organization - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organization/refreshlegacy

Organization - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organization/delete

Organization Association - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationassociation/store

Organization Association - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationassociation/refresh

Organization Association - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationassociation/refreshlegacy

Organization Association - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationassociation/delete

Organization Category - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategory/store

Organization Category - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategory/refresh

Organization Category - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategory/refreshlegacy

Organization Category - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategory/delete

Organization Category Membership - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategorymembership/store

Organization Category Membership - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategorymembership/refresh

Organization Category Membership - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategorymembership/refreshlegacy

Organization Category Membership - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationcategorymembership/delete

Organization Membership - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationmembership/store

Organization Membership - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationmembership/refresh

Organization Membership - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationmembership/refreshlegacy

Organization Membership - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/organizationmembership/delete

Person - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/person/store

Person - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/person/refresh

Person - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/person/refreshlegacy

Person - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/person/delete

Term - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/term/store

Term - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/term/refresh

Term - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/term/refreshlegacy

Term - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/term/delete

User Association - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/userassociation/store

User Association - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/userassociation/refresh

User Association - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/userassociation/refreshlegacy

User Association - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/userassociation/delete

User Secondary Institution Role - Store

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/secondaryinstrole/store

User Secondary Institution Role - Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/secondaryinstrole/refresh

User Secondary Institution Role - Complete Refresh by DSK

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/secondaryinstrole/refreshlegacy

User Secondary Institution Role - Delete

https://<servername.domain.edu>/webapps/bb-data-integration-flatfile-BB<DATABASEIDENTIFIER>/endpoint/secondaryinstrole/delete

If you are familiar with legacy Snapshot the SIS Framework Flat File type brings no change to the format other than the addition of headers for the new data objects supported. To learn more, see Snapshot Flat File Data Format and Snapshot Flat File Examples.

Snapshot operation endpoints for XML

Use this information to set up your SIS to POST XML feed files to this integration. Stored objects will continue to exist and retain their data. Deleted objects are removed. Complete Refresh will remove any existing objects that are not in the feed file. Posting an object containing an incorrect object type will fail.

Operation

Endpoint

Use Record Status

https://<servername.domain.edu>/webapps/bb-data-integration-ss-xml-BB<DATABASEIDENTIFIER>/endpoint

Complete Refresh

https://<servername.domain.edu>/webapps/bb-data-integration-ss-xml-BB<DATABASEIDENTIFIER>/endpoint/refresh

Delete Only

https://<servername.domain.edu>/webapps/bb-data-integration-ss-xml-BB<DATABASEIDENTIFIER>/endpoint/delete

Learn more

SIS Framework Overview

Data Source Key Overview