Export Framework

Overview

QBO supports the publication of templated data between QBO systems. Typically QBO client create and continually refine workflows in a UAT environment. When power users are happy with the behavior of a workflow in UAT, they can export it from UAT, and import it into their PROD environment.
All QBO modules support an Export method, which produces a XML document representing the object exporting and all related components. For example, DecisionTemplate/Export?ID=X will export a workflow template (ID = X) as well as:
  • All steps in the workflow (DecisionStepTemplate)
  • All dependencies in the workflow (DecisionDepenency)
  • Related tasks (ImportFormTemplate)
    • Related questions (ImportFormQuestion)
    • Related XSLTs (Attachment)
    • Related method signatures (Statements)
  • Related documents (AttachmentTemplate)
    • Related template documents (Attachment)
    • Related method signatures (Statements)
  • Child workflows (DecisionTemplate)
  • etc., recursively
Ideally, power users should create and refine workflows in a UAT environment, and always promote the workflow via Export. Power users should not modify a workflow directly in PROD.

SubscriberIDs

QBO systems typically do not have exactly the same number of template rows, created in the same order. For example, some workflow in UAT may be created but never promoted to PROD, or a workflow created on 1/1/2017 may be deployed to PROD after a workflow created on 2/1/2017. This means that the identity columns for the template tables almost never match between QBO environments.  In order to properly maintain relationships between rows (e.g. workflow vs workflow steps), the Export command replaced IDs and foreign keys with a SubscriberID.

For example, instead of:

<DecisionTemplateItem>
  <DecisionTemplateID>sample.quandis.net-DecisionTemplate-REO Sample Workflow</DecisionTemplateID>
  <DecisionTemplate>REO Sample Workflow</DecisionTemplate>
  ...
</DecisionTemplateItem>

the Export will produce something like:

<DecisionTemplateItem>
  <SubscriberID>sample.quandis.net-DecisionTemplate-f04d466f-ce86-4f29-b4fb-6543734c469c</SubscriberID>
  <DecisionTemplate>REO Sample Workflow</DecisionTemplate>
  ...
</DecisionTemplateItem>

The replacement of identity values with subscriber values runs deep; foreign keys and 'soft' keys like Object/ID are included in Export's mapping.

Handling SubscriberID values are baked into all QBO modules. When importing this XML into another environment such as PROD, PROD will check to see if the SubscriberID already exists. If it does, PROD will retrieve the 'correct' ID for the record, and update it.  If not, PROD will create a new record, and add the SubscriberID to the newly created record, so the next time it's seen, the correct ID can be retrieved.

This pattern enables:
  • ignoring differences in ID values between environments
  • ability to change the name of templates, and have the updated name promulgate between environments
  • ability to delete records, and have the delete promulgate between environments

Export Options

The Export method accepts the following parameters:
  • ID: a list of identity values for the module being exported (e.g. DecisionTemplate/Export?ID=17,22,253)
  • SubscriberUrn (string): the base of object subscription records to be used (e.g. uat.client.com)
    • defaults to the application setting for the site
    • if ObjectSubscription rows starting with the SubscriberUrn already exist, they will be used by the export
    • if ObjectSubscription rows starting with the SubscriberUrn do not yet exist, one will be created using a GUID to uniquely identify the row
  • Subscribe (bool): if true, ObjectSubscription rows will be created in the system running Export on (typically UAT)
    • defaults to false
  • ExcludeAccess (bool): if true, ObjectAccess and related security rows are exported
    • defaults to true

Reconciling Workflows

Working strictly in a UAT environment is ideal, but often not the "real world". Clients that manually created workflows in BOTH a UAT and PROD environment (instead of exporting and importing) may be faced with a challenge of reconciling two workflows. It is possible to automatically reconcile these workflows as long as the naming is identical between environments. For example, a workflow called "REO Sample Workflow" in UAT and in PROD can be reconciled automatically.  If the workflow is called "REO Sample Workflow" in UAT and "REO Workflow" in PROD, manual intervention is required.

To automatically reconcile a workflow between UAT and PROD, let's assume a REO Sample Workflow in UAT is ID=27, and in PROD is ID=22.
  • Export the workflow from UAT, using a SubscriberUrn and Subscribe=true
    • e.g. DecisionTemplate/Export?ID=27&SubscriberUrn=master.client.net&Subscribe=true&ExcludeAccess=true
    • note: at this point, the subscription records in UAT have GUIDs
  • Modify the related ObjectSubscription records in UAT via a query
    • e.g. DecisionTemplate/SubscribeByName?FromUrn=master.client.net
    • note: at this point, the subscription records in UAT have unique names instead of GUIDs
  • Export the workflow from PROD, using the same SubscriberUrn and Subscribe=true
    • e.g. DecisionTemplate/Export?ID=22&SubscriberUrn=master.client.net&Subscribe=true&ExcludeAccess=true
    • note: at this point, the subscription records in PROD have GUIDs
  • Modify the related ObjectSubscription records in PROD via a query
    • e.g. DecisionTemplate/SubscribeByName?FromUrn=master.client.net
    • note: at this point, the subscription records in PROD have unique names instead of GUIDs
      as long as these unique names match the unique names from UAT, automatic reconciliation will work!
  • Export the workflow from UAT
    • e.g. DecisionTemplate/Export?ID=27&SubscriberUrn=master.client.net&Subscribe=true&ExcludeAccess=true
    • note: the XML produced will have SubscriberIDs with unique names, instead of with GUIDs
      again, as long as these names match PROD names, automatically reconciliation will occur
  • Import the UAT XML file into PROD
Considerations:
  • If a workflow step name in UAT and PROD do no match:
    • ideally, rename the SubscriberID instances in the PROD ObjectSubscription table to match UAT
    • if the import is run with mismatched name, the import will create a 'duplicate' record. At this point, the 'original' record can be deleted.
  • You can export from PROD to UAT as well, once the SubscribeByName has been called in both systems

Assumptions

The definition of "unique names" differs for different tables. The SubscribeByName statement makes the following assumptions:
  • DecisionTemplate: DecisionTemplate is unique
  • DecisionStepTemplate: DecisionTemplate + DecisionStepTemplate is unique
  • DecisionDependency: DecisionTemplate + DecisionStepTemplate + DependencyStepTemplate is unique
  • ImportFormTemplate: ImportFormTemplate is unique
  • ImportFormQuestion: ImportFormTemplate + ImportFormQuestion is unique
  • SmartWorklistTemplate: SmartWorklistTemplate is unique
  • Attachment: {Object} + {Object Label} + Attachment is unique
  • Collection: {SourceObject} + Collection is unique
  • ObjectStatus: {Object} + {Template Label} + ObjectStatus is unique
  • SystemRole: SystemRole is unique
If there are other table that need to be covered by this statement, determine the unique pattern required for the table, and extend the statement.


Comments