Configuration‎ > ‎

Batch Processing

posted Sep 23, 2011, 12:43 PM by Eric Patrick
Many uses cases entail batch processing of portfolios, such as:
  • Ordering Military search for a portfolio of Contact records
  • Ordering Bankruptcy search for a portfolio of Debt records
  • Launching a Recovery workflow for a portfolio of Loan records
  • Generating Affidavits for a portfolio of Skiptrace records
The QBO Decision module now includes a series of Product classes to handle such batch ordering. 

Setting up Product Templates

Decide what "products" are to be made available by inserting ProductTemplate rows. For example, assume that EventID 27 is a Contact-based Military Search event:

INSERT INTO ProductTemplate (ProductTemplate, AppliesTo, SourceTemplate, SourceTemplateID)
SELECT Event, Object, 'Event', EventID
FROM Event
WHERE Event.EventID = 27

To make all QDS-related events products:

INSERT INTO ProductTemplate (ProductTemplate, AppliesTo, SourceTemplate, SourceTemplateID)
SELECT Event, Object, 'Event', EventID
FROM Event
WHERE Event.EventService = 'Quandis Data Services'

Setting up Contact-based Portfolios
  • Extend the skin to include a ContactPortfolio node, (see the LoanPortfolio node in Mortgage.Loan.SiteStructure.xml)
  • Create a ContactPortfolio.xslt container page, including a Product panel (see Templates > Mortgage > Servicing > LoanPortfolio.xslt)
  • Create a Collection Template called 'Contacts', and set the Template Type to 'Portfolio', and the SkinNode to ContactPortfolio
  • Create a Collection called 'GMAC Batch 001', using the 'Contacts' Template
  • Add a bunch of Contacts to the GMAC Batch 001 collection

Ordering Products

From the GMAC Batch 001 portfolio, click on the Products tab. This will execute pProductListByTemplate list all Product Templates that apply to the Collection.

FROM     Collection
    INNER JOIN ProductTemplate
        ON    ProductTemplate.AppliesTo = Collection.SourceObject
        ON    Product.CollectionID = Collection.CollectionID
            AND    Product.ProductTemplateID = ProductTemplate.ProductTemplateID
WHERE    Collection.CollectionID = @CollectionID

When you click on a Product hyperlink, a web service call to execute pProductLaunch is called.  This will:
  • create a Product row for the Collection, 
  • create ProductMember rows for each CollectionMember, and
  • queue an EventQueue row for each ProductMember to process itself
Note that this EventQueue row is not the Military Search event; it's an event to process some ProductMember!

-- Create the Product (including a default EventID and ReportID)
INSERT INTO Product (CollectionID, ProductTemplateID, ...)
SELECT @CollectionID, ProductTemplateID, ...
FROM ProductTemplate
WHERE    ProductTemplateID = @ProductTemplateID

-- Create ProductMembers
INSERT INTO ProductMember (...)
FROM    Product
    INNER JOIN CollectionMember ...

-- Queue processing of ProductMembers
INSERT INTO EventQueue (EventID, Object, ObjectID, ...)
SELECT Product.EventID, 'ProductMember', ProductMemberID, ...)
FROM Product
    INNER JOIN ProductMember
        ON    ProductMember.ProductID = Product.ProductID
WHERE Product.ProductID = @ProductID

Now wait for completion!

Product Processing

The "real work" of batch processing is being done by the ProductMember class. When you "launch" a Product, a ProductMember row is created for each CollectionMember row, and then an EventQueue row is created for each ProductMember.  At this point, the Event Queue Service takes over by processing the ProductMember events.

The ProductMember ProcessEvent method will process the underlying IDecisionStep class as it executes. This is not the normal Decision Queue processing that workflow does.  Instead, the ProductMember class uses the same factory to create an "underlying object" that the DecisionStep class does.  No DecisionStep row is created or required.

If the underlying class is an event, it will be fired synchronously.

If the underlying class is an IDecisionStep class, it will have ProcessStep called, and choose to complete itself or not, as it sees fit.  If the class reports itself as completed, the ProductMember will be marked completed as well.

The pProductMemberUpdate stored procedure will update the Product table indicating a SuccessCount and FailureCount, so as they are executed, the Product table reflect the current status of the "batch"

Product Dependencies

In some cases, order of one Product may be inappropriate until completion of another Product. This can be configured with a DecisionTemplate.  In this case, the DecisionTemplate is being used to define dependencies, but is not used to actually create new Decision rows.  The pProductListByTemplate stored procedure accepts a @DecisionTemplateID parameter, and includes 'DependencyPending' and 'DependencyComplete' columns.  The UI does not allow a user to create a new Product unless the 'DependencyComplete' column is NULL.

General Info

  • Portfolio: a group of records related by a Collection, with CollectionMember rows mapping to underlying records (Contact, Loan, Debt, etc.)
  • Product: some sort of IDecisionStep instance that will be batch ordered
    • e.g. a Military Search Event
    • e.g. a SkipTrace Affidavit Attachment Template
    • e.g. a Bankruptcy Search Event
    • e.g. a Recovery Workflow Decision Template
  • ProductTemplate: this table defines the subset of IDecisionStep template instances that constitute "Products"
  • Product: this table relates ProductMembers to a Collection
  • ProductMember: this table relates an IDecisionStep instance to a Product