qbo.Matrix: Rules Engine

The Matrix module allows power users to set up 'fuzzy logic' rules in QBO. At it's core, given a bunch of inputs ('dimensions'), a matrix will return the best matches based on weighted criteria.

Use Case: Worklist Assignment

Each Smart Worklist Template may optionally specify a Matrix to determine outputs (measures):

• • a Team,

  • a Person, and

  • a Priority

for each item in the worklist.

For example, a QBO client chose to distribute foreclosure work based on the following inputs (dimensions):

• • Investor

  • State

  • Term Digit

Use Case: Product Pricing

A client with several dozen service offerings ("products") tiers pricing by State, Client, Investor, and Loan UPB Range. In the table below, Product, Sttae, Client, Investor and UPB Range are 'inputs' (dimensions), while Cost and Price are 'outputs' (measures):

In this use case, the blue columns are dimensions, and the green columns are measurements.  The cost of a BPO is:

• • $45 in TX if the UPB < 100K, else

  • $60 in CA, else

  • $50 in all other cases

Pricing is more complicated:

• • $95 for Wells Fargo loans with Goldman as the investor, else

  • $90 for Wells Fargo loans, else

  • $110 for UPBs over 500K, else

  • $120 for CA, else

  • $100 in all other cases

Part of the challenge is determining which row wins if there are 'ties'.  For example, should a BPO done in CA for Wells Fargo be priced at $90 or $120.  The answer can be configured by applying weights to dimensions.  For example, dimensional weights in this example are:

• • State: 5

  • Client: 10

  • Investor: 3

  • UPB Range: 1

For a BPO where the state is CA, client is Wells Fargo, investor is Chase, and UPB is 200K, each row is weighted as follows:

Note that row 3 has a weight of 10, beating row 2 with a weight of 5, and row 2 beats row 1 with a weight of 0.   Rows 4 and 5 are not eligible, since they require criteria that the subject BPO does not contain.

Dimension XPath Hints

Dimensions based on repeatable values often should select the most recent item. The pattern for this is: (//MyXPath)[position()=last()]

Third Party Integration

Matrix functionality can be invoke in several ways. The Matrix/Lookup operation will return results in either JSON or XML format, with the best-matched row returned first. In most use cases, third party systems will elect to treat the first row as the 'best result' and ignore all subsequent rows. There are use cases, however, where a third party system may wish to take some action based on more than the first row. For this reason, the Lookup operation will return the top 25 rows by default.

When returning these 'rows', there are 3 columns that reflect the relevance of the row to the input columns:

• MisMatchCount: count of inputs that conflict with the row's values. Generally, any rows with a MisMatchCount > 0 should not be considered as 'matched'

• Weight: the sum of the weight of all matching dimensions; the higher the weight, the better the match

Storing Matrix Results

The output of a matrix can be stored as a Score, by using the Matrix Scoring Engine. To do this:

• From a Matrix, choose Options > Create a Score

  • this will create a Score Template with the same name as the Matrix, and a Score Item Template for each Matrix Measure (output column)

From this point, you can call:

Score/Score.ashx/Calculate?Template={Matrix}&{Dimension}={Value}...

Score/Score.ashx/Calculate?Template=Product Pricing&Product=BPO&State=CA&Client=Wells Fargo

One can upload a spreadsheet of items to evaluate, and view the results in a Score Analysis tab as follows:

• Create an Import File Template using the Batch Engine (e.g. "Product Pricing Import")

  • the Result Method should be Score/Analysis?InImportFileID={ImportFileID}

• Upload a spreadsheet for import, where the spreadsheet contains these columns:

  • ClassName: (set the value to Score for each row)

  • Operation: (set the value to Calculate for each row)

  • Template: (set the value to your Matrix / Score Template name for each row)

  • Score: (set the value to be whatever you want)

  • {Dimension 1}

  • ...

  • {Dimension n}

• From Dashboards > Import Files > {Your Import}, click on the Results tab to view the results

Bulk Lookup

Matrix/BulkLookup supports "batch" lookup operations. Assume we have a Matrix called 'Required Docs' like this:

One can call Matrix.ashx/BulkLookup?Matrix=Required Docs passing a payload like this: 

<Batch>

    <Input>

        <Loan>123</Loan>

        <AppType>HAMP Loan Mod</AppType>

        <PropertyState>WA</PropertyState>

    </Input>

    <Input>

        <Loan>234</Loan>

        <AppType>Liquidation</AppType>

        <PropertyState>MA</PropertyType>

    </Input>

</Batch>

and you will get back something like this:

<MatrixBulkLookup>

    <Output>

        <Loan>123</Loan>

        <AppType>HAMP Loan Mod</AppType>

        <PropertyState>WA</PropertyState>

        <Package>HAMP Package</Package>

    </Output>

    <Output>

        <Loan>234</Loan>

        <AppType>Liquidation</AppType>

        <PropertyState>MA</PropertyType>

        <Package>MA Closeout</Package>

    </Output>

</MatrixBulkLookup>

Note the Loan node in there for human eyes; it will be regurgitated in the output, but is useless to the Matrix unless you have a Loan dimension.

The reasons for using BulkLookup include:

• You get back only the 'best match', Matrix/Lookup is giving you all matches (more effort, more data over the wire) and more crap for UI purposes

• Significant reduction in HTTP connection chatter vs calling Lookup for each input node

If you have a lot of items to look up, make the call asynchronous:

Application/Matrix.ashx/AsyncLookup?Matrix=Required Docs

This will save the XML input payload to imaging, and queue the job for processing asynchronously. The output payload will be saved to imaging, and made available for retrieval.

A call to Matrix/AsyncLookup will return an ImportFile XML node:

<?xml version="1.0" encoding="utf-8"?>

<ImportFileItem xmlns:i="http://www.w3.org/2001/XMLSchema-instance">

<Object>Matrix</Object>

<ObjectID>225</ObjectID>

<ImportFile>Matrix Async Lookup</ImportFile>

<ImportFileID>5035</ImportFileID>

<ImportFileTemplateID>36</ImportFileTemplateID>

<Items />

<PreTransformID>117557</PreTransformID>

</ImportFileItem>

You can then poll for results via:

Import/ImportFile.ashx/Select?ID={ImportFileID}&Output=Xml

looking for a Status node of 'Complete':

<?xml version="1.0" encoding="utf-8"?>

<ImportFileItem xmlns:i="http://www.w3.org/2001/XMLSchema-instance">

<Status>Complete</Status>

<Object>Matrix</Object>

<ObjectID>225</ObjectID>

<ImportComplete>2014-06-17T11:46:35.077</ImportComplete>

<ImportFile>Matrix Async Lookup</ImportFile>

<ImportFileID>5035</ImportFileID>

<ImportFileTemplateID>36</ImportFileTemplateID>

<ImportResultID>117558</ImportResultID>

<ImportStart>2014-06-17T11:43:02.183</ImportStart>

<Items />

<PreTransformID>117557</PreTransformID>

</ImportFileItem>

Nested Matricies

Some matrices may become very complex, particularly if they're being used to field requirements from multiple business units. For example:

• • In some states, loan characteristics (UPB, days delinquent) may require creation of a breach letter

  • Other states may require launching of a default workflow

  • Breach letter templates may be based upon investor, state, and other criteria

  • Default workflow templates may be based upon additional loan characteristics (UPB, loan type, agency backing)

Creating a single matrix to field all these requirements gets a bit unweildy:

• • Dimensions: State, Breach UPB Range, Breach Days Delinquent, Investor, Default UPB Range, Loan Type, Agency Backing

  • Rows that are 'breach centric' populate only State, Breach UPB Range, Investor, and Breach Days Delinquent

  • Rows that are 'default centric' populate only State, UPB, Loan Type, and Agency

In such cases, you may instead configure a nested matrix: a matrix that calls other matrices. For the example above:

• • Create a matrix called 'Breach Letter'

    • inputs are State, UPB Range, Investor, Days Delinquent

    • output is an Attachment Template that determines which beach letter to launch

  • Create a matrix called 'Default Workflow'

    • inputs are State, UPB Range, Loan Type, Agency

    • output is a Decision Template that determines which default workflow to launch

  • Create a matrix called 'Delinquent Chooser' (the master matrix)

    • inputs are State, UPB, and Days delinquent

    • output is 'Matrix': if a row is breach centric, the output is 'Breach Letter'; if the row is default centric, the output is 'Default Workflow'

This allows the breach letter power users to maintain the breach matrix independently of the default workflow power users.

When calling Matrix/Lookup?ID={Delinquent Choose}&State=CA&UPB=127522.78&DaysDelinquent=35, assuming this matches a breach row, the output will include:

• • Matrix: Breach Letter

  • Attachment Template: {Some Attachment Template}

  • {note there are no 'default workflow' outputs here to clutter things up}

When calling Matrix/Lookup?ID={Delinquent Choose}&State=NY&UPB=127522.78&DaysDelinquent=62, assuming this matches a default row, the output will include:

• • Matrix: Default Workflow

  • Decision Template: {Some Decision Template}

  • {note there are no 'breach letter' outputs here to clutter things up}

To leverage nested matrices, the output of the master matrix must:

• be a string output type,

• include the word 'Matrix' in the output name, and

• its value must be the name of an existing matrix

Updating Objects From a Matrix

The AbstractObject/UpdateFromMatrix method can be used to change the properties of any QBO object by evaluating it against a Matrix. Examples include:

• • Set a Loan.AssignedPersonID based on a Matrix output

  • Set an Organization.Status based on a Matrix output

To leverage this feature, create a Matrix with:

• • 1 or more inputs (dimensions) that consider data associated with an object

  • 1 or more outputs that exactly match the field names of an object

  • Call {ClassName}/UpdateFromMatrix?ID={ID List}&FromMatrix={Matrix}

Parameters:

• • ID (Int64): one or more identity values for the objects being updated.

  • Overwrite (boolean)

    • When true, any Matrix outputs will overwrite the Object's values

    • When false, any Matrix outputs will only overwrite the Object's values if the Object's values are NULL

As with all standard methods, this can be used from a workflow step.

Loan Assignment Example

• • Create a Matrix named Loan Assignment

  • Create an input called UPBRange, with a value type of Money Low - Money High

  • Create outputs called Status and AssignedPersonID

  • Add rows as you deem fit

  • Call Loan/UpdateFromMatrix?ID=X&FromMatrix=Loan Assignment&Overwrite=true

Thinking Like a Business Analyst - Not a Programmer

A QBO Client tasked their IT department with creating a matrix to assess whether breach letters should not be sent on eligible loan. The rules were:

• • Don't send if more than 60 days delinquent, except for Investor ABC

  • Don't send if more than 45 days delinquent for Investor ABC

  • Don't send if more than 45 days delinquent, the state is AR, CA, FL, MA, NM, NY, RI, TX, WA, ME, and Hold is true

  • Don't send if the state is NJ, they don't have FC signing authority, and there is no investor

  • Don't send if more than 60 days delinquent and the state is PA

Their IT department create a 300 row matrix, covering all possible permutations of the rules; that is a row for every state, investor, and signing authority. Running against their entire breach-eligible loan portfolio took hours.

When their business unit reviewed the matrix, a business analyst redesigned the matrix to be 5 rows, with a row matching each bullet point listed in the requirements. Running against their entire breach-eligible loan portfolio took 6 minutes.

The Matrix module is intended to be configured in a manner that makes sense to a business analyst!

Programmers often tend to think in procedural manner, a mindset that frequently does not map "naturally" to business requirements. All too often, we let the inmates run the asylum. In this case, programmers genuinely did not think to use pattern matching and weights.  Because they did not, their procedural trained minds looked at 5 bullet points of requirements, and came up with 300 lines of all possible permutations.