IQueue: Managing Work in the Background

Background

The IQueue interface enables QBO 3 to offload work to background processes. This happens by calling AbstractObject.Queue(), instead of AbstractObject.Invoke(). The queue method will instantiate an IQueue plugin, and pass the operation and parameters to be invoked to some queue.  Example plugins include:
  • MSMQ: Microsoft Message Queue which is a queue based on local disk access
  • SQL Service Broker: the preferred option, this is a queue based on tables in SQL Server
  • Amazon Simple Queuing Service: this is a queue that enables cross-organization communication of queued items
  • CurrentQueue, FutureQueue, FailureQueue (ObjectQueue): these are queues based on the QBO 3 ObjectQueue table that handle non-FIFO (e.g. future-scheduled) items

Performance Considerations

All QBO 3 systems come pre-configured with a 'DefaultQueue' entry (using MSMQ), and a Current, Future and Failure queue (using ObjectQueue). This is rarely a sufficient setup to manage a production application. New queues can be configured via Design > Configuration > Queuing, and should be created with the following guidelines:
  • To offload queue from a web server to app server(s), use SQL Service Broker
    • this enables multiple machines to monitor a single queue
  • Critical operations (such as deliver of data to clients) should have their own queue
    • this ensures critical operations are not 'stuck' waiting for non-critical operations
  • Long running operations (such as import routines) should have their own queue
    • this ensure long running operations don't interrupt routine operations
  • Operations dependent on a third party (such as interfacing with a servicing system) should have their own queue
    • this enables a queue to be stopped should there be an interface problem, without impacting the flow of other operations
  • High throughput operations (such a DecisionStep/Start) should have their own queue
    • this enables allocation of multiple threads to very common operations, so you can control how much of a machine's resources are dedicated to such things