Queue Service 2.1

Queue Service has been updated to version 2.1 which allows searching of queues, recovery of any queue, ability to move messages, increased security.

Release Notes

Features

Summary of Features

• • Increased management of queues which includes: queue searching for unprocessed, in-process or processed records, ability to move messages to other queues, recover in-process messages in any queue

• Increased logging of queue activity: QueueLog now saves all parameters used to process a given item and they are searchable. QueueLog can show the life cycle of an item and split based on message iteration. All queue command execution is logged to QueueLog for auditing

  • All queues can be recovered. In Queuing 2.0 items that originated only from ObjectQueue could be recovered

  • Increased security: All actions on Queue.ashx are controlled by system functions

  • Processing views show items processing for all queues

  • UI Improvements: Queue.ashx will now automatically refresh relevant panels when taking manual actions

  • Performance enhancements for queue count checks (applicable to ServiceBroker and ObjectQueue) 

  • Simplified, condensed logging: RetryQueue removed and mapped to FailureQueue. System/Debug logging now occurs without enabling DebugLog for traceability

  • QueueLog Purge/Expiration custom statement

  • Bug Fixes (see below)

Searching Explained

• • Queue messages have been extended to contain:

    • ParentMessageID - this is established at message inception and is carried through a message as it is processed throughout queues (assuming item is scheduled or reprocessed)

    • Iteration - represents lifecycle of a message withing a given queue. For example. If a scheduled message enters the ObjectQueue first, the entire process (and logging) through ObjectQueue would be represented as Iteration 1 and when the message is submitted to the target queue (E.g. DefaultQueue) Iteration would be incremented to 2. 

  • Ability to search queued messages. Clicking on "Count" will display the queued items in the "Queued" tab below the summary tab. Queued Messages can be searched by: Attributes, Label, ClassName (Object), Operation, ObjectID, Queued User, ExecuteCount and QueuedDate. This allows searching and locating queued messages that users can take actions on (future queue, move, delete)

  • Ability to search in-process messages. Clicking on "Processing " under a queue will refresh the "Processing" panel. Items can be removed or recovered with the ability the future date if desired

  • Ability to track a queued item throughout it's lifecycle. Clicking on the search icon within a message label will pop up a history panel that shows all message activity which can then be filtered as desired. This is a single view of an item to show exact audit trail.

  • Processed queue item searching has been extended to search by Parameters. This allows for increased searching and better reconciliation since multiple calls to the same operation can now be distinguished based on parameters

Bug Fixes

• • Changes to CurrentQueue will now save to ConfigurationEntry instead of Queuing.config

  • Security failure for requeuing/rescheduling items due to queue caching fixed

QueueLog Purging

QueueLog now contains a Purge statement that purges QueueLog records with the following algorithm:

• • Any QueueLog record with an expiration date. Expiration date is set for all non-scheduled queued item which are logged. Expiration date can be adjusted by qbo.Application.Properties.Settings.Default.QueueLogExpirationInDays and defaults to 14 days

  • Any scheduled items with an occurrence of more than 100 records. For example, for a scheduled item such as a data listener which runs every 15 minutes, records will be saved to QueueLog based on ParentMessageID. The purge statement will purge any records that are older than the most recent 100 occurances for a given ParentMessageID. This setting can be adjusted by setting the parameter "ScheduledOccurances" in QueueLog Purge statement.

Steps to update to version 2.1:

1. Ensure Steps to update to version 2.0 have been met.

2. Get latest from trunk:

   1. Qbo.Application/qbo.ApplicationWeb

   2. Qbo.Logging/qbo.LoggingWeb

   3. Plugins:

      1. Qbo.Queue.ObjectQueue

      2. Qbo.Queue.MSSQL

   4. /qbo v3/Installation/qbo.QueueService.2.10.msi

   5. V2 – qbo.Application.Database/Tables:

• • 1. ObjectQueue.sql

    2. QueueLog.sql

    3. QueueLog.indexes.sql

    4. ObjectQueue.Indexes.sql

3. Ensure Queue Service is stopped

4. Backup

• 1. QueueService.exe.config

  2. /Config/Queuing.config

  3. /Config/QueueLog.config

  4. Web.config

5. Remove qbo.Queue.MSMQ.dll from any bin folders

6. Run table update / index scripts - ensure they run successfully

7. Build and deploy all projects

8. web.config – add categorySource ‘QueueLog’

9. Queuing.config

• 1. Save current queuing.config to alternate location

  2. use Queuing.config in ApplicationWeb

10. Log into site and cycle web.config

11. Navigate to /Application/ServiceBroker.ashx/ToggleRetention. This will toggle all user created ServiceBroker queues to enable message retention.

12. Ensure that SystemRole 'Public' with permission 'QueueLogInsert', 'HolidaySearch', 'ScheduleSelect', 'HolidayInstanceSearch', 'ConfigurationEntryUpdate' granted

13. Uninstall Queue Service

14. Install qbo.QueueService.2.10.msi

15. Replace QueueService.exe.config with backup version

16. Run Test Scripts from browser

• 1. Application – Queue Testing - All steps must pass to use this build. Note in some testing the DB Server time and Web Server time had to be manually synchronized for script to succeed:

     1. Control Panel->Set the time and date->Internet Time->Change Settings->Update Now

  2. Any additional test scripts you desire

17. Start Queue Service

18. Manually queue an item for immediate processing 

Upon starting the Queue Service refresh the Queue list from Queue.ashx. If you observe any "Paused" queues this indicates a problem. Inspect (Edit) each queue individually and observe the settings. Common issues to look for are:

• • ServiceBroker message retention has not been enabled

  • Queue is set to ExecuteAs "AdminUser" and no Admin Person is specified and/or account is inactive or locked out

  • Queue threshold was set to x and there were x consecutive processing errors