Queue Service 2.0

Queue Service has been updated to version 2.0 which increases robustness, redundancy and ease of configuration.

Release Notes

• Queues now use Schedules from the Schedule module to determine execution time. They only use “Continuous” schedules as these are based on a time range. Daily Continuous schedules are most common but weekly and monthly schedules are supported as well.

• Queuing.config and any other qbo.Config entries that inherit from DbConfigurationElementCollection (eg. DBStatement) now utilize SqlDependency. This allows for automatic configuration change propagation to all subscribing application. For example, if the application has 1 web server and 2 application servers, the Queuing settings can be modified from the web server and the changes will automatically propagate the application servers.

• Queue Service can be installed in a redundant scenario (multiple app servers monitoring the same queues). ObjectQueue.PopItem is now thread safe so the number of threads can be increased to 5 (total across all app servers). See Greg for setup.

• Addition of StagedQueue. Staged Queue is a view into ObjectQueue that shows any queue item that originated from ObjectQueue (Eg. CurrentQueue) and has been added to the target queue store but has not started processed .

• Addition of ProcessingQueue. ProcessingQueue is a view into ObjectQueue that shows any staged item that is currently processing. Since it’s currently processing,  the item is not viewable in the target queue as it’s in memory.

• Queue Recovery

  • Recover All – From Queue Summary recovery can be executed based on a date range. The recovery observes all items in the ProcessingQueue (based on date range) and will release records back to ObjectQueue so they can be processed again.

  • Recovery Individual – Can be executed on individual records in staged and processing queue. A check will be made to ensure item does not exist in target queue.

• Queue Management – Additional “safe-guard” logic has been implemented:

  • Queue must be stopped to be Purged

  • Queue must be empty and stopped to be Removed

  • Queue Removal will now remove the underlying queue store

  • Errors raised from Queuing.ashx will show as alerts for ease of use

• QueueLog

  • Now tracks what machine name and application name processed item

  • Filters (‘Success’, ‘Failure’, ‘Debug’, etc.) now functional when accessing QueueLog

• Schedule module now has a working UI to create, modify and test schedules. The schedule module can now calculate next run date which includes observing the Calendar associated with a schedule.

Steps to Update to Current Version:

1. Ensure ServiceBroker is functional on application’s target database

2. Get latest from trunk:

   1. Qbo.Application/qbo.ApplicationWeb

   2. Qbo.Queue.MSMQ

   3. Qbo.Queue.ObjectQueue

   4. Qbo.Queue.MSSQL

   5. Qbo.Logging/qbo.LoggingWeb

   6. Qbo.Security

   7. Qbo.Exception

   8. /qbo v3/Installation/qbo.QueueService.2.00.msi

   9. V2 – qbo.Application.Database/Tables:

      1. ObjectQueue.sql

      2. QueueLog.sql

3. Ensure Queue Service is stopped

4. Backup

• 1. QueueService.exe.config

  2. /Config/Queuing.config

  3. Web.config

  4. All config files - optional

5. Run table update scripts

6. Build and deploy all projects

7. web.config – add categorySource ‘Configuration’

8. Queuing.config

• 1. Save current queuing.config to alternate location

  2. use Queuing.config in ApplicationWeb

9. Log into site and cycle web.config

10. Ensure that Schedule, Calendar, Holiday are implemented in Entity view

11. Navigate to /Application/Queue.ashx/Setup. This should seed Schedule table with default Queue Schedule in new Queuing.config

12. Ensure that SystemRole 'Public' with permission 'QueueLogInsert', 'HolidaySearch', 'ScheduleSelect', 'HolidayInstanceSearch' granted. Ensure that ObjectQueue* (except ObjectQueueSearch) is granted to all roles.

13. Manually add custom queues via Queue.ashx

14. Manually update ObjectQueues with custom settings using Queue.ashx

• 1. Note that all customization (eg. New queues or updates to system queues) should be represented in the ConfigurationEntry table

15. Uninstall Queue Service

16. Install qbo.QueueService.2.00.msi

17. Replace QueueService.exe.config with backup version

18. Run Test Scripts from browser

• 1. Application – Queuing

  2. Application – Schedule

  3. Application – Sql Pattern

  4. Any additional test scripts you desire

19. Start Queue Service

20. Manually queue an item for immediate processing 

Upon starting the Queue Service refresh the Queue list from Queue.ashx. If you observe any "Paused" queues this indicates a problem. Inspect (Edit) each queue individually and observe the settings. Common issues to look for are:

• • No schedule specified

  • Queue is set to ExecuteAs "AdminUser" and no Admin Person is specified and/or account is inactive or locked out

  • Queue threshold was set to x and there were x consecutive processing errors