Queue Service has been updated to version 2.6 which includes: Queue item retry enhancements, Queue pause / auto restart enhancements, Queue Herding, Logic Monitor Notification strategies and minor optimizations/bug fixes.
Release Notes
Queue item retry behavior allows for additional configuration and now supports:
Retry Models - The retry model determines the calculation model of retry interval. Options are:
Straight - No exponential back-off. Uses same retry interval every time an item is retried.
Exponential Models - Note with both models, maximum wait and maximum wait interval must be supplied.
Slow - Item retry interval will slowly increase starting with retry wait (initial retry amount) until the Retry Max Wait reached.
Fast - Item retry interval will quickly increase starting with retry wait (initial retry amount) until the Retry Max Wait reached.
Item retry count now allows for infinite retry along with specified tries
Queues Automatic Retry - Queues can now automatically restart based on settings. Pause/Restart settings allow for:
Pause a queue after X number of failures
Restart X times allowing infinite restarts if desired
Restart after X minutes|hours|days
Queues can now support the concept of a "Failure Strategy". The purpose of Failure Strategy is to choose a "preset" of commonly used retry strategies. Currently, there are 7 strategies:
Queue Herds - Queues can be specified to process under a given "Herd". The concept of herd implies that a single or group of machines will process specified queues. Application servers running queue service can be configured to process specific queues thus filtering what they process. This is useful for auto-scale strategies with volume spikes.
Each queue can be set to process by a specific Herd. The default Herd is Default
If a Queue Herd is set to a custom Herd, there must be an application server configured to process that Herd.
See below for more examples.
Windows Application Logging - Queue Pause and Restart Events now log to Event Viewer which are monitored by Logic Monitor and can notify via an escalation list. Queue Events have custom EventID mappings in the Windows Application Log:
All Queue Pause and Resume events are logged to Windows Application log. Queue Pause events will log with a severity of Information when a queue is paused and a restart is eligible. If a queue is paused and cannot restart, the severity is Error which allows for further escalation within Logic Monitor.
UI Updates
Queue Status - The Queue Summary page will now indicate if a queue is paused based on Queue color:
White - Queue is not paused and is in good health
Yellow - Indicates Queue is paused but will restart based on calculated restart date
Red - Queue is paused and will not restart
Queue Status now supports hover and will display queue status, pause count and a restart date.
Queue Logic - Item and Queue retry settings will now display when hovered over under Queue Logic. A search icon will also display the retry schedule calculated based on queue settings. This allows users to view and adjust settings to achieve a desired retry schedule
Use Cases
Failure Strategy (Includes Item restart and Queue resume)
Example 1 - Power user wants to adjust a given queue such that it's retry will:
Retry 10 times, retrying immediately and eventually waiting 5 minutes. They also want this queue to pause but restart infinitely waiting 30 minutes between restarts.
Suggestion: Advanced, applying above settings
Example 2 - Power user manages 3 queues (Default, MSP, FTP Upload) and wants the queue service to retry with different strategies which fit best. For their queues, we would suggest
Example 3 - Power user manages 3 queues (Default, Letter Generation, Credco) and wants the queue service to retry with different strategies which fit best. For their queues, we would suggest
Queue Herds
Use Case 1 - Simple Install. QBO install has 2 application servers for redundancy and no queues that volume spike beyond reasonable levels.
Recommended Herding: Default
Settings:
Application Servers 1 & 2 - QueueService Herd: Default
Queue settings: All queues set to Default
Reason: Both application servers can handle volume and do not need to be split out.
Use Case 2 - QBO install has 2 application servers for redundancy and a letter generation queue that periodically encounters volume spikes that must be met by a certain SLA.
Recommended Herding:
APP 1 & APP 2: Herd = Default, LetterGeneration
Auto Scale Servers: Herd = LetterGeneration
Settings:
Application Servers 1 & 2 - QueueService Herd: Default, LetterGeneration
Queue settings: All queues set to Default except for Letter Generation queue which is set to LetterGeneration
Reason: APP 1 & APP 2 can process all queues. When volume spikes occur, auto scale specific Letter Generation app servers. Note that LetterGeneration is a separate Herd so it must be specified in the Herd list if a given application server is going to monitor multiple herds. The auto scale machines will specifically process LetterGeneration and