Test Specifications

Background

QBO 3 uses the Jasmine Behavior Driven Development (BDD) framework for unit testing functionality. As project managers develop features, they shall draft a test spec following BDD guidelines.

For example, the notification behavior feature includes the following test spec:
  • notifications can be inserted (insert a message with priority = 1)
  • the notification badge indicates 1 notifications pending
  • clicking on the notification badge shows the inserted notification
  • notifications can be deleted (delete the message)
  • the notification badge indicates 0 notifications pending
Existing test specs can be viewed from the standard QBO 3 main menu's Design > Specifications.

To implement a test spec, the following steps should be taken:
  • In the appropriate web project, ensure a /Scripts/{Namespace}/Specs folder exists
  • Add a javascript file to the /Scripts/{Namespace}/Specs folder named Spec.{Feature}.js (e.g. Spec.Notifications.js)
  • Add the javascript code required to implement the test spec defined in the feature
See existing specs (such as qbo.ContactWeb > Scripts > Specs > Contact > Spec.Contact.js) for examples.

Useful Specs

The following specs can quickly tell you if a QBO system is installed and configured properly:
  • Application > Configuration: tests all classes in web.config for a matching js class and table structure
  • Attachment > File Objects: tests all configured file objects to ensure they are properly configured
  • Security > Login: ensure new user accounts can be created and password resets work
  • Decision > Smart Worklists: ensures smart worklists are properly configured, including use of matricies
Sample Specification

The Security > Login spec performs the following tests:
  • creates a new user
  • issues a password reset
  • allows resetting of a password via a secure link
  • does not allow short passwords
  • does not allow weak passwords
  • does not allow key words
  • does allow strong passwords
  • cleans up the data created
In order to perform these tests, the script will invoke the Login, Person, and SecurityAccess modules. The javascript code to wire that is:

var login = qbo3.getJasmineObject('LoginObject', 'results');
var person = qbo3.getJasmineObject('PersonObject', 'results');
var access = qbo3.getJasmineObject('SecurityAccessObject', 'results');

The qbo3.getJasmineObject is a helper method (found in /Templates/Specs/qbo.SpecHelpers.js). It instantiates a qbo-based javascript class (e.g. new qbo3.PersonObject()), and automatically wires the class to be used in a spec. This includes tying into event handlers for JSON responses and failures.

To test creation of a person:

it('New users can be created', function () {
person.save({
'Person': 'Spec Test User - ' + person.spec.uniqueID,
'Email': testEmail,
'FirstName': person.spec.uniqueID,
'LastName': 'SpecUser',
'Active': 1
});
expect(person.properties.PersonID).toBeDefined();
});

In this case, we simply invoke the Person.ashx/Save to create a new user. Note that the Person.Person includes a unique ID, so we can later search for that unique name if we need to. If we get a PersonID in the response, we consider the test a success.

To test the password reset request functionality:

it('Can request a password reset', function () {
login.resetPassword({ 'Person': 'Spec Test User - ' + person.spec.uniqueID });
expect(login).toHaveElement('!body div.alert-success');
});

In this case, success is defined as receiving an alert-success message in the UI.

The next test checks to see if a one-time access (SecurityAccess) row was properly created for the new user:

it('allows resetting of a password via a secure link', function () {
access.invokeJson('Search', {
'PersonID': person.properties.PersonID
});
expect(access.spec.json.SecurityAccessCollection.SecurityAccessItem[0].SecurityAccessID).toBeDefined();
login.invoke('Impersonate', { 'PersonID': person.properties.PersonID });
expect(login.spec.errors).toEqual(0);
});

Note the use of Impersonate here; this is incredibly useful for testing various security scenarios. Impersonation is logged, and can be disabled site-wide for PROD sites. However, it's very useful for testing.

The next 4 tests simply call Login.ashx/SetPassword, 3 with invalid passwords, and the 4th with a valid password:

it('does not allow short passwords', function () {
login.invokeJson('SetPassword', { 'Password': 'Easy' });
expect(login.spec.json.success).toBeFalsy();
});

it('does not allow weak passwords', function () {
login.invokeJson('SetPassword', { 'Password': 'EasyPeasy' });
expect(login.spec.json.success).toBeFalsy();
});

it('does not allow key words', function () {
login.invokeJson('SetPassword', { 'Password': 'QuandisRocks!' });
expect(login.spec.json.success).toBeFalsy();
});

it('does allow strong passwords', function () {
login.invokeJson('SetPassword', { 'Password': login.spec.uniqueID + 'A!' });
expect(login.spec.json.success).toBeTruthy();
});

Lastly, we clean up the data recently inserted:

it('Can delete users', function () {
login.invoke('UnImpersonate');
expect(login.spec.errors).toEqual(0);
person.invoke('Delete', { 'PersonID': person.properties.PersonID });

});

Helper Methods

When calling qbo3.getJasmineObject, the object returns has a 'spec' object attached:

instance.spec = {
uniqueID: String.uniqueID(), // used to track data so you know where it came from
idList: new Array(), // keep a list if IDs inserted, so you can delete them as part of cleanup
errors: 0, // each time an error is caught, this is incremented
data: null, // stores XML documents received when calling invokeXml
json: null // stores JSON objects received when calling invokeJson
}

The spec.json is probably the most useful feature; it allows you to easily process any JSON responses from the server very quickly.

Additional helper methods in the qbo3.jasmine object include:
  • hasAttribute(field, attribute): returns true if a field exists has an attribute or class (e.g. 'readonly' or 'data-behavior')
  • isReadOnly(field): calls hasAttribute(field, 'readonly')
  • isRequired(field): calls hasAttribute(field, 'required')
  • getJsonItems(json, column, value): parses a JSON array for items with a column (property) matching value
  • getJsonItem(json, column, value): returns the first value found by getJsonItems()

Logging

Spec test results log to the Message table:
  • Spec test results can be identified by a 'SpecLog' value in the MessageType column
  • The BodyText column holds the folder/name of the spec test file that was processed
  • The Object column holds the 'suite' name, or grouping, for the specification
  • The Message column holds the 'it statement' or description of the specification
  • The Status column holds the pass/fail value for the specification
Pattern for querying test spec results:
--pattern: 

--SELECT * FROM Message WHERE MessageType = 'SPECLOG' 
--AND OBJECT = [SUITE NAME]
--AND MESSAGE = [IT STATEMENT]

--examine STATUS column for PASS/FAIL results

Sample queries for test spec results:
--examples of pulling data related to Decision/Spec.Decision.js:

--get all messages relevant to suite 'Workflow (Decision Processing)', most recent first
SELECT * FROM MESSAGE WHERE MESSAGETYPE = 'SPECLOG' AND OBJECT = 'WORKFLOW (DECISION PROCESSING)'order by MessageID desc

--get all messages relevant to SUITE = 'Workflow (Decision Processing)' and It Statement = 'can create an ImportForm step', most recent first
SELECT * FROM MESSAGE WHERE MESSAGETYPE = 'SPECLOG' AND OBJECT = 'WORKFLOW (DECISION PROCESSING)'AND Message = 'CAN CREATE AN IMPORTFORM STEP'order by MessageID desc

Road Map

Future versions of QBO3 will support the following spec enhancements:
  • migration of all specs into IFileObject storage, enabling power users to design specs and distribution across load balanced web servers
  • front-end GUI for editing specs, including use of CodeMirror for javascript coding
  • front-end creation of "spec pacakages": a list of specs that are commonly run together
    • packages can be stored in ConfigurationEntry, and contains ConfigurationXml that simply includes all the test specs to run as part of the package
  • extension of a Jasmine reporter to log spec results to a server-side logging sink (e.g. Message table)

Subpages (49): View All
Comments