When developing a data-entry intensive piece of software,
one inevitably has to deal with validation; ensuring that the data entered by
the user is of the proper type and conforms to any required business rules. The following is an explanation of how
validation works in qbo3, the challenges we faced, and how we resolved
them.
Hard and soft validators
Validation comes in two flavors, what we are calling “hard”
and “soft” validators, and is implemented by including the proper CSS class on
an HTML input/select element. A “hard”
validator inspects the quality of the data, for example ensuring that a numeric
input contains only a numeric value; e.g. an entry of “ABC” would be invalid
for a money-type field. At present, the
qbo3 “hard” validators are as follows: qbo-money,
qbo-int, qbo-numeric, qbo-percent, qbo-phone, qbo-ssn, qbo-zip, qbo-date and
qbo-investor-loan. The
implementation for these can be found in Scripts/qbo.Validation.js in the
qbo.ApplicationWeb project.
A “hard” validator is to be evaluated in all cases; we would
never want to allow an alpha character in a date or money field; doing so would
result in an exception being raised when the database tried to update the row
with that value.
On the other hand, a “soft” validator is a validation that we
want to bypass in certain situations. A
“soft” validator does not evaluate the quality of the data in the same way as
does a “hard” validator. “Soft”
validators are more concerned with ensuring the data conforms to a business
rule, leaving the validation of data quality to the “hard” validators. The current “soft” validators are as follows:
required, equality-validator,
range-validator-quantity, range-validator-percent, range-validator-lotsize,
lessthan-validator, greaterthan-validator.
The implementation for these can be found in Scripts/qbo.Validation.js
in the qbo.ApplicationWeb project.
IMPORTANT: For a validator to be considered a
“soft” validator, it must be included in the “softValidators” array within the
“Validator” behavior defined in qbo.Validation.js.
It is common for an input to have both “hard” and “soft”
validators. For instance, an input can
be a date field, requiring a “hard” date validation, and it can be required,
which is a “soft” validation. In such a
case, the “hard” validator is evaluated first, and the result of the “hard”
validation trumps the “soft” validator.
So if an input is both a date field (hard validation) and required (soft
validation), the data for the input cannot be saved if the “hard” validation
fails. In other words, while entering “abc” in a date field satisfies the
“required” validator, it fails the “hard” validator which says the value must be
a valid date.
Not all soft validators are created equal
This is a bit of an aside, but worth mentioning. The “soft” validator “required” is different
than the other “soft” validators. The
“required” validator checks only to see if an input has a value, it is not
concerned with what the value is. Any
value will satisfy the “required” validator. The remaining “soft” validators do
care about the value, as they are generally validating a business rule. For example, the “equality-validator” checks
to see if two values match. An input can
have both a “required” validator and any of the other “soft” validators,
meaning not only is it a value required, but it must also comply with a
business rule.
Also, as of this writing all “soft” validators with the
exception of “required” are used only
with justifications, which are explained below.
This is not a requirement nor by design; the nature of justifications
necessitated the use “soft” validators.
Why soft validators are necessary
Once a broker has completed the data entry for a given
section of the BPO, they click a “Submit” button, which completes the
underlying decision step, effectively marking that section as complete. This is
necessary because all of the sections must be marked as complete before the
broker can submit the form for review.
When “Submit” is clicked, all validators are evaluated,
“hard” and “soft” alike; if any fail, the submit function is cancelled and the
user must fix the fields that failed validation.
The amount of data required to complete a BPO is
substantial. It is common for a broker
begin to enter the data without having all of the required data. In this situation, the broker will want to save
what they have entered at that point and come back later to finish once they
have the remainder of the data. If the
“required” validator were treated as a “hard” validator, the broker would have
to complete all of the data entry for a given section in one sitting, which is
not always realistic.
For this reason, we introduced the concept of “save and
pause”, which allows the user to save the data they have entered, even though
the data may not be complete nor fully comply with stated business rules. “Save
and pause” does not attempt to complete the underlying decision step. To facilitate “save and pause”, we created
the “soft” validators.
Challenges
The challenges of validation were twofold.
1.
Evaluate all “hard” validators in all situations
to prevent the submission of data that would result in an exception being
raised.
2.
Simultaneously allow the saving of data from
inputs with “soft” validators, even if those validations have not been met
(save and pause scenario).
How validation works
“Hard” validators are fairly straightforward. When an input element loses focus, the
validators are invoked. A date input
must contain a valid date; a money field must contain a valid money value, and
so on. If validation fails, the
“validation-failed” CSS class is added to the input. Attempting to save the data will fail until
the hard errors are fixed.
If a broker wishes to complete a section of the BPO form,
they will click the “Submit” button.
Doing so invokes evaluation of both “hard” and “soft” validators.
Assuming all “hard” validators for an input have been satisfied, the “soft”
validators will then be evaluated. If
all “soft” validators pass, then it is determined that the input passed all
validation. If all inputs pass all
validations, the data will be submitted for persistence in the database.
If a broker does not have all of the necessary data but
wishes to save what they have, they will click the “Save” button, which is the
“save and pause” scenario. When they
click “Save”, only the “hard” validators are evaluated. If all of the “hard” validators pass, the
data will be submitted to the server to be persisted in the database; the
“soft” validators are ignored.
Here is the code from qbo.Validation.js that shows how
“soft” validators are ignored in the “save-and-pause” scenario:
validateNonRequired: function
() {
var softValidatorList = {};
this.options.softValidators.each(function(validator){
softValidatorList[validator]
= this.fields.filter(function
(f) { return f.hasClass(validator); }); //Create a list of inputs with soft validators
softValidatorList[validator].removeClass(validator);
}.bind(this)); // Remove the soft validators
result = this.validate(); //Evaluate remaining hard validators
this.options.softValidators.each(function(validator){
softValidatorList[validator].addClass(validator);
//Add soft validators back
to the inputs
});
return result;
}
The “validateNonRequired” method is executed when the user
clicks “Save”, which is “save-and-pause”.
It iterates through the array of “soft” validators, creating a list of
all inputs that have soft validators, and then removes the “soft” validator
classes from those inputs. The “hard”
validators like qbo-int, qbo-money, etc., are left in place. Once the “soft” validators have been removed,
validation is performed on all inputs that have “hard” validators. After validation is complete, the “soft”
validators are added back to the controls that need them, by iterating through
the list of controls with “soft” validators.
The result of the validation is then returned by the method. Keep in mind that the final result is derived
from evaluating only the “hard” validators.
Justifications
Some data points for a BPO may have business rules attached
to them. For example, the square footage for a comparable property must be
within some predefined percentage of the square footage for the subject
property. There are occasions when an
agent is unable to find a comparable property that conforms to a defined
business rule. When that happens the
agent must provide an explanation as to why they chose a property that fails
the rule. This explanation is called a
“justification”.
Justifications are nothing more than optionally required
fields. However, to satisfy a client
request, it was necessary to allow the user to be able to save the form (“save-and-pause”)
without having entered justifications.
It was this requirement that lead to the creation of “soft” validators,
but there is some debate as to the validity of this approach and it may change
so that justifications are not ignored in the “save-and-pause” scenario.
Conclusion
·
“Hard” validators such as qbo-int will be
evaluated in all cases. If a “hard”
validation fails, no data will be submitted to the database
·
“Soft” validators will be ignored in the
“save-and-pause” scenario.
·
Both “hard” and “soft” validators are evaluated
when data is submitted, as opposed to save-and-pause.
·
Justifications are just optionally required
fields.
·
The ability to perform a “save-and-pause”
without having entered justifications lead to the creation of “soft” validators.