Automated Testing

Running C# Unit Tests from Powershell

    There are two options for running C# unit tests from Powershell:
    1. RunMsTest.ps1                                  
      • allows you to run a single test, or a specific DLL of tests.
      • located at qbo.Test.Helpers/Scripts/
      • executed by calling .\RunMsTest.ps1 {physical path of test dll}
    2. qbo3.Core.Targets.proj
      • allows you to run blocks of unit test DLLs, in isolation or as part of a deploy
      • qbo3.Core.Targets.proj will currently run what are consider the 'core tests' for a qbo application
      • To run the tests without deploying, call: 
      • Run tests without deploy by calling:
        • & 'C:\Program Files (x86)\MSBuild\14.0\Bin\amd64\MSBuild' .\qbo3.Core.Targets.proj /t:TestCore /p:"test=true"
      • Run Tests as part of a deploy by calling:
        • & 'C:\Program Files (x86)\MSBuild\14.0\Bin\amd64\MSBuild' .\qbo3.Core.Targets.proj /p:"test=true"
      • You can still run a deploy without tests by leaving out the "test=true" parameter
    All of the above is predicated on the assumption that you are attempting to run tests written with the Microsoft Unit Testing framework.
    • If you are writing XUnit tests, we will need to extend this functionality.
    The results of your tests will be printed in the Powershell console, but you can also review a test report in Visual Studio.
    • Navigate to the qbo.Tests.Helpers/Scripts/TestResults folder, and order by Date Modified descending to find your most recent run.
    • Double clicking on a Test Results File will open the file in Visual Studio.

       Extending a custom MSBuild project to include custom C# tests

        Note that in qbo3.Core.Targets, we run the core tests by iterating over an ItemGroup within a Target:
        

Assuming that your MSBuild project imports qbo3.Targets.Core.proj, you can add custom (Theme specific) tests to your MSBuild project by instantiating this pattern:
<ItemGroup>
  <TestCustom Include="/testcontainer:$(MSBuildThisFileDirectory)..\qbo.Core\Tests\qbo.Application.Tests\bin\Debug\qbo.Application.Tests.dll"/>
</ItemGroup>
<Target Name="TestCustom" BeforeTargets="DeployCore" Condition="$(Test) == true">
  <Exec Command="$(MSTest) %(TestCustom.Identity)"/>
</Target>

In this instance, 'TestCustom' will be renamed to reflect your domain.

qbo.Jasmine.Tests - Running Jasmine specs from Powershell

    Note: This project is still in development, and will change frequently. Please review this blog post if you experience unexpected results.

    qbo.Jasmine.Tests uses a Selenium driver to launch a Chrome browser and run selected Jasmine specs.

    Launching from Powershell

  • If you are performing this operation for the first time, you may need to restore the Nuget packages for qbo.Jasmine.Tests.
  • Navigate to the qbo.Test.Helpers\Scripts folder.
    • On my machine, the path is: C:\trunk\qbo.3\qbo.Core\Tests\qbo.Test.Helpers\Scripts>
  • Run the RunJasmineXunit.ps1 script with the following parameters: website, username, password: 
    • .\RunJasmineXUnit.ps1 sls.quandis.net user@quandis.com "password" "filePathForTestDll"
    • website: this should be a domain name such as devsls.quandis.net, or adr.quandis.net.
    • username and password: this will be your login credentials for the website. Password needs to be in quotes
    • dllPath: the physical path for the DLL that you wish to execute on your machine
      • for example: "C:\trunk\qbo.3\qbo.Core\Tests\qbo.Jasmine.Tests\bin\Debug\qbo.Jasmine.Tests.dll"
    You will see a browser launch, log into the selected qbo website, and navigate to your selected specs.

    When the test runner has completed, you will see a report similar to this:


    There are a few items to note:
    • This test runner only reports on the results of each Jasmine script, not on each individual test.
      • For example, the first failure above notes that Application/Spec.Polyfill.js failed.
      • The next step is to log into your selected domain and run the relevant spec to debug further.
    • Trouble Shooting: This test runner relies upon functionality provided in qbo.SpecHelpers.js to render the test results such that Selenium can collect them.
      • If you see the message 'Spec did not populate JSON result in the Results div', it is likely that your qbo.SpecHelpers.js file is not up to date.
      • Occasionally this script will launch a browser, log into QBO, but will not navigate past the Loan screen.
        • If this occurs, you need to close the browser, close Powershell, and start over.
        • I don't yet have a diagnosis for this errant behavior.

    Selecting Specs

    1. Open qbo.Jasmine.Tests\JasmineTests.cs
    2. Edit the InlineData attributes above the Theory "public void JasmineTest" according to the following pattern:
      • [InlineData("module", "specName")]
        • Example: for the URL {*}/Theme.ashx/Specs?Library=Application\Spec.AbstractObject.js, the InlineData attribute will read:
      • [InlineData("Application", "AbstractObject")]

    Dependencies

    • This process requires that you have the latest versions of qbo.SpecHelpers.js installed on your site.

    Goals

    • This process will shortly be config driven, such that each developer can leave a config file with their preferred specs in the qbo.Jasmine.Tests config folder, rather than having to edit the .cs file.
    • This process is currently driven off of Environment variables which are stored and deleted in the .ps1 file, and referenced in the .cs file. I'm not crazy about this tight coupling, and hope to make this config or parameter driven shortly.
Comments