NAV Team: Application Test Toolset for Microsoft Dynamics NAV 2013

[Blog bot]

We recently shipped the Application Test Toolset for Microsoft Dynamics NAV 2013.

The supplement is applicable to the following country releases of Microsoft Dynamics NAV 2013:

W1, AU, CA, DE, DK, ES, FR, GB, IN, IT, MX, NL, NZ, SE and US

The supplement contains the following per country:

• Tools for managing and executing tests, capturing code coverage information, and selecting relevant tests out of available tests.
• Between 7,000 – 9,000 real regression tests that we run, not dumbed-down sample tests.
• Helper libraries for improving test development through reusing common functionality.
• Searchable documentation of helper libraries with examples of how to use library functionality.

You may attempt to apply the W1 version to other country versions, but you should expect parts of the toolset to not work.

Installation

To install the supplement, do the following:

1. Download the supplement from PartnerSource here.
2. Extract the content.
3. Import the FOB file found under your corresponding country version folder. For example: .AppTestToolsetNAV2013DEAppTestToolsetNAV2013-DE.fob
4. After experimenting with the toolset, don’t forget to fill out the survey form and send it to Microsoft. We really appreciate your feedback and we rely on it to improve future versions of the toolset.

How Do I Use This?

The simplest way to make use of this supplement is to run the Test Tool page (130021) directly from the development environment. This launches the following page:



Click Get Test Codeunits and then select All Test Codeunits.



After Microsoft Dynamics NAV finishes loading all test codeunits, they are displayed on the Test Tool page. To run them, click the Run action and then select All.



It will take about 1 – 2 hours depending on your machine and the setup to run all tests. When the run is completed it will show the results in the Test Tool page:



Any changes done to the database through running of tests from the Test Tool are automatically rolled back using the Test Isolation testability feature of Microsoft Dynamics NAV 2013. (See the Additional Resources section in this post.)

During typical development, it is unacceptable to have to wait hours to get results from tests, which is why we have built an advanced test selection feature to help identify the relevant tests. (See the Test Selection section in this post.)

Alternatively, you can run individual tests or codeunits by selecting them and choosing either Active Line or Active Codeunit after you click the Run action.

If any test fails, you can attach a debugger session and re-run the failing test. The debugger will then break at the line where the test failed and you will be able to inspect the call stack and examine variables to determine the underlying cause of the failure.

Extending the Toolset With Your Own Tests

After you have written your first test codeunit, you can easily integrate it into the tools we provide in this supplement.

To include your own tests, in the Test Tool page, simply run the page from the development environment and click the action Get Test Codeunits and choose Select Test Codeunits. This will display a page listing all available test codeunits, including your own:



Select the codeunits you would like to add to the tool and press OK. The new test codeunits appear at the bottom of the Test Tool list, and you can now select them and run them just like any of the tests we included.

Again, Test Isolation prevents your tests from persisting changes to the database. During development it may be beneficial to actually see the output produced by the tests. It is possible to disable Test Isolation just by running the test codeunit directly from the development environment, however, instead we recommend attaching a debugger session, breaking at the test entry point, then stepping through test execution and inspecting variables to determine if your test is behaving as expected.

Speeding Up Development of Your Own Tests

The tests that we have developed are built on top of a layer of libraries that contain helper functionality to automate many aspects of Microsoft Dynamics NAV. For example, the library named Library – Sales contains functionality related to working with customers and sales documents, including creating new customers, sales headers, sales lines and posting sales documents. The library is extensive and has functionality in many areas of the product, such as finance, service, jobs, warehousing, inventory, etc.

Instead of re-inventing the wheel when developing your own tests, we highly suggest that you look into our existing helper functionality for functions you can leverage.

To help you find your way around the libraries, we have shipped a Microsoft Compiled HTML Help file (*.chm), which is bundled together with the .fob file you installed. When you open the .chm file, you are prompted with the following window:



This lists all our libraries and the functions inside them. However, normally you don’t know which library to look for, You can search it from the Search tab. Try searching for "finance charge memo" and you will have a couple of choices to pick from:



Code Coverage Tools

Code coverage is the means of being able to track which part of the application code has been exercised during some activity. In Microsoft Dynamics NAV, code coverage is recorded by AL code line and in addition to knowing if a code line was exercised it also records the number of times it was recorded.

The code coverage activity that we record can be any interaction with Microsoft Dynamics NAV, be it manual user interaction, automated test execution, NAS, Web services, etc. You can, of course, record code coverage of your own tests exercising your own objects.

The toolset includes a page (130002), Code Coverage List, which you can use to track code coverage. Run the page from the development environment:



From this page you can start/refresh/stop the code coverage recorder. If you click the Start action, the code coverage engine is turned on and code coverage is captured. However, you will not be able to see any updated information before you click either Refresh or Stop, at which time you are presented with the code coverage information:



The information contains coverage of objects, triggers/functions and individual lines (code and empty) as determined by the column Line Type. Only lines of type Code can have coverage. Lines of type Trigger/Function show the average coverage of all code lines in the trigger/function. Lines of type Object show the average coverage of all code lines inside the object.

From the above picture, you can read that the activity exercised 33.93% of the Currency table (4). It covered 100% of the OnModify trigger and that comes from 100% of a single Code line.

It is often desirable to filter on Line Type = Object to first get a high-level overview of the coverage result:



Then from here, you can filter to look at individual objects and expand the Line Type filter to include triggers/functions as well:



This way you can drill-down into the results starting from a high-level view going to a low-level view.

Note #1: Code coverage is recorded globally for all sessions when using this tool, so make sure you are running in a controlled environment so you don’t have any activity from unaccounted sessions.

Note #2: Only objects that are touched by the activity are recorded, meaning the coverage of any object not in the list is implied to be zero. If you would like to force the recorder to include specific objects even if they are not covered, you can use
the Load objects action and select the relevant objects from the subsequent page. This forces the code coverage engine to load these objects and provide information on even when no lines are covered.

Test Selection

Now that we have all the building blocks in place, I’d like to talk about an advanced feature we included with the tooling.

As mentioned previously, having to wait hours to run all tests is not feasible from a development point of view. Therefore we shipped the Test Selection, which helps you narrow the set of tests down to the relevant tests.

The feature works by analyzing the code coverage data from individual test codeunits and comparing it to the set of objects that have the Modified field set to Yes in the database.

To use this feature, you run the Test Tool page and go to the Actions tab and click Import/Export Test Map action. On the request page, make sure the direction is Import and click OK. Browse to the bundled "AppTestToolsetNAV2013-<country
code>-Map.txt" file and import the file. This will take a couple of seconds. After it is done, click the Get Test Codeunits action. The prompt will now include a third option:



Select this third option and the tool will automatically detect the relevant tests to run and add them to your current suite.

Note #1: In typical circumstances you would want to make sure your suite is empty before using this feature.

Note #2: There is a small risk this feature will not identify all relevant tests in unusual circumstances. Thus we strongly recommend running the full regression suite before shipping anything to the customer.

This feature also integrates with you own tests. Once enabled (by loading the Test Map), the information will auto-update when any test is run &ndash; including your own tests. This means that you can load the map, run your tests and now export the map to another text file. You can then load the new map into another database and the test selection feature will now be able to suggest your own tests based on modified objects in this other database. If your test codeunit is not present in the database, you will be prompted with a list of missing test codeunits that could not be added. Import the missing test codeunits into the database and re-run the test selection feature.

Additional Resources

• Application Test Toolset for Microsoft Dynamics NAV 2013
• Application Test Toolset for Microsoft Dynamics NAV 2009 SP1
• Testability
• Page Testability
• Test Isolation

-Simon Ejsing



Источник: http://feedproxy.google.com/~r/Micro...-nav-2013.aspx