Показать сообщение отдельно
Старый 04.09.2015, 15:28   #14  
Link is offline
Link
Британский учённый
Аватар для Link
Соотечественники
 
568 / 513 (19) +++++++
Регистрация: 25.11.2005
Адрес: UK
Записей в блоге: 9
Цитата:
Сообщение от Morpheus Посмотреть сообщение
Каких инструментов? Почему эти инструменты недоступны партнерам и клиентам?
Кто интересуется темой почитайте Testing best practices (White paper) [AX 2012]:

Цитата:
Test phase best practices
Unit testing
SDEs created new unit tests and modified existing unit tests while developing new features. The source code check-in process required a minimum level for the code coverage of the unit tests that the developer created. For X++ development, these tests were developed by using the SysTest framework in MorphX. For managed code, these tests were developed by using an internal test harness and/or MSTest. Tens of thousands of unit tests were run during Microsoft Dynamics AX 2012
development. The number of lines of code for unit testing was nearly the same as the number of lines of code for the product.
A subset of the unit tests was defined as check-in tests (CITs). These tests were run and required to pass as part of the gated check-in system that all SDEs used for any check-in to the source code repository. This prevented major regressions from being introduced into the code base.
Function testing
The first hands-on use of a feature by the SDETs came as part of the scorecard testing of the testable units. This testing is a lightweight form of acceptance test–driven development (ATDD). SDETs and SDEs work very closely in this phase of the project.
Subprocess, process, integration, and user acceptance testing
In addition to testing that was focused on features, the SDETs broadened their testing efforts to focus on key interfaces with other areas of the system. As described earlier, end-to-end scenarios were a key part of this testing effort.
In addition to the scripted E2E scenarios, the test team coordinated numerous “interactive test sessions” in the later phases of the release. The goal of each session was to bring together SDETs,
PMs, and SDEs to focus on a particular business cycle or a particular area of the product. These sessions were critical to driving completeness into the product.
To automate or not to automate?
Because of the scale of the product and the long-term support requirements for Microsoft Dynamics
AX, the development team made a heavy investment in automated regression tests during the Microsoft Dynamics AX 2012 development cycle. The unit tests were one part of this regression suite.
Additionally, many functional tests were automated. For these tests, the user interface of the product was the primary interaction point. The automated regression tests for features fell into one of three categories:
- Build verification tests (BVTs) – These tests verify the most basic functionality in the product and can be considered “smoke tests.” These tests were run as part of the gated check-in system for every SDE change. Tens of BVT test cases were executed thousands of times during the development cycle.
- Build acceptance tests (BATs) – These tests verify core functionality in each functional area of the product. As core features of the product were created, a new test case was created, or an existing test case was modified. These tests were run together with every nightly build. They also were frequently run before an SDE check-in to verify that no functional breaks resulted from the checkin. Hundreds of BAT test cases were executed for each run.
- Weekly tests – These tests made up the remainder of the automated test suite. As the name suggests, these tests were run one time per week for much of the release. As Microsoft Dynamics AX 2012 approached its release to customers, the tests were run much more frequently. Tens of thousands of weekly test cases were executed for each run. The team has several milestone and release goals for automation – for example:
- A targeted percentage of priority 1 test cases must be automated.
- A targeted percentage of all test cases must be automated.
- A targeted percentage of code coverage must be met for each area of the system.
Another category of regression testing is the verification of bug fixes. The workflow that is associated with a bug can be summarized as follows:
1. The bug can be created by anyone on the team and is mapped to a feature team in the bug tracking tool. The bug is in an Active state.
2. The bug is triaged by a cross-discipline team (PM, SDE, and SDET). The triage result is one of the following:
- Fix
- Don’t Fix – The bug is put into a Resolved state, and a resolution must be specified. The resolution options include By Design, Duplicate, External, Fixed, Not Repro, Postponed, and Won’t Fix.
- Vnext Candidate – The bug is put into a Resolved state, and a resolution must be specified.
The options are the same as the options for Don’t Fix.
3. If the triage result is Fix, it is assigned to an SDE, who makes the changes that are required to address the issue. Upon check-in to source code control, the bug is in a Resolved state, and the resolution is Fixed.
4. Regardless of the triage result, all bugs that are in the Resolved state are assigned to an SDET so that the resolution can be reviewed. If the resolution is Fixed, the SDET tests the fix to verify correctness. The SDET also acts as a customer advocate, and provides a “check and balance” in the system for other resolution types. For example, an SDET may “push back” on a bug that has a resolution of Postponed, by providing more details or a stronger argument for the triage team to consider.
5. If the SDET supports the resolution, the SDET puts the bug into a Closed state. The SDET gets input from the original bug author before closing the bug. As part of the closing process, the SDET reviews the existing test case collateral and decides whether test cases must be updated to ensure that the product does not regression in the future.
6. Any bugs that have a resolution of Postponed are reactivated in the next release.
__________________
Людям физического труда для восстановления своих сил нужен 7-8 часовой ночной сон. Людям умственного труда нужно спать часов 9-10. Ну а программистов будить нельзя вообще.