Services: Software Testing and QA
Today's mission-critical applications employ sophisticated programming techniques to produce required results. Technologies are complex and time frames are shorter. Given contemporary market dynamics, poorly tested product releases without proper verification and validation can result in unexpected problems for customers. To minimize the risks of failure, testing, verification and validation have to be in place as early as possible, yet, employing proper testing processes and methodologies.
Auriga's hands-on experience with European and U.S.-based companies provides customers with the unique opportunity to perform comprehensive product testing and QA with low cost and high quality. The effectiveness of the services offered by Auriga has been proven in an array of projects involving large OEMs, such as Dräger Medical. The quality of the provided services is confirmed by several testing projects performed in the vertical domains where the highest level of quality is a must—medical devices and avionics systems. Auriga was able to deliver the testing services as a part of ISO 13485, 21 CFR Part 820, and DO-178B Level A projects.
Software testing as a separate process requires close integration with the core project work starting from the early stages. We achieve this through our emphasis on close relationships with, and attention to, the customers and their needs. Auriga provides expertise to enable clients to concentrate their own time and resources for high-level tasks, as well as lower the operating costs related to acquiring and maintaining necessary infrastructure. Our testing services have proved to completely meet customer requirements by providing flexibility and high level of customization targeted at ensuring quality outcome.
Key Services
Auriga provides a full range of testing services which include all methods and types of testing, verifications, and validation. The three main competence areas are:
The full list of services includes:
- Test Development and Optimization
- Test strategy definition and refinement
- Test case development, refactoring and maintenance
- Requirements coverage and traceability analysis
- Test Execution and Reporting
- Functional and non-functional testing for all phases: unit, regression, integration, system, acceptance, and other types of testing
- Full defect lifecycle support
- Gathering quantitative product quality and testing quality data
- Test Automation
- Developing and maintaining automated test suites and frameworks
- Test Consulting
- Defect root cause analysis and defect prevention measures
- Testing process improvement
- Feasibility study, 3rd party solutions evaluation
These services are provided for a broad set of testing areas for system-level and application-level software products:
- Functional Testing
- Functionality Testing—ensures that the hardware, software, portal, or internal application of the product meet expected functional requirements.
- Regression Testing—re-testing after fixes or modifications of the software or its environment with the focus on automated testing.
- Non-Functional Testing
- Load/Stress Testing—observes the system behavior under different conditions, including heavy load on the product interfaces.
- Performance Testing—verifies current system capacity and identifies system bottlenecks in terms of the load that can be sustained.
- Internationalization Testing—ensures that the hardware, software, portal, or internal applications are ready for global market release.
- Usability Testing—testing for consistency in user interface and ensuring user friendliness.
- Interoperability / Compatibility Testing—cross product / cross-platform testing to ensure that the application behaves as expected in different environments.
- Compliance / Certification Testing—ensures that the product or the product development process complies to a certain standard or methodology.
- Vulnerability / Security Testing—identifying vulnerabilities that may be introduced by patches and updates or by misconfiguration of servers, routers, and firewalls.
- Data Migration / Conversion Testing—ensures that the solution behavior didn’t change for worse after migration to a different underlying platform (hardware, OS, database server, application server, enterprise information system, etc) and that the data was not corrupt in the process of migration.
Automated Testing
Auriga has built a significant expertise in automating the testing tasks, including
- Testing of application-level and system-level products
- Development of automated test suites
- Development of testing frameworks supporting execution of the test suites written in
- Perl
- C/C++
- Tcl, Expect
- UNIX scripts
- Development of special hardware tools to allow automated testing
- Simultaneous control over execution of software on multiple targets
The list of the projects successfully completed in this area among others includes:
- Development of the automated test framework for testing AdvancedTCA Shelf Management and backplane networking products for Pigeon Point Systems
- Development of the automated test framework and automated test suites for the Hot Swap Kit and the Hot Swap Controller Kit products licensed by Motorola, Force Computers, Ziatech/Intel, and Microsoft
- Development of automated test suites for Motorola Computer Group’s/LynuxWorks’ High Availability Package
- Development of automated test suites for the next generation Draeger Medical’s meta-product development platform
- and more…
Embedded Systems Testing
Thanks to its unique experience in developing software for embedded and real-time systems, Auriga has built strong skills in the area of embedded systems testing. As a single example, since 2004 LynuxWorks Inc. has been outsourcing testing tasks to Auriga. Outsourcing this work to Auriga emerged as the best choice for LynuxWorks because of the comprehensive testing facilities existing at Auriga and procximity of these facilities to the development center run by Auriga for LynuxWorks since 1996. The areas of testing performed for LynuxWorks include:
- System and tools testing—to verify the basic system functionalities, POSIX conformance, network functionalities, compilers, debuggers, etc
- Performance testing
- Real-time compliance testing
Based on its 17-year experience, Auriga has developed its own time-proven approach to performing the embedded systems testing. Among other things, the approach introduces the concept of Virtual Laboratory (VL) that
- Provides remote access to and control of the hardware.
- Allows Controlling
- Multiple serial ports
- Multiple relay channels
- Power supply for multiple boards
- Serves as a base for execution of automated test suites
- Provides APIs that allow test suites using VL resources
Auriga has also developed its own hardware solution based on the idea of Virtual Laboratory.
More details can be found in the Testing of Embedded Systems whitepaper.
Load/Stress/Performance Testing for Web/Network Products
Testing of web/enterprise applications is another Auriga strength that, combined with the deep experience in testing system-level network components, provides a complete coverage for the needs in performance, load, and stress testing of network solutions. Two examples from the two main domains of this testing type provide a good idea of Auriga engineering skills and capabilities in this area:
- Load and stress testing (planning, design, development, execution, analysis) and performance optimization for a high performance web service (10,000+ simultaneous users), using Oracle 10g database servers, clustered J2EE application servers running Linux, a set of secured FreeBSD front-end machines and other components.
- Performance testing and optimization for an industrial backplane network emulation product that covered several hardware and software platforms:
- MS Windows, Linux, LynxOS, VxWorks, OSE
- x86, PowerPC
- CompactPCI, PCI, VME
Tools and Technologies
Auriga engineers have deep experience in using the following tools and technologies frequently involved in testing projects
- Test Suites
- Rational Robot/TestManager
- Mercury LoadRunner/WinRunner/TestDirector
- Segue SilkTest
- CppUnit and JUnit
- Configuration Management Tools
- CVS and Subversion
- Rational ClearCase
- Perforce
- ChangeSynergy
- Defect Management Tools
- Rational ClearQuest
- Bugzilla
- PVCS Tracker
- ProblemTracker
- Remedy
- Requirements Management Tools
- Rational RequisitePro
- DOORS
Methodology
Testing methodology used by Auriga is outlined as follows:
The methodology focuses on the following key aspects:
- Performing according to the pre-defined strategy
- Requirements-based test specification
- Traceability of tests to requirements
- Requirements-driven defect classification and release evaluation criteria
- Performance test approach based on workload models
- Regression test strategies
- Templates for test specifications, test logs, test reports, etc
- Defect tracking system to provide immediate feedback during testing
- Quantitative measurements
- Regular reports to the Customer and supervisor, QA audits
Please refer to the Skills and Technologies page for more information on the technical skills possessed by our experts.
