With Orangebeard’s Software Quality Intelligence Platform, the AFAS Test & Quality team no longer spends hours each day analyzing test results and selecting tests for the next run. All tests, test sets, and test results are now easily accessible. The outcome of each test run shows what went wrong and why. This saves the AFAS Test & Quality team a great deal of time every day and significantly speeds up the delivery process while ensuring quality remains high.
About AFAS
AFAS Software is a Dutch family-owned company that develops innovative ERP software for the business market. These smart solutions support the digital administrative processes of modern companies, from CRM to finance, and from HR to workflow management.
AFAS inspires better business practices with software and everything needed for that software to achieve a greater goal. With innovation as its core focus, AFAS is always looking for smart solutions to improve performance and free up time for what truly matters ; for both quality, and job satisfaction. That is why Orangebeard’s intelligent SaaS platform is such a great match
The challenge
AFAS works closely with customers to continuously improve its products. Throughout the year, new features and functionalities are developed and tested. Three times a year, the development team releases updates for their products.
Updates, bug fixes, and new releases are only launched after they have been thoroughly reviewed, documented, and tested by the Test & Quality team. The 16 software testers in the team test the software both manually and automatically to ensure that no bugs make it into production.
With so many products being continuously improved, automated testing takes up a large part of the testing work at AFAS. Before the introduction of Orangebeard, the Test & Quality team worked with a Jenkins server and SVN. The Ranorex testing tool was used to automate and run test sets overnight. Both Jenkins and Ranorex were used for reporting.
With a high volume of automated tests, monitoring and reporting on quality was one of the bottlenecks in AFAS’s continuous delivery pipeline. This was because the quality team had to manually check the results of the test runs — every single day. Each morning, the team went through the same tests, compared static HTML reports, and checked the red and green ticks in Ranorex to understand the outcomes.
The second bottleneck was that over years of test automation, AFAS had accumulated a large number of test scripts. With every new release, the number of tests and test sets continued to grow. It became increasingly difficult and time-consuming for AFAS to select the right test sets and maintain an overview of who was testing what, what was happening, and why. This repetitive checking and selecting was time-consuming, and all that effort still did not provide the team with clear, actionable information or the right reports.
That is why AFAS started looking for a tool that could help them produce clearer reports and save time by reducing the need for manual checking and analysis of test results. Most of the tools they found were too expensive for the added value they could provide. AFAS was searching for a tool that was more than just a dashboard.
Orangebeard as the solution
For AFAS, Orangebeard is much more than a dashboard. It is an intelligent automated testing platform. Orangebeard provides AFAS with better insights and a clear overview of the vast number of test sets and test results. This also helps reduce unnecessary repetitive manual test analyses and reporting, saving AFAS time and money while accelerating the development cycle.
Orangebeard connects directly with Ranorex and collects all test data from the tool. It uses machine learning to analyze the test data and classify the results. Bugs, errors, and issues are automatically categorized and displayed in the Orangebeard dashboard. The user-friendly interface ensures that testers can find all the answers they need in just a few clicks.
10x gains for AFAS
- 50% time saved for testers
- Faster delivery through shorter test cycles
- Better insight into existing test sets
- Unlimited test history for deeper understanding
- Flexible tool, adaptable to AFAS’s needs
- No more time-consuming repetitive analyses
- Automatic bug classification and test selection
- All test results in one tool
- All answers in just a few clicks
- User-friendly interface
With Orangebeard’s automatic analysis, testers no longer need to spend hours manually going through all the test results from the nightly runs to understand what happened. Testers can now see faster than ever where an error or bug is located or what is causing a failure. Orangebeard also provides everyone with unlimited access to the test history. Testers can review and compare results as far back as needed for an even deeper understanding.
By reducing these repetitive and time-consuming manual analyses, up to 50% of the testers’ time at AFAS has been freed up, time they can now spend on more important testing tasks.
“With what is still on Orangebeard’s roadmap, we expect the time and cost savings to continue growing. The next new feature from Orangebeard, Auto Test Pilot, will greatly help us accelerate our time-to-market by automatically selecting the best test sets for each new release.
Although Orangebeard is a standard application, the platform is highly flexible and can be tailored to your needs — for example, in the categorization of bugs. That flexibility is extremely valuable.
That flexibility is also evident in the collaboration with the people at Orangebeard. They are approachable, think along with us, and are eager to work together to improve their product. And if things do not go as smoothly as planned, they communicate quickly and always find a suitable solution.
We are very satisfied with both the product and the company.”