Time saved, faster deliveries
With the introduction of Orangebeard, the Software Quality Intelligence Platform, the Test & Quality team at AFAS software no longer needs to spend several hours every day analyzing test run data and selecting tests for the next run. With Orangebeard, all tests, test sets and test results are easily accessible. The outcomes of each test run now clearly show where what went wrong and why, saving the Test & Quality Team at AFAS several hours a day and speeding up their delivery pipeline considerably while maintaining quality.
About AFAS
AFAS Software is a Dutch family-owned ERP business software company. AFAS stands for Applications for Administrative Solutions. AFAS’s smart software solutions fully support the digital administrative processes of modern-day companies and businesses, from CRM to finance and from HR to workflow management.The Challenge
AFAS works closely with their customers to continuously improve their products. New features and functionalities are being developed and tested continuously throughout the year. Three times a year, the development team releases new updates for their products.
Updates, bug fixes, and new releases are only launched when they have been checked, documented, and tested thoroughly by the Test & Quality department. The 16 software testers test the software automatically and manually to ensure no bugs are released to production.
“Orangebeard seamlessly connects to our existing tooling and provides us with clear insights and overview of our extensive automated test sets.
The time we used to spent on investigating and understanding why test failed can now be spend on other things. Freeing up our testers, so they can focus more efficiently on test work that cannot be automated, further improving the quality of our software.”
Thymen Simons, Manager Test & Quality, AFAS
With so many products being improved continuously, automated testing has taken up a large chunk of the testing at AFAS. Prior to Orangebeard, the Test & Quality team worked with Jenkins server and SVN. The test tool Ranorex was used to automate and run the test sets during the night. Both Jenkins and Ranorex were used for reporting.
With a lot of tests automated, monitoring and reporting on the quality was one of the bottlenecks in AFAS’s continuous delivery pipeline. It was because the quality team had to check the results of the test runs manually each day. Every morning, they needed to go through the same tests, compare static HTML reports, and check the red and green lights in Ranorex to understand the outcomes.
The second bottleneck was that, after many years of test automation, AFAS accumulated a lot of test scripts. With each new delivery, the number of tests and test sets grew. It became increasingly more difficult and time-consuming for AFAS to select the right test sets and to keep the overview of who tested what, what happened, and why. This repetitive checking and selecting was time-consuming and still didn’t provide the team with clear and actionable information and the right reports.
Therefore, AFAS was looking for a tool that could help them with clearer reports and save time by reducing the need for manual checking and analysis of the test outcomes. However, most tools they found were too expensive for the limited added value that these tools could provide. AFAS was looking for a tool that was more than a dashboard.
Orangebeard as The Solution
For AFAS, Orangebeard is much more than just a dashboard. It’s an intelligent automated test platform. Orangebeard provides AFAS with better insights and a clear overview into the enormous amount of test sets and test results. It also helps reduce unnecessary repetitive manual test analysis and reporting, saving AFAS time, money, and speeding up their development cycle.
Orangebeard directly connects with Ranorex and collects all test run data from the tool. Orangebeard uses machine learning to analyze the test data and to classify the results. Bugs, failures, and errors are automatically categorized on product level and displayed in Orangebeard’s dashboard. The user-friendly interface of Orangebeard ensures that the testers can find all the answers in just a few simple clicks.
Top 10 benefits & gains
- 50% of testers time saved
- Faster deliveries through shorter test cycles
- Better insights into existing test sets
- Unlimited test history for better understanding
- Flexible tool; adaptable to fit AFAS’s needs
- No more time consuming repetitive analyses
- Auto bug classification and test selection
- All test results in one tool
- All answers in a couple of clicks
- User-friendly interface
Orangebeard’s auto analysis means that the testers no longer need to spend hours a day having to manually go through all the test results of the nightly runs to understand what has happened. Thanks to Orangebeard, the testers can now quickly determine where an error or bug is located or what has caused a failure. It also provides them with an unlimited test history, allowing the testers to view and compare results and outcomes as far back in time as they want for an even better understanding.
Reducing this repetitive and time-consuming manual labor of analyzing results has freed up to 50% of AFAS’s testers’ time. Now they can spend the time saved on analysis on other important test tasks at hand. “Based on what is still on Orangebeard’s roadmap, we are expecting the amount of time and costs we are saving to continue to grow.
Orangebeard’s next new feature, Auto Test Pilot, will greatly help us to further accelerate our time-to-market by autoselecting the best test sets for every new release.
Even though Orangebeard is a standard application, the platform is still very flexible and can be adapted to fit your needs. For instance, in the categorization of bugs. That flexibility is very valuable. That flexibility is also present when we work with the people from Orangebeard.
They are easy to approach, think with us, and like to work together with their customers to further improve their product. And if things don’t go as smoothly as planned, they are quick to communicate and find a suitable solution for everyone.
We are really satisfied with both the product and the company,” says Thymen Simons, Manager Test & Quality at AFAS Software.
