With Orangebeard’s Test Intelligence Platform, AFAS’s Test & Quality team no longer spends hours every day analyzing test results and selecting tests for the next run. All tests, test sets, and test results are now easily accessible. The outcome of every test run shows what went wrong, why, and who is responsible.This gives AFAS a double time saving every day — both for the Test & Quality team and for the internal audit process — while also significantly speeding up the delivery process, with quality remaining assured.
About AFAS
AFAS Software is a Dutch family-owned company that develops innovative ERP software for the business market. These smart solutions support the digital administrative processes of modern companies, from CRM to finance, and from HR to workflow management.
AFAS inspires better business practices with software and everything needed for that software to achieve a greater goal. With innovation as its core focus, AFAS is always looking for smart solutions to improve performance and free up time for what truly matters ; for both quality, and job satisfaction. That is why Orangebeard’s intelligent SaaS platform is such a great match
The challenge
AFAS works closely with customers to continuously improve its products. New features and functionalities are developed and tested throughout the year. Three times a year, the development team releases new updates for its products. Updates, bug fixes, and new releases are only launched after they have been thoroughly checked, documented, and tested by the Test & Quality team department. The 16 software testers on the team test the software both manually and automatically to ensure that no bugs make it into production.
With so many products being continuously improved, automated testing accounts for a large share of AFAS’s testing work. Before introducing Orangebeard, the Test & Quality team worked with a Jenkins server and SVN. The testing tool Ranorex was used to automate and run test sets overnight. Both Jenkins and Ranorex were used for reporting.
With many automated tests in place, monitoring and reporting on quality had become one of the bottlenecks in AFAS’s continuous delivery pipeline. This was because the quality team had to manually check the results of the test runs — every single day. Every morning, the team went through the same tests, compared static HTML reports, and checked the red and green check marks in Ranorex to understand the outcomes.
A second bottleneck was that, over all those years of test automation, AFAS had accumulated a large number of test scripts. And with every new delivery, the number of tests and test sets grew. It became increasingly difficult and time-consuming for AFAS to select the right test sets and maintain an overview of who was testing what, what was happening, and why. This repetitive checking and selection process was time-consuming, as was collecting the information required for audits.
Even after all that work, the team still did not have clear, usable information or the right reports. That is why AFAS was looking for a system that could help them create a consistent overview of quality and provide clear insight. They also wanted a tool that could save time by reducing the need for recurring manual checks, time-consuming audit data collection, and analysis of test results. Most of the tools they found were too expensive for the added value they could provide. AFAS was also looking for a system that was more than just a dashboard.
Test Intelligence as the solution
Orangebeard is an intelligent automated testing platform. Orangebeard connects directly to all test and build tools and collects all test data and metadata. Orangebeard uses machine learning, RAG technology, and GenAI to analyze the test data, classify results, discover trends and patterns, and give the team all the insights needed for a smooth and comprehensive testing process.
Bugs, errors, and issues are automatically categorized, and their history is stored indefinitely for traceability. Orangebeard’s user-friendly interface ensures that users can find all the answers in just a few clicks.
10x gains for AFAS
- 50% time savings on testing
- Faster delivery through shorter test cycles
- Better insight into existing test sets
- Unlimited test history for traceability
- Flexible tool; adaptable to what AFAS needs
- No more time-consuming repetitive analyses
- Automatic bug classification and test selection
- All test results in one tool
- Complete audit trail, visible in real time
- User-friendly interface
With Orangebeard’s automatic analysis, testers no longer have to spend hours manually going through all the test results from the overnight runs to understand what happened. Testers now see faster than ever where an error or bug is located, or what is causing a failure. Every morning, they immediately see what needs their attention that day. Orangebeard also gives everyone unlimited insight into the test history.
As a result, every user can look back at and compare results and outcomes as far into the past as needed, for even better understanding and full demonstrability and traceability for audits and compliance.
By reducing these repetitive and time-consuming manual analyses, up to 50% of AFAS’s IT team’s time has been freed up. Time they can now spend on more important testing tasks..
“With what is still on Orangebeard’s roadmap, we expect the time and cost savings to continue growing. The next new feature from Orangebeard, Auto Test Pilot, will greatly help us accelerate our time-to-market by automatically selecting the best test sets for each new release.
Although Orangebeard is a standard application, the platform is highly flexible and can be tailored to your needs — for example, in the categorization of bugs. That flexibility is extremely valuable.
That flexibility is also evident in the collaboration with the people at Orangebeard. They are approachable, think along with us, and are eager to work together to improve their product. And if things do not go as smoothly as planned, they communicate quickly and always find a suitable solution.
We are very satisfied with both the product and the company.”