Software Testing Statistics (2026): ECC to S/4HANA Migration, Quality and AI Benchmarks

Testing numbers for SAP programs sit across user-group surveys, analyst releases, consultancy studies and vendor reports, and those sources rarely measure the same thing. This roundup keeps only the figures we could trace to a named publisher, dates each one, and flags where sources disagree.

Key takeaways

How far along is the S/4HANA move?

User-group data shows the migration accelerating, but each survey asks a slightly different question, so the headline numbers should not be stacked on one another.

• ASUG and Precisely's 2026 trends report, announced in November 2025, finds 59% of companies fully or partly live on S/4HANA, up 13 percentage points from 2024. The sample size was not stated in the coverage we reviewed.
• Automation adoption has plateaued at 57% as migration demands pull resources away from automation work.
• ASUG data from 2025 shows 56% of customers either live on S/4HANA or have begun their migration, up from 45% in 2024.
• The share planning to wait more than two years fell from 22% in 2023 to 9% in 2025.
• In SAPinsider's 2025 research, 32% of respondents said their organization had already transitioned to S/4HANA. Another 27% were in the implementation stage, and 21% were still in the evaluation phase.

Source: Precisely and ASUG, 2025; SAPinsider, 2025

Source: CIO, 2025 (citing ASUG member data)

For the deadline context behind these numbers, see our explainer on ECC end of maintenance, and for a wider view of the migration by the numbers, read The S/4HANA migration by the numbers.

Timelines and budgets

Overrun data is the most consistent theme across sources, though the sample and definitions vary.

• Horváth surveyed 200 top executives from six countries about their S/4HANA transformations.
• Projects took on average 30 percent longer than originally planned. Fewer than one in ten companies that had already completed the transformation stayed on schedule.
• The respondents' main reasons for deviations were expanding scope during the project, weak project management, underestimated testing and data migration phases, revision loops, and lack of decision-making.
• Two-thirds of respondents were dissatisfied with the quality of the result.
• In ASUG's 2024 member research, 49% of respondents already live on S/4HANA said costs exceeded their original budgets, up 17% from 2023. That survey drew on 208 members of the ASUG community. Treat it as historical.

Source: heise, 2025, reporting the Horváth study, April 2025

The broader ERP picture comes from Panorama Consulting. Its 2026 report is not SAP-specific, but the cost patterns are useful context for a program business case. The 2026 ERP Report covered 170 companies in a worldwide survey from January 2025 to January 2026, with a median project timeline of nine months. More than a quarter of organizations exceeded their project budgets, with additional technology needs cited as the leading cause.

Source: ERP Statistics 2026, summarizing Panorama Consulting, 2026 (secondary summary; the primary report is gated)

Historical project-outcome data

Older benchmarks still get quoted in SAP business cases. They are useful for framing but weak for benchmarking.

• In the original 1994 Standish CHAOS report, only 16.2% of software projects were successful, 52.7% were challenged, and 31.1% were impaired and cancelled. Historical.
• Secondary reporting of the 2020 CHAOS edition gives 31% successful, 50% challenged and 19% failed. The full report is paywalled, so the figures come from open secondary sources.
• An academic critique argues that Standish's successful and challenged project results are meaningless for benchmarking. Read the Standish series as directional only.
• The Consortium for Information and Software Quality put the cost of poor software quality in the US at $2.41 trillion in its 2022 report.
• An earlier NIST-commissioned study, quoted in a USPTO patent filing, estimated software bugs cost the US economy about $59 billion annually in 2002. More than a third of those costs, an estimated $22.2 billion, could have been eliminated by an improved testing infrastructure. This is historical data, and the NIST figure is read through a patent document, not the original report.

Testing and quality

The 2025 World Quality Report, the most cited global quality engineering survey, shows that automation has not kept pace with adoption intent.

• Nearly half of organizations were still in the planning stage for automation, and coverage averaged only a third of test cases.
• Although 94% of organizations review production data, nearly half struggle to turn those insights into testing strategy.
• Synthetic data use in testing rose from 14% in 2024 to an average of 25% in 2025. See our guide to test data for S/4HANA for how to handle copies, masking and synthetic data.
• The same report's public summary gives two ERP-specific figures that look contradictory. Almost all (95%) organizations now use GenAI to generate test data, but only 10% have fully embedded it into their development lifecycles. Only 6% of enterprises use GenAI in ERP testing. The summary does not define the base for each figure, so do not combine them.
• On testing centers of excellence, a reviewer of the 2024–25 edition reported that only 27% of respondents still used a traditional testing CoE, down from 70% the year before. The reviewer suggested the drop may reflect conflicting interpretations of what counts as a TCoE. For a practical model, see our SAP Testing CoE operating model.

We could not find a sourced figure for the share of total ECC-to-S/4HANA effort that goes to testing. Until a credible source publishes one, plan from the Horváth finding that testing and data migration were among the underestimated phases.

AI in testing

AI adoption is high in the surveys, but enterprise-scale use is much lower. Vendor numbers run noticeably higher than independent ones.

• The 2025 World Quality Report found 89% of responding organizations piloting or deploying GenAI-augmented workflows, with 37% in production and 52% in pilot phases.
• Only 15% had achieved enterprise-wide implementation, while 43% remained in the experimental phase and 30% operated within limited use cases.
• The share of non-adopters rose to 11% in 2025, up from 4% in 2024, though still well below 31% in 2023.
• The report puts the average productivity boost at 19%, but one third of respondents saw minimal gains.
• Top barriers in 2025 were integration complexity (64%), data privacy risks (67%), and hallucination and reliability concerns (60%).
• DORA's 2025 research, which surveyed nearly 5,000 technology professionals, found that 90% of respondents report using AI at work, and more than 80% believe it has increased their productivity. The same research found that AI adoption still has a negative relationship with software delivery stability. It also points to strong automated testing, mature version control and fast feedback loops as the controls that prevent that instability.
• Vendor-run surveys report higher figures. BrowserStack's 2026 State of AI in Software Testing report found 94% of teams use AI in testing, and 61% use it across most testing workflows. The same vendor survey found that 12% had reached full autonomy. BrowserStack's own write-up notes that the data is self-reported and not an audit of test environments.

Source: World Quality Report, Capgemini and Sogeti, 2023–2025

Source: World Quality Report 2025, Capgemini and Sogeti

Source: Capgemini, 2025; Capgemini via PR Newswire, 2025

Source: BrowserStack and PractiTest 2026 reports, as summarized by Testdino, 2026; BrowserStack, as summarized by Quash, 2026; Katalon 2025, as summarized by TestMax, 2026

Source: Google Cloud DORA, 2025; Splunk review of DORA 2025; Stack Overflow Developer Survey 2025, as summarized by Shipper, 2026

Forecasts are also inconsistent. One 2026 compilation attributes to Gartner a projection that 80% of software engineering organizations will use AI-augmented testing tools by 2027, up from fewer than 20% in 2023. Another compilation attributes to Gartner a projection that 70% of enterprises will integrate AI-augmented testing tools into toolchains by 2028, up from about 20% in early 2025. We could not retrieve the original Gartner releases, so treat both as secondary.

For SAP-specific AI use cases, see SAP AI for practitioners.

People and skills

• Half of respondents report that their organizations lack AI and ML expertise, a figure unchanged from 2024.
• Generative AI ranked as the top skill for quality engineers at 63%, followed closely by core quality engineering skills at 60%.
• Soft skills, both verbal and written, ranked fifth in importance, with 51% of respondents emphasizing them.

For how to staff and fund this work, read our SAP Testing CoE operating model and playbook for building a Testing CoE before your S/4HANA move.

Where the numbers conflict

• S/4HANA adoption. ASUG and Precisely's 59% counts companies fully or partly live. SAPinsider's 32% counts organizations that have transitioned. These are different questions and samples.
• On-schedule completion. Horváth's own wording is fewer than one in ten. Some secondary sources repeat 8%, and heise reports 10%. We use Horváth's wording.
• Gartner AI-testing forecasts. The 80%-by-2027 and 70%-by-2028 projections above cannot both be read at face value without the original Gartner documents.
• AI testing adoption. Published figures range from 76% to 94% depending on whether a respondent counts any use or use across most workflows. Vendor surveys sit at the high end.
• Standish success rates. The series reports one-third success across decades, but methodology changes and a critique in the literature mean year-to-year comparisons are weak.
• ERP failure rates. A widely repeated claim that 73% of ERP projects fail is attributed to Panorama, but one analysis reports that Panorama's 2026 report says something narrower. We do not use the 73% figure.

What this means for your sign-off

• Put testing and data migration on the steering committee dashboard. Horváth's respondents named underestimated testing and data migration among the causes of their overruns, so your plan should carry explicit exit criteria for those phases. Use our phase-by-phase ECC to S/4HANA test plan as the starting point.
• Do not plan on the automation curve moving by itself. With automation adoption plateaued at 57% in the ASUG and Precisely data, expect migration work to crowd out automation build time. Prioritize with the S/4HANA regression automation shortlist.
• Budget for test data as its own workstream. Synthetic data supplies a growing share of test data in the World Quality Report, but tooling and ownership are fragmented. Assign a named owner before the first mock cutover.
• Treat AI tool adoption as a controlled change. DORA links AI adoption to lower delivery stability when strong automated testing is missing. Gate AI-generated test assets through review before they enter the regression suite. The template library includes an AI-generated test review checklist and an agent guardrails register.
• Use the number that matches your question. If the question is how many companies are live, cite ASUG and Precisely. If the question is how much of your plan is at risk, cite Horváth. Do not mix survey populations in a single slide.

Frequently asked questions

What share of SAP customers are live on S/4HANA?

It depends on the survey. ASUG and Precisely report 59% of companies fully or partly live on S/4HANA. SAPinsider's 2025 research found 32% had transitioned. The two measure different things.

How often do S/4HANA projects run late?

In Horváth's 2025 study of 200 companies, projects took on average 30 percent longer than originally planned. Fewer than one in ten companies that had completed the transformation stayed on schedule.

What did respondents say caused the overruns?

The main reasons cited were scope expansion during the project, weak project management, underestimated testing and data migration phases, and lack of decision-making.

How many testing teams use AI, and how many have scaled it?

Independent and vendor figures run from 76% to 94% for any use. Only 15% of World Quality Report 2025 respondents had achieved enterprise-wide implementation. Vendor survey data shows 12% had reached full autonomy.

Does AI improve delivery stability?

Not on its own. DORA's 2025 research found AI adoption still has a negative relationship with delivery stability, and it tied the risk to weak controls such as automated testing.

What is the most cited barrier to scaling AI in quality engineering?

In 2025, data privacy risks (67%) topped the list, followed by integration complexity (64%) and hallucination and reliability concerns (60%).

Sources

1. Study shows: SAP S/4HANA transformations rarely go as planned, Horváth, April 2025
2. SAP S4/HANA: Changeover takes on average 30 percent longer than planned, heise, 2025
3. New Research Reveals SAP S/4HANA Migration Momentum, Precisely and ASUG, November 2025
4. SAP S/4HANA migration accelerates despite automation challenges, Process Excellence Network, 2025
5. New Research Reveals SAP S/4HANA Migration Momentum (release text), Precisely, 2025
6. SAP customers say migration is eating their budgets, and AI is next in line, CIO, 2025
7. The State of SAP S/4HANA Adoption: Trends, Successes, and Challenges, ASUG, 2024
8. SAP S/4HANA Migration 2025, SAPinsider, 2025
9. Panorama Consulting Group Releases Latest Study of ERP Implementation Outcomes, Panorama Consulting Group, March 2026
10. ERP Statistics 2026: Market Size, Costs, Adoption, summarizing Panorama Consulting Group, 2026 (secondary)
11. The ERP "Failure Rate" Myth vs. Panorama's Real Data, NawaOne, 2026 (secondary)
12. Standish CHAOS Report by Year: Every Edition 1994 to 2020, BudgetOverrun, 2026 (secondary; original reports 1994 and 2020)
13. Standish CHAOS Report: 69% IT Project Failure Rate Explained, BudgetOverrun, 2026 (secondary)
14. The rise and fall of the Chaos report figures, academic paper, date not captured in our search
15. CISQ Publishes the Cost of Poor Software Quality in the US: A 2022 Report, Consortium for Information and Software Quality, December 6, 2022
16. Method and system for software defect reporting, USPTO patent text citing a 2002 NIST study (historical)
17. World Quality Report 2025: AI adoption surges in Quality Engineering, Capgemini, November 2025
18. World Quality Report 2025 (PR Newswire release), Capgemini, 2025
19. World Quality Report 2025, Sogeti, 2025
20. World Quality Report 2025-26, Capgemini, 2025
21. World Quality Report 17th Edition 2025-26 (PDF excerpt), Capgemini, Sogeti and OpenText, 2025
22. Reviewing Capgemini's "World Quality Report 2024-25", independent review, 2024
23. Announcing the 2025 DORA Report: State of AI-Assisted Software Development, Google Cloud, 2025
24. State of DevOps 2025: Review of the DORA Report, Splunk, 2025
25. AI testing statistics with a link to every source, Shipper, 2026 (summarizes Stack Overflow 2025, DORA 2025 and WQR 2025-26)
26. AI Testing Statistics: Insights from World Quality Report, Quash, 2026 (summarizes BrowserStack, a vendor survey)
27. AI Testing: Statistics, Trends and Adoption Data (2026), Quash, 2026 (summarizes Gartner projections, secondary)
28. AI Quality Assurance Statistics 2026, StealthAgents, 2026 (secondary)
29. State of AI in Test Automation 2026, Testdino, 2026 (summarizes BrowserStack and PractiTest, vendor surveys)
30. QA Automation Adoption Statistics 2026, TestMax, 2026 (summarizes Katalon, vendor survey)
31. The State of Software Quality and Testing 2026, SmartBear, 2026 (vendor)

Last updated: October 29, 2026. Statistics are refreshed as new research is published.