If you’re a QA lead, automation engineer, or developer responsible for test automation, you’ve probably dealt with false positives. Tests fail, but when you investigate, you realize there’s nothing wrong with the application, just with the test itself. It happens so often that false positives become part of the daily routine, eating up hours of work without actually improving software quality.
In a recent webinar, Tobias Müller, Founder of TestResults, explained why false positives happen, why teams often focus on the wrong issues, and what actually needs to change to make automation more reliable. If you’re struggling with flaky tests, constant maintenance, or automation that seems to break for no reason, this session is for you.
Why False Positives Get Worse Over Time
At first, false positives seem like a minor inconvenience. A test fails, someone reruns it, and it passes the second time. Maybe it was slow test data, a network issue, or an unstable environment. No big deal.
But as test suites grow, failures happen more often. The same tests keep failing randomly. Debugging them doesn’t reveal any real problems with the application. Fixing one failure just causes another. The more automation expands, the more time is spent maintaining tests instead of running them.
This is when teams start questioning whether automation is helping at all. Instead of freeing up testers to focus on higher-value work, test automation becomes just as demanding as manual testing. Or worse, an unreliable system that no one fully trusts.
What Causes False Positives?
A lot of teams assume that false positives happen because of flaky environments or bad test data. While that’s sometimes true, the bigger issue is usually automation design.
Many test failures happen because of decisions made early in the automation process, decisions that seemed fine at the time but ended up creating long-term instability. Tobias covered the most common automation traps teams fall into:
1. Sleep Commands and Hardcoded Waits
One of the first things automation teams do when they run into timing issues is add sleep commands or fixed waits. The idea is simple: if a test is failing because a page loads too slowly, adding a delay should give it enough time to work.
The problem? This approach doesn’t actually fix timing issues, it just hides them.
- If the delay is too short, the test still fails randomly.
- If the delay is too long, the test wastes time and slows down the entire pipeline.
- If conditions change (e.g., an update makes the app slower or faster), the test becomes unreliable again.
Instead of creating stability, sleep commands introduce inconsistent test execution and increase false positives.
2. Recorded Tests That Break with UI Changes
Recorded tests seem like an easy way to build automation, especially for teams that want quick results without writing test scripts from scratch. But while recording tools work for small projects, they don’t scale.
Why? Because recorded tests are tied to the UI’s exact structure. Even small design changes—like a button moving or a field label being updated, can cause the test to fail. Instead of detecting real bugs, teams end up dealing with tests that need to be re-recorded constantly.
This creates a cycle of maintenance instead of progress. Every UI update requires someone to check and fix dozens of tests, slowing down automation instead of making it faster.
3. Brittle Locators (Element IDs, XPaths, and CSS Selectors)
Most automated tests rely on locators (element IDs, XPaths, CSS selectors) to find and interact with buttons, text fields, dropdowns, and other UI elements. But these locators are not stable over time.
- Developers might update the front-end framework, changing how elements are structured.
- IDs that were static in one release might be generated dynamically in the next.
- A small UI redesign might shift elements around, causing XPaths to break.
Locators work well when the UI doesn’t change, but modern applications are constantly evolving. Relying on element IDs and XPaths means that as soon as the UI changes, tests start failing, not because of real bugs, but because the test automation can’t adapt.
Why Test Data Isn’t the Real Problem
Many teams assume that if tests are failing, bad test data must be the issue. And sometimes, that’s true. Test data problems, like missing records, unexpected API responses, or inconsistent databases, can cause false positives.
But here’s the key difference: test data issues improve over time.
As teams refine their test data strategy, whether through better environment control, data resets, or API mocking, data-related false positives tend to decrease.
What doesn’t improve over time? Bad automation design.
If automation is built on sleep commands, recorded tests, or brittle locators, false positives will only increase as the test suite grows. What seemed like a minor issue in the beginning turns into a long-term maintenance nightmare.
The Solution: Visual Steering
False positives aren’t just frustrating, they make automation unusable. If teams can’t trust their tests, automation doesn’t add value.
The solution isn’t to constantly fix broken tests, but to design automation that doesn’t break in the first place. That’s where automated testing tools with visual steering comes in.
Visual steering works differently from traditional locator-based automation. Instead of relying on element IDs, XPaths, or recorded scripts, it recognizes UI elements the same way a human would, by their appearance, context, and behavior rather than their underlying code.
This makes automation more resilient because:
- Tests don’t break when locators change—Visual steering doesn’t depend on element IDs.
- Automation works even when UI elements shift—It recognizes buttons, fields, and icons based on how they look.
- False positives decrease—Because tests aren’t tied to fragile locators, they only fail when something is actually wrong.
Visual steering eliminates the root causes of false positives, allowing teams to focus on real testing instead of fixing automation.
Watch the Webinar Recording
If your team is spending more time fixing tests than writing them, or if automation is creating more work than it saves, this session will help you fix the problem at its source.
In this webinar, Tobias Müller breaks down why false positives happen, why common automation shortcuts make them worse, and how visual steering allows teams to automate with confidence.
Watch the full webinar recording here:
