The pursuit of "undetectable bots" for testing is a symptom of a deeper malady. Teams chase sophisticated AI detection evasion not because it fundamentally improves quality, but because their current testing strategies are so fragile they look like bot traffic to any discerning system. This isn't a problem of sophisticated anti-bot measures; it's a failure to define what a "test" actually is.
If It Looks Like a Duck, It's Probably a Bot (And That's Bad)
When your automated tests are flagged by AI detection, it's a loud, clear signal that your tests are performing actions in a way that's statistically anomalous for human users. This usually stems from overly predictable, rapid, or unnatural sequences of interactions. Think about it: a human browsing a catalog doesn't click through fifty product pages in two seconds, nor do they always navigate in the exact same sequence. They pause, they scroll, they might open a new tab.
When your tests are specifically designed to mimic human unpredictability to evade detection, you're building tests that are inherently less repeatable and more complex to debug. The goal shouldn't be to fool AI; it should be to validate system behavior under realistic, and reproducible, conditions.
"User Behavior" Isn't a Test Strategy
I've seen teams spend considerable effort trying to inject random delays, mouse movements, and scrolling patterns into their UI automation to make it appear more human. This is a misguided effort. The goal of testing is not to perfectly replicate a single user's experience, but to assert that the system behaves correctly under various conditions. If your tests are failing because a random delay wasn't long enough, or the mouse cursor didn't move quite right, you're not testing your application; you're testing your test framework's ability to simulate human quirks.
This is where the distinction between an "emulation" and a "test" becomes critical. Emulation attempts to mimic something. A test attempts to verify a property or behavior. When feder-cr/invisible_dots, or any other LLM-based tool, is introduced, the temptation is to treat it as just another UI element to interact with via traditional E2E flows. This is a mistake.
The Real Failure Mode: Missing State Transitions
The core issue with many E2E and UI tests is their inability to reliably test complex state transitions. Imagine a banking application. Testing the "transfer funds" feature involves a series of steps: login, select accounts, enter amount, confirm. If your test framework, say Playwright 1.4x, relies solely on clicking buttons and entering text, you’re implicitly trusting that every intermediate state is correct.
What if the system fails after the confirmation button is clicked but before the database transaction is committed? A traditional UI test might not catch this if it simply waits for the "transfer successful" page to load. The page might load, but the underlying data is inconsistent. This is where focusing on "undetectable bots" distracts from the real problem: asserting the integrity of the system's state.
Shift Focus to System Assertions, Not Bot Evasion
Instead of asking "how do we make our tests look less like bots?", ask "how can we verify the system's core invariants and state changes?" This means moving beyond superficial UI interactions and looking for deeper integration points.
For instance, when testing an LLM interaction, instead of just verifying the final text output, consider these assertions:
- API Response Integrity: If the LLM is accessed via an API, assert the structure and expected values within the API response itself. Tools like Postman or even simple
requestsin Python can do this reliably. - Database State: After an LLM interaction that modifies data (e.g., saving a summary), directly query the database to ensure the expected record was created or updated with the correct information.
- Event Streams: If your system uses event-driven architecture, assert that the correct events are published to message queues (e.g., Kafka, RabbitMQ) with the expected payloads.
Consider a scenario where you're testing the integration of an LLM to summarize user feedback. Your test should not just look for a summarized text string on a UI. It should verify that a new record containing this summary exists in your feedback database, and perhaps that an event indicating "feedback summarized" was published to your internal event bus.
Here's a simplified Python example using pytest and requests to test an API endpoint that interacts with an LLM:
import pytest
import requests
import json
API_URL = "http://localhost:8080/api/v1/summarize"
# Assume this is your internal API endpoint for triggering summarization.
# The actual LLM interaction happens on the server side.
def test_successful_summarization_api_response():
"""
Tests that the API returns a valid response structure for summarization.
"""
payload = {
"text": "This is a very long piece of user feedback that needs summarization. It talks about the usability of the application and some minor bugs. Overall, the experience was positive but could be improved.",
"user_id": "user-123"
}
response = requests.post(API_URL, json=payload)
assert response.status_code == 200
response_data = response.json()
assert "summary" in response_data
assert isinstance(response_data["summary"], str)
assert "user_id" in response_data
assert response_data["user_id"] == "user-123"
# Further assertions could check for expected summary length or keywords
# if these are deterministic based on the input (which is rare for LLMs).
# For LLMs, it's often better to assert properties of the *process*
# or the downstream effects rather than the exact text output.
def test_api_handles_empty_text():
"""
Tests API behavior with empty input text.
"""
payload = {
"text": "",
"user_id": "user-456"
}
response = requests.post(API_URL, json=payload)
assert response.status_code == 400 # Assuming a 400 for invalid input
assert "error" in response.json()
# In a real scenario, you'd also have tests that query a database
# or check event streams after the API call, e.g.:
#
# def test_summary_persisted_to_database():
# # ... perform API call ...
# # Then, query your database (e.g., using SQLAlchemy or a direct DB connector)
# # to assert that a summary record exists for user-123 with relevant content.
# pass
This approach sidesteps the "bot detection" problem entirely by focusing on the verifiable outcomes of the system's behavior, not the perceived authenticity of the interaction.
Where This Breaks Down
The primary tradeoff with this approach is that it requires a deeper understanding of the system's architecture and data flows. You can't simply point a record-and-playback tool at the UI and expect it to be sufficient. It necessitates developers and QA engineers working together to identify critical system invariants and how to assert them. It also means that tests might require access to databases or message queues, which can complicate test environments. For purely client-side, non-stateful applications with no backend interaction, this approach offers less value, and UI-level assertions become more prominent.
The Danger of Obscurity as a Feature
The idea of a tool like feder-cr/invisible_dots being "undetectable" is a dangerous allure. It suggests that obscurity is the solution, rather than robust verification. When an anti-bot system flags your tests, it's not necessarily a sign of sophisticated detection; it's often a sign that your tests are behaving in a way that deviates from typical human patterns. If your primary concern is not being flagged, you're optimizing for the wrong thing. You're building tests that pass because they are invisible, not because they are correct.
The Mendix platform, for example, allowed for rapid application development where business logic could be visually modeled. While this accelerated development, it also meant that the underlying code execution paths could become complex and opaque. Testing these complex, visually defined processes required going beyond simple UI checks to ensure the logic itself was sound, regardless of how a user interacted with the interface. Similarly, my time at Liberty Global involved testing complex network devices. The behavior of these devices wasn't just about the front-end interface; it was about the underlying network protocols, state management, and data flow. A test that only looked at the device's web UI would miss critical failures happening at the packet level.
What This Costs You
Chasing "undetectable bots" costs you engineering time and focus. It leads to brittle tests that are hard to maintain and debug because they are designed to be unpredictable or to mimic human idiosyncrasies rather than verify system state. This distracts from the real work of ensuring your application functions as intended. It creates a false sense of security when tests pass, as they may be passing because they are indistinguishable from normal traffic, not because they have thoroughly validated the system's behavior.
What's the Real "AI" in Testing?
The true power of AI in testing won't be in making our automated scripts stealthy. It will be in its ability to analyze system behavior, identify complex failure patterns, and help us craft more precise assertions. Tools like Claude claude-sonnet-4-6 can help generate test ideas, analyze logs, or even suggest potential invariants to test, but they don't replace the fundamental need for verifiable system state.
Actionable Next Step
This week, identify one critical user flow in your application. Instead of asking "how can I make my E2E test for this flow pass more reliably?", ask "what are the key data states that must be true before, during, and after this flow?" Then, write a direct API or database assertion that verifies one of those states. Skip the UI for this specific assertion.