Practical case · Analysis and method development
When AI misses the point
From a support exchange to a method for evaluating reliable AI communication
A support case was going round in circles. I suspected that AI was writing the replies and wanted to take a closer look. Then something unexpected happened during the analysis with an AI assistant: after a safety interruption and a clarification, we ended up examining the wrong event. That became the starting point for a test design: how well does an assistant use context, when does it ask questions and how does it get back on track after an error?
This started with my own support case. At some point I wondered: is anyone actually reading this? That frustration led to a closer analysis - and things did not all go smoothly in my work with the AI assistant either. For this case study, I have anonymised the exchange and condensed it to the key steps.
The sequence
Support loops
Repeated questions and referrals did little to move my issue forward.
Suspicion and analysis
Suspected AI replies prompted an examination of text patterns, use of context and progress towards resolution.
Developing methods
The findings were to inform future evaluation work. A safety message appeared during this process.
Clarifying the task
I clarified the task: analyse existing communications and develop evaluation procedures from them.
Requesting a test
I then asked for a test based on ‘the incident’. I meant the earlier safety interruption.
Identifying the wrong reference
Without asking, the assistant linked that phrase to the later misunderstanding and developed an inappropriate test focus.
Expanding the test design
The correction made reference clarification a separate evaluation area: ask a focused question when several meanings are plausible.
What stood out in the support exchange
Known information requested again
I was repeatedly asked for information I had already supplied. This led to the test question: does the reply take the previous exchange into account?
Replies without visible progress
Understanding of my frustration was often expressed. What was missing was a clear next step. The analysis therefore examines what new information, decision or traceable action a reply actually provides.
Responsibility passed on
Referrals between participants left it unclear who would act next. The method therefore records who should act and whether there is evidence for a claimed handover.
My suspicion was not proof. Repetition and evasive replies can also result from templates, time pressure or missing information. So I separated two questions: what does the reply actually achieve? And what do we know about how it was produced? The first can be examined through the exchange. The second needs additional evidence.
This became a simple, reusable analysis process: what was asked? What information was already available? What does the message answer, what does it claim and what remains unresolved? Each conclusion can then be traced back to a specific part of the exchange.
From an interruption to the wrong reference
When I wanted to turn the findings into methods for future evaluation work, a safety message appeared. I clarified that the task was to analyse existing support messages. Work then continued. The interruption is documented; its precise technical trigger remained unresolved.
I then asked the assistant to make a test based on ‘the incident’. I meant the earlier safety interruption. By then, however, several events in the conversation were plausible references. Instead of asking, the assistant selected the later misunderstanding and continued with the wrong focus. It had understood the task but linked it to the wrong event.
I pointed out the incorrect association. We corrected the design and added four reference-clarification cases. An error in our own work thus became a subject for evaluation.
Do you mean the earlier safety interruption or the later misunderstanding?
This also shows why recorded return points matter: what was the task, what did it refer to and which state had last been confirmed? Keeping that information makes it possible to return to a known point after a wrong turn. In this case, my intervention prompted the correction. That experience provides a concrete requirement for future workflows.
The resulting test design
Clarify the reference
4 comparison cases
Distinguish several plausible incidents, ask focused questions, recognise unambiguous references and correct an incorrect association.
Resume after clarification
4 comparison cases
Proceed with permitted document analysis, clarify ambiguous access requests and continue appropriately after an explicit clarification.
Substantiate response patterns
4 comparison cases
Distinguish repetition, unanswered questions, context fidelity and claimed actions using fictional support replies.
The outputs are a documented analysis process and twelve prepared test cases, each with an expected response. Model runs for this new test set are still outstanding. They should establish whether the cases produce meaningful, repeatable results.
Contribution and value
I reconstructed the exchange, separated observations from possible explanations and worked with the AI assistant to turn them into concrete test questions. When the work went in the wrong direction, I corrected the reference and expanded the test design accordingly.
For businesses, there is a very practical question behind this: does the assistant understand my concern, take account of what has already been said and help me move forward? This case shows how a frustrating experience can become a useful test design - especially when errors in your own analysis are taken seriously too.
Clarifying questions before consequential associations. Evidence for claimed actions. A recorded starting point for corrections.