Aller au contenu

Essay · Mr1000xGrowth Lab

Voice agents: collecting without retaining.

The fear is legitimate: an AI on the phone, listening, recording, keeping everything. That is the default image of the voice agent, and it is also the worst possible design. But it is not inevitable, it is a choice.

I built one of my first voice agents for a study conducted across an entire profession. The principle fit in one sentence: collect without retaining. And that is precisely what reassured people who were wary of AI.

8 min readCharles Gautier

01

The default fear

We picture the voice agent as an ear that stores everything: the conversation, the voice, the hesitations, anything that might one day prove useful or leak. It is the most common design, out of laziness more than malice. We keep everything « just in case », and we will deal with compliance later.

This reflex is exactly the opposite of what should be done. The best protected sensitive data is the data we never kept.

The best protected sensitive data is the data we never kept.

02

The reverse principle: keep only the synthesis

We can design the opposite. Retain only the structured synthesis of what we are looking for, never the raw transcript. Explain at the start of the call what will be kept and what will not. Anonymize by default, from the design stage.

In the study I built, the database contained only structured and anonymized points, extracted from the dialogue, not the conversation. A genuine analysis of what was said, not a simple capture of keywords, but without keeping the raw material that would have created a risk.

03

Why wary people agree

The most telling part is not technical. These were rather older profiles, not at ease with these technologies, who agreed to answer an AI. Not because they were forced, but because the framing was clear, respectful and useful.

Respect is felt. When you explain honestly what you keep and what you discard, when you do not try to take more than necessary, distrust falls away. Transparency reassures far more than avoidance or vagueness.

04

Privacy is not a constraint, it is a design

You do not « add » privacy at the end, through a legal amendment or a checkbox. You design it into the architecture: what you capture, what you discard, what you synthesize, what you can prove. It is an engineering decision as much as a legal one.

A system built not to retain is structurally safer than a system that retains everything and then tries to protect it. The first has almost nothing to leak; the second carries a permanent risk.

You do not add privacy. You design it.

05

AI Act and sovereignty

In Europe, with the AI Act, these questions have stopped being theoretical. Compliance is not obtained after the fact. It is earned upstream, in the way the system is designed, in what it has decided not to know.

It is more demanding, and that is precisely where the difference will be made. The agentic systems that will last in Europe will not be the most talkative. They will be the most data-sober, the ones that turned respect into an architectural principle, not a constraint endured.

Coda

Coda

The right question is not « can the AI listen? ». It is « what have we decided it should retain? ». The answer is made at the design stage, not in the terms of use.

Collecting without retaining is not a slogan. It is an architect's choice, and it is often what makes an AI more respectful than a human taking notes.

Read next

  1. Case studies

    The concrete case behind this essay: surveying an entire profession through a voice agent, without keeping any transcript.

  2. Governance and cost

    The real driver of the agentic migration: knowing who decides, and what it costs.