The intake pipeline
Five stages from raw call recordings to knowledge-base documents.
Flatten the source calls
976 source JSON files become 976 records of exactly three fields: a unique call id, a timestamp, and the transcript. Everything else in the source is either personal data or derived content the pipeline does not want — agent names, customer contacts, the recording URL (whose path embeds both), and performance-review notes.
Narrowing here makes the privacy boundary auditable: any personal data downstream must be inside the transcript prose, because no structured field survives to carry it.
Extract atomic claims
One call per transcript. Each yields self-contained factual statements, tagged with the subject they concern, the kind of knowledge they are, and — for procedure steps — their position in the sequence as that call described it.
That position is what later lets scattered steps be reassembled: several calls each describe part of a procedure, and the step numbers say how the fragments fit together.
Claims about only one customer are separated out and never published. Of 6,569 claims extracted, 1,221 were case-specific and dropped; 2 more were withheld by the privacy gate.
Consolidate subjects
Extraction names subjects freely, so the same product arrives under many labels — 229 of them. One call consolidates these into 103 canonical subjects, each becoming a candidate document.
It works from evidence rather than wording: every label is presented with sample claims and with the other labels it co-occurs with in the same call. Package and tier names are grouped per product, so every tier of a product lands in one packages document rather than scattering.
Review the plan
Before anything is written, the full document plan is emitted with every claim visible — which subjects become documents, what each will contain, and which fall below the threshold. Grouping mistakes are cheap to fix at this point and expensive afterwards.
Synthesize documents
One call per section rather than per document, so the largest document is written in pieces instead of one oversized request. Each call receives every claim for that section and writes the section body: grouping facts into tasks, merging the many restatements of one fact into a single line, and rebuilding procedures from the per-call step positions.
Where calls genuinely disagree, both readings are kept and the disagreement stated — never averaged, never silently resolved.
What each stage produced
| Stage | In | Out |
|---|---|---|
| Flatten | 976 call files | 976 transcripts |
| Extract claims | 976 transcripts | 5,346 facts |
| Consolidate subjects | 229 labels | 103 subjects |
| Synthesize | 5,280 facts | 74 documents |
Personal data
Excluded at two points
Structured personal data never enters the pipeline: it is dropped when the source call is flattened. Prose personal data is handled by instruction during extraction and then by a deterministic gate applied to both the fact and its subject — the subject becomes a document title and reaches every chunk of that document.
The company’s own published channels pass deliberately: a support knowledge base should contain them. On this corpus the gate withheld 2 facts out of 5,348, both false positives.