semantic-contradictions-cluster-before-llm
IN premise
The `--semantic` flag on `detect-contradictions` embeds beliefs via sentence-transformers, clusters with KMeans via `list_clusters()`, and sends each cluster to the LLM as a batch — ensuring topically related beliefs are analyzed together instead of being scattered across random batches of 50.
Summary
The contradiction detection system has a smart batching mode where instead of sending arbitrary groups of 50 beliefs to the LLM, it first uses embedding-based clustering to group topically related beliefs together. This means the LLM sees beliefs about the same subject in the same batch, making it far more likely to catch genuine contradictions between related claims that would be missed if those claims ended up in different random batches.
Details
| Source | entries/2026/05/10/semantic-contradiction-detection.md |