Why do high-throughput vector search engines require all embedding vectors to undergo L2 unit normalization prior to indexing?
Day 8: Select and Evaluate an Embedding Model
Select and Evaluate an Embedding Model
Loading lesson content...
In the Matryoshka embedding utility above, why is re-normalizing the truncated vector to unit length mandatory before indexing or executing dot-product queries?
The FieldDesk team selects an embedding model based on its #1 position on the public MTEB leaderboard. In production, technicians report that queries for part code SK-840-C return irrelevant manuals for SK-840-A, while descriptions like 'rumbling gravel sound in suction line' fail to retrieve the cavitation dynamics chapter. Why did the top public leaderboard model fail on this industrial workload?