Reducing embedding dimensions with matryoshka representation

The text-embedding-3-large model supports reducing dimensions via the dimensions parameter. I tested how this affects retrieval quality.

Benchmark results (MTEB retrieval subset)

| Dimensions | NDCG@10 | Storage per vector | Cost reduction | |-----------|---------|-------------------|---------------| | 3072 (full) | 0.642 | 12.3 KB | baseline | | 1536 | 0.637 | 6.1 KB | 50% | | 768 | 0.621 | 3.1 KB | 75% | | 256 | 0.589 | 1.0 KB | 92% |

Going from 3072 to 1536 dimensions loses almost nothing (0.5% NDCG drop) while halving storage costs. The sweet spot for most applications seems to be 1536.

At 256 dimensions, quality starts to degrade noticeably, but it's still usable for rough semantic search where recall matters more than precision.