Embedding retrieval finds 50 candidates in milliseconds. A cross-encoder on two L4 GPUs reads each one against your question and reorders them.