how do you sample it behind the scenes? usually best of X means you generate X outputs and you choose best result.

if you could do this automatically, it would be game changer as you could run top 5 best models in parallel and select best answer every time

but it's not practical because you are the bottleneck as you have to read all 5 solutions and compare them

> if you could do this automatically, it would be game changer as you could run top 5 best models in parallel and select best answer every time

remember they have access to the RLHF reward model, against which they can evaluate all N outputs and have the most "rewarded" answer picked and sent

I think the idea is they use another/same model to judge all the results and only return the best one to the user.

I think the idea is they just feed each to the RLHF reward model used to train the model and return the most rewarded answer.

I believe it is a majority vote kinda thing, rather than a best single result.