> Modern BERT with the extended context has solved natural language web search.
I doubt this. Embedding models are no panacea even with a lot simpler retrieval tasks like RAG.
RAG is literally what Google Search is.
Unlike the natural language queries that RAG has to deal with, Google searches are (usually) atomic ideas and encoder-only models have a much easier time with them.