What is the recommended model to process a RAG of PDF text documents? I've seen some recommendations for Mistral:7b. Looking to run on a consumer pedestrian home PC (ollama) with a Nvidia 4060ti and Ryzen 5700x.
Apparently there are two versions of the 4060Ti, with 8GB and 16GB of VRAM respectively. I've got an 8GB 3060 that runs gemma2:9b nicely, and that will parse PDF files; gemma3:4b also seems to analyze PDFs decently.
I also have the 8GB 4060ti variant. Want to upgrade to a 4070 super, but prices on them are still ridiculous. Could be had for $599 a handful of months ago, now on ebay going for $750 plus.
Thanks for the recommendations. I'll give gemma3:12b a try and if needed go down to gemma:4b.
May I ask why you don’t get a used 3090 with 24GB VRAM?