I'm using open-webui at home with a couple of different models. gemma2-9b fits in VRAM on a NV 3060 card + performs nicely.
> performs nicely
Do you have rough indication of token/s ?
What is the memory of your NV3060? 8GB?
12GB (edit: that is what mine is)