>According to that figure, fine-tuning helps. And examples help. But it’s examples that make fine-tuning redundant, not the other way around.
This is extremely interesting. In this specific case at least, simply giving examples is equivalent to fine-tuning. This is a great discovery for me, I'll try using examples more often.
To me this is very intuitively true.
I can't explain why.I always had the intuition that fine-tuning was overrated.
One reason perhaps is that examples are "right there" and thus implicitly weighted much more in relation to the fine-tuned neurons.
Agreed on providing examples is definitely a useful insight vs fine-tuning.
While it is not very important for this toy case, it's good to keep in mind that each provided example in the input will increase the prediction time and cost compared to fine-tuning.