>Theory 1: Large enough base models are good at chess, but this doesn’t persist through instruction tuning to chat models.

I lean mostly towards this and also the chess notations - not sure if it might get chopped during tokenization unless it's very precisely processed.

It's like designing an LLM just for predicting protein sequence because the sequencing matters. The base data might have it but i don't think that's the intention for it to continue.