you get video tokens through those seemingly dumb tiktok shorts
Of all the types of tokens in the world video is not the one that comes to mind as having a shortage.
By setting a a few thousand security cameras in various high traffic places you can get almost infinite footage.
Instagram, Youtube and Snapchat have no shortage of data too.
except 1) tiktok is video stream data many orders of magnitude larger than any security cam data, that's attached to real identity 2) china doesn't have direct access to Instagram reels and shorts, so yeah
Why does tying it to identity help LLM training?
It's pretty unclear that having orders of magnitude more video data of dancing is useful. Diverse data is much useful!