This is a really interesting idea! I'll be honest, it took me a minute to really get what it was doing. The GitHub page video doesn't play with any audio, so it's not clear what's happening.
Once I watched the video, I think I have a better understanding. One thing I would like to see is more of a breakdown of how this solves a problem that just a big model itself wouldn't.
Thank you!
Yeah we rushed to create a "Plexe in action" video for our Readme. We'll put a link to the YouTube video on the Readme so it's easier.
Using large generative models enables fast prototyping, but runs into several issues: generic LLMs have high latency and cost, and fine-tuning/distilling doesn’t address the fundamental size issue. Given these pain points, we realized the solution isn’t bigger generic models (fine-tuned or not), but rather automating the creation, deployment, and management of lightweight models built on domain-specific data. An LLM can detect if an email is malicious, but a classifier built specifically for detecting malicious emails is orders of magnitude smaller and more efficient. Plus, it's easier to retrain with more data.