This Playwright test script uses AI to test if there's smoke coming out of the Sistine Chapel chimney and whether that smoke is white. The test only passes if the smoke is white.
Currently, set to use Google Gemini Flash 2.0, but you can switch it to use other LLM providers/models by setting the environment variable in the Github actions workflow: https://github.com/donobu-inc/donobu-papal-election-tests/bl...
I've set it to run every minute during the Papal Conclave election times - https://github.com/donobu-inc/donobu-papal-election-tests/ac...
Alternative version: check for dings on my phone from every news outlet sending a notification about it.
Instead of AI looking at your code and browser and writing Playwright scripts, AI is directly controlling browser and asserting over tests. Do we have to wait for on-prem multimodal low latency AI for this to be viable?
And nice "smoke test" and making me curious about your product.
Yes, Google gemini flash and other models are reasonably fast but on-prem multimodal models will make these dramatically better. We are prepping for that future by being local-first including a desktop app.
What was your motivation to use AI for this instead of simple image analysis?
Writing prompts is much simpler than image analysis plus they make a AI test framework thingy
Image analysis would be better. I just wanted to test it quickly with different models (Gemini, GPT 4o, etc.) using an AI testing framework I am building
Test should be passing now
It passed right when the smoke started coming out - https://github.com/donobu-inc/donobu-papal-election-tests/bl...
