If you have any feedback/questions I would love to hear! I hope this kicks off a generation of new interesting devices. If you aren't familiar with WebRTC it can do some magical things. Check out WebRTC for the Curious[1] and would love to talk about all the cool things that does also.
* Making a toy. I have had a lot of fun putting a silly/sarcastic voice in toys. My 4 year old thinks it is VERY funny.
* Smart Speaker/Assistant. I want to put one in each room. If I am in the kitchen it has a prompt to assist with recipes.
I have A LOT more in the future I want to do. The microcontrollers I was using can't do video yet BUT ESP32 does have newer ones that can. When I pull that I can do smart cameras, then it gets really fun :)
But yeah, once I figured out that this enables streaming speech-to-speech applications on embedded devices, then it's easy to think up use cases.
Beyond that, while it does seem like its primarily vision is for speech-to-speech interfaces, it could easily be stretched to do things like send a templatized text prompt that was constructed based on toggle states, sensor readings, etc and (optimistically) asking for a structured response that could control lights or servos or whatever.
Generally, this looks like a very early stage in a hobby project (the code practices fall short of my expectations for good embedded work, being presented as a library would be better than as an application, the README needs lots of work, etc), but something more sophisticated isn't too far out of reach.
Even though the README isn’t completely done, give it a chance I bet you can have fun with it :)
Really - any physical place where people are easily overwhelmed, have something like that would be really nice.
With some work - you can probably even run RAG on the questions and answer esoteric things like where the food court in an airport or the ATM in a hotel.
Even if you trust OpenAI's models more than your trained, certified, and insured pharmacist -- the pharmacists, their regulators, and their insurers sure won't!
They've got a century of sunk costs to consider (and maybe even some valid concern over the answers a model might give on their behalf...)
Don't be expecting anything like that in an traditional regulated medical setting any time soon.
I think it would help to either have a freertos example, or if you want to go real crazy create a zephyr integration! It would be a lot of fun to work on AI and microcontroller combination - what a cool niche!
I love my Airthings. It don’t know if it’s actionable, but it would be cool to see what conclusions would come up from sending co2 and radon readings in. Could make understanding your home a lot easirr
I have talked with incredibly creative developers that are hampered by domain knowledge requirements. I hope to see an explosion of cool projects if we get this right :)