Small AI Models Gain Traction In places with unreliable networks
31 points
3 hours ago
| 3 comments
| spectrum.ieee.org
| HN
tim-fan
58 minutes ago
[-]
Is anyone making LLM-in-a-box for emergency supply kits yet?

I feel that would be handy in all sorts of situations when networks are down.

reply
Terr_
3 minutes ago
[-]
[delayed]
reply
SwellJoe
16 minutes ago
[-]
This is couched in prepper nonsense, but it's got LLM, WikiPedia, maps, etc. A bunch of genuinely useful stuff to keep on a USB stick or whatever: https://www.projectnomad.us/

But, the current model you really want for an emergency kit is Gemma 4 12B QAT 4-bit. At ~7GB on disk, it's small enough to run on a tablet or any modern computer, slowly if you don't have a GPU or modern Apple silicon, but exceedingly smart for its size, excellent vision capabilities, good tool user, surprisingly good reasoning.

reply
cdnsteve
37 minutes ago
[-]
Can you expand what you mean?
reply
wahnfrieden
26 minutes ago
[-]
They want to ask the iOS Foundation model (frontier on device intelligence for something small) for instance about emergency procedures and life-saving info. I wouldn’t trust that model with much at all though. More likely to find what you need from miniature survival guides.
reply
bombcar
1 hour ago
[-]
99% of the model "work" (meaning the connection to your computer) is just spinning a spinner - something that makes me want to wrap it with a mosh shell so I can just keep moving from network to network.
reply
enoint
1 hour ago
[-]
Fascinating to wonder whether the bigger model finds fewer or more counterfeits than the on-device one.
reply