Neuromorphic chips won't help because we don't even know what topology makes sense. Searching for topologies is unbelievably slow. The only thing you can do is run a simulation on an actual problem and measure the performance each time. These simulations turn into tar pits as the power law of spiking activity kicks in. Biology really seems to have the only viable solution to this one. I don't think we can emulate it in any practical way. Chasing STDP and membrane thresholds as some kind of schematic on AI is absolutely the wrong path.
We should be leaning into what our machines do better than biology. Not what they do worse. My CPU doesn't have to leak charge or simulate any delay if I don't want it to. I can losslessly copy and process information at rates that far exceed biological plausibility.
> Cause and Effect: If Neuron A fires just a few milliseconds before Neuron B, the brain assumes A caused B. The synapse between them gets stronger.
A recent study from Stanford found that it's more complex than this rule, some synapses followed it, some did the opposite, etc.
Source?
* taking less than 18 years to produce a new chip
* able to task-switch instantly (being a doctor one minute and being a lawyer the next, scaling up/down instantly based on current workload)
* having millions of identical clones that people intuitively understand how to work with
* with no need for toilet breaks, sleep, family emergencies, holidays, weekends and all that
It would be pretty damn useful.
Because LLMs users use LLMs for everything
It seems very premature to say "let's build spiking NN hardware!" (or a million core cluster) before we even know how to write the software.
Spiking NNs need their Alexnet before it makes any sense to make dedicated hardware IMO.