undefined | Better HN

0 pointsFrereKhan4y ago0 comments

If you bring activation sparsity into the mix, the advantage of SNN processors over GPUs/TPUs becomes more clear. Loss-gradient-based optimisation approaches are great because they give you a tool to include e.g. sparsity regularisation into the loss. Encouraging sparse activity makes simple linear algebra a poor fit for network activation, and SNN processors a much better fit.

0 comments

periheli0n4y ago

But is sparse activation sufficient to motivate the use of SNNs? In my opinion one needs a temporal component as well.

Sparse activations that don't also have a time component (i.e. are sparse in space and time) can be very well implemented without events.

Granted, SNN processors can handle sparse activations better than matrix accelerators. But then again, SNN accelerators might carry lots of SNN overhead that is not required for sparse activations alone.

Edit: A good example for a non-spiking sparse activation accelerator is the NullHop architecture [1].

[1] https://ieeexplore.ieee.org/abstract/document/8421093

FrereKhanOP4y ago

I agree. The use case needs to justify having state, otherwise the ideal architecture is something like NullHop. Temporal signal processing / vision processing tasks are ideal for SNNs, especially if the inputs can also be sparse.

j / k navigate · click thread line to collapse