Current researchIn progress, October 2026

When does a spiking network earn its place at the edge?

An experiment comparing a spiking neural network with conventional convolutional networks, running on an Android tablet with no cloud connection, on video from the EdgeRaptor field rig.

The aim is not to show that one approach wins. It is to find out when, if ever, event-driven inference gives a practical advantage for edge video, and to report the answer whichever way it falls.

The hypotheses Progress
Frame-driven CNN every frame processed Event-driven SNN most work where change fires time something changes
Illustration of the idea being tested, not a result. On a phone processor both approaches still cost computation.

What a spiking network promises, and what it needs

A spiking neural network passes information as discrete spikes over time. Neurons that receive nothing new stay silent, so when little in the scene changes, little computation is needed. The promise is less work per hour of video, not a faster answer for any single frame.

That saving depends on two conditions, and this experiment can test only one of them directly.

  • Event input. The network must receive changes rather than whole frames: from an event camera, or from frame differences computed on ordinary video. This experiment uses frame differences.
  • Hardware that skips silent neurons. A neuromorphic processor does no work where nothing fires. A phone processor computes every operation regardless, so on the tablet the spike count is measured as an estimate of computation, not as proof of energy saved.

The task: can this frame be assessed?

Both networks perform the same job: an instant, on-device check of whether each frame is good enough to assess. Today EdgeRaptor's cloud review asks for a recapture when evidence is unusable. Running this check on the tablet makes that decision immediately, with no signal needed.

The labels come from the data itself. Good frames from storage-tank footage recorded in Texas and from the gasholder field test are degraded in code, so every training label is known. Final testing uses 200 to 300 real frames, labelled by hand, from a site kept entirely out of training.

  • Usablesharp, well exposed, unobstructed
  • Blurredcamera shake or poor focus
  • Too dark or brightunder or over exposed
  • Obstructedthe subject is blocked from view

Hypotheses, and predictions recorded before testing

The expected outcomes are written down now, so the results can show whether they were right.

  • H1 · Feasibility

    A compact network can classify inspection video on the tablet within 100 milliseconds per frame, with no network connection.

    Predicted: confirmed for the conventional network, and probably for the spiking network too.

  • H2 · Advantage

    During continuous capture, an event-driven spiking network needs less computation per minute of footage than a CNN on every frame, or a CNN that runs only when change is detected, with no loss of accuracy and no delay to alerts.

    Predicted: not confirmed while the camera moves, because almost every pixel changes as the inspector walks. The advantage is expected to appear with a fixed camera watching a quiet scene.

Two kinds of footage

Camera motion is the variable most likely to decide the result, so it is tested directly.

  • Moving

    Walking a structure with the pole camera

    The inspection case. Steel, paint and corrosion all slide across the image, so changes fire almost everywhere, whether or not anything is wrong.

  • Fixed

    The tablet on a tripod watching a quiet scene

    A gate, a yard or a road junction, where little changes. This is the case where event-driven processing should save the most, and the case closest to fixed surveillance and monitoring.

Diagrams show the expected pattern of change, not measurements.

Next stage: the drone

Once the tablet comparison is complete, the same models and measurements move to aerial video from a Parrot drone. A drone adds the two conditions that matter most for event-driven processing: hovering over one scene, which is close to the fixed camera, and flying a route, where every pixel changes all the time.

The drone's video is streamed to the tablet through Parrot's software development kit, so the comparison runs on the same device. Running a model on the drone itself depends on whether the airframe supports onboard applications.

  1. Tablet stageFixed cameraLittle changes. Largest expected saving.
  2. Drone stageDrone hoveringSome drift. The persistent-watch case, over a site, perimeter or route.
  3. Tablet stageWalking with the pole cameraMost of the frame changes as the inspector moves.
  4. Drone stageDrone on a routeEverything changes. Smallest expected saving.

Bars show the expected ordering, not measurements. The prediction to test: the spiking network's advantage shrinks from left to right. Hovering is where it matters most in practice, because long watches on battery power are where saving computation counts.

Three approaches, one test set

All three are trained on the same data and judged on the same hand-labelled frames from the held-out site.

  • CNN on every frame

    MobileNetV3-small, quantised to INT8

    The optimised conventional baseline. A spiking network has to beat a strong baseline, not a weak one, for any advantage to count.

  • CNN when change is detected

    Same model, gated by frame difference

    The cheap engineering answer to the same idea. If this matches the spiking network, the simpler approach wins.

  • Spiking network

    Leaky integrate-and-fire neurons, snnTorch

    A small convolutional spiking network trained with surrogate gradients, taking frames converted to spikes and frame-difference events as input.

From training to the tablet

Exporting a spiking network with its state over time cannot be assumed to work, so it is tested first, before any training effort is spent.

  1. 1. Go or no-goExport a throwaway spiking model to ONNX, check its outputs match PyTorch, then run it on the tablet with a dummy frame and time it.
  2. 2. TrainIn PyTorch and snnTorch, with surrogate gradients for the spiking network.
  3. 3. ExportA hand-written leaky neuron with a fixed number of timesteps and a hard threshold, so the network exports as ordinary operations.
  4. 4. Run on the deviceONNX Runtime on the Android tablet, inside the EdgeRaptor capture app, with ExecuTorch or TensorFlow Lite as fallbacks.

How it is measured

  • Accuracy and F1 on real, hand-labelled frames from a site held out from training.
  • Latency, memory and model size measured on the tablet itself, with the runtime route recorded, because it affects the figures.
  • Spikes per frame and estimated operations for the spiking network, reported as estimates. Real energy figures need neuromorphic hardware.
  • Several training runs with different random seeds, reporting the average and the spread, not the single best run.
  • One variable at a time: the number of timesteps, the firing threshold, and how images are turned into spikes.
  • Code released with the write-up, so the comparison can be repeated.

Progress

  1. Experiment design and hypothesesDone
  2. Go or no-go: spiking model exported and run on the tabletIn progress
  3. Real test set labelled by hand from the held-out sitePlanned
  4. Conventional baseline trained and quantisedPlanned
  5. Spiking network trained and tunedPlanned
  6. Both models running in the capture appPlanned
  7. Evaluation on moving and fixed footagePlanned
  8. Write-up and code releasePlanned
  9. Next stage: drone video, hovering and flying a routePlanned

Results

No results yet. Measured figures will be published here when the evaluation is complete.

They will be reported in full, including any that go against the hypotheses. A clear negative result, showing where spiking networks do not help, is as useful as a positive one.

Related work

This continues a long-standing aim: real-time inference on live video, close to the sensor. The doctoral research used a fixed camera at road junctions, which is exactly the low-motion case where event-driven processing should help most.

Research: intent prediction How the capture system is built EdgeRaptor