Sensors Reference

All 37 built-in sensors, the signal phenomena each claims, the accepted source types, and the peer-dep footprint (three optional – sharp for image decode, face-api.js for faces, @xenova/transformers for scene). Plus createSensorStream for live audio.

How to read this page

The sense phase runs one or many sensors over a NavigateResult whose content is audio, video, or text bytes. Each sensor implements accepts(source) to tell the registry "I can read this," plus sense(source, options) to produce a SenseResult with typed findings.

• One optional peer (sharp) for image sensors. The audio + text + video sensors are pure TypeScript. The 5 image sensors (v0.9) load sharp lazily on first call and throw MISSING_PEER_DEP if it isn't installed – users who don't touch images don't pay for the native bindings. The DSP primitives (FFT, autocorrelation, envelope detection, Goertzel, Hilbert, Welch, windowing, text-DSP helpers like chi-squared / Kasiski / index-of-coincidence, 2D convolution + FFT) live in packages/core/src/sense/ and every sensor composes from them. That still makes Pluck the only signal-analysis library in the JS ecosystem whose audio + text + video sensors run on Cloudflare Workers, Vercel edge, or any other runtime without native bindings.
• Resolution knob. Every sensor respects SenseOptions.resolution: "fast" | "standard" | "deep" – trade throughput for confidence.
• pluck.dowse() runs every sensor in fast mode and ranks findings by confidence. Use it when you don't yet know what's in a signal.

See Concepts: Sense for the phase mental model and SenseResult shape.

The 37 built-in sensors

All sensors are registered in a deterministic order; pluck.sense(uri, { detect: [...] }) picks the ones whose names match your list.

Spectral analysis

Foundational frequency-domain primitives. Everything else composes from these.

Decoded signals

Decoders that read sub-perceptual data out of audio.

Band presence + diagnostic

Boolean-ish "is there energy in this band" sensors and the "is the mic live?" sanity check.

Identity

Physiological + periodicity

Outlier detection

Text domain (cipher + steganography)

Operates on text/* sources (plain text, JSON, XML, YAML, .txt / .md / .csv / .log URLs). All four enforce a 1 MB pre-DSP input cap (TEXT_TOO_LARGE) and honour AbortSignal.

Every StegoTextHit.offset indexes the original text so operators can surgically strip the payload.

Image domain (forensics + screen-recording detection)

Operates on image/* sources (PNG / JPEG / WebP / TIFF / GIF / BMP via the optional sharp peer). Decodes go through a two-phase cap: sharp.metadata() is checked against MAX_IMAGE_PIXELS (8192²) BEFORE .raw().toBuffer(), defending against image-bomb inputs. All five sensors honour AbortSignal inside their Sobel / FFT inner loops (poll every 64 rows).

CV domain (ML-backed + aerial / thermal)

The v0.10 CV sensors wrap optional face-api.js / @xenova/transformers peers, lazy-loaded so consumers who don't run them don't pay for the ~40 MB install. Every sensor that runs Sobel / CCL pre-resizes to a 1024-longest-edge analysis cap; detection boxes + reported dimensions are scaled back to the ORIGINAL image space so consumer overlays always align.

The image foundation module (toLuminance, applyKernel, sobelMagnitude, fft2d, connectedComponents, sourceToImage, resizeImage, resizeToAnalysisMax, recompressJpeg, MAX_IMAGE_PIXELS, MAX_FFT_PADDED_PIXELS, CV_ANALYSIS_MAX_DIM) is exported for consumers building custom image sensors – defineSensor({ name, accepts, sense }) composes as naturally as it does for audio / text.

Live streaming – createSensorStream

For mic / SDR / SIP / live-tail sources, the streaming primitive runs the same sensors over an incoming ReadableStream<Float32Array> and emits per-window events.

import { createSensorStream } from "@sizls/pluck";

const stream = createSensorStream(micAudio, {
  sampleRate: 44_100,
  detect: ["fft", "heartbeat", "breathing"],
  windowSize: 4096,        // default
  hop: 2048,               // default = 50% overlap
  signal: abortSignal,     // optional; aborts both directions
});

for await (const event of stream) {
  // { time: number, feature: SenseFeature, result: Partial<SenseFeatures> }
}

Memory-bounded (rolling buffer; ~40 KB peak regardless of stream length), backpressured (highWaterMark: 1 queuing strategy), abort-aware. Non-audio sensors like rppg are silently skipped on audio streams.

Planned sensors (not yet shipped)

PlannedSenseFeature is a tag union in the type layer; calling pluck.sense(uri, { detect: ["watermark"] }) today throws NO_SENSOR. Check availability before calling:

pluck.sensors.whichHandles("dtmf");     // "dtmf"
pluck.sensors.whichHandles("flicker");  // "flicker" (v0.9 – now ships)
pluck.sensors.whichHandles("watermark"); // undefined – planned, not shipped

Tracked: watermark (content-ID style spectral watermarks), voiceprint (speaker identity), engine (RPM / cylinder fingerprint), seismic (earthquake precursors), animalsong (broader-than-birds bioacoustic coverage), steganography-image (LSB channel extraction – v0.9 ships ELA which covers the tampering half of the image-stego diagnostic surface; the pure LSB decoder is still planned).

Custom sensors

import { createPluck, defineSensor } from "@sizls/pluck";
import { goertzel } from "@sizls/pluck/dsp"; // re-exported DSP primitive

const lisnr = defineSensor({
  name: "lisnr",
  accepts: (source) => source.contentType.startsWith("audio/"),
  sense(source, options) {
    // Proprietary Lisnr ultrasonic beacon decoder
    const { sampleRate, samples } = readAudioSource(source);
    const decoded = decodeLisnrFrames(samples, sampleRate);

    return {
      features: { lisnr: decoded },
      confidence: decoded.length > 0 ? 0.9 : 0.0,
      decoded: decoded.map((d) => ({
        kind: "lisnr",
        data: d.payload,
        startTime: d.t0,
        endTime: d.t1,
        confidence: d.snr,
      })),
    };
  },
});

const pluck = createPluck({ sensors: [lisnr] });
await pluck.sense("./store-visit.wav", { detect: ["lisnr"] });

Custom sensors prepend to the registry. The community signature registry proposal (@sizls/signatures-*) in IDEAS makes the pattern a shipping primitive – each npm package declares patterns the main sense API picks up at registration.

pluck.dowse() – zero-config scan

When you don't know what's in a signal, dowse is the one-liner:

import { pluck } from "@sizls/pluck";

const scan = await pluck.dowse("./mystery.wav");

console.log(scan.topFinding?.summary);
// → 'Decoded dtmf: "0123456789"'

for (const finding of scan.findings) {
  console.log(`${finding.sensor} ${finding.confidence.toFixed(2)} ${finding.summary}`);
}

dowse runs every shipped sensor at resolution: "fast", returns findings sorted by confidence descending, and aliases findings[0] as topFinding. See Concepts: Sense → pluck.dowse().

Introspection

import { createPluck } from "@sizls/pluck";

const pluck = createPluck();

pluck.sensors.list();
// ["fft", "spectrogram", "dtmf", "pitch", "tempo", ...]

pluck.sensors.whichHandles("dtmf");
// "dtmf"

pluck.sensors.find("dtmf", audioSource);
// Sensor – narrows by name AND accepts() probe

What's next

• Concepts: Sense – phase model, SenseResult, DSP primitive catalog.
• Recipe: Snitch Privacy – composing ultrasonic + fft + spectrogram into a forensic privacy audit.
• IDEAS backlog – Sense Receipts, pluck.senseDiff, the Community Signature Registry.