LiDAR-Whisper

Consumer-grade LIDAR sensors (e.g. iPhone Pro, Livox Mid-360) measure range with sub-millimeter precision. When pointed at a window from across the street, the sensor can recover audio-band vibration from the surface and reconstruct speech. LiDAR-Whisper produces signed attestations on both sides: an operator's recovery, and a defender's detection of an inward-pointed surveillance sweep.

Posture: 🟣 Red & Blue (dual-use)   ·   Status: alpha (moonshot)

What it does

LIDAR (LIght Detection And Ranging) sensors fire laser pulses and measure the round-trip time to derive sub-millimeter range to a target. Modern consumer LIDAR (the iPhone Pro's camera array, the Livox Mid-360 puck) samples at ~30 Hz with sub-mm precision. With phase-unwrap math you can recover audio-band vibration off any reflective surface. A glass window vibrates with the speech in the room. A keyboard vibrates with the keystrokes. Across the street, with no infrared laser visibly aimed at the target, a $999 phone can recover speech and keystrokes from the surface vibration of a window.

This works. Researchers have demonstrated versions of it since the Lamphone paper in 2020. What's new is consumer-grade LIDAR (no specialized RF gear) plus a phase-unwrap pipeline that fits in a JS/TS process. LiDAR-Whisper plants the flag at the boundary by shipping the signed-attestation shape for two opposite use cases:

• Red-team. An operator running consumer LIDAR captures frames, pipes them through a reconstruction pipeline, and gets per-class confidence scores (speech, keystroke, ambient). When speechConfidence > 0.6 the Bureau emits a signed speech-recovered proof; when keystrokeConfidence > 0.7, it emits keystroke-recovered.
• Blue-team. Operators run their own LIDAR pointed at their own windows. When the pipeline detects an external LIDAR sweep painting the windows (the signature of someone else aiming at the building), remote-surveillance-detected fires.

Who would use it

• A red-team contractor evaluating an executive office's surveillance posture, who needs court-exhibit-grade attestation that yes, speech could be recovered from across the street.
• A blue-team CISO at a hedge fund, embassy, law firm, or biotech who runs their own LIDAR pointed at their own conference-room windows and wants to know when external sweeps are painting the building.
• An investigative journalist or NGO documenting state-sponsored corporate espionage who needs cryptographically-signed observations of a recovery rather than re-recordable audio.
• A privacy researcher publishing an academic paper on consumer-LIDAR surveillance who wants reproducible signed reconstructions auditors can verify.

What you'll need

• The Pluck CLI installed (npm i -g @sizls/pluck-cli).
• For real deployment: an iPhone Pro (LIDAR-equipped) or Livox Mid-360, plus a host machine to run the pipeline. The hardware bridge to read raw LIDAR phase frames is deferred today.
• For blue-team use: a LIDAR placed inside your own building, pointed at the windows you care about.

Step-by-step

pluck bureau lidar-whisper demo

The demo synthesizes four captures: an ambient baseline (no proof), a speech-energy capture (high speech confidence – fires speech-recovered), a keystroke-rhythm capture (high keystroke confidence – fires keystroke-recovered), and a blue-team capture with surveillanceSweepDetected: true (fires remote-surveillance-detected).

lidar-whisper/demo: ingesting 4 LidarCaptures -> 3 LidarWhisperProofs.
[Bureau/LIDAR-WHISPER] proof=09fdb3a3… kind=remote-surveillance-detected
[Bureau/LIDAR-WHISPER] proof=d6f4e822… kind=speech-recovered
[Bureau/LIDAR-WHISPER] proof=c0e648ae… kind=keystroke-recovered

Production CLI (capture from the LIDAR device, reconstruct to run the pipeline, verify to check a published reconstruction) lands in a follow-up.

Run it yourself

Drop this into a Node 18+ project (npm install @sizls/pluck-bureau-lidar-whisper @sizls/pluck-bureau-core tsx). Real LIDAR phase-unwrap math is research-required; the example feeds the deterministic stub classifier with a synthetic surveillance-sweep frame profile.

// index.ts
import { createHash } from "node:crypto";
import {
  createLidarWhisperSystem,
  fingerprintPrivateKey,
  signCanonicalBody,
} from "@sizls/pluck-bureau-lidar-whisper";
import { generateOperatorKey } from "@sizls/pluck-bureau-core";

const sha256 = (s: string) => createHash("sha256").update(s).digest("hex");
const flush = (n = 60) => new Promise<void>((r) => { let i = 0; const tick = () => (++i >= n ? r() : setImmediate(tick)); setImmediate(tick); });

async function main() {
  const op = generateOperatorKey();
  const opFp = fingerprintPrivateKey(op.privateKeyPem);

  const system = createLidarWhisperSystem({
    signingKey: op.privateKeyPem,
    disablePausePoll: true,
    disableLogging: true,
  });

  // Surveillance-sweep frames: alternating peaks of 0.98 with high-baseline 0.82.
  const frames = Array.from({ length: 64 }, (_, i) => (i % 4 === 0 ? 0.98 : 0.82));
  const captureBody = {
    schemaVersion: 1 as const,
    frames, sampleRateHz: 30,
    locationHash: sha256("apartment-3B"),
    hardwareFingerprint: sha256("iphone-pro-lidar"),
    targetWindowFingerprint: sha256("window:home-office"),
    operatorFingerprint: opFp,
    surveillanceSweepDetected: true,
    capturedAt: "2026-04-15T11:30:00.000Z",
  };
  const captureId = sha256(JSON.stringify(captureBody));
  const { signature } = signCanonicalBody({ ...captureBody, captureId }, op.privateKeyPem);

  try {
    system.observeCapture({ ...captureBody, captureId, signature });
    await flush();
    for (const p of system.facts.proofs()) {
      console.log(`proof kind=${p.kind} id=${p.proofId.slice(0, 16)}…`);
    }
  } finally {
    await system.shutdown();
  }
}

main().catch((err) => { console.error(err); process.exit(1); });

Run with tsx index.ts. Expected output:

proof kind=remote-surveillance-detected id=…

▶ Open in StackBlitz – runs in your browser, no install required.

What you get

A LidarCapture envelope per session containing the frame digest, hardware fingerprint, location-hash (bucketed), target-window fingerprint, and a surveillance-sweep flag. A Reconstruction envelope per recovery containing the feature-vector digest, SNR (signal-to-noise ratio in dB), and per-class confidence scores. Plus a Proof envelope for each fired detector.

The signed body never carries recovered audio bytes, recovered keystroke characters, or the raw location string. Only digests, frequency-bin magnitudes, and confidence values appear.

Three classes of proof:

• speech-recovered – speech confidence > 0.6.
• keystroke-recovered – keystroke confidence > 0.7.
• remote-surveillance-detected – operator's own LIDAR caught an external sweep painting the windows.

What it can't do

• Real phase-unwrap math (Itoh's 1D, branch-cut 2D, residue tracking) is deferred. The alpha's reconstructAudio() is a deterministic feature-vector mapping, not a real phase unwrapper. Production phase-unwrap is research-required.
• Real Livox Mid-360 / iPhone-Pro hardware bridge is deferred.
• Real adversarial-flicker / surveillance-sweep DSP detector is deferred – the alpha sets surveillanceSweepDetected from operator input rather than a real DSP pipeline.
• This is attestation, not attack code. The package does not include exploit code for vendor LIDAR APIs. Pointing a LIDAR at someone else's window may be illegal in your jurisdiction.
• Privacy property is asymmetric: signed bodies never carry recovered content, but they DO carry confidence scores. A high-confidence proof reveals that recovery succeeded, even if not what was recovered.

A real-world example

In late 2027, a biotech executive's office in Cambridge, MA is the suspected target of corporate-espionage activity by a competitor. The CISO installs a Livox LIDAR inside the conference room pointed outward at the floor-to-ceiling glass. Four days later, between 11:32 PM and 11:34 PM, the LIDAR detects a high-frequency target sweep across the windows from a building 400 meters away. surveillanceSweepDetected: true fires, the proof is signed and Rekor-anchored. The CISO files a complaint with the FBI's Boston field office. The Bureau cassette includes the timestamp, the frequency-band signature of the external sweep, and the bucketed location of the office (without precise address). The signed timeline serves as third-party-verifiable evidence supporting the parallel investigation.

For developers

Predicate URIs

Programs composed

• Pluck core's DSSE in-toto envelopes + Sigstore Rekor client.
• Directive's facts/constraints/resolvers.
• Bureau core's pause kill-switch, operator-key tooling, and PII redactor.

Threat model + adversary

Red-team threat: a foreign-intelligence or corporate-espionage actor pointing consumer LIDAR at a target's windows. Blue-team defense: operator's own LIDAR catches the external sweep. The package does not stop the recovery – it makes the recovery's existence attestable and the external sweep detectable given an inward-pointing LIDAR.

What's stubbed (alpha – moonshot)

• Real Itoh / branch-cut / residue-tracking phase-unwrap math deferred.
• Real Livox Mid-360 / iPhone-Pro hardware bridge deferred.
• Real adversarial-flicker / surveillance-sweep DSP detector deferred.
• dsseSign / notarizeAttestation Rekor integration stubbed.

Verify a published cassette

pluck bureau verify <bundle-dir>
cosign verify-blob --key <pubkey.pem> --signature <sig> \
  --type https://pluck.run/LidarWhisper.Reconstruction/v1 <body.json>

See also

• Bureau Foundations
• Threat Model
• Verify a dossier
• Lamphone (LIDAR / lightbulb-vibration audio recovery)
• Itoh 1D phase unwrap