DangNH Studio
NEWSEN

AI Police Surveillance Tools: Human Cost and Accountability Gaps

21/10/2026 507 views
AI Police Surveillance Tools: Human Cost and Accountability Gaps

What Flock’s AI Search Tool Actually Does

Flock’s platform lives inside a web‑based console that officers open on a laptop or tablet. The interface presents three main search modes:

1. FreeForm (plain‑language) search – an officer types a description such as “person wearing a red hoodie and a tattoo of a dragon on the left forearm.” The text is vectorised and compared against numeric embeddings of every frame captured by the city’s cameras. The system returns a ranked list of video snippets that best match the description. 2. Smart Sort – after the initial list appears, the officer can vote on a few results (thumbs‑up or thumbs‑down). Each vote re‑weights the underlying similarity scores, pushing footage that looks more like the approved clips to the top. This iterative loop lets a single officer refine a massive result set in seconds. 3. Watch‑list / area‑wide search – an officer draws a polygon on a city map, selects a description, and tells the system to run continuously across all cameras inside that zone. The backend streams live frames to the model and flags any matches in real time.

The UI also shows case codes (a short alphanumeric tag the officer must enter) and a confidence score (0‑100) for each result. A tooltip warns, “Results may be inaccurate; verify with other evidence before acting.”

> *Example walkthrough*: An officer in Chicago wants to locate a suspect described only by clothing. They select FreeForm, type “person in a blue denim jacket, black backpack,” set the case code “CJ‑2026‑001,” and hit Search. Within seconds the panel lists ten clips, each with a confidence rating. The officer clicks the first clip, sees the suspect on a street corner, and uses Smart Sort to up‑vote it, instantly reshuffling the list so similar footage rises to the top.

The underlying model is a proprietary vision‑language transformer trained on millions of public‑domain images; the exact architecture is not disclosed by Flock. For a technical deep‑dive see the WIRED reverse‑engineering report Source.

The Promise of Guardrails: Policies vs. Practice

Flock’s public documentation (updated August 2026) lists three core safeguards:

* Data‑retention limits – raw video and search results are automatically deleted after 30 days unless a case code is attached, in which case they are kept for up to 90 days for investigative use. * Mandatory case codes – every search must be tagged with a pre‑approved case identifier. The UI disables the Search button until a code is entered. * Automated audit logs – each query, the officer’s description, the model’s confidence score, and any subsequent Smart Sort votes are logged. An internal dashboard flags searches that exceed a confidence‑threshold of 80 % without a case code, generating an email alert to the department’s compliance officer.

What the code actually does

The JavaScript bundle captured by WIRED shows that the audit‑log call (/api/audit) is optional until the end‑of‑year rollout. Until then, departments can toggle the “enforce‑audit” flag off, allowing searches to proceed without any server‑side record. Moreover, the retention timer is hard‑coded to 90 days for *any* case code, regardless of local policy, and there is no cryptographic proof that deleted footage is unrecoverable.

Real‑world implementation gaps

* Several police departments (e.g., Austin PD) reported that their IT staff could disable the case‑code requirement via a hidden config file, effectively reverting the UI to a “free search” mode. * A Freedom of Information Act request in Illinois revealed that audit logs for 2025 contained null entries for 12 % of all searches, indicating that the logging service was down for weeks. * The promised “automated audits” are limited to flagging high‑confidence searches; they do not block the officer from proceeding, nor do they trigger any external oversight.

In short, the policies read like a compliance checklist, but the underlying code leaves critical knobs open for agencies to bypass them. The gap between promise and practice fuels the controversy highlighted in congressional letters to Flock’s CEO Source.

Accountability Gaps: Who Monitors the Monitors?

Direct answer: Flock’s AI police surveillance tools lack transparent oversight; only the company and a handful of agency officials can view search logs, and retention periods remain vague. The system records every query, the officer’s description, and the model’s confidence score, but the public cannot see how long these logs are kept or who accesses them.

The Wired investigation shows that audit logs exist, yet the code reveals no mandatory expiration date for the data, and internal access controls are undocumented【https://www.wired.com/story/flock-ai-search-user-interface】. Moreover, the AI can flag sensitive categories (race, religion) but only slows the search for political expression, leaving a loophole for potential abuse.

Key findings

These gaps make it difficult for communities to hold law‑enforcement accountable for mistaken or discriminatory searches.

Policy Paths Forward: Building Real Safeguards

To close the accountability gaps, legislators and city councils can adopt a layered framework that combines technical controls with independent oversight.

| Recommendation | Why It Matters | Example Implementation | |----------------|----------------|------------------------| | Mandate independent audit logs | Guarantees that third‑party auditors can verify who accessed data and when. | Require quarterly audits by a certified privacy firm, with results posted publicly. | Set strict data‑retention limits | Reduces the risk of historic misuse and limits exposure if a breach occurs. | Enforce a maximum 30‑day retention for raw footage and 90‑day retention for query metadata, unless a court order extends it. | Require transparent model documentation | Allows researchers to assess bias and accuracy before deployment. | Publish a model‑card describing training data sources, performance metrics across demographic groups, and known limitations. | Create a civilian oversight board | Provides community voice and rapid response to complaints. | Board members include civil‑rights advocates, technologists, and local residents; they receive real‑time alerts for flagged searches. | Implement “stop‑list” categories | Prevents searches based on protected characteristics. | Extend the existing speed‑bump for political expression to a hard stop for race, religion, gender, and sexual orientation.

Action steps for policymakers 1. Draft ordinances that embed the above requirements into any contract with AI surveillance vendors. 2. Allocate budget for independent audits and community board operations. 3. Require vendors to certify compliance before deployment, with penalties for violations. 4. Encourage public‑record requests for audit logs to increase transparency.

By coupling technical safeguards with robust civilian oversight, municipalities can harness the investigative benefits of AI police surveillance tools while protecting civil liberties.

Use the tool

Open the tool →

This article was edited with AI assistance based on publicly available sources and reviewed before publishing.

PreviousWhy the Redmi Note 17’s Price Jump Isn’t Just a Xiaomi Issue
20/10 588