PashuSaarthi: turning farm footage into reviewable welfare evidence

Overview

Farm and facility operators can spend hours scanning CCTV for a small number of welfare-relevant moments. PashuSaarthi narrows that footage into timestamped search results, evidence clips, and an observation packet that a human can check and pass to a veterinarian.

The workflow has three main screens: a workspace for upload and indexing, search and evidence review, and the packet page. Each packet keeps the source clip, observed behavior, context, uncertainty notes, and review status together.

Problem

Manual review is slow, and notes can become separated from the footage that supports them. A search result without a timestamp or source trail is also hard to trust. Operators need a quick route back to the actual video before they decide whether to escalate an observation.

Safety language matters here. A model can help locate a scene, but visual footage alone cannot establish a cause or diagnosis. The interface has to keep that boundary visible throughout search, chat, packet review, and export.

Approach

VideoDB handles upload, scene indexing, semantic search, evidence streams, transcripts, and grounded text generation. The application wraps those capabilities in a short review flow. Users search in plain language, inspect the returned clip, select relevant evidence, add context, and create a packet.

Human review is part of the data model rather than a disclaimer added at the end. Observation language stays descriptive, uncertainty is editable, provenance remains attached, and packet status shows whether someone has reviewed the evidence.

Architecture

The prototype is a Next.js App Router application with server routes around the VideoDB Node.js SDK. A local JSON store keeps video and packet records, which kept the hackathon build small and easy to inspect.

• - Workspace page for file and URL ingestion with indexing status.
• - Search page for semantic queries, timestamped results, and evidence playback.
• - Assistant route for multi-turn chat grounded in indexed footage.
• - Packet routes and pages for provenance, notes, review status, and handoff.
• - jsPDF export for a portable observation document.
• - Vercel deployment with a seeded demo mode when API access is unavailable.

What I Built

I built the complete path from ingest to handoff. The demo can upload footage or accept a URL, trigger scene indexing, search the indexed video, play an evidence segment, and turn selected results into a packet.

• - Upload and URL ingestion with stored VideoDB identifiers.
• - Scene indexing controls and asynchronous status feedback.
• - Natural-language search with timestamps, scores, and playable evidence.
• - Grounded chat that includes earlier turns as conversation context.
• - Editable observation packets with provenance and uncertainty notes.
• - PDF export for human and veterinary review.

Challenges & Tradeoffs

Semantic search helps a reviewer find a likely scene, but a low or high score is not a clinical result. The interface always brings the reviewer back to the footage and uses descriptive language such as observed behavior and review recommended.

The prototype uses a JSON store instead of authentication and a production database. That kept the hackathon scope focused on the VideoDB workflow. A real deployment would need access controls, retention rules, audit logs, and explicit footage permissions before handling facility data.

Results

PashuSaarthi completed the upload, search, evidence, packet, and export flow. It was selected as one of ten finalists in VideoDB's Unlock the Footage hackathon and was featured by VideoDB Labs.

The useful outcome is deliberately modest: a reviewer gets a shorter, traceable evidence trail and can decide what needs human or veterinary attention. PashuSaarthi does not make that decision for them.

Tech Stack

• Next.js 16
• React 19
• TypeScript
• Tailwind CSS 4
• VideoDB SDK
• jsPDF
• Vercel

The application uses Next.js 16, React 19, TypeScript, and Tailwind CSS 4. VideoDB provides video ingestion, indexing, search, streams, transcripts, and grounded text generation. jsPDF creates packet exports, a JSON file stores prototype records, and Vercel hosts the app.

Links

The live demo shows the review workflow. The public repository includes the architecture, demo data attribution, VideoDB adapter, evaluation script, packet code, and safety constraints.