Ethos Guard: Multimodal Speech Risk Analytics

The Problem

Public speakers and influencers in Pakistan operate in a code-switched mix of Urdu, Roman Urdu and English. A register that mainstream moderation tooling handles poorly. One careless generalisation about an ethnic or religious group can do real damage before anyone reviews it. Ethos Guard, built as part of a doctoral research programme at the University of Karachi, gives a speaker an advisory review of their own recorded session before they publish it.

What Makes the Analysis Different

• Identity is never treated as harm. Words like ethnic or religious group names never raise a flag alone, only the combination of a group reference, a hostile predicate and a generaliser ("yeh sab… ghaddar hain") does.
• Negation is respected, including Urdu grammar. "Hum nafrat ke khilaf hain" is recognised as counter-speech, not hostility, post-positional negation is handled, which keyword systems get wrong.
• Advisory only, by design. The system tells the speaker a segment touches a sensitive area and should be reviewed. It never issues legal determinations. A deliberate safety constraint, documented in the research.

Every flagged sentence is located in time, typed (group targeting, gendered contempt, sensitive domain…), scored for severity, and paired with a safer-wording hint, so the output is coaching, not censorship.

Multimodal, Not Text-Only

Text carries most of the signal, but not all of it. The engine also reads prosody (pitch, intensity, pace, pauses) and facial-expression probabilities, then fuses the three channels into one risk score that re-weights itself when a modality is missing.

Engineering Under the Hood

• Python analysis engine (negation-aware lexicons, pluggable toxicity model interface) behind a FastAPI service, with a React dashboard consuming live per-segment findings.
• A full research pipeline around it: Whisper-based transcription and segmentation, a rule-based Urdu-script → Roman Urdu transliterator, prosodic and facial feature extraction, and an inter-annotator agreement calculator verified against the literature.
• An evaluation harness with grouped cross-validation, bootstrap confidence intervals and paired significance tests. Every number the research reports is reproducible from one command.
• 56 automated tests, including regression tests that pin the safety behaviours (counter-speech never flags; no statute names ever appear in output).

The Outcome

• A working end-to-end prototype: upload a session JSON, get an explained, per-segment risk review in seconds.
• The classic keyword-system failure (flagging "we reject hatred and violence" as incitement) demonstrably fixed and locked by tests.
• A defensible foundation for the research corpus and benchmark now being built on top of it.

This is an internal R&D project from our founder's doctoral research at the University of Karachi, shown here because it reflects how Baydot approaches ML products: honest naming, measurable claims, and safety constraints written into the code, not the marketing.