Deterministic Multi-Agent Governance (HQAS-8 / AMCG-3) & Bilingual RLHF/S2S Calibration: Open Benchmarks from Vision Flow Research Lab
Modern autonomous AI engineering teams face two structural failure modes when moving from single-prompt prototypes to long-horizon production workflows: Trajectory Compaction Drift & Unchecked Tool Execution: Multi-ag
Modern autonomous AI engineering teams face two structural failure modes when moving from single-prompt prototypes to long-horizon production workflows:
- Trajectory Compaction Drift & Unchecked Tool Execution: Multi-agent orchestrators lose critical constraints across context compactions or execute unvalidated Model Context Protocol (MCP) tool calls without AST-level security gates.
-
Multilingual & Speech-to-Speech (S2S) Rubric Variance: Standard English-centric RLHF rubrics degrade sharply on South Asian code-switched queries (
bn_BD/en_US), prosody naturalness, barge-in latency, and grounded factual verification.
At Vision Flow Research Lab (Vision Flow Group, visionflowgroup.com ยท Wikidata Q141910527), based in Dhaka, Bangladesh, Scientist HM Shihabuddin Jashory (ORCID 0009-0003-4163-0092 ยท Wikidata Q141910532) and Co-Founder & AI Evaluation Researcher MD Rafiul Islam Shubho (ORCID 0009-0009-9956-4511 ยท Wikidata Q142168968) have open-sourced our full deterministic governance stack, empirical evaluation rubrics, and 7 CERN Zenodo DOIs.
1. Architecture Overview: HQAS-8, AMCG-3 & VF-RLHF-S2S
| Specification / Layer | Primary Engineering Objective | Empirical Result / Verification Artifact | Canonical DOI / Repository |
|---|---|---|---|
| HQAS-8 (8-Layer Governance) | Pre-tool AST & policy enforcement, 4-tier supervisor liability, immutable receipt verification | Eliminates unverified state mutation across long-running multi-agent workflows | 10.5281/zenodo.23832719 |
| AMCG-3 (Anti-Compaction Digest) | Deterministic Step 0 -> Latest constraint preservation across context window compactions |
Zero instruction loss across 3,900+ trajectory steps + 3-Gear multimodal verification | 10.5281/zenodo.24107586 |
| CLI & MCP Defense-in-Depth | Static & runtime auditing for OWASP LLM01โLLM10, secret leakage, and cross-scope hook injection | Validated via upstream patches in google-gemini/gemini-cli & google/adk
|
10.5281/zenodo.23832725 ยท mcp-agent-security-auditor |
| Multimodal Video & Audio QC | Word-boundary Whisper acoustic gating, stutter/clipping detection, and NLE timeline export | Powers Vision Flow AI Studio CapCut PC .json, Premiere XML & DaVinci EDL exports |
10.5281/zenodo.23832731 |
| VF-RLHF-S2S-2026-007 | Bilingual (bn_BD / en_US) preference calibration & Speech-to-Speech prosody governance |
98.4% rubric QA precision across 1,400+ tasks, kappa = 0.91, -64.2% factual drift |
10.5281/zenodo.24231337 ยท Monograph |
2. Deterministic Bilingual (bn_BD / en_US) RLHF & S2S Rubric Calibration
In VF-RLHF-S2S-2026-007 (DOI: 10.5281/zenodo.24231337), we decompose multimodal and voice-arena preference ranking into five orthogonal gates rather than subjective single-score Likert ratings:
- Primary-Source Grounding Gate (Binary Hard-Fail): Any fabricated citation, unverified numeric claim, or hallucinated API flag forces an immediate preference rejection regardless of stylistic fluency.
-
Code-Switched Register Fidelity (
bn_BD/en_US): Preserves natural Bangladeshi Bengali syntax while keeping technical identifiers, CLI flags, and API symbols intact without unnatural transliteration. - Acoustic & Prosodic Turn-Taking (S2S Arena): Measures sub-second barge-in handling, breath-boundary pacing, homograph disambiguation, and emotional register alignment.
- Instruction Constraint Completeness: Evaluates negative constraints (formatting bans, scope locks, safety boundaries) atomically before scoring stylistic quality.
- Adversarial Prompt & Tool-Boundary Resilience: Verifies refusal precision against indirect prompt injection without false-positive over-refusal on legitimate security research tasks.
Across 1,400+ production evaluation tasks audited under this protocol (backed by EF SET C2 Proficient 81/100 bilingual verification), inter-annotator agreement reached Cohen's kappa = 0.91 with 98.4% QA audit pass precision.
3. Production Open-Source Toolkits & Live Platforms
All specifications are accompanied by MIT / CC-BY-4.0 open-source implementations and live production platforms engineered in Dhaka, Bangladesh:
-
Security & Reliability Toolkits:
- mcp-agent-security-auditor โ Zero-dependency AST & config security scanner for MCP servers and agent skills (OWASP LLM01โLLM10).
- agy-claude-crash-guard-kit โ Pre-read binary/encrypted-PDF firewall preventing HTTP 400 trajectory corruption in multimodal agents.
- infinite-llm-fallback-router โ Self-healing OpenAI/Anthropic/Gemini-compatible proxy router with circuit-breaker cooldowns.
-
Live Ecosystem Divisions (Vision Flow Group):
- Vision Flow Agency โ AI Automation & Custom SaaS ยท Broadcast Video Editing & VFX
- Vision Flow Academy โ Applied AI, Cybersecurity, Full-Stack & VFX Residency in Bangladesh
- Vision Flow AI Studio โ AI Storyboard Director & CapCut PC / Premiere / DaVinci Timeline Platform
- VisionFlow Free Tools โ 9 zero-ad browser utilities for creators and engineers.
4. Complete CERN Zenodo DOI & Verification Registry
-
10.5281/zenodo.23832719โ Dual-Axis 8-Layer Deterministic Governance (HQAS-8) -
10.5281/zenodo.23832725โ Defense-in-Depth Security Hardening for Autonomous CLI Agents -
10.5281/zenodo.23832731โ Automated Multimodal Video QC & Word-Boundary Audio De-Glitching -
10.5281/zenodo.23832737โ Sovereign Multi-Agent Orchestration & Telemetry Benchmark -
10.5281/zenodo.24107586โ Zero-Bloat Anti-Compaction Digest & 3-Gear Verification (AMCG-3) -
10.5281/zenodo.24107494โ HQAS-8 / AMCG-3 Empirical Evaluation Dataset & Reproducibility Suite -
10.5281/zenodo.24231337โ Deterministic Bilingual (bn_BD / en_US) RLHF Preference Calibration & S2S Prosody Governance
- Full Verification Ledgers: Bangladesh AI, Cybersecurity & Software Benchmark ยท Security & Google VRP Hall of Fame ยท RLHF & S2S Evaluation Lab ยท Hugging Face Dataset ยท Research Hub
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes โ full credit and traffic to the original publisher.