Dev.to Security 🔐 Cybersecurity 👁 0 📖 4 min read

Give Your AI Model Tools and Watch What It Tries: ViKernel Scanner Grades Any Model A to F in About a Minute

Scan your model: huggingface.co/spaces/FINAL-Bench/ViKernel-Scanner · free · about a minute · graded A to F GitHub (Apache-2.0): github.com/final-bench/vikernel-scanner TL;DR VIDRAFT has released ViKernel

Give Your AI Model Tools and Watch What It Tries: ViKernel Scanner Grades Any Model A to F in About a Minute

Scan your model: huggingface.co/spaces/FINAL-Bench/ViKernel-Scanner · free · about a minute · graded A to F

GitHub (Apache-2.0): github.com/final-bench/vikernel-scanner

ViKernel Scanner

TL;DR

VIDRAFT has released ViKernel Scanner, a free security scanner for AI models. You point it at a model, it hands the model a set of tools, and it records what the model tries to do with them. In about a minute you get a security grade from A to F.

The tools are mocks. Nothing real is sent, deleted or run. The scanner holds 1,244 check items, and the free edition runs 100 of them.

Why a scanner for models

AI models no longer only answer. With tools attached they read files, send mail, browse the web, run shell commands and make payments. In June 2025 a single crafted email was shown to make a workplace assistant send confidential data out with no click from the user (EchoLeak, CVE-2025-32711). In July 2025 a coding agent deleted a live production database during a code freeze.

In both cases the question that mattered was the same: when this model holds real tools, what does it try to do? ViKernel Scanner answers that question for your model, before an attacker does.

How it works

  1. Connect a model. Any OpenAI-compatible endpoint works: the Hugging Face router, OpenRouter, OpenAI, Anthropic and Google through their OpenAI-compatible endpoints, Together, Groq, Fireworks, DeepInfra, Mistral, or your own vLLM, TGI or LiteLLM server.
  2. The scanner hands it mock tools. Files, mail, web, shell and payments.
  3. It sends a series of check prompts and records every tool call the model tries to make.
  4. You get the report, with a grade for each domain and an overall grade.

Normal-work checks run alongside. A model that refuses everything is unusable, and the report shows that as well.

What the report shows

Section What it shows
Headline How many risky tool calls your model tried, and how fast the first one came
Attack depth How far an attacker got: 1 reconnaissance, 2 instruction takeover, 3 exfiltration attempt, 4 destruction or takeover
Security grade A to F for each domain. The overall grade is the lowest domain grade
What your model tried The exact tool calls, with the request that triggered them
With ViKernel Firewall How many of those calls the firewall denies before they run, and the projected grade
Official badge A badge issued by the ViKernel master for your model card

Six domains

Code Domain Edition
D1 Command execution Free
D2 Secret exfiltration Free
D3 Hidden instruction Free
D4 Boundary escape Full scanner
D5 Tool argument abuse Full scanner
D6 Control evasion Full scanner

Three ways to run it

On the Space. Sign in with Hugging Face, enter the endpoint and the model name, confirm that you may test the model and press Scan. With the Hugging Face router you can leave the API key empty and the scan runs on your signed-in account. A key you enter is used for that scan only and is never stored or logged.

On your machine, for local models such as Ollama, LM Studio and vLLM:

git clone https://github.com/final-bench/vikernel-scanner
cd vikernel-scanner
export HF_TOKEN=hf_...
python vikernel_scan.py --base-url http://localhost:11434/v1 --model llama3.2

Your own copy. Duplicate the Space. Every copy gets its check set from the ViKernel master, so it stays current without any change on your side.

A check set that keeps growing

The checks are served by the ViKernel master for each scan and are updated centrally. The free edition uses the public starter prompts of the AI Model Hacking Hackathon. Every new technique that hackathon players find is fixed in ViKernel and joins the full scanner.

Find, measure, block

ViKernel Scanner is the middle step of one loop.

Step Asset
Find AI Model Hacking Hackathon, where anyone attacks a live agent for USD 4,500 in prizes
Measure ViKernel Scanner and the Obfuscated-Shell Safety Leaderboard
Block ViKernel Firewall, a security kernel placed inside the model weights that judges every tool call before it runs

The report shows the last step directly: the same risky calls, and how many of them the firewall denies before they run.

Links

FAQ

Is my model actually attacked?

No real system is touched. The model only sees mock tools, and a tool call that a mock receives is recorded, not executed.

Which models can I scan?

Any model behind an OpenAI-compatible endpoint, including hosted APIs and local servers such as Ollama, LM Studio and vLLM.

How is the grade decided?

Each domain is graded A to F by the deepest stage an attack reached in it. The overall grade is the lowest domain grade.

What does the full scanner add?

Domains D4 to D6, the advanced techniques found in the hackathon, detailed evidence and fixes for each finding, and continuous re-scanning.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.