BiBoyang/dsh-eval-harness
dsh-eval-harness is a tool for DeepSeek Harness. DSH plugin eval tool: YAML case-driven real agent regression eval + baseline comparison PASS/WARN/FAIL gates｜Regression eval harness for DeepSeek Harness plugins
REVIEW SCORECARD
✓ No risk detected · Automated scan results are for reference only, not an official endorsement
AI DEEP REVIEW
A well-engineered regression eval harness for DSH plugins with real-agent tracing, baseline gating, and CI exit codes; w
README documents a zero-dependency YAML subset parser, strict schemaVersion validation with legacy fallback, per-attempt result history, and unbiased pass@k/pass^k estimators rather than the biased (c/n)^k, indicating deliberate engineering rigor.
The tool spawns local dsh subprocesses with isolated session roots and workspaces, but the LLM judge sends outputs to an OpenAI-compatible endpoint configured via EVAL_JUDGE_API_KEY/DEEPSEEK_API_KEY environment variables, so no plaintext secrets or download-and-execute patterns are visible.
It addresses a concrete, common pain point for plugin and skill authors by turning YAML cases into real-agent regression runs with PASS/WARN/FAIL gating and CI exit codes, plus a judge-calibration tool reporting TPR/TNR.
Last updated 2026-08-26 with an active repository, a documented deprecation of the stale npm channel in favor of GitHub distribution, and schema versioning that anticipates future changes.
The README covers installation with a mount-verification command, full tool and skill tables, an annotated YAML case format, parameter tables with defaults, and explicit parsing limitations, though the excerpt is truncated mid-section on eval_judge_validate.
Generated by AI after reading the project README — a decision aid, not an endorsement. Neutral scores are given when information is limited.
Loading GitHub README...
The GitHub README is temporarily unavailable. Please try again later.
Open repositoryINSTALL REFERENCE
dsh plugin --profile headless add github:BiBoyang/dsh-eval-harnessThis command was extracted from an explicit README install reference and is display-only; the DSH host will not execute it automatically.
PROJECT TOPICS
VALIDATION LADDER
FAQ
DSH plugin eval tool: YAML case-driven real agent regression eval + baseline comparison PASS/WARN/FAIL gates｜Regression eval harness for DeepSeek Harness plugins
dsh-eval-harness has 13 stars and 1 forks on GitHub, last updated 2026-09-25.
dsh-eval-harness is listed in the DSH Universe directory as a tool for DeepSeek Harness. This plugin is listed in the DSH Universe directory and covered by its validation pipeline.
CLASSIFICATION EVIDENCE
The system checks GitHub Topics against the site taxonomy and keyword rules. Current matches:regression-testing。
nexu-io
🎨 Best DeepSeek Harness Design Plugin. The open-source Claude Design alternative. 🖥️ Local-first desktop app. 🖼️ Your coding agent becomes the design engine: prototypes, landing pages, dashboards, slides, images & video — real files, HTML/PDF/PPTX/MP4 export. 🤖 Claude Code / Codex / Cursor / DeepSeek Harness / OpenCode & 20+ CLIs via BYOK.
titanwings
Distilly — Distill how they think into reusable Skills for any Agent or Bot. Formerly Colleague Skill (originally a colleague Skill).
tt-a1i
Agent skill for beautiful, verifiable architecture, workflow, sequence, data-flow, and lifecycle diagrams—self-contained HTML with motion and crisp export.
YaoApp
✨ All your agents and workspaces in one place, on every device you own. Track tasks on a board, accessible from desktop, mobile, browser, or API. Self-hosted.
Devin-AXIS
Enterprise-grade, local-first Agent Workbench for people and agent teams. A unified multi-engine workspace for Codex Harness, DeepSeek Harness, and OpenCode, with unified plugins and Skills, multi-agent projects and tasks, and editable code, documents, presentations, design, and video.
MeteorNOX
DeepSeek Harness（DSH）A little whale girl living in the bottom-right corner of the DSH interface, watching your DeepSeek account balance. Bouncy, supports drag snapping, left-snap flipping, and rolling number animation, auto-enables with the interface, recommend directly summoning your dsh to install