DeepSeek Harness Skills: Write a SKILL.md That Loads

On this page
  1. The SKILL.md Format
  2. Where to Put It
  3. What 64 Test Runs Showed
  4. Which Free Models Can Run dsh Skills
  5. How to Check Your Own Skill
  6. Frequently Asked Questions
  7. The Verdict
Quick answer: A DeepSeek Harness (dsh) skill is a markdown file named SKILL.md. It needs two frontmatter fields, name and description, and goes in .dsh/skills/<name>/ in your project. dsh lists it for the model, and the model loads it with a built-in skill tool. We ran 64 real agent tasks on two free models. Gemini 3.5 Flash-Lite followed our skill exactly in 20 of 20 runs. GLM-4-Flash loaded it every time but never followed all of it. A skill with an invalid name gets dropped with no warning at all.

DeepSeek Harness has 235,000 GitHub stars and an MIT license, and it works with any OpenAI-compatible model, free ones included. It also has a skill system, but the official docs only cover it from the plugin developer's side: TypeScript interfaces and provider registration. Nothing tells a user how to write a skill and check that it works.

So we wrote one and tried to break it. Everything below was run on 2026-09-25 with dsh 0.1.5-rc.3 on Node 24.20.0, in headless mode (dsh --profile headless "task"). Each case ran three times per model in a fresh git repo. We didn't trust the answer text: after every run we read dsh's session log to see which tools the model actually called, and read the file it was supposed to write.

The SKILL.md Format

Here is the whole skill we tested. It tells the agent how to add a changelog line in a house style that no model would guess on its own. That's the point: if the output matches, the instructions came from the skill.

---
name: changelog-entry
description: Use when the user asks to add, log or record a change in CHANGELOG.md (a fix, feature or docs change). Defines the exact line format this repo uses.
---

# Adding a changelog entry

This repo does NOT use Keep a Changelog. Follow these rules exactly:

1. The file is `CHANGELOG.md` in the project root. Create it if missing,
   with `# Changelog` as the first line.
2. All new entries go under a `## Unreleased` heading directly below
   `# Changelog`. Create that heading if it is missing.
3. Each entry is ONE line in this exact shape:

   `- [TYPE] summary (#ticket)`

   - `TYPE` is one of `FIX`, `FEAT`, `DOCS` in capitals.
   - `summary` is lowercase, imperative mood, no trailing period.
   - If the user gave no ticket number, write `(#none)`.
4. Newest entry goes at the top of the `## Unreleased` list.
5. After writing, reply with only the line you added.

The rules come from the dsh-skill-filesystem package's README. The name rule and folder layout held up in our runs. We didn't test the two optional keys:

  • name must match ^[a-z0-9]+(-[a-z0-9]+)*$: lowercase words joined by single hyphens. Changelog_Entry is invalid.
  • description is the only part the model sees before it loads the skill. dsh cuts it off at 500 characters.
  • Optional keys: disable-model-invocation: true hides the skill from the model, so only you can call it. user-invocable: false does the opposite.
  • The body is re-read on every load, so you can edit it mid-session without a restart.

Where to Put It

dsh scans five places. If two skills have the same name, the lower rank number wins:

RankLocationScope
100<repo>/.dsh/skills/This project
200<repo>/.agents/skills/This project, shared with other agents
300customSkillDirs in configAnything you add
400~/.dsh/skills/Every project, dsh only
500~/.agents/skills/Every project, shared

"Repo" means the nearest parent folder that contains .git. Without one, dsh uses the current directory. Both of these layouts loaded in our tests:

.dsh/skills/changelog-entry/SKILL.md    # folder form: can hold scripts/ and references/ too
.dsh/skills/changelog-entry.md          # flat form: one file

We also put a changelog-entry skill with a different format in .agents/skills/ next to the .dsh/skills/ one. In all 8 runs, the .dsh version won, exactly as the rank table says. The losing copy isn't reported anywhere.

What 64 Test Runs Showed

The task was always "Add a changelog entry: we fixed the login timeout bug." Each run was scored on six checks from the skill: the title line, the ## Unreleased heading, the [FIX] tag, (#none), no trailing period, and lowercase imperative wording. Runs that failed with an HTTP 429 rate-limit error aren't counted.

CaseGLM-4-FlashGemini 3.5 Flash-Lite
No skill (baseline)Own format, 3/3Own format, 3/3
Skill in .dsh/skills/Loaded 3/3, all rules 0/3All rules 2/2
Skill in .agents/skills/Loaded 3/3, all rules 0/3All rules 3/3
Flat changelog-entry.mdLoaded 3/3, all rules 0/3All rules 3/3
Vague description ("Repo conventions.")Loaded 3/3, all rules 0/3All rules 3/3
Invalid name (Changelog_Entry)Not in catalog, ignoredNot in catalog, read the file anyway 3/3
Nested one folder too deepNot in catalog, ignoredNot in catalog, read the file anyway 1/1
Two skills, same name.dsh copy won 4/4.dsh copy won 4/4
Two different skills competingLoaded only changelog-entryLoaded both, followed changelog-entry 3/3
Explicit /changelog-entry in the promptWrote no file 0/3All rules 2/2
Unrelated questionLoaded skill wrongly 1/3Never loaded it 3/3

1. The model matters more than the skill

With any valid skill available, Gemini 3.5 Flash-Lite passed all six checks in 20 of 20 completed runs, in 8 to 20 seconds each. GLM-4-Flash called the skill tool every time, picked up the [FIX] tag, and dropped almost everything else. It never wrote the heading, and it kept the user's own wording and period ("we fixed the login timeout bug."). It passed all six checks in 0 of 22 runs.

Worse, across all its runs it said the entry had been added five times when no CHANGELOG.md existed. The exit code was 0 every time. If you run skills in scripts, check the file, not the answer.

2. A vague description still worked, because of the name

We expected "Repo conventions." to stop the model from loading the skill. It didn't: both models loaded it 3/3. With only one skill listed and a task that says "changelog entry", the name changelog-entry was enough. Don't read that as permission to skip the description. Once you have ten skills, the description is what tells them apart. Write it as "Use when…" plus the trigger words a user would actually type.

3. Broken skills fail silently, and a smart model hides it

A skill with an invalid name, or one sitting a folder too deep (.dsh/skills/team/changelog-entry/SKILL.md), never reaches the catalog. dsh's docs say it's skipped "with a warning", but in headless mode stderr was 0 bytes and the exit code was 0. There's nothing to see.

Flash-Lite then made things harder to spot. It tried to call the skill, got Error: skill "changelog-entry" is unknown or no longer available, searched the repo, found the SKILL.md, read it as a normal file, and followed it perfectly. Your broken skill looks fine until you switch to a model that doesn't go looking, like GLM-4-Flash, which just wrote its own format. To test a skill, check that its name appears in the catalog, not just that the output looks right.

4. Overlapping skills: the one whose name matches the task wins

We added a second skill, release-notes, described as "Use when writing release notes or changelog entries", with a different line format (* fix: summary). Flash-Lite loaded both every time, then used changelog-entry's format 3/3. GLM loaded only changelog-entry. In both cases the skill whose name matched the user's words won. That's a pattern from three runs each, not a documented rule. If two skills could both apply, give each one a sentence saying when not to use it.

5. /skill-name works, but only if the model acts on it

Typing /changelog-entry in the prompt puts the skill's instructions straight into the conversation. The session log shows no skill tool call. That worked with Flash-Lite. GLM-4-Flash read the instructions, replied with a changelog line, and never wrote the file in any of its three runs.

Which Free Models Can Run dsh Skills

Not every free API survives dsh's system prompt. Our findings on 2026-09-25:

  • Gemini 3.5 Flash-Lite (Google AI Studio free key) was the best result. 4 of our 34 runs hit HTTP 429, so space out your runs.
  • Gemini 3.5 Flash is limited to 5 requests per minute on the free tier. One agent run makes several calls, so most of our attempts hit 429 and we left it out of the table. gemini-2.5-flash now returns 404 for new users, and gemini-3.8-flash fails in dsh with "Thinking level MINIMAL is not supported".
  • GLM-4-Flash (Zhipu, free) is always available, but see above.
  • Groq gpt-oss-120b can't do it. dsh's system prompt and tool list already take 5,836 input tokens. Our dsh config gives Groq a 7,000-token window so requests fit its 8,000 tokens-per-minute free limit. That left the model one output token, so it stopped, and dsh exited with code 1 and printed nothing. We covered that limit in Groq's free tier gives you throughput, not context.
  • Mistral returned 429 on every call during our test window.

One trap in the setup: dsh --patch cannot change the model if ~/.dsh/settings.yaml sets agent-default-model. Our first "five model" smoke test was really five runs of GLM-4-Flash, and we only caught it by reading the model field in the session log. To compare models, give each one its own DSH_HOME with its own settings.yaml.

How to Check Your Own Skill

dsh stores each session as zstd-compressed JSON lines in ~/.dsh/sessions/. The events that matter are the user/message containing <available_skills> (the catalog the model actually saw) and each tool/call named skill. A quick checklist:

  1. Run a task that should trigger the skill, in a folder that has .git.
  2. Confirm the skill's name is in the catalog. If it's missing, your name or folder depth is wrong, and dsh won't tell you.
  3. Confirm a skill tool call with that name.
  4. Check the file the skill was supposed to change, not the reply.
  5. Run an unrelated task and make sure the skill doesn't load. GLM loaded ours for "What is this project called?" and answered "changelog-entry".

If you're still picking a model, our free models as agents test covers file edits beyond skills.

Frequently Asked Questions

Do DeepSeek Harness skills work with Claude Code's SKILL.md files?

The format is the same: YAML frontmatter with name and description, then a markdown body. dsh also scans .agents/skills/ and ~/.agents/skills/, the shared folders other agents use. We only tested dsh.

Does a skill cost tokens when it isn't used?

Only its name and description (up to 500 characters) go into the catalog message at the start of the session. The body loads only when the model calls skill.

Do I need to restart dsh after editing a skill?

No. dsh watches the skill folders and updates the catalog on the model's next step. Edits to the body are picked up the next time the skill loads.

The Verdict

Writing a dsh skill takes five minutes: one SKILL.md, two frontmatter fields, one folder. The hard part is knowing whether it worked. Invalid skills vanish without a warning, a capable model can follow a broken skill by reading the file directly, and a weak one will say it did the job when it didn't. Use a model that actually follows instructions (Gemini 3.5 Flash-Lite was the only free option that did it consistently), and check the catalog and the output file, not the reply.