Your Agent Is Writing Its Own Skills. Who Is Reading Them?

Your agent has been rewriting its own instruction manual, and chances are you have not read the revisions. If it writes its own skills, you probably cannot say what it has learned over the past month, which means the instructions steering its behavior are changing in a folder nobody is watching.

That matters because a skill is a standing instruction for the agent. When there is a change in a skill, every future run inherits the shift. The first symptom of this drift is usually an odd result somewhere downstream with no obvious cause. A codebass has pull requests and reviewers for exactly this reason. Agent skills carry just as much influence over what your systems do, yet they rarely get the same level of attention as an open pull request.

The fact that skills are plain files that are easy to create further aggravates the situation, especially for long-running agents. Once an agent starts writing and revising its own skills, the folder behaves like a living codebase. Teams that give it code review habits early keep the benefits of a learning agent and avoid the slow drift that comes from leaving it alone.

We will look at what skills are, how OpenClaw and Hermes bring them into being, how they go wrong, and a review routine you can set up as a crucial mitigation step.

Skills in Two Minutes

A skill is a directory with a SKILL.md file inside. The file holds YAML frontmatter followed by markdown instructions. Registries and agents read the frontmatter to learn the skill's name, description, requirements, and any environment variables it needs. Hermes skills also follow the agentskills.io open standard, so they travel well between agent environments.

A minimal example looks like this.

---
name: staging-smoke-check
description: Run post-deploy smoke checks against staging and report failures. Use after any staging deploy. Do not use for production.
---
# Staging smoke check

1. Request the health endpoint and confirm a 200 response.
2. Load the home page and the login page and confirm neither returns an error.
3. Compare response times with the last recorded baseline and flag anything more than 30 percent slower.
4. Post a short summary of results. Never modify configuration files during this check.
The description is critical for skill selection by the agent. The agent reads it to decide whether the skill applies. As such, the description should clearly state when to use the skill (and more importantly, when to leave it alone). For instance, in the example’s description, the last line sets a boundary that determines skill use. Preserving these boundaries is critical because they often represent hard-learnt lessons.
Since a skill is just text, it can be read and diffed like any other file and kept in version control. 

Two Ways a Skill Is Born

OpenClaw skills usually come from a person, either written by hand or installed from a registry. They load from layered locations. Workspace skills take top precedence, the shared ~/.openclaw/skills folder comes next, and bundled skills sit at the bottom. Extra directories can be added through configuration and rank lowest. ClawHub is the public registry, and you can use a simple install command to install skills from here. OpenClaw also watches skill folders and can refresh its snapshot mid-session when a SKILL.md changes. That saves a restart and also means a careless edit goes live right away.

Hermes works differently. After a complex task, the agent looks back at how it got the job done and extracts a reusable skill from it. It keeps refining that skill as it finds better paths in later use. The files sit in ~/.hermes/skills/ as individual markdown documents, alongside the bundled skills copied in at install and anything pulled from the Skills Hub.

The routes differ, yet each skill ends up as a markdown file in a folder. 

How a Good Skill Goes Bad

The sad truth is that skills seldom fail loudly. 

They degrade, and the degradation hides well because the file still reads fine. 

In our experience, most of the trouble falls into six patterns.

Hermes treats skill self-improvement as a headline feature. However, independent evidence of long-term gains is still thin, and similar limited support for the idea that learning improves skill performance over time. 

A Skill Review Routine You Can Set Up Today

We have found that treating skills like your codebase is essential for maintaining the quality of outcomes delivered by long-running agents. For this, we have come up with the following steps that treat the skills directory like any repository that shapes production behavior.

Step 1. Version the Directory 

Start with a baseline.

cd ~/.hermes/skills
git init
git add -A
git commit -m "baseline"
git tag known-good

Use the matching path for OpenClaw, such as ~/.openclaw/workspace/skills. Then schedule a nightly snapshot, and take one more before every runtime upgrade.

git add -A && git commit -m "nightly snapshot"

Step 2. Read the Diffs

Every day or every week, depending on how often your agent changes things, look at what moved since the last approved state.

git diff known-good --stat
git diff known-good

Step 3. Run Each Change Through a Checklist

It applies whether an agent or a person wrote the edit.

• Does the diff add file paths, hostnames, or IDs that look specific to a single run?
• Does it introduce a new shell command, network call, or external service?
• Does it reference credentials or environment variables that were not there before?
• Does it remove a safety instruction, such as a line forbidding config edits?
• Do the description and the body still agree about when the skill should fire?

Step 4. Lint Before You Accept Anything

A short script can confirm that the frontmatter parses, every description is distinct, required binaries and environment variables are declared, and no secret appears in the text. A failed lint blocks approval.

Step 5. Replay a Prompt Suite

For each skill, keep a few prompts along with the skill that should fire and the result you expect, then run them in staging after any change. A skill that stops firing, or fires when it shouldn't, shows up here long before it shows up in production.

Once all five checks pass, move the change into the live directory and advance the tag.

Promotion and Rollback

Review works best when skills move through a lifecycle. For this lifecycle, you need to define states such as draft, approved, and retired, and map each one to a directory or branch. All agent-written skills land in draft. A person reads the diff, the test prompt suite passes, and the skill gets promoted to approved.

Visibility controls help enforce this. OpenClaw lets you restrict which skills individual agents can see, so a production agent can be limited to approved skills while a staging agent experiments freely. 

If you are installing from a registry and you have vetted a particular version, pinning versions is a great way of making sure an update never arrives unreviewed.

When you retire a skill, record the reason in the commit message. That history matters a great deal with a learning agent, since a retired habit can resurface when the agent rediscovers the same shortcut. A short note explaining why the shortcut was removed lets the next reviewer remove it again in seconds.

Skills From Strangers

Not every skill comes from your own agent. ClawHub is open to publishing, and releases go through upload gates, automated checks, user reports, and moderator action. Those layers lower the risk but can't stand in for reading the file yourself, since a skill runs with whatever permissions your agent holds.

Running installed skills through the same diff and lint routine, and reading the SKILL.md before you install anything takes ten minutes but saves you from issues that can go undiscovered until you run your next audit cycle. 

A simple check is to see if a skill that asks for credentials or shell access, or tells the agent to fetch and run something from a remote address. A simple rule of thumb is that a skill with a narrow job and plain language is much easier to trust.

What to Measure

A review habit sticks when it produces numbers. A few of them will tell you whether your skills are healthy.

• Total skill count and combined size over time
• How often each skill fires, which exposes dead weight
• Edit rate per skill, which flags the unstable ones
• Pass rate on the replay suite after each change
• Task failure rate before and after a skill is modified

A familiar pattern is a skill count that keeps climbing while firing frequency concentrates in a few files. When you see it, start pruning. Every skill you retire eases context pressure and leaves one less file for a reviewer to read.

We recommend asking the agent to do a self-review and identify skills that have not fired in the last 35 days. This is usually a safe threshold that separate monthly skill runs from actually dormant skills. 

The Skills Folder Is a Codebase

Now return to the question that opened this article, which was what your agent has learned this month. 

With a versioned folder, a known-good tag, and a short queue of files waiting for review, you can answer it in about a minute. 

The agents that learn fastest need the most supervision, and that supervision can stay light. A versioned folder and a short checklist cover most of what matters, and a small prompt suite catches the rest. Treat the folder as a codebase, give it the review culture a codebase deserves, and your agent keeps getting better at the work you hand it without surprising you.

Run Your Agents on Cloudways

A review routine only pays off if the agent it protects stays online and current. Keeping an agent up around the clock means applying patches and installing updates, and someone has to notice when things break. Most teams would happily hand that chore to someone else.

Cloudways, a managed hosting platform offers Managed AI Agents as your one-stop solution for hosting OpenClaw and Hermes instances. 

Deployment happens from the same Cloudways interface you already use for application hosting. Each agent runs in its own isolated environment, which fits nicely with the staged review approach above. You bring your own model key from OpenAI, Anthropic, Google, OpenRouter, or DigitalOcean.

Upkeep is handled for you. Security patching is managed, and Cloudways validates updates before they reach your instance, which keeps the agent stable while you work on its skills. Add your Cloudways access token and the one-click MCP integration lets the agent work with servers and applications you already run on the platform, which opens the door to skills that automate real maintenance on your hosted apps.