Lessons Learnt in AI Adoption

I have been working with AI coding agents seriously for a while now. In this post I’ll collect the things I have learnt along the way; Some of it is settled, some of it I am still working out.

Agent skills

The first wave of AI adoption is usually about picking a tool and getting it to do something useful. That part is covered well elsewhere. What gets less attention is what happens after, once the novelty wears off and you start noticing the same friction every day.

The agent doesn’t know your codebase, your conventions, or how your team operates, so you repeat the same context every session.

A skill is a reusable artefact the agent can reach for when a recognisable situation comes up. The analogy I find useful is a library. You don’t copy utilities into every file, you extract them, give them a clean interface, and import them where needed. A skill is packaged knowledge the agent can invoke instead of rediscovering it from scratch each time.

Here is roughly how the pieces fit together:

The LLM reasons and decides, but it doesn’t act. The agent is the runtime that acts on those decisions. Tools are primitive, atomic operations. Skills sit above them; They are higher level processes that compose multiple tool calls and MCP calls together, and that composability is what makes them useful.

The key design idea is progressive disclosure. The agent holds a list of skill descriptions, i.e. what each skill does and when to use it. The LLM decides which ones are relevant for a given task and loads only those; Everything else stays out of the context window. Same principle as lazy loading.

One thing I got wrong early on. I tried to design skills upfront: think of a problem, imagine the skill, write the spec. That did not work well.

What works better is starting from the actual problem and collaborating with the agent on solving it. Let it explore the tools on connected MCPs, build scripts, try approaches. Once you notice the agent reaching for the same sequence of steps, that is the point to extract a skill. The skill is the distilled version of what you worked out together.

Testing skills

I am still working this one out.

The intuition is clear enough. A skill is code, or at least structured instructions, and code should be tested. You want to know whether it triggers in the right situations, produces useful output, and doesn’t regress when you update it.

The mechanics are harder than unit testing. Skills invoke an LLM, so the same input doesn’t always produce the same output. Testing a skill is closer to running evals: you define scenarios, run against them, and assess whether the outcomes are good enough rather than whether they are identical.

I am currently thinking about it in three dimensions: trigger accuracy (does the agent call this skill when it should?), output quality (is what it produces actually useful?), and regression coverage (do I know if I have broken something after an update?). I don’t have a clean answer on implementation yet. If you have solved this better than I have, I would like to hear about it.

Sharing skills with the team

This one is clearer, because the software world has already solved the same problem for libraries. When you write something useful you don’t keep it on your machine; You publish it, versioned with semver, with a clear interface, somewhere the rest of the team can depend on it. A 1.0.0 is stable, a 2.0.0 signals a breaking change, and a patch is safe to pick up.

Skills work the same way. The artifact is the skill folder, and the publish step is committing it to a shared repository that your team’s agents can pull from. When a senior engineer works out a better way to handle a recurring task, that knowledge should not stay in their session history.

Agent skills are team knowledge, the same as your internal libraries and your runbooks. Review them, version them, maintain them.

Managing skills locally

Different tools look for skills in different places. Claude uses ~/.claude/skills, Cursor uses ~/.cursor/skills. If you use more than one agent, and most people do, you end up maintaining the same skills in several locations, which gets messy fast.

Skills are also shared in different ways. The most common is a git repository you clone; Your team might have one, or you might pull some in from the community.

My approach is to keep a single source of truth at ~/.agents/skills, and symlink from there into wherever each tool expects to find them.

For project-specific skills, I keep them in the repo under .agents/skills/, then link them into the central location on setup:

for skill_dir in $PROJECT_ROOT/.agents/skills/*/; do
  skill_name="$(basename "$skill_dir")"
  ln -snf "$skill_dir" "$HOME/.agents/skills/$skill_name"
  echo "linked: $skill_name"
done

The skill then lives with the project, is versioned alongside the code, and is still visible to any agent tool on the machine. Updating it in the repo updates what the agent sees, with no manual syncing. It is a small thing, but having one place to look makes the whole setup easier to manage.

Where I’m at

The tools will keep changing; Today’s best option probably won’t be the best option in six months.

What has stayed useful is the approach underneath: pick tools for what they are actually good at, build reusable knowledge instead of starting from scratch every session, and treat your skills as the engineering artefacts they are.

Still working a lot of this out.