Lessons Learnt in AI Adoption

I’ve been working with AI coding agents seriously for a while now. This is a collection of things I’ve learnt along the way — some of it figured out, some of it still in progress.

Agent skills

The first wave of AI adoption is usually about picking a tool and getting it to do useful things. That part is well-covered elsewhere. What’s less talked about is what happens after — once the novelty wears off and you start noticing the same friction every day.

The agent doesn’t know your codebase, your conventions, or how your team operates. You end up repeating the same context every session.

This is where agent skills come in. A skill is a reusable artefact the agent can reach for when a recognisable situation arises. The mental model I find useful is a library: you don’t copy utilities into every file — you extract them, give them a clean interface, and import them where needed. Skills work the same way. They’re packaged knowledge the agent can invoke rather than rediscovering from scratch each time.

To understand how all the pieces fit together, here’s a rough picture:

The LLM reasons and decides — but it doesn’t act. The agent is the runtime that acts on those decisions. Tools are primitive, atomic operations. Skills sit above that — they’re higher-level processes that can compose multiple tool calls and MCP calls together. That composability is what makes them useful.

The key design idea is progressive disclosure. The agent holds a list of skill descriptions — what each skill does and when to use it. The LLM decides which skills are relevant for a given task and loads only those. Everything else stays out of the context window. Same principle as lazy loading.

One thing I got wrong early: I tried to design skills upfront. I’d think of a problem, imagine the skill, write the spec. That didn’t work well.

What works better is to start with the actual problem. Collaborate with the agent on solving it. Let it explore the available tools on connected MCPs, build scripts, try approaches. Once a pattern repeats — once you notice the agent reaching for the same sequence of steps — that’s when you extract a skill. The skill becomes the distilled version of what you figured out together.

How do I test my skills?

Honest answer: I’m still working this out.

The intuition is clear. A skill is code — or at least structured instructions — and code should be tested. You want to know if it triggers in the right situations, produces useful output, and doesn’t regress when you update it.

But the mechanics are harder than unit testing. Skills invoke an LLM, so the same input doesn’t always produce the same output. “Testing” a skill is closer to running evals: you define scenarios, run against them, and assess whether the outcomes are good enough, not whether they’re identical.

I’m currently thinking about it in terms of trigger accuracy (does the agent call this skill when it should?), output quality (is what it produces actually useful?), and regression coverage (do I know if I’ve broken something after an update?). But I don’t have a clean answer on implementation yet.

If you’ve solved this better than I have, I’d genuinely like to know.

Sharing skills with the team

This one I have a clearer view on, because the software world has already solved the same problem for libraries.

When you write something useful, you don’t keep it on your machine. You publish it — versioned with semver, with a clear interface, somewhere the rest of the team can depend on it. A 1.0.0 is stable. A 2.0.0 signals a breaking change. A patch is safe to pick up.

Skills should work the same way. The artifact is the skill folder. The publish step is committing it to a shared repository your team’s agents can pull from. When a senior engineer figures out a better way to handle a recurring task, that knowledge shouldn’t stay in their session history. It should become a versioned, reviewable skill.

The interesting implication: agent skills are a form of team knowledge — as much as your internal libraries or your runbooks. They deserve the same care. Review them, version them, maintain them.

Managing skills locally

Different tools look for skills in different places. Claude uses ~/.claude/skills, Cursor uses ~/.cursor/skills. If you use more than one agent — and most people do — you end up maintaining the same skills in multiple locations. That gets messy fast.

Skills are also shared in different ways. The most common is a git repository you clone. Your team might have one, or you might pull in skills from the community.

My approach: keep a single source of truth at ~/.agents/skills, and symlink from there into wherever each tool expects to find them.

For project-specific skills, I keep them in the repo itself under .agents/skills/, then link them into the central location on setup:

for skill_dir in $PROJECT_ROOT/.agents/skills/*/; do
  skill_name="$(basename "$skill_dir")"
  ln -snf "$skill_dir" "$HOME/.agents/skills/$skill_name"
  echo "linked: $skill_name"
done

This way the skill lives with the project, is versioned alongside the code, and is still accessible to any agent tool on the machine. Updating the skill in the repo automatically updates what the agent sees — no manual syncing.

It’s a small thing, but having one place to look makes the whole system feel more manageable.

Where I’m at

The tools will keep changing. The best option today probably won’t be the best option in six months.

What’s stayed useful is the underlying approach: pick tools based on what they’re actually good for, build reusable knowledge rather than starting from scratch every session, and treat your skills like the engineering artefacts they are.

Still figuring a lot of this out.