Ensuring the reliability of custom skills for Anthropic Claude Code AI agents presents a significant challenge: these Markdown files, incorporating YAML frontmatter, are prone to silent failures that can go unnoticed for extended periods. A developer recently shared their experience addressing this pain point with `pulser eval`, a new tool integrated into GitHub Actions.
The problem became evident when a missing `name` field in one skill file within a shared repository silently disabled 14 skills for a week. Users simply assumed Claude lacked the capability, and the team spent hours debugging before tracing the issue to a minor YAML syntax error.
Silent failures manifest in several ways. A missing `name` field means Claude won't load the skill, offering no error or warning. A vague `description`, such as "useful for various tasks," prevents Claude from effectively triggering the skill. Furthermore, malformed YAML frontmatter (e.g., forgotten closing `---`, incorrect indentation, or unquoted colons) can lead to the file being parsed as raw Markdown, rendering the skill body invisible.
To tackle these issues, the developer built `pulser eval`, a zero-dependency CLI tool designed to scan Claude Code skill files and report structural problems before they reach production. It can process over 40 skill files in under 200 milliseconds, providing a clear pass/fail output.
`pulser eval` performs a comprehensive suite of checks:
- **YAML frontmatter parsing:** Catches syntax errors, missing delimiters, and type mismatches.
- **Required field validation:** Ensures essential fields like `name` and `description` exist and are non-empty.
- **Description quality scoring:** Flags vague descriptions that hinder Claude's activation logic.
- **File structure analysis:** Detects orphaned files, empty skill bodies, and issues with naming conventions.
By wrapping `pulser eval` in a GitHub Action, the team integrated it into their CI/CD pipeline. This ensures that any skill breakage is caught automatically before code merges. In its first week of deployment, the system identified 23 issues that would otherwise have been silently deployed, significantly improving the robustness of AI agent skills and development efficiency.