feat: agl-skill - #534
Open
dalongbao wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an installable Agent Skills package (agl-optimizer) to the repository, along with a skills catalog README, so compatible agents can discover and load the skill via the Agent Skills format.
Changes:
- Added
skills/agl-optimizer/SKILL.mddefining theagl-optimizerskill (metadata + optimization guidance, action space, and boundaries). - Added
skills/README.mddocumenting the skill, installation options, and published evaluation context/results.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| skills/README.md | Introduces the skills catalog, installation instructions, and benchmark/result context for agl-optimizer. |
| skills/agl-optimizer/SKILL.md | Adds the Agent Skills spec file (frontmatter + content) for the agl-optimizer skill. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ultmaster
reviewed
Jul 31, 2026
| @@ -0,0 +1,90 @@ | |||
| --- | |||
| name: agl-optimizer | |||
ultmaster
reviewed
Jul 31, 2026
|
|
||
| Our evaluation uses a different protocol: Claude Code, Codex, and Copilot act as agentic optimizers that can edit the whole agent. Results are held-out test means pooled across \$5, \$10, and \$25 budgets (`n = 9`) and are shown as mean ± standard deviation. Higher is better; bold marks the better mean within each row. | ||
|
|
||
| | Benchmark (train/test) | Optimizer | agl-skill v9.7 (%) | No-skill control (%) | Change (pp) | |
ultmaster
reviewed
Jul 31, 2026
| Our evaluation uses a different protocol: Claude Code, Codex, and Copilot act as agentic optimizers that can edit the whole agent. Results are held-out test means pooled across \$5, \$10, and \$25 budgets (`n = 9`) and are shown as mean ± standard deviation. Higher is better; bold marks the better mean within each row. | ||
|
|
||
| | Benchmark (train/test) | Optimizer | agl-skill v9.7 (%) | No-skill control (%) | Change (pp) | | ||
| | :--- | :--- | ---: | ---: | ---: | |
ultmaster
reviewed
Jul 31, 2026
| | ALFWorld (3553/134) | Claude Code | **89.1** ± 26.4 | 88.2 ± 33.1 | +0.9 | | ||
| | ALFWorld (3553/134) | Codex | 97.5 ± 4.7 | **97.8** ± 5.9 | −0.3 | | ||
| | ALFWorld (3553/134) | Copilot | 97.5 ± 7.2 | **99.9** ± 0.3 | −2.4 | | ||
|
|
Contributor
There was a problem hiding this comment.
columns:
accuracy cost
rows:
before optimizer
claude code w/ skill
codex w/ skill
copilot w/ skill
ultmaster
reviewed
Jul 31, 2026
| | ALFWorld (3553/134) | Codex | 97.5 ± 4.7 | **97.8** ± 5.9 | −0.3 | | ||
| | ALFWorld (3553/134) | Copilot | 97.5 ± 7.2 | **99.9** ± 0.3 | −2.4 | | ||
|
|
||
| Because the two tables use different protocols and data splits, compare results within a table, not across tables. Full per-budget scores, cost breakdowns, and caveats are available in [RESULTS_V9_7.md](https://github.com/agent-lightning/agl-skill/blob/verifier-v8/RESULTS_V9_7.md). |
Contributor
There was a problem hiding this comment.
don't put links here. it won't be publicly accessible
ultmaster
reviewed
Jul 31, 2026
Author
|
@microsoft-github-policy-service agree company="Microsoft" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.