TICKET-611: Pass in optional batch_id to autotester

[Naragod]

Proposed Changes

Pass in optionalbatch_id to autotester for use in the ai-telemetry-gateway.

Screenshots of your changes (if applicable)

Associated documentation repository pull request (if applicable)

Type of Change

(Write an X or a brief description next to the type or types that best describe your changes.)

Type	Applies?
🚨 Breaking change (fix or feature that would cause existing functionality to change)
✨ New feature (non-breaking change that adds functionality)	X
🐛 Bug fix (non-breaking change that fixes an issue)
🎨 User interface change (change to user interface; provide screenshots)
♻️ Refactoring (internal change to codebase, without changing functionality)
🚦 Test update (change that only adds or modifies tests)
📦 Dependency update (change that updates a dependency)
🔧 Internal (change that only affects developers or continuous integration)

Checklist

(Complete each of the following items for your pull request. Indicate that you have completed an item by changing the [ ] into a [x] in the raw text, or by clicking on the checkbox in the rendered description on GitHub.)

Before opening your pull request:

• I have performed a self-review of my changes.
  • Check that all changed files included in this pull request are intentional changes.
  • Check that all changes are relevant to the purpose of this pull request, as described above.
• I have added tests for my changes, if applicable.
  • This is required for all bug fixes and new features.
• I have updated the project documentation, if applicable.
  • This is required for new features.
• If this is my first contribution, I have added myself to the list of contributors.

After opening your pull request:

• I have updated the project Changelog (this is required for all changes).
• I have verified that the pre-commit.ci checks have passed.
• I have verified that the CI tests have passed.
• I have reviewed the test coverage changes reported by Coveralls.
• I have requested a review from a project maintainer.

Questions and Comments

(Include any questions or comments you have regarding your changes.)

[coveralls]

Coverage Report for CI Build 30331322134

Coverage increased (+0.01%) to 90.479%

Details

• Coverage increased (+0.01%) from the base build.
• Patch coverage: 31 of 31 lines across 2 files are fully covered (100%).
• No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	51511
Covered Lines:	47635
Line Coverage:	92.48%
Relevant Branches:	2484
Covered Branches:	1219
Branch Coverage:	49.07%
Branches in Coverage %:	Yes
Coverage Strength:	128.28 hits per line

---

💛 - Coveralls

[donny-wong]

Hi @Naragod. Please review CI errors and update branch. thank you.

[Naragod]

@donny-wong please take another look

[donny-wong]

Thanks @Naragod — the batch_id change looks good and the specs cover both the batch and solo cases. Please update the branch and I'll approve.

[donny-wong]

@Naragod, I've noticed an issue with the usage logs table when running tests in MarkUs: the first run gets recorded, but subsequent runs don't show up at all — it's not intermittent, it consistently stops logging after the first call. Have you seen anything similar on your end? For context, I'm testing against the Azure Gateway platform.

[Naragod]

@Naragod, I've noticed an issue with the usage logs table when running tests in MarkUs: the first run gets recorded, but subsequent runs don't show up at all — it's not intermittent, it consistently stops logging after the first call. Have you seen anything similar on your end? For context, I'm testing against the Azure Gateway platform.

@donny-wong can you run this on your instance?

SELECT c.max_budget, SUM(u.total_cost) AS spent
FROM aitg.course_budgets c
LEFT JOIN aitg.usage_logs u USING (instance, course_id)
WHERE c.course_id = <your course>
GROUP BY c.max_budget;

If spent reaches max_budget, that explains the stop. The gateway turns down every call past the cap, meaning that the first run logs a row and the rest stop.

The cause stays hidden for a second reason. The gateway sends a plain 400 that names the budget. That 400 lands on the last line of a 60-line Python trace.

I have a fix in ai-autograding-feedback. Gateway failures now print one line:

Error: Course budget exhausted for course_id=1: spent CAD 0.02 of CAD 0.01.

A gateway that goes down prints one line too. It used to print its own trace.