Input processor for removing empty toolcall list from input#4383
Input processor for removing empty toolcall list from input#4383przepeck wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds an input-processing step to sanitize chat messages by removing empty "tool_calls": [] entries from ov::genai::ChatHistory, preventing empty tool-call blocks from being rendered downstream (e.g., by chat templates).
Changes:
- Added
EmptyToolCallsArrayRemovingProcessorto remove emptytool_callsarrays from chat-history messages. - Wired the new processor into the chat-path
InputProcessorchain early in processing. - Added a new gtest suite covering empty vs non-empty
tool_callsbehavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/test/llm/input_processing/empty_tool_calls_array_removing_processor_test.cpp |
Adds unit tests for the new processor behavior. |
src/llm/io_processing/input_processors/empty_tool_calls_array_removing_processor.hpp |
Declares the new processor (doc comment currently mismatches implementation). |
src/llm/io_processing/input_processors/empty_tool_calls_array_removing_processor.cpp |
Implements removal of empty tool_calls arrays from ChatHistory. |
src/llm/io_processing/input_processor.cpp |
Inserts the new processor into the chat-path processing chain. |
src/llm/BUILD |
Adds the new processor sources/headers to the Bazel target. |
mzegla
left a comment
There was a problem hiding this comment.
Do we want this processor to be unconditional? Like not tied to chat template caps?
I wonder if resolving this won't introduce another problem when we get chat template that always expects something in the tool calls and empty array would be somewhat expected.
In that scenario usual curl request e.g. |
|
Okay, we can always adjust if such template appears. Let's move forward. |
🛠 Summary
CVS-190935
We are removing empty tool call lists from messages to prevent rendering it in input e.g.
It will help us, to not implement the same workaround in chat templates.
🧪 Checklist
``