Skip to content

Input processor for removing empty toolcall list from input#4383

Open
przepeck wants to merge 5 commits into
mainfrom
przepeck/empty_toolcalls_rm
Open

Input processor for removing empty toolcall list from input#4383
przepeck wants to merge 5 commits into
mainfrom
przepeck/empty_toolcalls_rm

Conversation

@przepeck

Copy link
Copy Markdown
Collaborator

🛠 Summary

CVS-190935
We are removing empty tool call lists from messages to prevent rendering it in input e.g.

[...]
<role>assistant
<|tool_call_start|>[]<|tool_call_end|>
[...]

It will help us, to not implement the same workaround in chat templates.

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an input-processing step to sanitize chat messages by removing empty "tool_calls": [] entries from ov::genai::ChatHistory, preventing empty tool-call blocks from being rendered downstream (e.g., by chat templates).

Changes:

  • Added EmptyToolCallsArrayRemovingProcessor to remove empty tool_calls arrays from chat-history messages.
  • Wired the new processor into the chat-path InputProcessor chain early in processing.
  • Added a new gtest suite covering empty vs non-empty tool_calls behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/test/llm/input_processing/empty_tool_calls_array_removing_processor_test.cpp Adds unit tests for the new processor behavior.
src/llm/io_processing/input_processors/empty_tool_calls_array_removing_processor.hpp Declares the new processor (doc comment currently mismatches implementation).
src/llm/io_processing/input_processors/empty_tool_calls_array_removing_processor.cpp Implements removal of empty tool_calls arrays from ChatHistory.
src/llm/io_processing/input_processor.cpp Inserts the new processor into the chat-path processing chain.
src/llm/BUILD Adds the new processor sources/headers to the Bazel target.

@mzegla mzegla left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want this processor to be unconditional? Like not tied to chat template caps?
I wonder if resolving this won't introduce another problem when we get chat template that always expects something in the tool calls and empty array would be somewhat expected.

@przepeck

Copy link
Copy Markdown
Collaborator Author

Do we want this processor to be unconditional? Like not tied to chat template caps? I wonder if resolving this won't introduce another problem when we get chat template that always expects something in the tool calls and empty array would be somewhat expected.

In that scenario usual curl request e.g. {type:"text", content:"Hello"} would fail, I don't think that is possible for chat template to be that bad.

@mzegla

mzegla commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Okay, we can always adjust if such template appears. Let's move forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants