Python: Bug: Anthropic, Bedrock, Google AI and Vertex AI tool formatters ignore include_in_function_choices=False
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
Research direction
Start with the OpenAI formatter in semantic_kernel/connectors/ai/function_calling_utils.py, which already filters on include_in_function_choices, and copy that filter into the four formatters the issue names: anthropic/services/utils.py, bedrock/services/model_provider/utils.py, google/google_ai/services/utils.py and google/vertex_ai/services/utils.py. Filter both properties and required. Done when the reproduction script in the issue shows only query for every provider and the new tests fail on main and pass with the change.
Written by the indexing model from the issue text.
Description
Describe the bug
A parameter marked with {"include_in_function_choices": False} is hidden from the model by the OpenAI formatters. The Anthropic, Bedrock, Google AI and Vertex AI formatters ignore the flag. They still send the parameter in the tool schema.
The decorator docs say a parameter with this flag "will not be included in the function choices". In function_calling_utils.py both OpenAI formatters filter on include_in_function_choices. The four other formatters read metadata.parameters without the filter.
Files on current main:
python/semantic_kernel/connectors/ai/anthropic/services/utils.pypython/semantic_kernel/connectors/ai/bedrock/services/model_provider/utils.pypython/semantic_kernel/connectors/ai/google/google_ai/services/utils.pypython/semantic_kernel/connectors/ai/google/vertex_ai/services/utils.py
To Reproduce
I ran this on microsoft/semantic-kernel main at 7a92cd4e7 (Python 3.13).
from typing import Annotated
from semantic_kernel import Kernel
from semantic_kernel.functions import kernel_function
from semantic_kernel.connectors.ai.function_calling_utils import kernel_function_metadata_to_function_call_format as oa
from semantic_kernel.connectors.ai.anthropic.services.utils import kernel_function_metadata_to_function_call_format as an
from semantic_kernel.connectors.ai.google.google_ai.services.utils import kernel_function_metadata_to_google_ai_function_call_format as ga
from semantic_kernel.connectors.ai.google.vertex_ai.services.utils import kernel_function_metadata_to_vertex_ai_function_call_format as va
from semantic_kernel.connectors.ai.bedrock.services.model_provider.utils import update_settings_from_function_choice_configuration as bd
from semantic_kernel.connectors.ai.bedrock.bedrock_prompt_execution_settings import BedrockChatPromptExecutionSettings
from semantic_kernel.connectors.ai.function_choice_type import FunctionChoiceType
from semantic_kernel.connectors.ai.function_call_choice_configuration import FunctionCallChoiceConfiguration
class P:
@kernel_function(name="lookup", description="Look up a user")
def lookup(
self,
query: Annotated[str, "What to look up"],
token: Annotated[str, "Internal token", {"include_in_function_choices": False}] = "",
) -> str:
return query
meta = Kernel().add_plugin(P(), "p").functions["lookup"].metadata
print("OpenAI :", list(oa(meta)["function"]["parameters"]["properties"]))
print("Anthropic :", list(an(meta)["input_schema"]["properties"]))
print("GoogleAI :", list(ga(meta)["parameters"]["properties"]))
print("VertexAI :", list(type(va(meta)).to_dict(va(meta))["parameters"]["properties"]))
s = BedrockChatPromptExecutionSettings()
bd(FunctionCallChoiceConfiguration(available_functions=[meta]), s, FunctionChoiceType.AUTO)
print("Bedrock :", list(s.tools[0]["toolSpec"]["inputSchema"]["json"]["properties"]))
Output:
OpenAI : ['query']
Anthropic : ['query', 'token']
GoogleAI : ['query', 'token']
VertexAI : ['token', 'query']
Bedrock : ['query', 'token']
Expected behavior
All formatters hide token, as the OpenAI one does. The model should only see query.
Impact
The parameter name and description are sent to the model on these providers. The model can then try to fill a value that the caller meant to supply itself. It also adds tokens to every request.
Fix
The fix is small. Filter on include_in_function_choices in the four formatters, for both properties and required. I have a branch with tests that fail on main and pass with the change: https://github.com/VANDRANKI/semantic-kernel/tree/fix/non-openai-formatters-include-in-function-choices
I can open a PR from this branch if the maintainers want it.
- Dominant language
- C#
- Stars
- 28.6k
- Forks
- 4.8k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 22
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/semantic-kernel
-
Python: Bug: AnnotationContent and StreamingAnnotationContent to_element writes CitationType.X, so annotations in a rendered chat history fail to parsePossibly taken @CRYPTONIKAV claimed this 3 days ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
microsoft/semantic-kernel#14590 ·
Maintainers usually reply within 3 days
-
Python: Bug: TimePlugin timeZoneOffset and timeZoneName always return an empty stringPossibly taken @jeevan1098 claimed this 1 day ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
microsoft/semantic-kernel#14583 · 1 comment ·
Maintainers usually reply within 3 days
-
.Net: gpt-image-1 is the default image model in the .NET OpenAI connector, and OpenAI shuts it down on October 23Possibly taken @nightcityblade claimed this 13 days ago. Open.NET triage
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/semantic-kernel#14526 · 1 comment ·
Maintainers usually reply within 3 days
-
Python: VolatileMemoryStore.get_batch and get_nearest_matches ignore with_embeddings=False (deepcopy result is discarded)Possibly taken @VANDRANKI claimed this 15 days ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
microsoft/semantic-kernel#14522 ·
Maintainers usually reply within 3 days
-
Python: VolatileMemoryStore.get_nearest_match returns an un-awaited coroutine instead of a (MemoryRecord, score) tuplePossibly taken @VANDRANKI claimed this 15 days ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
microsoft/semantic-kernel#14521 ·
Maintainers usually reply within 3 days
All issues in microsoft/semantic-kernel
Similar issues
-
ASP.NET Core container quickstart: `docker-run: debug` task sample is missing its closing braceOpen
Difficulty 1/5 Under an hour Newbie friendliness 88/100
microsoft/vscode-docs#10499 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
vzeol/DeployCraft#125 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/arrow-adbc#4915 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
fboucher/2d6-dungeon-app#245 · 1 comment ·
Maintainers usually reply within 1 day