tests.system.vertexai.test_tokenization.TestTokenization: many tests failed
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- google-cloud, python
- Domain
- ai, testing-qa
Research direction
Start by running tests.system.vertexai.test_tokenization.TestTokenization and inspect the listed failures for gemini-1.5-pro-002 against PROD_ENDPOINT. Trace the tokenization behavior for local counts, computed tokens, system instructions, tools, and function calls or responses. Done means the package has no remaining failures and at least one test passes.
Written by the indexing model from the issue text.
Description
Many tests failed at the same time in this package.
- I will close this issue when there are no more failures in this package and
there is at least one pass. - No new issues will be filed for this package until this issue is closed.
- If there are already issues for individual test cases, I will close them when
the corresponding test passes. You can close them earlier, if you prefer, and
I won't reopen them while this issue is still open.
Here are the tests that failed:
- test_count_tokens_local[_get_tokenizer_for_model_preview-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
- test_count_tokens_local[get_tokenizer_for_model-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
- test_compute_tokens[_get_tokenizer_for_model_preview-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
- test_compute_tokens[get_tokenizer_for_model-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
- test_count_tokens_system_instruction[gemini-1.5-pro-002-PROD_ENDPOINT]
- test_count_tokens_system_instruction_is_function_call[gemini-1.5-pro-002-PROD_ENDPOINT]
- test_count_tokens_system_instruction_is_function_response[gemini-1.5-pro-002-PROD_ENDPOINT]
- test_count_tokens_tool_is_function_declaration[gemini-1.5-pro-002-PROD_ENDPOINT]
- test_count_tokens_content_is_function_call[gemini-1.5-pro-002-PROD_ENDPOINT]
- test_count_tokens_content_is_function_response[gemini-1.5-pro-002-PROD_ENDPOINT]
commit: ca6b45e3fa09bfa53c2f2c1b1d44f9a3c7aa79d7
buildURL: Build Status, Sponge
status: failed
- Dominant language
- Python
- Stars
- 907
- Forks
- 467
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 40
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from googleapis/python-aiplatform
-
api: vertex-ai
Difficulty 1/5 Under an hour Newbie friendliness 85/100
googleapis/python-aiplatform#7132 ·
-
api: vertex-ai
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
googleapis/python-aiplatform#7097 ·
-
CustomContainerTrainingJob.run drops max_wait_duration=0 instead of requesting indefinite DWS wait Openapi: vertex-ai
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
googleapis/python-aiplatform#7067 · 1 comment ·
-
api: vertex-ai
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
googleapis/python-aiplatform#6877 ·
-
api: vertex-ai
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
googleapis/python-aiplatform#6865 · 1 comment ·
All issues in googleapis/python-aiplatform
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/skills#1811 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
speaches-ai/speaches#678 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
datalayer/mcp-compose#42 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
conda-forge/spacy-feedstock#177 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
UKGovernmentBEIS/inspect_evals#2523 ·