Local evaluation buckets percentage splits with the server key, so results differ from Flagsmith
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
Research direction
Start with Client#get_identity_model in lib/flagsmith.rb, especially the composite-key construction linked in the issue, and trace how the environment API key is made available in local evaluation. Compare the resulting key with the reproduction's client_key_account_two expectation. Update the focused tests for local percentage-split evaluation and run them; done means the SDK matches the dashboard's split result.
Written by the indexing model from the issue text.
Description
In local evaluation mode, the client builds the identity's composite key from the environment_key it was configured with:
Local evaluation requires a server-side key, so environment_key is always ser.…. The composite key ends up as ser.<env key>_<identifier>, and that's what PERCENTAGE_SPLIT conditions hash, together with the segment ID.
Flagsmith itself appears to hash splits with the environment's client-side key, the api_key field of the environment document, giving <env key>_<identifier>. The two keys produce different percentages, so the SDK puts a different set of identities in every partial percentage split than the Flagsmith dashboard shows for the same identity and traits.
We found this when an identity that the dashboard showed in a 5% split segment evaluated as outside it in our app. Hashing <env key>_<identifier> gave 1.36 (in the percentage-based segment). Hashing ser.<env key>_<identifier>, what the SDK uses, gave 5.23 (outside the segment).
Reproduction
require "flagsmith"
document = {
id: 1,
api_key: "client_key",
project: {
id: 1, name: "Project", hide_disabled_flags: false,
organisation: {id: 1, name: "Org", feature_analytics: false, stop_serving_flags: false, persist_trait_data: true},
segments: [{
id: 1, name: "60% rollout",
rules: [{type: "ALL", rules: [], conditions: [{operator: "PERCENTAGE_SPLIT", value: "60", property_: "$.identity.key"}]}],
feature_states: [{django_id: 2, featurestate_uuid: "segment-state", enabled: true, feature_state_value: nil,
feature: {id: 1, name: "rollout", type: "STANDARD"}, multivariate_feature_state_values: []}]
}]
},
feature_states: [{django_id: 1, featurestate_uuid: "default-state", enabled: false, feature_state_value: nil,
feature: {id: 1, name: "rollout", type: "STANDARD"}, multivariate_feature_state_values: []}],
identity_overrides: []
}
environment = Flagsmith::Engine::Environment.build(document)
handler = Object.new
handler.define_singleton_method(:environment) { environment }
client = Flagsmith::Client.new(
environment_key: "ser.server_key",
offline_mode: true,
offline_handler: handler,
enable_local_evaluation: true
)
client.get_identity_flags("account_two").is_feature_enabled("rollout")
# => false. Hashing "client_key_account_two" gives 57.05, which is inside the 60% split;
# the SDK hashes "ser.server_key_account_two", which gives 62.53.
Expected
The SDK should match the dashboard and the other evaluation paths by building the composite key from environment.api_key.
Affected versions
4.1.1 and 4.3.0 (latest on RubyGems). It's also present in v5.0.0 and on main. In v5, Mappers::Identity.build_environment_context sets key: identity.composite_key, so the correct fallback in Segments::Evaluator.get_enriched_context, which builds "#{context[:environment][:key]}_#{identifier}" from the environment's api_key, never runs.
Suggested fix
In Client#get_identity_model, pass environment_api_key: environment.api_key instead of environment_key. We're running that change as a monkeypatch.
- Dominant language
- Ruby
- Stars
- 12
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Flagsmith/flagsmith-ruby-client
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Dependency DashboardOpen
Difficulty 3/5 1-2 days Newbie friendliness 15/100
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
All issues in Flagsmith/flagsmith-ruby-client
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
JetBrains/homebrew-utils#21 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ublue-os/homebrew-experimental-tap#774 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
r-uby-dev/llm#251 · 2 comments ·
Maintainers usually reply within 1 day
-
accessibility
Difficulty 1/5 Under an hour Newbie friendliness 72/100
slovensko-digital/ops-portal#212 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100