[BUG] RequestValidator cannot validate webhooks that repeat a form parameter (e.g. `MessagingBinding.Address`)
まだ誰も着手していません。
評価
調査の方向性
The relevant code is in src/main/java/com/twilio/security/RequestValidator.java: validate and the private getValidationSignature build the signed string from a Map<String, String>, so each name appears once. Read the existing RequestValidatorTest and the reproduction test in the issue, which uses the twilio-node vector (token 12345, expected signature 83O6e2vORAoJHUNzJjDWN1jz+BA=). Done means a multi-valued entry point (for example, values per name sorted and de-duplicated, with the name repeated before each value) validates that vector, and the existing tests still pass.
索引モデルが issue の本文から書いたものです。
説明
Describe the bug
Location (twilio/twilio-java @ 0155569):
src/main/java/com/twilio/security/RequestValidator.java:28-36:validateonly acceptsMap<String, String>, so a parameter can carry one value.src/main/java/com/twilio/security/RequestValidator.java:70-84:getValidationSignatureappends each parameter name once.
public boolean validate(String url, Map<String, String> params, String expectedSignature) {
...
private String getValidationSignature(String url, Map<String, String> params) {
try {
StringBuilder builder = new StringBuilder(url);
if (params != null) {
List<String> sortedKeys = new ArrayList<>(params.keySet());
Collections.sort(sortedKeys);
for (String key : sortedKeys) {
builder.append(key);
String value = params.get(key);
builder.append(value == null ? "" : value);
}
}
Some webhooks send the same form parameter more than once. The Conversations onConversationAdd webhook sends MessagingBinding.Address once per address for group messages. For such a request Twilio signs every value, repeating the name each time, with the values of one name sorted and de-duplicated:
<url>MessagingBinding.Address+1325xxxxxxxMessagingBinding.Address+1415xxxxxxx
RequestValidator has no way to express this. With Map<String, String> there is only one value per name, and whatever single value is passed (first, last, or both concatenated), the computed data string never contains the name twice, so a genuine request is rejected. Servlet users get a Map<String, String[]> from getParameterMap() and have to drop values to call validate.
The sort/de-duplicate rule for repeated values is not in the webhook security docs; it comes from reports on the Node and Python libraries (twilio/twilio-node#722, twilio/twilio-python#613) and is what twilio-node, twilio-python and twilio-php implement today.
Code snippet
Using the test vector from twilio-node (token = "12345", values +1325xxxxxxx and +1415xxxxxxx, signature 83O6e2vORAoJHUNzJjDWN1jz+BA=; recomputing the HMAC-SHA1 of the string above gives the same value):
RequestValidator validator = new RequestValidator("12345");
String url = "https://mycompany.com/myapp.php?foo=1&bar=2";
String signature = "83O6e2vORAoJHUNzJjDWN1jz+BA=";
Map<String, String> params = new HashMap<>();
params.put("MessagingBinding.Address", "+1325xxxxxxx"); // or "+1415xxxxxxx", or "+1325xxxxxxx+1415xxxxxxx"
validator.validate(url, params, signature); // false in all three cases
Test used to check this (tries all three single-value representations; comments trimmed)
package com.twilio.security;
import org.junit.Assert;
import org.junit.Test;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.util.*;
public class RequestValidatorArrayParamTest {
private final String token = "12345";
private final String url = "https://mycompany.com/myapp.php?foo=1&bar=2";
private final RequestValidator validator = new RequestValidator(token);
private String computeExpectedSignature(String url, String key, List<String> values) throws Exception {
StringBuilder builder = new StringBuilder(url);
for (String value : values) {
builder.append(key);
builder.append(value);
}
Mac mac = Mac.getInstance("HmacSHA1");
mac.init(new SecretKeySpec(token.getBytes(), "HmacSHA1"));
byte[] rawHmac = mac.doFinal(builder.toString().getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(rawHmac);
}
@Test
public void testValidateWithArrayParameter() throws Exception {
List<String> addressValues = Arrays.asList("+1325xxxxxxx", "+1415xxxxxxx");
String correctSignature = computeExpectedSignature(url, "MessagingBinding.Address", addressValues);
Map<String, String> paramsFirstValue = new HashMap<>();
paramsFirstValue.put("MessagingBinding.Address", "+1325xxxxxxx");
boolean validWithFirst = validator.validate(url, paramsFirstValue, correctSignature);
Map<String, String> paramsLastValue = new HashMap<>();
paramsLastValue.put("MessagingBinding.Address", "+1415xxxxxxx");
boolean validWithLast = validator.validate(url, paramsLastValue, correctSignature);
Map<String, String> paramsConcatenated = new HashMap<>();
paramsConcatenated.put("MessagingBinding.Address", "+1325xxxxxxx+1415xxxxxxx");
boolean validWithConcatenated = validator.validate(url, paramsConcatenated, correctSignature);
Assert.assertTrue(
"RequestValidator should be able to validate webhooks with array parameters",
validWithFirst || validWithLast || validWithConcatenated
);
}
}
Actual behavior
validate returns false for a correctly signed request whose form repeats a parameter name; there is no method that accepts several values per name.
Expected behavior
RequestValidator offers a way to validate such requests, e.g. a method taking Map<String, List<String>> that signs every value as name + value, values sorted and de-duplicated, like the other Twilio helper libraries.
twilio-java version
13.1.0 (main @ 0155569)
Java version
OpenJDK 21.0.2
Logs or error messages
Output of the test above (mvn test -Dtest='RequestValidator*Test', Apache Maven 3.9.16); the existing RequestValidatorTest cases all pass:
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.529 s <<< FAILURE! -- in com.twilio.security.RequestValidatorArrayParamTest
[ERROR] com.twilio.security.RequestValidatorArrayParamTest.testValidateWithArrayParameter -- Time elapsed: 0.143 s <<< FAILURE!
java.lang.AssertionError: RequestValidator should be able to validate webhooks with array parameters
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at com.twilio.security.RequestValidatorArrayParamTest.testValidateWithArrayParameter(RequestValidatorArrayParamTest.java:79)
Environment: commit 01555696c76a2c515fd424eabcf547bdad1f78af (current main), OpenJDK 21.0.2, Apache Maven 3.9.16.
Additional context
Already fixed in twilio/twilio-node
PR: https://github.com/twilio/twilio-node/pull/687 — "fix: webhook validation with an array parameter" (merged 2021-08-16T22:14:08Z, merge commit 6f35b2a305a). It repeats the parameter name for every element of an array value when building the signed string. Follow-ups made the value order match Twilio's: https://github.com/twilio/twilio-node/pull/723 (merged 2022-08-26, sorts the values, fixes twilio/twilio-node#722) and https://github.com/twilio/twilio-python/pull/617 (merged 2022-09-20, removes duplicate values); current twilio-node does Array.from(new Set(paramValue)).sort().
How twilio-node fixed it (relevant hunks, verbatim):
--- a/lib/webhooks/webhooks.js
+++ b/lib/webhooks/webhooks.js
@@ -50,6 +50,22 @@ function removePort(parsedUrl) {
return parsedUrl.toString();
}
+/**
+ Utility function to convert request parameter to a string format
+
+ @param {string} paramName - The request parameter name
+ @param {string|array<string>} paramValue - The request parameter value
+ @returns {string} - Formatted parameter string
+ */
+function toFormUrlEncodedParam(paramName, paramValue) {
+ if (paramValue instanceof Array) {
+ return paramValue
+ .map(val => toFormUrlEncodedParam(paramName, val))
+ .reduce((acc, val) => acc + val, '');
+ }
+ return paramName + paramValue;
+}
+
/**
Utility function to get the expected signature for a given request
@@ -65,7 +81,7 @@ function getExpectedTwilioSignature(authToken, url, params) {
var data = Object.keys(params)
.sort()
- .reduce((acc, key) => acc + key + params[key], url);
+ .reduce((acc, key) => acc + toFormUrlEncodedParam(key, params[key]), url);
return crypto
.createHmac('sha1', authToken)
--- a/lib/webhooks/webhooks.js
+++ b/lib/webhooks/webhooks.js
@@ -60,6 +60,7 @@ function removePort(parsedUrl) {
function toFormUrlEncodedParam(paramName, paramValue) {
if (paramValue instanceof Array) {
return paramValue
+ .sort()
.map(val => toFormUrlEncodedParam(paramName, val))
.reduce((acc, val) => acc + val, '');
}
Suggested fix
Add a separate method for multi-valued parameters (an overload of validate taking Map<String, List<String>> would clash with the existing one after erasure) and share the HMAC step; the existing validate methods are unchanged:
diff --git a/src/main/java/com/twilio/security/RequestValidator.java b/src/main/java/com/twilio/security/RequestValidator.java
--- a/src/main/java/com/twilio/security/RequestValidator.java
+++ b/src/main/java/com/twilio/security/RequestValidator.java
@@ -35,6 +35,24 @@ public class RequestValidator {
secureCompare(signatureWithoutPort, expectedSignature);
}
+ /**
+ * Validates a request whose POST parameters can have several values for the same name, e.g.
+ * {@code MessagingBinding.Address} in Conversations webhooks. As when Twilio signs the request, the values
+ * of each parameter are sorted and de-duplicated, and the parameter name is repeated before every value.
+ *
+ * @param url the full URL Twilio requested, including the query string
+ * @param params the POST parameters, each mapped to all of its values
+ * @param expectedSignature the value of the X-Twilio-Signature header
+ * @return true if the signature is valid
+ */
+ public boolean validateWithMultiValueParams(String url, Map<String, List<String>> params,
+ String expectedSignature) {
+ String signatureWithPort = sign(appendMultiValueParams(addPort(url), params));
+ String signatureWithoutPort = sign(appendMultiValueParams(removePort(url), params));
+ return secureCompare(signatureWithPort, expectedSignature) ||
+ secureCompare(signatureWithoutPort, expectedSignature);
+ }
+
public boolean validate(String url, String body, String expectedSignature) throws URISyntaxException {
Map<String, String> empty = new HashMap<>();
List<NameValuePair> params = new URIBuilder(url).getQueryParams();
@@ -68,25 +86,50 @@ public class RequestValidator {
}
private String getValidationSignature(String url, Map<String, String> params) {
- try {
+ StringBuilder builder = new StringBuilder(url);
+ if (params != null) {
+ List<String> sortedKeys = new ArrayList<>(params.keySet());
+ Collections.sort(sortedKeys);
+
+ for (String key : sortedKeys) {
+ builder.append(key);
- StringBuilder builder = new StringBuilder(url);
- if (params != null) {
- List<String> sortedKeys = new ArrayList<>(params.keySet());
- Collections.sort(sortedKeys);
+ String value = params.get(key);
+ builder.append(value == null ? "" : value);
+ }
+ }
+ return sign(builder.toString());
+ }
- for (String key : sortedKeys) {
- builder.append(key);
+ private String appendMultiValueParams(String url, Map<String, List<String>> params) {
+ StringBuilder builder = new StringBuilder(url);
+ if (params != null) {
+ List<String> sortedKeys = new ArrayList<>(params.keySet());
+ Collections.sort(sortedKeys);
- String value = params.get(key);
- builder.append(value == null ? "" : value);
+ for (String key : sortedKeys) {
+ List<String> values = params.get(key);
+ if (values == null) {
+ continue;
+ }
+ SortedSet<String> sortedValues = new TreeSet<>();
+ for (String value : values) {
+ sortedValues.add(value == null ? "" : value);
+ }
+ for (String value : sortedValues) {
+ builder.append(key).append(value);
}
}
+ }
+ return builder.toString();
+ }
+ private String sign(String data) {
+ try {
Mac mac = Mac.getInstance(HMAC);
mac.init(signingKey);
- byte[] rawHmac = mac.doFinal(builder.toString().getBytes(StandardCharsets.UTF_8));
+ byte[] rawHmac = mac.doFinal(data.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(rawHmac);
} catch (Exception e) {
Regression test for the new method
package com.twilio.security;
import org.junit.Assert;
import org.junit.Test;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
* Test class for {@link RequestValidator} with POST parameters that have several values for the same name.
*/
public class RequestValidatorArrayParamTest {
private final String url = "https://mycompany.com/myapp.php?foo=1&bar=2";
private final RequestValidator validator = new RequestValidator("12345");
// Signature of url + "MessagingBinding.Address+1325xxxxxxx" + "MessagingBinding.Address+1415xxxxxxx"
private final String signature = "83O6e2vORAoJHUNzJjDWN1jz+BA=";
@Test
public void testValidateWithArrayParameter() {
Map<String, List<String>> params = new HashMap<>();
params.put("MessagingBinding.Address", Arrays.asList("+1325xxxxxxx", "+1415xxxxxxx"));
Assert.assertTrue(validator.validateWithMultiValueParams(url, params, signature));
}
@Test
public void testValidateWithArrayParameterSortsAndDeduplicatesValues() {
Map<String, List<String>> params = new HashMap<>();
params.put("MessagingBinding.Address", Arrays.asList("+1415xxxxxxx", "+1325xxxxxxx", "+1415xxxxxxx"));
Assert.assertTrue(validator.validateWithMultiValueParams(url, params, signature));
}
@Test
public void testValidateWithArrayParameterFailsWhenValueMissing() {
Map<String, List<String>> params = new HashMap<>();
params.put("MessagingBinding.Address", Collections.singletonList("+1325xxxxxxx"));
Assert.assertFalse(validator.validateWithMultiValueParams(url, params, signature));
}
@Test
public void testValidateWithSingleValues() {
// Same parameters and signature as RequestValidatorTest.testValidate
Map<String, List<String>> params = new HashMap<>();
params.put("Digits", Collections.singletonList("1234"));
params.put("CallSid", Collections.singletonList("CA1234567890ABCDE"));
params.put("To", Collections.singletonList("+18005551212"));
params.put("Caller", Collections.singletonList("+14158675309"));
params.put("From", Collections.singletonList("+14158675309"));
Assert.assertTrue(validator.validateWithMultiValueParams(url, params, "RSOYDt4T1cUTdK1PDd93/VVr8B8="));
}
}
Verified on 0155569 (OpenJDK 21.0.2, Maven 3.9.16): the regression test passes with this patch (without it, validateWithMultiValueParams does not exist and the test in the bug description fails as shown above); no other test results change (920 tests, 7 skipped, as before).
I can submit a PR for this.
Checked: searches for array/multi-value/duplicate parameters, getParameterMap, MessagingBinding and Conversations signatures found only test-coverage work (#645, #902) and unrelated RequestValidator changes (URL encoding #601/#823/#901, HMAC-SHA256 #986).
- 主要言語
- Java
- スター
- 528
- フォーク
- 462
- 平均マージ
- 2日 11時間
- マージ済み PR(30日)
- 4
環境構築
- Dockerfile または Docker Compose ファイルあり
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
twilio/twilio-java のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
twilio/twilio-java#988 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
twilio/twilio-java#976 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
twilio/twilio-java#975 ·
-
[BUG] Sip.Builder(String) drops SIP destinations with an explicit port when the scheme is omittedオープンtype: bug
難易度 3/5 1〜2日 初心者へのやさしさ 55/100
twilio/twilio-java#982 ·
-
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
twilio/twilio-java#960 ·
twilio/twilio-java の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 77/100
-
RegistryExceptionHandler uses self-referencing basePackageClasses instead of a controller classオープンbug finsim-registry good-first-issue
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
メンテナーはふだん 1 日以内に返信
-
status:pending type:bug
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
-
bug level:trivial
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
exoego/keycloak-fuhito#666 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 79/100
hammy275/immersive-mc#669 ·