[BUG] RequestValidator cannot validate webhooks that repeat a form parameter (e.g. `MessagingBinding.Address`)
还没有人认领这个 Issue。
评估
调研方向
The relevant code is in src/main/java/com/twilio/security/RequestValidator.java: validate and the private getValidationSignature build the signed string from a Map<String, String>, so each name appears once. Read the existing RequestValidatorTest and the reproduction test in the issue, which uses the twilio-node vector (token 12345, expected signature 83O6e2vORAoJHUNzJjDWN1jz+BA=). Done means a multi-valued entry point (for example, values per name sorted and de-duplicated, with the name repeated before each value) validates that vector, and the existing tests still pass.
由索引模型根据 Issue 内容生成。
描述
Describe the bug
Location (twilio/twilio-java @ 0155569):
src/main/java/com/twilio/security/RequestValidator.java:28-36:validateonly acceptsMap<String, String>, so a parameter can carry one value.src/main/java/com/twilio/security/RequestValidator.java:70-84:getValidationSignatureappends each parameter name once.
public boolean validate(String url, Map<String, String> params, String expectedSignature) {
...
private String getValidationSignature(String url, Map<String, String> params) {
try {
StringBuilder builder = new StringBuilder(url);
if (params != null) {
List<String> sortedKeys = new ArrayList<>(params.keySet());
Collections.sort(sortedKeys);
for (String key : sortedKeys) {
builder.append(key);
String value = params.get(key);
builder.append(value == null ? "" : value);
}
}
Some webhooks send the same form parameter more than once. The Conversations onConversationAdd webhook sends MessagingBinding.Address once per address for group messages. For such a request Twilio signs every value, repeating the name each time, with the values of one name sorted and de-duplicated:
<url>MessagingBinding.Address+1325xxxxxxxMessagingBinding.Address+1415xxxxxxx
RequestValidator has no way to express this. With Map<String, String> there is only one value per name, and whatever single value is passed (first, last, or both concatenated), the computed data string never contains the name twice, so a genuine request is rejected. Servlet users get a Map<String, String[]> from getParameterMap() and have to drop values to call validate.
The sort/de-duplicate rule for repeated values is not in the webhook security docs; it comes from reports on the Node and Python libraries (twilio/twilio-node#722, twilio/twilio-python#613) and is what twilio-node, twilio-python and twilio-php implement today.
Code snippet
Using the test vector from twilio-node (token = "12345", values +1325xxxxxxx and +1415xxxxxxx, signature 83O6e2vORAoJHUNzJjDWN1jz+BA=; recomputing the HMAC-SHA1 of the string above gives the same value):
RequestValidator validator = new RequestValidator("12345");
String url = "https://mycompany.com/myapp.php?foo=1&bar=2";
String signature = "83O6e2vORAoJHUNzJjDWN1jz+BA=";
Map<String, String> params = new HashMap<>();
params.put("MessagingBinding.Address", "+1325xxxxxxx"); // or "+1415xxxxxxx", or "+1325xxxxxxx+1415xxxxxxx"
validator.validate(url, params, signature); // false in all three cases
Test used to check this (tries all three single-value representations; comments trimmed)
package com.twilio.security;
import org.junit.Assert;
import org.junit.Test;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.util.*;
public class RequestValidatorArrayParamTest {
private final String token = "12345";
private final String url = "https://mycompany.com/myapp.php?foo=1&bar=2";
private final RequestValidator validator = new RequestValidator(token);
private String computeExpectedSignature(String url, String key, List<String> values) throws Exception {
StringBuilder builder = new StringBuilder(url);
for (String value : values) {
builder.append(key);
builder.append(value);
}
Mac mac = Mac.getInstance("HmacSHA1");
mac.init(new SecretKeySpec(token.getBytes(), "HmacSHA1"));
byte[] rawHmac = mac.doFinal(builder.toString().getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(rawHmac);
}
@Test
public void testValidateWithArrayParameter() throws Exception {
List<String> addressValues = Arrays.asList("+1325xxxxxxx", "+1415xxxxxxx");
String correctSignature = computeExpectedSignature(url, "MessagingBinding.Address", addressValues);
Map<String, String> paramsFirstValue = new HashMap<>();
paramsFirstValue.put("MessagingBinding.Address", "+1325xxxxxxx");
boolean validWithFirst = validator.validate(url, paramsFirstValue, correctSignature);
Map<String, String> paramsLastValue = new HashMap<>();
paramsLastValue.put("MessagingBinding.Address", "+1415xxxxxxx");
boolean validWithLast = validator.validate(url, paramsLastValue, correctSignature);
Map<String, String> paramsConcatenated = new HashMap<>();
paramsConcatenated.put("MessagingBinding.Address", "+1325xxxxxxx+1415xxxxxxx");
boolean validWithConcatenated = validator.validate(url, paramsConcatenated, correctSignature);
Assert.assertTrue(
"RequestValidator should be able to validate webhooks with array parameters",
validWithFirst || validWithLast || validWithConcatenated
);
}
}
Actual behavior
validate returns false for a correctly signed request whose form repeats a parameter name; there is no method that accepts several values per name.
Expected behavior
RequestValidator offers a way to validate such requests, e.g. a method taking Map<String, List<String>> that signs every value as name + value, values sorted and de-duplicated, like the other Twilio helper libraries.
twilio-java version
13.1.0 (main @ 0155569)
Java version
OpenJDK 21.0.2
Logs or error messages
Output of the test above (mvn test -Dtest='RequestValidator*Test', Apache Maven 3.9.16); the existing RequestValidatorTest cases all pass:
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.529 s <<< FAILURE! -- in com.twilio.security.RequestValidatorArrayParamTest
[ERROR] com.twilio.security.RequestValidatorArrayParamTest.testValidateWithArrayParameter -- Time elapsed: 0.143 s <<< FAILURE!
java.lang.AssertionError: RequestValidator should be able to validate webhooks with array parameters
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at com.twilio.security.RequestValidatorArrayParamTest.testValidateWithArrayParameter(RequestValidatorArrayParamTest.java:79)
Environment: commit 01555696c76a2c515fd424eabcf547bdad1f78af (current main), OpenJDK 21.0.2, Apache Maven 3.9.16.
Additional context
Already fixed in twilio/twilio-node
PR: https://github.com/twilio/twilio-node/pull/687 — "fix: webhook validation with an array parameter" (merged 2021-08-16T22:14:08Z, merge commit 6f35b2a305a). It repeats the parameter name for every element of an array value when building the signed string. Follow-ups made the value order match Twilio's: https://github.com/twilio/twilio-node/pull/723 (merged 2022-08-26, sorts the values, fixes twilio/twilio-node#722) and https://github.com/twilio/twilio-python/pull/617 (merged 2022-09-20, removes duplicate values); current twilio-node does Array.from(new Set(paramValue)).sort().
How twilio-node fixed it (relevant hunks, verbatim):
--- a/lib/webhooks/webhooks.js
+++ b/lib/webhooks/webhooks.js
@@ -50,6 +50,22 @@ function removePort(parsedUrl) {
return parsedUrl.toString();
}
+/**
+ Utility function to convert request parameter to a string format
+
+ @param {string} paramName - The request parameter name
+ @param {string|array<string>} paramValue - The request parameter value
+ @returns {string} - Formatted parameter string
+ */
+function toFormUrlEncodedParam(paramName, paramValue) {
+ if (paramValue instanceof Array) {
+ return paramValue
+ .map(val => toFormUrlEncodedParam(paramName, val))
+ .reduce((acc, val) => acc + val, '');
+ }
+ return paramName + paramValue;
+}
+
/**
Utility function to get the expected signature for a given request
@@ -65,7 +81,7 @@ function getExpectedTwilioSignature(authToken, url, params) {
var data = Object.keys(params)
.sort()
- .reduce((acc, key) => acc + key + params[key], url);
+ .reduce((acc, key) => acc + toFormUrlEncodedParam(key, params[key]), url);
return crypto
.createHmac('sha1', authToken)
--- a/lib/webhooks/webhooks.js
+++ b/lib/webhooks/webhooks.js
@@ -60,6 +60,7 @@ function removePort(parsedUrl) {
function toFormUrlEncodedParam(paramName, paramValue) {
if (paramValue instanceof Array) {
return paramValue
+ .sort()
.map(val => toFormUrlEncodedParam(paramName, val))
.reduce((acc, val) => acc + val, '');
}
Suggested fix
Add a separate method for multi-valued parameters (an overload of validate taking Map<String, List<String>> would clash with the existing one after erasure) and share the HMAC step; the existing validate methods are unchanged:
diff --git a/src/main/java/com/twilio/security/RequestValidator.java b/src/main/java/com/twilio/security/RequestValidator.java
--- a/src/main/java/com/twilio/security/RequestValidator.java
+++ b/src/main/java/com/twilio/security/RequestValidator.java
@@ -35,6 +35,24 @@ public class RequestValidator {
secureCompare(signatureWithoutPort, expectedSignature);
}
+ /**
+ * Validates a request whose POST parameters can have several values for the same name, e.g.
+ * {@code MessagingBinding.Address} in Conversations webhooks. As when Twilio signs the request, the values
+ * of each parameter are sorted and de-duplicated, and the parameter name is repeated before every value.
+ *
+ * @param url the full URL Twilio requested, including the query string
+ * @param params the POST parameters, each mapped to all of its values
+ * @param expectedSignature the value of the X-Twilio-Signature header
+ * @return true if the signature is valid
+ */
+ public boolean validateWithMultiValueParams(String url, Map<String, List<String>> params,
+ String expectedSignature) {
+ String signatureWithPort = sign(appendMultiValueParams(addPort(url), params));
+ String signatureWithoutPort = sign(appendMultiValueParams(removePort(url), params));
+ return secureCompare(signatureWithPort, expectedSignature) ||
+ secureCompare(signatureWithoutPort, expectedSignature);
+ }
+
public boolean validate(String url, String body, String expectedSignature) throws URISyntaxException {
Map<String, String> empty = new HashMap<>();
List<NameValuePair> params = new URIBuilder(url).getQueryParams();
@@ -68,25 +86,50 @@ public class RequestValidator {
}
private String getValidationSignature(String url, Map<String, String> params) {
- try {
+ StringBuilder builder = new StringBuilder(url);
+ if (params != null) {
+ List<String> sortedKeys = new ArrayList<>(params.keySet());
+ Collections.sort(sortedKeys);
+
+ for (String key : sortedKeys) {
+ builder.append(key);
- StringBuilder builder = new StringBuilder(url);
- if (params != null) {
- List<String> sortedKeys = new ArrayList<>(params.keySet());
- Collections.sort(sortedKeys);
+ String value = params.get(key);
+ builder.append(value == null ? "" : value);
+ }
+ }
+ return sign(builder.toString());
+ }
- for (String key : sortedKeys) {
- builder.append(key);
+ private String appendMultiValueParams(String url, Map<String, List<String>> params) {
+ StringBuilder builder = new StringBuilder(url);
+ if (params != null) {
+ List<String> sortedKeys = new ArrayList<>(params.keySet());
+ Collections.sort(sortedKeys);
- String value = params.get(key);
- builder.append(value == null ? "" : value);
+ for (String key : sortedKeys) {
+ List<String> values = params.get(key);
+ if (values == null) {
+ continue;
+ }
+ SortedSet<String> sortedValues = new TreeSet<>();
+ for (String value : values) {
+ sortedValues.add(value == null ? "" : value);
+ }
+ for (String value : sortedValues) {
+ builder.append(key).append(value);
}
}
+ }
+ return builder.toString();
+ }
+ private String sign(String data) {
+ try {
Mac mac = Mac.getInstance(HMAC);
mac.init(signingKey);
- byte[] rawHmac = mac.doFinal(builder.toString().getBytes(StandardCharsets.UTF_8));
+ byte[] rawHmac = mac.doFinal(data.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(rawHmac);
} catch (Exception e) {
Regression test for the new method
package com.twilio.security;
import org.junit.Assert;
import org.junit.Test;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
* Test class for {@link RequestValidator} with POST parameters that have several values for the same name.
*/
public class RequestValidatorArrayParamTest {
private final String url = "https://mycompany.com/myapp.php?foo=1&bar=2";
private final RequestValidator validator = new RequestValidator("12345");
// Signature of url + "MessagingBinding.Address+1325xxxxxxx" + "MessagingBinding.Address+1415xxxxxxx"
private final String signature = "83O6e2vORAoJHUNzJjDWN1jz+BA=";
@Test
public void testValidateWithArrayParameter() {
Map<String, List<String>> params = new HashMap<>();
params.put("MessagingBinding.Address", Arrays.asList("+1325xxxxxxx", "+1415xxxxxxx"));
Assert.assertTrue(validator.validateWithMultiValueParams(url, params, signature));
}
@Test
public void testValidateWithArrayParameterSortsAndDeduplicatesValues() {
Map<String, List<String>> params = new HashMap<>();
params.put("MessagingBinding.Address", Arrays.asList("+1415xxxxxxx", "+1325xxxxxxx", "+1415xxxxxxx"));
Assert.assertTrue(validator.validateWithMultiValueParams(url, params, signature));
}
@Test
public void testValidateWithArrayParameterFailsWhenValueMissing() {
Map<String, List<String>> params = new HashMap<>();
params.put("MessagingBinding.Address", Collections.singletonList("+1325xxxxxxx"));
Assert.assertFalse(validator.validateWithMultiValueParams(url, params, signature));
}
@Test
public void testValidateWithSingleValues() {
// Same parameters and signature as RequestValidatorTest.testValidate
Map<String, List<String>> params = new HashMap<>();
params.put("Digits", Collections.singletonList("1234"));
params.put("CallSid", Collections.singletonList("CA1234567890ABCDE"));
params.put("To", Collections.singletonList("+18005551212"));
params.put("Caller", Collections.singletonList("+14158675309"));
params.put("From", Collections.singletonList("+14158675309"));
Assert.assertTrue(validator.validateWithMultiValueParams(url, params, "RSOYDt4T1cUTdK1PDd93/VVr8B8="));
}
}
Verified on 0155569 (OpenJDK 21.0.2, Maven 3.9.16): the regression test passes with this patch (without it, validateWithMultiValueParams does not exist and the test in the bug description fails as shown above); no other test results change (920 tests, 7 skipped, as before).
I can submit a PR for this.
Checked: searches for array/multi-value/duplicate parameters, getParameterMap, MessagingBinding and Conversations signatures found only test-coverage work (#645, #902) and unrelated RequestValidator changes (URL encoding #601/#823/#901, HMAC-SHA256 #986).
- 主要语言
- Java
- 星标
- 528
- 派生
- 462
- 平均合并
- 2 天 11 小时
- 30 天内合并 PR
- 4
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
twilio/twilio-java 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 88/100
twilio/twilio-java#988 ·
-
难度 2/5 1-3 小时 新手友好度 68/100
twilio/twilio-java#976 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
twilio/twilio-java#975 ·
-
[BUG] Sip.Builder(String) drops SIP destinations with an explicit port when the scheme is omitted未关闭type: bug
难度 3/5 1-2 天 新手友好度 55/100
twilio/twilio-java#982 ·
-
难度 5/5 一周以上 新手友好度 35/100
twilio/twilio-java#960 ·
查看 twilio/twilio-java 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 77/100
-
bug finsim-registry good-first-issue
难度 1/5 1 小时以内 新手友好度 88/100
维护者通常 1 天内回复
-
status:pending type:bug
难度 2/5 1-3 小时 新手友好度 75/100
-
bug level:trivial
难度 2/5 1-3 小时 新手友好度 72/100
exoego/keycloak-fuhito#666 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 79/100
hammy275/immersive-mc#669 ·