Contribution Submission: MQTT v5 Input and Output components
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 需要澄清
- 活跃度
- 冷清
- 技术栈
- go
调研方向
先阅读 CONTRIBUTING.md §3.1.1 以及现有的 mqtt 输入和输出组件,然后审查基于 paho.golang 的拟议配置和元数据范围。完成的标志是 Redpanda 确认需要 MQTT v5 支持、批准范围,并定义新 mqtt5 组件的实现和测试要求。
由索引模型根据 Issue 内容生成。
描述
Background
We are evaluating Redpanda Connect for use in our data pipelines, but this use case requires MQTT v5 support, which currently does not exist for Redpanda Connect. Currently only MQTT v3.1.1 is supported via the mqtt input and output using the paho.mqtt.golang library.
Prior Work
There were two prior attempts to contribute MQTT v5 support to Redpanda Connect:
- PR #2197: Was closed due to withdrawal of the contribution on change from Benthos to Redpanda Connect.
- PR #3731: Was closed due to unaccepted CLA.
Kafka Connect Equivalent
There is no Kafka Connect equivalent since the Kafka Connect MQTT source connector is also based on an MQTT v3.1.1 paho library.
Proposal
We would like to contribute a new mqtt5 input and output component. Per §3.1.1 of the CONTRIBUTING.md, code written by non-Redpanda engineers needs to be reviewed and scoped by Redpanda. We are opening this issue to ask for comments on two points:
- Is an MQTT v5 input and output component still desired/wanted by Redpanda?
- Is the following scope properly sized for a MQTT v5 input and output component?
Intended Scope
- A new, separate
mqtt5input and output component. - Written in Go and based on paho.golang. This library is written by the same authors as the mqtt.paho.golang library that is already in use with the
mqttconnector. - Code structure will mirror the existing
mqttconnector where it makes sense. - All configuration fields shared with the existing
mqttconnector are carried over unchanged - New
mqtt5input config options:
| Field | Notes |
|---|---|
session_expiry_interval |
How long the broker retains session state after disconnect. |
receive_maximum |
Maximum in-flight QoS 1/2 messages the broker may send to this client simultaneously. |
maximum_packet_size |
Maximum packet size this client will accept from the broker. |
no_local |
Subscription option: do not receive messages published by this client. |
retain_as_published |
Subscription option: preserve the retain flag from the original publisher. |
retain_handling |
Subscription option: controls whether retained messages are sent on subscribe. |
topic_alias_maximum |
Maximum number of topic aliases this client will accept from the broker. 0 disables topic aliases. |
subscription_identifiers |
Enables the use of subscription identifiers by the client. |
will_* |
Full v5 will message config. |
- Metadata set by the
mqtt5input:
| Metadata key | Source |
|---|---|
mqtt5_user_properties_<key> |
User Properties from the PUBLISH packet. If a key appears only once, it is stored as-is. |
mqtt5_user_properties_<key>_<idx> |
User Properties from the PUBLISH packet can have the same key multiple times. If this happens they receive a zero-based index as a suffix |
mqtt5_content_type |
Content Type |
mqtt5_payload_format_indicator |
Payload Format Indicator |
mqtt5_response_topic |
Response Topic |
mqtt5_correlation_data |
Correlation Data |
mqtt5_subscription_identifier |
Subscription identifier sent by the broker |
- New
mqtt5output config options:
| Field | Notes |
|---|---|
session_expiry_interval |
How long the broker retains session state after disconnect. |
message_expiry_interval |
Per-message TTL. Broker drops the message if undelivered before expiry. |
topic_alias_maximum |
Maximum number of topic aliases this client will use when publishing. 0 disables topic aliases. |
will_* |
Full v5 will message config. |
- Metadata read by the
mqtt5output:
| Metadata key | Written to |
|---|---|
mqtt5_user_properties_<key> |
User Properties on the PUBLISH packet. Plain keys (mqtt5_user_properties_<key>) are written as single entries. Indexed keys (mqtt5_user_properties_<key>_0, _1, etc.) are reassembled into multiple User Properties with the same key, preserving order. |
mqtt5_content_type |
Content Type |
mqtt5_payload_format_indicator |
Payload Format Indicator |
mqtt5_response_topic |
Response Topic |
mqtt5_correlation_data |
Correlation Data |
- Other MQTT v5 Behavior:
- Shared Subscriptions are supported by the underlying library.
- Reason codes will be surfaced in error messages.
- Server-initiated DISCONNECT is handled by the underlying library.
- 主要语言
- Go
- 星标
- 8.8k
- 派生
- 975
- 平均合并
- 2 天 8 小时
- 30 天内合并 PR
- 72
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
redpanda-data/connect 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
redpanda-data/connect#4837 ·
维护者通常 1 天内回复
-
Iceberg output: the `row_operation` example uses a non-existent bloblang ternary可能已有人在做 @Tyagiquamar 于 25 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 84/100
redpanda-data/connect#4711 ·
维护者通常 1 天内回复
-
URL Secrets Exposure in Logs可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 85/100
redpanda-data/connect#4705 ·
维护者通常 1 天内回复
-
opensearch output: action does not support "create"可能已有人在做 @highlyavailable 于 94 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
redpanda-data/connect#4515 · 1 个 reaction ·
维护者通常 1 天内回复
-
Minor issue with slack_post component in Redpanda Connect.可能已有人在做 @hsdfat 于 72 天前认领。 未关闭needs investigation
难度 2/5 1-3 小时 新手友好度 65/100
redpanda-data/connect#4511 · 1 条评论 ·
维护者通常 1 天内回复
查看 redpanda-data/connect 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 85/100
维护者通常 1 天内回复
-
kind/bug
难度 1/5 1-3 小时 新手友好度 85/100
kubernetes-sigs/vsphere-csi-driver#4351 ·
维护者通常 1 天内回复
-
component/gpu-kubelet-plugin component/helm needs-triage
难度 2/5 1-3 小时 新手友好度 62/100
kubernetes-sigs/dra-driver-nvidia-gpu#1559 ·
维护者通常 1 天内回复
-
workflow: redundant goroutine in scheduleRetry可能已有人在做 @moodbased 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
JuliusBrussee/caveman#1266 · 2 条评论 ·
维护者通常 1 天内回复