elastic/beats

Update MongoDB protocol with new opcodes

Open

#6191 opened on Jan 26, 2018

View on GitHub
 (24 comments) (0 reactions) (0 assignees)Go (9,173 stars) (3,451 forks)batch import
PacketbeatTeam:Service-Integrationsgood first issuehelp wanted

Description

Please post all questions and issues on https://discuss.elastic.co/c/beats before opening a Github Issue. Your questions will reach a wider audience there, and if we confirm that there is a bug, then you can open a new issue.

For security vulnerabilities please only send reports to security@elastic.co. See https://www.elastic.co/community/security for more information.

For confirmed bugs, please report:

  • Version: packetbeat-6.1.2-linux-x86_64
  • Operating System: Linux contents-mongo-dev-01 4.9.20-11.31.amzn1.x86_64 #1 SMP Thu Apr 13 01:53:57 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • Mongodb Version: db version v3.2.13
  • Steps to Reproduce:

[root@contents-mongo-dev-01 packetbeat-6.1.2-linux-x86_64]# /beat/packetbeat-6.1.2-linux-x86_64/packetbeat -c /beat/packetbeat-6.1.2-linux-x86_64/packetbeat.yml -e

Q) When packetbeat is executed, the following error occurs.

2018/01/26 06:30:01.311175 mongodb_parser.go:42: ERR Unknown operation code: 2018/01/26 06:30:01.311872 mongodb_parser.go:42: ERR Unknown operation code: 2018/01/26 06:30:01.311902 mongodb_parser.go:42: ERR Unknown operation code: 2018/01/26 06:30:01.311928 mongodb_parser.go:42: ERR Unknown operation code: 2018/01/26 06:30:01.403019 mongodb_parser.go:42: ERR Unknown operation code: 2018/01/26 06:30:01.403631 mongodb_parser.go:42: ERR Unknown operation code:

Q) My setup is as below and I simply linked it to the logstash. Please check the cause of the above error.

[root@contents-mongo-dev-01 packetbeat-6.1.2-linux-x86_64]# cat packetbeat.yml | grep -v '#'

packetbeat.interfaces.device: any

packetbeat.flows: timeout: 30s period: 10s

packetbeat.protocols:

  • type: mongodb ports: [27017] send_request: true send_response: true max_docs: 0 max_doc_length: 0

setup.template.settings: index.number_of_shards: 3 setup.kibana:

output.elasticsearch: hosts: ["internal-bh-elasticsearch-lb-1602908268.ap-northeast-2.elb.amazonaws.com:9200"]

logging.level: error

Contributor guide

Tech stack
gomongodb
Domain
backenddatabase
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Go programmingMongoDB protocolPacketbeat codebase
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
Investigate the MongoDB wire protocol to identify new opcodes added in later versions (e.g., MongoDB 3.6+). Update the parser in https://github.com/elastic/beats/blob/master/packetbeat/protos/mongodb/mongodb parser.go to handle these opcodes. Verify the fix by reproducing the error with a MongoDB 3.2.13 setup and ensure no 'Unknown operation code' errors appear.