yugabyte/yugabyte-db

yb-admin should not show a FATAL with a stack trace if it is unable to contact master leader

Open

#1,504 opened on Jun 8, 2019

View on GitHub
 (0 comments) (0 reactions) (0 assignees)C (8,229 stars) (1,003 forks)batch import
area/docdbgood first issuekind/bugpriority/medium

Description

Jira Link: DB-2354 F20190607 16:58:04 ../../src/yb/tools/yb-admin_client.cc:165] Check failed: _s.ok() Bad status: Timed out (yb/rpc/rpc.cc:199): Could not locate the leader master: GetLeaderMasterRpc(addrs: [127.0.0.1:7100], num_attempts: 336) passed its deadline 14656.514s (passed: 60.128s) @ 0x10cd76500 google::LogDestination::LogToSinks() @ 0x10cd7550b google::LogMessage::SendToLog() @ 0x10cd75eb5 google::LogMessage::Flush() @ 0x10cd7a9bf google::LogMessageFatal::~LogMessageFatal() @ 0x10cd76ea9 google::LogMessageFatal::~LogMessageFatal() @ 0x109480638 yb::tools::ClusterAdminClient::Init() @ 0x1094a7237 yb::tools::enterprise::ClusterAdminClient::Init() @ 0x10949d4e1 yb::tools::ClusterAdminCli::Run() @ 0x10949f342 main @ 0x7fff73f33ed9 start

Contributor guide

Tech stack
cpp
Domain
backendcli
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
C++Glog loggingError handling
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45
Research direction
Investigate the file src/yb/tools/yb admin client.cc around line 165 where a CHECK failure occurs. The issue is that when the tool cannot contact the master leader, it logs a fatal error and crashes. The desired behavior is to handle the timeout gracefully, perhaps by returning an error status or logging an error without a stack trace. Check if there are other places in the same file or similar admin tools that handle such errors appropriately, and implement a consistent approach. Also verify if the issue is still reproducible with the current codebase.