Quoted scalar continuation line inside a mapping is accepted when it starts with a tab instead of the required space indentation

Start in ScanQuotedScalar in src/scantoken.cpp, where the scanning parameters (indent, onTabInIndentation) are set, and compare with ScanPlainScalar in the same file. The shared ScanScalar loop in src/scanscalar.cpp handles the indentation exit. Reproduce with YAML::Load on foo: "bar\n\tbaz" and the single-quote variant. Done when both inputs throw YAML::ParserException and the existing scanner tests still pass.

Location (jbeder/yaml-cpp @ default branch c5e6536):

• src/scantoken.cpp:343-353 — Scanner::ScanQuotedScalar parameters
• src/scantoken.cpp:306-312 — ScanPlainScalar, for comparison

  // setup the scanning parameters
  ScanScalarParams params;
  RegEx end = (single ? RegEx(quote) & !Exp::EscSingleQuote() : RegEx(quote));
  params.end = &end;
  params.eatEnd = true;
  params.escape = (single ? '\'' : '\\');
  params.indent = 0;
  params.fold = FOLD_FLOW;
  params.eatLeadingWhitespace = true;
  params.trimTrailingSpaces = false;
  params.chomp = CLIP;
  params.onDocIndicator = THROW;

Problem
A multi-line quoted mapping value whose continuation line begins with a tab at column 0 is accepted and folded:

YAML::Load("foo: \"bar\n\tbaz\"");  // no exception, foo == "bar baz"
YAML::Load("foo: 'bar\n\tbaz'");      // same for single quotes

ScanQuotedScalar sets params.indent = 0 and doesn't set onTabInIndentation, whereas ScanPlainScalar uses GetTopIndent() + 1 and onTabInIndentation = THROW. Expected: a YAML::ParserException for both inputs.

This is not a request to forbid tabs in quoted scalars. The problem is that the tab replaces the indentation the spec requires. The value of foo (a mapping entry at indentation 0) is s-l+flow-in-block(0) = ns-flow-node(1,FLOW-OUT) (YAML 1.2.2 8.2.3, [197]), so each continuation line of the quoted scalar starts with [69] s-flow-line-prefix(n) ::= s-indent(n) s-separate-in-line? with n = 1 (6.3 Line Prefixes). s-indent(1) is one space. Tabs are only allowed in the optional separation after it. Per 6.1, tab characters must not be used in indentation.

So these should keep working unchanged:

foo: "bar
 	baz"      # one space, then a tab: valid, folds to "bar baz"

and root-level quoted scalars (where the required indentation is 0) are not affected. Flow context (inside [...]/{...}) has different rules and is out of scope here.

Reproduction

#include <iostream>
#include "yaml-cpp/yaml.h"

int main() {
  for (const char* in : {"foo: \"bar\n\tbaz\"", "foo: 'bar\n\tbaz'"}) {
    try {
      std::cout << "accepted: [" << YAML::Load(in)["foo"].as<std::string>() << "]\n";
    } catch (const YAML::ParserException& e) {
      std::cout << "rejected: " << e.what() << "\n";
    }
  }
}

Observed (gtest on the commit above):

Expected: Load("foo: \"bar\n\tbaz\"") throws an exception of type ParserException.
  Actual: it throws nothing.
Expected: Load("foo: 'bar\n\tbaz'") throws an exception of type ParserException.
  Actual: it throws nothing.

Environment:

• Commit: c5e65360373d1d99d6b6848c991cd4fcee435009 (master)
• Toolchain: GCC 11.5.0

Already fixed in goccy/go-yaml
goccy/go-yaml is another YAML implementation of the same spec; it fixed the same case. PR: https://github.com/goccy/go-yaml/pull/592 — "Fix parsing of tab characters" (merged 2024-12-14T16:29:05Z, merge commit a37e684cac8). In both the single- and double-quoted scanners, a tab at the start of a continuation line is now rejected ("tab character cannot be used for indentation in ... text") when it is not past the enclosing indentation column (s.lastDelimColumn >= s.column); leading spaces are still skipped as before. The PR also changes tab handling in the general scanner; the quoted-scalar hunks below are the part that addresses this behaviour.

How go-yaml fixed it:

--- a/scanner/scanner.go
+++ b/scanner/scanner.go
@@ -250,7 +250,17 @@ func (s *Scanner) scanSingleQuote(ctx *Context) (*token.Token, error) {
 				}
 			}
 			continue
-		} else if isFirstLineChar && (c == ' ' || c == '\t') {
+		} else if isFirstLineChar && c == ' ' {
+			continue
+		} else if isFirstLineChar && c == '\t' {
+			if s.lastDelimColumn >= s.column {
+				return nil, ErrInvalidToken(
+					token.Invalid(
+						"tab character cannot be used for indentation in single-quoted text",
+						string(ctx.obuf), s.pos(),
+					),
+				)
+			}
 			continue
 		} else if c != '\'' {
 			value = append(value, c)
@@ -338,7 +348,17 @@ func (s *Scanner) scanDoubleQuote(ctx *Context) (*token.Token, error) {
 				}
 			}
 			continue
-		} else if isFirstLineChar && (c == ' ' || c == '\t') {
+		} else if isFirstLineChar && c == ' ' {
+			continue
+		} else if isFirstLineChar && c == '\t' {
+			if s.lastDelimColumn >= s.column {
+				return nil, ErrInvalidToken(
+					token.Invalid(
+						"tab character cannot be used for indentation in double-quoted text",
+						string(ctx.obuf), s.pos(),
+					),
+				)
+			}
 			continue
 		} else if c == '\\' {
 			isFirstLineChar = false

Suggested fix
In block context, scan quoted scalars with the parent indentation + 1 and onTabInIndentation = THROW, as ScanPlainScalar already does, while still ending them only at the closing quote:

diff --git a/src/scanscalar.cpp b/src/scanscalar.cpp
--- a/src/scanscalar.cpp
+++ b/src/scanscalar.cpp
@@ -195,8 +195,9 @@ std::string ScanScalar(Stream& INPUT, ScanScalarParams& params) {
     moreIndented = nextMoreIndented;
     pastOpeningBreak = true;
 
-    // are we done via indentation?
-    if (!emptyLine && INPUT.column() < params.indent) {
+    // are we done via indentation? (quoted scalars only end at their closing
+    // quote)
+    if (!params.eatEnd && !emptyLine && INPUT.column() < params.indent) {
       params.leadingSpaces = true;
       break;
     }
diff --git a/src/scantoken.cpp b/src/scantoken.cpp
--- a/src/scantoken.cpp
+++ b/src/scantoken.cpp
@@ -345,12 +345,13 @@ void Scanner::ScanQuotedScalar() {
   params.end = &end;
   params.eatEnd = true;
   params.escape = (single ? '\'' : '\\');
-  params.indent = 0;
+  params.indent = (InFlowContext() ? 0 : GetTopIndent() + 1);
   params.fold = FOLD_FLOW;
   params.eatLeadingWhitespace = true;
   params.trimTrailingSpaces = false;
   params.chomp = CLIP;
   params.onDocIndicator = THROW;
+  params.onTabInIndentation = THROW;
 
   // insert a potential simple key
   InsertPotentialSimpleKey();

Verified on c5e6536 (GCC 11.4.0, bundled GoogleTest 1.16.0): the test above fails without this patch and passes with it; no other test results change.

I can submit a PR for this.

Related: searched issues and PRs (all states) for tab + quoted scalar / tab indentation / multiline; no matching report (#1506/#1508 are about emitting control characters, unrelated).