highlightjs/highlight.js

Add ability to skip markup tests for edge cases that aren't supported yet

Open

#1,855 opened on Oct 8, 2018

View on GitHub
 (5 comments) (0 reactions) (0 assignees)JavaScript (22,960 stars) (3,497 forks)batch import
enhancementgood first issuehelp welcome

Description

I have a question about the Mocha tests, it's not clear to me if it's a condition that the commited tests in test/markup/<language>/ must all pass, or if failing edge cases can be kept there as a remainder of the fixes that must be implemented for a syntax.

For example, I noticed that the AsciiDoc syntax fails on many edge cases and advanced Asciidotor markup. So I was thinking of first creating all the tests with expected results, and then start to work on the syntax until all tests pass.

Ideally, even failing tests should be commited to master branch, because while working on the syntax the more tests are available the easier it is to see if and how new changes affect the expected results on all previous tests. In some cases, a small tweak to the syntax could fix many failed tests; in AsciiDoc this is quite likely, for example when introducing better support for verbatim and passthrough contents this might affect a whole range of elements currently being parsed incorrectly.

Since complex syntaxes like AsciiDoc might require quite a long time to get all the edge cases straight, having all the expected tests results in the Mocha test folder (regardlessly wether they pass or fail) would be a good way to keep track of the work, as well as to offer a sort of roadmap of missing features for the syntaxes.

This way, any contributor could use Mocha to see what unresolved syntax problems are pending, and anyone who notices that a syntax doesn't behave as expected could add a test for the fail case, even if he/she isn't planning to fix it him/herself.

I didn't find much on this topic in the documentation, as it focuses mainly on how to carry out tests on one's own new syntax. It doesn't say, for example, what to do if a new syntax is in a usable stage although some edge cases tests still fail — should these tests be removed from the test folder, or be kept there for future improvements? Obviously, developing a syntax definition can often be a multi-step job, where the first usable syntax might not cover edge cases or advanced features, but be usable nevertheless.

Contributor guide