lark-parser/lark

Using lark for a language server

Open

#684 opened on Sep 7, 2020

View on GitHub
 (22 comments) (1 reaction) (0 assignees)Python (4,275 stars) (384 forks)batch import
discussionenhancementhelp wanted

Description

Hey, I'm wondering how conducive lark would be for creating a language server implementation for a given DSL.

For some background:

  • I have a custom DSL I'm using to describe device interfaces and then using jinja2 to generate the specific interface into the target language
  • The main implementation of this obviously in Python
  • I'm using antlr4 to define the grammar, which is used to generate a python and typescript implementations.
  • I have written a basic language server extension for vscode in typescript using the antlr completion engine antlr4-c3. This provides live validation and auto-completion which is really useful.

Problems:

  • Antlr's python implementation is quite slow (~350ms to parse + transform a sample file compared to ~50ms with lark)
  • I'm having to duplicate a lot of work between python & typescript, though admittedly they are solving slightly different problems.

I would quite like to use to Lark and implement the language server as part of the main python package if possible.

I realize a language sever requires some specific parser features namely:

  • Error tolerance: This can partly be resolved by making the grammar itself more error tolerant and solving those errors later. I notice lark also an error puppet which can maybe be used for this purpose.
  • Support for auto completion information: Its is namely done by parsing an incomplete source file up to the current caret position and extracting both the next possible tokens and the currently parser rule context from the parser. This can be combined with info from the last successful parse to produce auto completion options.

I'm wondering if lark as any support for these features? And If so how I might go about implementing them?

If something like this is possible, it might also be cool to write a language server for the lark grammar itself to aid development.

Cheers, Rob

Contributor guide

Tech stack
python
Domain
backenddeveloper experiencetooling
Issue type
research
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
Knowledge of Lark parsing libraryUnderstanding of Language Server ProtocolFamiliarity with ANTLR
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Explore Lark's error tolerance capabilities (e.g., 'error puppet' feature) and its ability to parse incomplete source files for auto completion. Investigate if Lark's parsing can be used to extract context and possible tokens at a given position. Consider implementing a prototype language server for a simple DSL using Lark to validate feasibility. Look into existing work or discussions in the Lark community about LSP integration.
Using lark for a language server · lark-parser/lark#684 | Good First Issue