LSP: Support `textDocument/semanticTokens/full` and `textDocument/semanticTokens/full/delta`

Summary

The new LSP only have support for the textDocument/semanticTokens/range due to performance reason. As quoted in one of the comment in the code:

// Using only range handling has shown to be more performant than using a combination of full/edits/range // handling, especially for larger files. With range handling, we only need to compute tokens for whatever // is in view, while with full/edits handling we need to compute tokens for the entire file and then // potentially run a diff between the old and new tokens.

It would be awesome if the LSP can support other types of semanticTokens requests as well.

Background and Motivation

Other editors such as Neovim or Emac that want to integrate with the new LSP might not be able to utilize the semantic token support because those clients have not implement the range request. It would be great if the the LSP should support all of these capabilities and let's the client decide if they want to improve performance by using the range request. If clients chose to use the full alternative, they must live with the performance hit. Before making this issue, I understand that the priority of supporting other editors is low, nor semanticTokens is a deal breaker for the coding experience. Just want to note down something that we can improve upon.

Proposed Feature

The LSP should support either textDocument/semanticTokens/full or textDocument/semanticTokens/full/delta or both

Alternative Designs