Storia-AI/sage
Feature request: Linkify code symbols in the model response
Offen
#54 geöffnet am 04.10.2024
hacktoberfest🔥 difficulty: 3
Repository-Metriken
- Stars
- (1.267 Sterne)
- PR-Merge-Metriken
- (PR-Metriken ausstehend)
Beschreibung
Given an LLM response that contains a code symbol (e.g. class name, method, etc.), we should link it to its exact location on GitHub. For instance, if you're chatting with Hugging Face's Transformers library:
-
Original response: "To define a BERT model using the Hugging Face Transformers library, you can use the BertModel class."
-
Link-ified response: "To define a BERT model using the Hugging Face Transformers library, you can use the BertModel class."
This should be done as a post-processing step in chat.py, once the model is done streaming.
Work items:
- Produce an AST (Abstract Syntax Tree) for every file in the repository. This can be done via the
tree_sitterlibrary, which we're already using in chunk.py. - Identify class / method / file names in the model response, potentially using regular expressions.
- Look up the strings matched by the regular expressions in the per-file ASTs.
- Map the code symbol to a github URL.