Llama3 instruct prompt template missing BOS token

The issue is in the Llama3 chat prompt template. Look at the file llama_cpp/llama_chat_format.py for the format_llama3 function. Compare the output between versions 0.2.77 and 0.2.78 to see the missing BOS token '<|begin_of_text|>'. Add the token back to the prompt construction. Test by importing the function and checking the output matches the expected format from the Meta documentation.

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Llama3 models using the prompt template in llama-cpp-python>0.2.77 are missing the BOS token and then the model quality is degraded.

For example, the Meta doc on Llama 3 has prompt template examples here: https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/

Current Behavior

Result of an empty format_llama3({}) after from llama_cpp.llama_chat_format import format_llama3:

0.2.78:

ChatFormatterResponse(prompt='<|start_header_id|>assistant<|end_header_id|>\n\n', stop='<|eot_id|>', stopping_criteria=None, added_special=False)

0.2.77:

ChatFormatterResponse(prompt='<|begin_of_text|><|start_header_id|>assistant<|end_header_id|>\n\n', stop='<|eot_id|>', stopping_criteria=None)

Environment and Context

• Operating System, e.g. for Linux:

22.6.0 Darwin Kernel Version 22.6.0: Mon Apr 22 20:49:37 PDT 2024; root:xnu-8796.141.3.705.2~1/RELEASE_ARM64_T6000 arm64

• SDK version, e.g. for Linux:

Python 3.10.13
GNU Make 3.81
$ g++ --version

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

Steps to Reproduce

Install versions 0.2.77 and 0.2.78 and test anything with a Llama 3 model and llm.create_chat_completion. In addition, import the templates and check the difference.

I am using https://huggingface.co/bartowski/Meta-Llama-3-8B-Instruct-GGUF/tree/main as a model for testing, particularly the Q5_K_M quantization, but this should affect all models.