Prompt template

[d5423197]

Hello, community,

From the source code, the language and code snippet are two major inputs to build the prompt. I have trouble reading rust code. So I am looking for help with the format of the prompt sent to the model. I want to check how language is used in building the prompt.

Related code block:

tabby/crates/tabby/src/services/completion/completion_prompt.rs

Line 48 in 99d49a9

pub fn build(&self, language: &str, segments: Segments, snippets: &[Snippet]) -> String {

Any help will be appreciated!

Thanks,

ZD

[wsxiaoys]

The unit test should be self-explanatory for this case.

tabby/crates/tabby/src/services/completion/completion_prompt.rs

Line 349 in 99d49a9

fn test_build_prefix_readable() {

If you run tabby locally with repository context enabled, you could find the final prompt being feed to LLM in side completion events of ~/.tabby/events, which could be used as a cross reference for understanding the code.

[d5423197]

Hello there,

Thanks for your reply. Will do.

ZD

[ANYMS-A]

Hi, followed the describtion above, I found the demo prompt template in the ~/.tabby/events However, I have a few quesitons about the template based on the content in #2674

Q1: It says that the prefix is only the code before the cursor, but during my test，I found some repo level codes are also added to the fim_prefix texts, e.g: the code below is what I copy from the ~/.tabby/events under the fim_prefix filed, the import statements started with the "#" is actually from another .py file in the repo,may I ask how this part of code(imports from the neighbor .py file) is selected and added to the prefix for the code-completion task， thanks!

# import torch
# from transformers import (
#     AutoModelForCausalLM,
#     AutoTokenizer,
#     HfArgumentParser,
#     Trainer,
#     TrainingArguments,
# )
# from datasets import Dataset, load_dataset
#
#
# class ConstantLengthDataset:
#     """
#     Iterable dataset that returns constant length chunks of tokens from stream of text files.
#         Args:
from torch.utils.data import IterableDataset
from torch.utils.data.sampler import Sampler

class EvalDataset(IterableDataset):

[wsxiaoys]

For more update to date information, check #2674 to see how prompt is constructed for completion requests.

[ANYMS-A]

Thank you very much, also I noticed in the rust source code, there 2 terminologies which called:

1. Segments
2. Snippets
   May I ask what is the difference between them? or what are the objects they refer to? Snippets IMO looks like the real code snippet， then what is a Segment？