About This Task

Prompt Explorer is a sandbox environment where you can:

• Test different versions of your prompts before bringing them to production.
• Modify your prompts.
• Compare the same prompt across up to three pipelines.
• Update your pipelines with refined prompts.
• Save custom prompts to the library and reuse them later.

Accessing Prompt Explorer

Prompt Explorer is only available for deployed pipelines that use at least one Generator component. (You use LLMs through Generators in deepset Cloud.)

In the navigation, click Prompt Studio:

It opens Prompt Explorer, with Prompt Editor and a template repository.

Testing and Modifying Prompts

Use the Prompt Editor within the Prompt Explorer to modify your prompts and observe how they affect the results. You can experiment with different versions of prompts for the same query and review the history of your results.

Prompts must conform to the Jinja2 template language. Prompt Explorer supports Markdown formatting of prompts.

Using the expanding list at the top, you can choose a pipeline. The pipeline must have at least one PromptBuilder component. Once you choose a pipeline, its prompt immediately shows in the Prompt Editor at the bottom. If multiple PromptBuilders are in the pipeline, you can choose the one you want to work with.

Use the Editor to modify your prompt and run queries. To check the history of your queries, click the clock icon next to the query (it's only available if you select a single pipeline).

You can preview the rendered prompt by clicking the More Actions icon next to the generated answer.

Legend:

1. Pipeline selection. You can only select pipelines that have at least one PromptBuilder component.
2. Prompt template preview.
3. Rendered prompt preview. This shows the prompt with variables replaced with real values.
4. PromptBuilder selection. This is available if there is more than one PromptBuilder in the pipeline.
5. Query.
6. Query history.
7. Editable prompt.
8. Update the prompt in your pipeline. This replaces the original prompt in the pipeline with the one you currently have in Prompt Editor.

Using Prompt Templates (Coming Soon)

📘

Prompt templates are currently only available for v1 pipelines. Support for v2 pipelines is coming soon.

Choose a prompt from a library and test it in Prompt Explorer. The prompt you choose shows in Prompt Editor where you can further modify it if needed.

In Prompt Editor, click Templates. The Custom tab lists the prompts you saved, and the deepset tab lists prompts curated and tested by deepset.

Saving Custom Prompts

Save your prompts in the library so you can use them in the future.

1. In Prompt Explorer, click Templates.

2. On the Custom tab, click Create Custom Prompt.

3. Name the prompt, type the prompt text, and save your prompt. Add variables to the prompt using Jinja2 syntax, for example:

   Python

   Given these documents, answer the question.\nDocuments:
       {% for doc in documents %}
           {{ doc.content }}
       {% endfor %}
   
       \nQuestion: {{query}}
       \nAnswer:
   

Updating the Prompt in the Pipeline

Once you've tested the new prompt and decided you want to use it in your pipeline, click the Update button to replace the prompt in the pipeline with the one you've just tested in Prompt Explorer.

Comparing Pipelines

You can compare up to three pipelines. This is useful if you have pipelines using different models to understand how they perform. You pass the same query to all the pipelines you're comparing, but you can modify the prompts for each pipeline.

To add a pipeline to comparison, click Add Pipeline at the top of the Playground.

You can then add or remove pipelines using the plus (+) and minus (-) icons:

Each pipeline receives the same query, but you can change the prompt for each pipeline.