Create a Pipeline in Studio

Use an intuitive drag-and-drop interface to build your pipelines.

About Studio

Studio is an easy way to build and visualize your pipelines. In Studio, you simply drag components from the components library and drop them onto a canvas, where you can customize their parameters and define connections. It helps you visualize your pipeline and offers guidance on component compatibility. You can also switch to the YAML view any time; everything you do in Studio is synchronized with the pipeline YAML configuration.

ℹ️

Studio is available for 2.0 pipelines only. For 1.0 pipeline, you can use visualizer.

Using Studio

This image shows how to access some functionalities in Studio. The numbers in the list below correspond to the numbers in the image.


  1. Component library. Expand a component group and drag a selected component onto the canvas.
  2. Click the component name to change it.
  3. Click a component to access the menu for deleting, duplicating, and accessing component's documentation.
  4. Draw lines joining components outputs and inputs.
  5. Switch to the YAML view.

Considerations for Building Pipelines

There are a couple of things you should know when building in Studio:

  • Pipeline start: Your pipeline must start with an input component. By default, there are two input components: Query and Filters (DictInput component), but you can use other components from the Input group. This is needed so that we can map the API inputs to your pipeline.

  • Pipeline end: Your pipeline must end with an output component. The output of a pipeline is answers, documents, or both. This lets us map API outputs to your pipeline.

  • Complex parameters: Some components take parameters that are not Python primitives. These parameters are configured as YAML. For example, PromptBuilder's template or ConditionalRouter's routes. These paramteres' configuration can affect the component's inputs and outputs. For instance, if you add Query and Documents as variables in the PromptBuilder's template, they'll be listed as required inputs. Otherwise, they won't be.
    For configuration examples, check the component's documentation.

  • Indexing and query pipelines: When creating a pipeline, you can see two tabs:

    • Indexing: Here you're building your indexing pipeline that defines how your files are preprocessed. Whenever you add a file, it is preprocessed by all deployed pipelines.

    • Query: Here you're building your query pipeline that describes how the query is resolved.

      Pipeline Studio with indexing and query tabs highlightedh

Prerequisites

  • To learn about how pipelines and components work in deepset Cloud, see Pipeline Components and Pipelines.
  • To use a hosted model, Connect to Model Providers first so that you don't have to pass the API key within the pipeline. For Hugging Face, this is only required for private models. Once deepset Cloud is connected to a model provider, just pass the model name in the model parameter of the component that uses it in the pipeline. deepset Cloud will download and load the model. For more information, see Language Models in deepset Cloud.

Create a Pipeline From an Empty File

  1. Log in to deepset Cloud and go to Pipelines.

  2. Click Create Pipeline>From empty file.

  3. Give your pipeline a name, choose deepset Cloud 2.0 as the version, and click Create Pipeline.
    Your pipeline shows on the Pipelines page in the All tab.

  4. Click the More Actions menu beside your pipeline and choose Studio.

    The more actions menu expanded and the edit option highlighted
  5. First, build your indexing pipeline:

    1. Add the inputs for your pipeline. For indexing pipelines, the input is usually FilesInput or TextInput.
    2. Choose Preprocessors and Converters, and any other components as needed.
    3. Connect the components by dragging a line from one component's input to another component's output. The connections are immediately validated.
    4. Add the pipeline output.
  6. Switch to the Query tab:

    1. Add the inputs for your pipeline. For query pipelines. this is usually Query and, optionally, DictInput (filters).
    2. Add components from the components library and define their connections.
    3. Add the pipeline output.
  7. Save your pipeline.

Create a Pipeline From a Template

  1. Log in to deepset Cloud and go to Pipelines.

  2. Click Create Pipeline > Select template.
    There are templates available for various tasks. They work out of the or you can use them as a starting point for your pipeline.

  3. Switch to the deepset Cloud 2.0 tab, find a template that best matches your use case, hover over it, and click Use Template.

    The template selection process shown, first you click on a template group, then you choose Use template on the template card.
  4. Give your pipeline a name and click Create Pipeline. Your pipeline shows on the Pipelines page in the All tab.

  5. Depending on what you want to do:

    1. To test your pipeline, deploy it first. Click Deploy next to your pipeline name, wait until it's indexed and then test your pipeline in Playground.
    2. To edit your pipeline, see Step 4 in Create a pipeline from an empty file.

What To Do Next

  • To use your pipeline, deploy it. Click Deploy in the top right corner of Pipeline Studio.
  • To test your pipeline, wait until it's indexed and then go to Playground. Make sure your pipeline is selected, and type your query.
  • To view pipeline details, such as statistics, feedback, or logs, click the pipeline name. This opens the Pipeline Details page.
  • To let others test your pipeline, share your pipeline prototype.