Create a Pipeline in Pipeline Builder

Use an intuitive, low code, drag-and-drop interface to build your pipelines. Easily switch between visual and code representations.

---

About Builder

Builder is an easy way to build and visualize your pipelines. In Builder, you simply drag components from the components library and drop them onto a canvas, where you can customize their parameters and define connections. It helps you visualize your pipeline and offers guidance on component compatibility. You can also switch to the YAML view anytime; everything you do in Builder is synchronized with the pipeline YAML configuration.

Using Builder

This image shows how to access the basic functionalities in Builder. The numbers in the list below correspond to the numbers in the image.

1. Component library. Expand a component group and drag a selected component onto the canvas to add it to your pipeline. You can also use the search bar to find components. By default, the component library hides legacy components. To show them, click More Actions next to the search field and choose Show deprecated components.
2. A component card. Click the component name to change it.
3. Component connections. Click an input connection on one component and an output connection on another to link them.
   
4. Click a component card to access the menu for deleting, running, duplicating, and accessing the component's documentation.
   Tip: Use the Run action to run a component in isolation and check if it works correctly.
5. Use this menu to:
   • Enable autosave.
   • View, restore, and deploy pipeline versions.
   • Run the pipeline without needing to deploy it.
   • Delete all components from the canvas.
   • Download pipeline YAML or Python configuration.
   • Switch between YAML and visual editors.
6. Use this menu to:

• Zoom in and out of the canvas.
• Show components in full screen.
• Lock the canvas.
• Show or hide optional parameters.
• Return to the default positioning of components.

Finding Components with Semantic Search

Builder includes an AI-powered semantic search feature to help you find the right components for your pipeline. When you search in the component library, the system uses both traditional keyword matching and semantic understanding to surface relevant components. Type natural language queries like "generate text from documents" or "rank search results by relevance" to find relevant component suggestions.

Running Components and Pipelines

You can run single components to understand their inputs and outputs and check if they work correctly before you save and deploy the whole pipeline. Click a component card and choose Run.

You can also run the whole pipeline without needing to deploy it first. Just click the Run button on the navigation bar.

Organizing Components with Groups

You can group components visually to organize complex pipelines and improve readability. Groups help you keep related components together and you can collapse them to reduce visual clutter.

To create a group, click the Groups button (group icon) in the bottom right corner of the canvas. Then click and drag on the canvas to draw a rectangle around the components you want to group.

Keyboard shortcut: Press Cmd+G (Mac) or Ctrl+G (Windows/Linux) to activate group drawing mode, then draw the rectangle. Press Esc to cancel drawing mode.

Groups are saved with your pipeline and restored when you reload the builder.

Considerations for Building Pipelines

There are a couple of things you should know when building in Builder:

• Pipeline start: Your pipeline must start with an input component. Query pipelines always take Input as the first component.
• Pipeline end: Query pipelines finish with the Output component connected to a component that passes answers and often also documents to it.
• Complex parameters: Some components take parameters that are not Python primitives. These parameters are configured as YAML.
  For example, PromptBuilder's template or ConditionalRouter's routes use Jinja2 templates. These parameters configurations can affect the component's inputs and outputs, depending on the variables you add to the template. For instance, if you add Query and Documents as variables in the PromptBuilder's template, they'll be listed as required inputs. Otherwise, they won't be.
  For configuration examples, check the component's documentation in the Pipeline Components section.

Finding Components with Semantic Search

Builder includes an AI-powered semantic search feature to help you find the right components for your pipeline. When you search in the component library, the system uses both traditional keyword matching and semantic understanding to surface relevant components. Type natural language queries like "generate text from documents" or "rank search results by relevance" to find relevant component suggestions.

Running Components and Pipelines

You can run single components to understand their inputs and outputs and check if they work correctly before you save and deploy the whole pipeline. For information on how it works, see Run Components and Pipelines in Builder.

You can also run the whole pipeline without needing to deploy it first. Just click the Run button on the navigation bar.

Organizing Components with Groups

You can group components visually to organize complex pipelines and improve readability. Groups help you keep related components together and can be collapsed to reduce visual clutter.

Create a Group

Draw a group rectangle:

1. Click the Groups button (group icon) in the bottom right corner of the canvas.
2. Click and drag on the canvas to draw a rectangle around the components you want to group.
3. Release to create the group.

Group selected components:

1. Select the components you want to group by clicking on them while holding Shift.
2. Press Cmd+G (Mac) or Ctrl+G (Windows/Linux) to group the selected components.

Keyboard shortcut: Press Cmd+G (Mac) or Ctrl+G (Windows/Linux) to activate group drawing mode, then draw the rectangle. Press Esc to cancel drawing mode.

Work with Groups

• Rename a group: Click on the group name and type a new name.
• Change group color: Click the color picker on the group to choose a different color.
• Move components: Drag components in and out of groups. Components inside a group move with the group.
• Collapse/expand: Click the collapse button to hide components inside the group for a cleaner view. Collapsed groups show a solid border.
• Delete a group: Click the delete button on the group. This removes only the group container—components inside remain on the canvas.

Groups are saved with your pipeline and restored when you reload the builder.

Understanding the Pipeline YAML Format

Expand to view the YAML explanation

Every pipeline is backed by a YAML configuration. You can access and edit it directly by switching to the YAML editor in Builder. Any changes you make in the visual editor are reflected in the YAML, and the other way around.

A pipeline YAML has four main sections:

• components: Defines the components in your pipeline and their parameters.
• connections: Defines how components pass data to each other.
• inputs: Maps pipeline-level inputs to the inputs of specific components.
• outputs: Maps component outputs to the pipeline's final output.

Components

Each entry under components defines a component by its name and type. The name is optional and is a label you choose; you reference it in connections, inputs, and outputs. The type is the fully qualified class path of the component, which you can find on the component's documentation page.

Component Information

You can drag a component from the Component Library onto the visual canvas and then switch to the YAML editor to see the component's definition.

Use init_parameters to configure the component. Nested components, such as document stores, are also defined inline here:

components:
  query_embedder:
    type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
    init_parameters:
      normalize_embeddings: true
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          embedding_dim: 768
      top_k: 20

Connections

The connections section defines how data flows between components. Each connection specifies a sender and a receiver in the format component_name.output_name and component_name.input_name:

connections:
- sender: query_embedder.embedding
  receiver: embedding_retriever.query_embedding
- sender: embedding_retriever.documents
  receiver: ranker.documents

Inputs

The inputs section maps pipeline-level inputs to the inputs of specific components. Each key is a pipeline input name (for example, query or filters). The list under each key specifies which component inputs receive that value:

inputs:
  query:
  - "query_embedder.text"
  - "ranker.query"
  filters:
  - "embedding_retriever.filters"

Outputs

The outputs section maps the pipeline's named outputs to specific component outputs. Each key is the name of a pipeline output, and the value is a component_name.output_name reference:

outputs:
  documents: "ranker.documents"

Example: Semantic Document Search Pipeline

Here's a complete query pipeline that embeds the query, retrieves semantically similar documents, and re-ranks the results:

components:
  query_embedder:
    type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
    init_parameters:
      normalize_embeddings: true
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          embedding_dim: 768
      top_k: 20

  ranker:
    type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
    init_parameters:
      model: tomaarsen/Qwen3-Reranker-0.6B-seq-cls
      top_k: 20

connections:
- sender: query_embedder.embedding
  receiver: embedding_retriever.query_embedding
- sender: embedding_retriever.documents
  receiver: ranker.documents

inputs:
  query:
  - "query_embedder.text"
  - "ranker.query"
  filters:
  - "embedding_retriever.filters"

outputs:
  documents: "ranker.documents"

This pipeline works as follows:

1. query_embedder converts the query into a vector embedding.
2. embedding_retriever uses that embedding to find the most semantically similar documents in the document store.
3. ranker re-ranks the retrieved documents by relevance.
4. The pipeline returns the re-ranked documents as its output.

Prerequisites

• To learn about how pipelines and components work, see Pipeline Components and Pipelines.
• To use a hosted model, connect to model providers first so that you don't have to pass the API key within the pipeline. For Hugging Face, this is only required for private models. Once Haystack Enterprise Platform is connected to a model provider, just pass the model name in the model parameter of the component that uses it in the pipeline. Haystack Enterprise Platform will download and load the model. For more information, see Language Models in Haystack Enterprise Platform.
• To reuse custom model definitions when you configure components in Pipeline Builder, make sure they're added on the Integrations page in Settings. For details, see Add Custom Model Definitions.

Create a Pipeline From a Template

1. Log in to Haystack Enterprise Platform and go to Pipeline Templates.
   There are templates available for various tasks. They work out of the box or you can use them as a starting point for your pipeline.

2. Find a template that best matches your use case, hover over it, and click Use Template.

   

3. Give your pipeline a name and click Create Pipeline. You're redirected to Pipeline Builder, where you can view and edit your pipeline. Make sure you choose an index for your pipeline on the document store card.

4. Depending on what you want to do:

   1. To test your pipeline, deploy it first. Click Deploy in the upper right corner, wait until it's deployed, and then test your pipeline in Playground.
   2. To edit your pipeline, see Step 5 in Create a pipeline from an empty file.

Create a Pipeline From an Empty File

1. Log in to Haystack Enterprise Platform and go to Pipeline Templates.
2. In the top right corner, click Create empty pipeline.
3. Give your pipeline a name and click Create Pipeline.
   You're redirected to Pipeline Builder.
4. Add the inputs for your pipeline. Query pipelines must start with the Input component.
5. Add components from the components library and define their connections.
6. To give your pipeline access to files from a Haystack Platform workspace, add Retrievers with a matching Document Store. For example, to use OpenSearchDocumentStore, add OpenSearchEmbeddingRetriever or OpenSearchBM25Retriever to your pipeline and connect them to the OpenSearchDocumentStore. Choose the index for the document store.
7. Add the Output component as the last component in your pipeline and connect it to the component generating answers (in LLM-based pipelines, this is DeepsetAnswerBuilder). Optionally, connect the documents output to it if you want them included in the pipeline's output.
8. Save your pipeline.

What To Do Next

• To use your pipeline, deploy it. Click Deploy in the top right corner of Pipeline Builder.
• To test your pipeline, wait until it's indexed and then go to Playground. Make sure your pipeline is selected, and type your query.
• To view pipeline details, such as statistics, feedback, or logs, click the pipeline name. This opens the Pipeline Details page.
• To let others test your pipeline, share your pipeline prototype.
• If your pipeline contains components that work better on a GPU, turn on GPU acceleration. For details, see Enable GPU Acceleration for Pipelines.