Skip to main content

Haystack Enterprise Platform and Haystack Open Source

Haystack is Haystack Platform's open source Python framework for creating production-ready AI-based systems. It's also the underlying technology for Haystack Enterprise Platform.

Haystack Enterprise Platform uses the same components, pipelines, and data classes as the Haystack open source framework. Let's explain what this means and how you can use it in your day-to-day work.

If you're not familiar with the open source version of Haystack, check the following resources:

We bump Haystack version used in Haystack Enterprise Platform with every Haystack release, so that it's always up to date.

Pipelines​

Haystack Enterprise Platform uses Haystack pipelines, giving you the full flexibility of Haystack features, including loops and multiple branches in your workflows.

Pipelines Start and End​

Initial and final components: The components you can use at the start and end of indexes and query pipelines are restricted:

  • Indexes must start with FilesInput, which acts as a placeholder for the files you upload to Haystack Enterprise Platform. There are no restrictions on the final component, though it is typically DocumentWriter.
  • Query pipelines must start with Input and end with Output.

For more information, see Pipelines.

Components​

Haystack Enterprise Platform combines open source Haystack components with its own unique components. Components specific to Haystack Enterprise Platform have names that start with Deepset. We introduced these custom components to support features tailored specifically for Haystack Enterprise Platform.

Some components may appear to be duplicates, for example AnswerBuilder and DeepsetAnswerBuilder. Typically, these "duplicates" extend the original component by adding functionality needed for Haystack Enterprise Platform. For instance, we created DeepsetAnswerBuilder to ensure document references display correctly in Haystack Platform's interface. This is enabled by an additional parameter, reference_pattern, which you can configure in DeepsetAnswerBuilder.

The list of components is constantly growing giving you more possibilities. Additionally, you can create your own custom components. For more information, see Custom Components.

Data Classes​

Haystack Enterprise Platform uses Haystack's objects, or data classes namely:

  • Answer, including ExtractedAnswer and GeneratedAnswer
  • ByteStream
  • ChatMessage
  • Document
  • StreamingChunk
  • ImageContent

Each of these objects has properties you can access. For detailed description of the data classes and their properties, see Haystack's Data Classes.

Data Classes in Jinja2 Templates​

It's useful to know the objects you can use and their attributes when working with components that use Jinja2 templates: PromptBuilder, OutputAdapter, and ConditionalRouter. You can access an object's attribute in the template, for example:

{% for document in documents %}
Document[{{ loop.index }}]:
{{ document.content }}
{% endfor %}

This expression iterates through all documents to fetch their content. This is achieved by accessing the content attribute of the document data class using document.content.

You can also use the document's meta attribute to work with metadata. The syntax for this is: document.meta.metadata_key. For more examples, see Use Metadata in Your Search System.

The same approach applies to other data classes and their attributes.

Data Classes as Output and Input Variables​

Additionally, certain components use or output data classes, for example AnswerBuilders output a list of GeneratedAnswer objects, or Rankers accept a list of document objects. It's good to be aware of that when connecting components to make sure the connections are compatible. You can check the details on each component documentation page. For details, see Pipeline Components.

Haystack Experimental Features

The haystack-experimental package offers experimental features you can try out. You can find it on GitHub. For more information, check the Haystack documentation.

Components in the haystack-experimental package are not included in the Pipeline Builder's component library by default. You can add them as custom components, but keep in mind that experimental features are not intended for production pipelines. They're likely to change or be deprecated. For details, see Custom Components Using Haystack Experimental features.