Skip to main content

Synchronize Data Using Airbyte

Airbyte is an open source data integration tool you can use to synchronize your data from various sources to destinations. We've added a connector for the Haystack Enterprise Platform destination so that you can seamlessly transfer your data from various sources to Haystack Platform.

About this Task​

With Airbyte, you can easily transfer data from your data sources to a Haystack Platform workspace. You can also schedule regular data syncs to automate the process. To learn more about Airbyte, visit their webpage.

You can upload your data from a source that supports the DocumentFileType format to a Haystack Platform workspace. The currently supported sources are:

  • Microsoft OneDrive
  • Microsoft SharePoint
  • Google Drive
  • S3

Using the Haystack Platform destination connector, you can specify:

  • The Haystack Platform API key.
  • The workspace where you want to upload the data.
  • The URL for your Haystack Enterprise Platform deployment.

For details, see the connector documentation.

Prerequisites​

  • You must have an Airbyte account.
  • You must have a Haystack Platform API key. For details, see Generate an API Key.

Synchronize Data with Airbyte​

Airbyte is an external service, so we recommend you check their documentation (you can find the links below). Synchronizing data involves three steps:

  1. Add a source.
  2. Add a destination. Choose the Haystack Platform destination connector. You can check its documentation at Haystack Platform.
  3. Set up a connection between source and destination.