MSSharePointFetcher

Fetch the full content of Microsoft SharePoint and OneDrive items via the Microsoft Graph API. Use this component with MSSharePointRetriever to download file content after search, or pass SharePoint URLs directly.

Key Features

• Downloads files (driveItem) as raw bytes in ByteStream objects.
• Returns list items (listItem) as JSON ByteStream objects with column values.
• Renders SharePoint pages (sitePage) as HTML ByteStream objects from web parts.
• Accepts Document objects from MSSharePointRetriever or raw SharePoint and OneDrive URLs.
• Resolves items through the Microsoft Graph shares endpoint.

Configuration

Add Workspace-Level Integration

1. Click your profile icon and choose Settings.
2. Go to Workspace>Integrations.
3. Find the provider you want to connect and click Connect next to them.
4. Enter the API key and any other required details.
5. Click Connect. You can use this integration in pipelines and indexes in the current workspace.

Add Organization-Level Integration

1. Click your profile icon and choose Settings.
2. Go to Organization>Integrations.
3. Find the provider you want to connect and click Connect next to them.
4. Enter the API key and any other required details.
5. Click Connect. You can use this integration in pipelines and indexes in all workspaces in the current organization.

1. Drag the MSSharePointFetcher component onto the canvas from the Component Library.
2. Wire an upstream OAuthTokenResolver to provide a delegated Microsoft Graph bearer token on the access_token input.
3. Connect MSSharePointRetriever.documents or SharePoint URLs to the targets input.
4. Connect the streams output to downstream converters such as PyPDFToDocument, DOCXToDocument, or HTMLToDocument.

The access token must carry delegated Microsoft Graph permissions, for example Files.Read.All for files and Sites.Read.All for list items and pages.

Connections

MSSharePointFetcher receives an access_token and a list of targets (Document objects or web_url strings). It outputs streams as a list of ByteStream objects. Each stream's metadata includes url, file_name, content_type, and entity_type.

Source Code

To check this component's source code, open fetcher.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  MSSharePointFetcher:
    type: haystack_integrations.components.fetchers.microsoft_sharepoint.fetcher.MSSharePointFetcher
    init_parameters: {}

Using the Component in a Pipeline

# haystack-pipeline
components:
  oauth_resolver:
    type: haystack_integrations.components.connectors.oauth.resolver.OAuthTokenResolver
    init_parameters:
      token_source:
        type: haystack_integrations.utils.oauth.sources.OAuthRefreshTokenSource
        init_parameters:
          token_url: https://login.microsoftonline.com/common/oauth2/v2.0/token
          client_id: your-client-id
          refresh_token:
            type: env_var
            env_vars:
            - MS_REFRESH_TOKEN
            strict: false
          scopes:
          - https://graph.microsoft.com/Files.Read.All
          - https://graph.microsoft.com/Sites.Read.All

  retriever:
    type: haystack_integrations.components.retrievers.microsoft_sharepoint.retriever.MSSharePointRetriever
    init_parameters:
      top_k: 5

  fetcher:
    type: haystack_integrations.components.fetchers.microsoft_sharepoint.fetcher.MSSharePointFetcher
    init_parameters: {}

connections:
- sender: oauth_resolver.access_token
  receiver: retriever.access_token
- sender: oauth_resolver.access_token
  receiver: fetcher.access_token
- sender: retriever.documents
  receiver: fetcher.targets

inputs:
  query:
  - retriever.query

outputs:
  streams: fetcher.streams

Parameters

Inputs

Parameter	Type	Description
access_token	str | Secret	Delegated Microsoft Graph bearer token for the user whose content is fetched.
targets	List[Document | str]	Items to fetch as Document objects from MSSharePointRetriever or raw SharePoint and OneDrive URLs.

Outputs

Parameter	Type	Description
streams	List[ByteStream]	Fetched content. Each stream's metadata includes url, file_name, content_type, and entity_type.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
graph_url	str	https://graph.microsoft.com/v1.0	The Microsoft Graph base URL. Override for sovereign clouds.
timeout	float	30.0	HTTP timeout in seconds for each Graph API request.
max_retries	int	3	Maximum retries for throttled or transient server errors.
max_concurrent_requests	int	5	Maximum concurrent item fetches in run_async.
raise_on_failure	bool	True	If True, a fetch failure raises an exception. If False, the item is skipped.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
access_token	str | Secret		Delegated Microsoft Graph bearer token.
targets	List[Document | str]		Items to fetch.

Related Information

• MSSharePointRetriever
• OAuthTokenResolver
• Add Secrets