# Node SDK
Source: https://docs.getmetal.io/api-reference/node

Overview of the Node Metal SDK.

<Card title="Node SDK" icon="github" href="https://github.com/getmetal/metal-ts">
  Visit the GitHub Repo
</Card>

<Note>
  You'll need to [Create an API Key](/misc-get-keys) to
  [Authentication](/api-reference/introduction#authentication) with Metal.
</Note>

## Installation

```bash theme={null}
npm install @getmetal/metal-sdk
# or
yarn add @getmetal/metal-sdk
```

# Retrieval

<Note>
  You'll need to [Create an API Key](/misc-get-keys) to
  [Authenticate](/api-reference/introduction#authentication) with Metal.
</Note>

## App Setup

| Param      | Type   | Description                                                                     |
| ---------- | ------ | ------------------------------------------------------------------------------- |
| `apiKey`   | string | The API Key for your Org. [Learn more.](/misc-get-keys)                         |
| `clientId` | string | The Client ID for your Org. [Learn more.](/misc-get-keys)                       |
| `indexId`  | string | The ID of the index you want to connect with. [Learn more.](/misc-create-index) |

```ts theme={null}
import { Metal } from "@getmetal/metal-sdk";
const metal = new Metal("pk_123", "ci_123", "index-id");
```

## `addApp()`

#### Parameters

| Param  | Type   | Description      |
| ------ | ------ | ---------------- |
| `name` | string | Name of the app. |

```ts theme={null}
const app = await metal.addApp({
    name: "My App",
    indexes: ["index-id"]
});
```

## `getApp()`

#### Parameters

| Param   | Type   | Description             |
| ------- | ------ | ----------------------- |
| `appId` | string | Identifier for the app. |

```ts theme={null}
const app = await metal.getApp("app-id");
```

## `getApps()`

```ts theme={null}
const apps = await metal.getApps();
```

## `updateApp()`

#### Parameters

| Param     | Type              | Description                                          |   |
| --------- | ----------------- | ---------------------------------------------------- | - |
| `appId`   | string            | The unique identifier of the app you want to update. |   |
| `name`    | string (optional) | The updated name for the App.                        |   |
| `indexes` | array (optional)  | An updated array of the index connected to the app.  |   |

```ts theme={null}
const app = await metal.updateApp("app-id", { name: "updated name", indexes: ["index-id"] });
```

## `addIndex()`

#### Parameters

| Param        | Type   | Description                                                          |
| ------------ | ------ | -------------------------------------------------------------------- |
| `model`      | string | Identifier for the model being used, e.g., "text-embedding-ada-002". |
| `name`       | string | Name of the index.                                                   |
| `datasource` | string | Unique identifier for the Datasource.                                |
| `indexType`  | string | Type of index structure, e.g., "FLAT", "HNSW".                       |
| `dimensions` | number | Number of dimensions for the index. e.g.1536                         |
| `filters`    | array  | An array of filters specifying fields for advanced searching.        |

#### Filters

| Param   | Type   | Description                                       |
| ------- | ------ | ------------------------------------------------- |
| `field` | string | Name of the attribute you wish to filter by.      |
| `type`  | string | Type of the attribute (e.g., "string", "number"). |

```ts theme={null}
const payload = {
    model: "text-embedding-ada-002",
    name: "my index",
    datasource: "<datasource_id>",
    indexType: "FLAT",
    dimensions: 1536,
    filters: [
        {
            field: "name",
            type: "string"
        },
        {
            field: "age",
            type: "number"
        }
    ]
};

const newIndex = await metal.addIndex(payload);

```

## `getIndex()`

#### Parameters

| Param     | Type   | Description               |
| --------- | ------ | ------------------------- |
| `indexId` | string | Identifier for the index. |

```ts theme={null}
const index = await metal.getIndex("index-id");
```

## `index()`

Add a single embedding to an index. When invoked, we will generate an embedding with one of the below embedding params and store it into a vector db.

<Note>
  You can only pass one of the following fields: `text`, `imageUrl`, or
  `embedding`. We only embed one field in a single request.
</Note>

```ts theme={null}
const textEmbedding = await metal.index({ text: "Rocket" });
// or
const imgEmbedding = await metal.index({ imageUrl: "https://path-to.img" });
// or
const rawEmbedding = await metal.index({ embedding: [0.1, 0.2, 0.3] });
```

You can also pass an optional `id` and `metadata` object to be stored with the embedding. Eg.

```ts theme={null}
const opts = {
  id: "rocket1",
  metadata: {
    category: "space",
    another: "field",
  },
};

const embedding = await metal.index({
  text: "Rocket",
  ...opts,
});
```

#### Parameters

| Param               | Type      | Description                                                        |
| ------------------- | --------- | ------------------------------------------------------------------ |
| Embed - `embedding` | number\[] | Raw embedding to be indexed.                                       |
| Embed - `imageUrl`  | string    | A URL for an image to be embedded and indexed.                     |
| Embed - `text`      | string    | The text to be embedded and indexed.                               |
| `id`                | string    | Optional. Identifier for your embedding.                           |
| `metadata`          | object    | Optional. A flexible metdata object to be stored w/ the embedding. |

## `indexMany()`

Bulk add multiple embedding documents to an index. When invoked, we will generate an embedding with one of the below "Embed" params and store it in a vector index.

<Note>
  You can only pass one of the following fields: `text`, `imageUrl`, or
  `embedding`. We only embed one field in a single request.
</Note>

#### Parameters

| Param       | Type      | Description                                                        |
| ----------- | --------- | ------------------------------------------------------------------ |
| `index`     | string    | Required. Index id.                                                |
| `embedding` | number\[] | Raw embeddings to be indexed.                                      |
| `imageUrl`  | string    | A URL for an image to be vectorized and indexed.                   |
| `text`      | string    | The text to be vectorized and indexed.                             |
| `id`        | string    | Optional. identifier for your embedding.                           |
| `metadata`  | object    | Optional. A flexible metdata object to be stored w/ the embedding. |

```typescript theme={null}
await metal.indexMany([
  { text: "Megadeth", index: "index-id" },
  { text: "Metallica", index: "index-id" },
]);
```

## `search()`

#### Parameters

| Param       | Type      | Description                                       |
| ----------- | --------- | ------------------------------------------------- |
| `embedding` | number\[] | Raw embeddings to be searched.                    |
| `imageUrl`  | string    | A URL for an image to be vectorized and searched. |
| `text`      | string    | The text to be vectorized and searched.           |
| `filters`   | Filter    | Filters included for filtered search.             |
| `indexId`   | string    | Optional index id where record will get indexed.  |
| `idsOnly`   | boolean   | Return only the ids of the documents.             |
| `limit`     | number    | Number of documents returned by the API           |

**Filter Object**

| Param | Type          | Description                                         |
| ----- | ------------- | --------------------------------------------------- |
| `and` | FilterItem\[] | And clause, all members have to be satisfied.       |
| `or`  | FilterItem\[] | Or clause, at least one member has to be satisfied. |

**Filter Item**

| Param      | Type             | Description                             |
| ---------- | ---------------- | --------------------------------------- |
| `field`    | string           | Name of the field to filter             |
| `value`    | string \| number | Value to match the filter               |
| `operator` | string           | One of: `eq`, `gt`, `gte`, `lt`, `lte`. |

```typescript theme={null}
const results = await metal.search({
  text: "term-to-search",
  filters: { and: [{ field: "favoriteNumber", value: 666, operator:"lt"}] },
  indexId: "indexId",
  idsOnly: false,
  limit: 100,
});
```

## `getOne()`

Retrieve a single embedding document.

#### Parameters

| Param     | Type   | Description                                       |
| --------- | ------ | ------------------------------------------------- |
| `id`      | string | The ID of the document to retrieve.               |
| `indexId` | string | Optional. index id where record will get indexed. |

```typescript theme={null}
const document = await metal.getOne("documentId-123");
```

## `getMany()`

#### Parameters

| Name      | Type          | Description                                                                                               |
| --------- | ------------- | --------------------------------------------------------------------------------------------------------- |
| `ids`     | Array of str  | An array of document IDs to retrieve.                                                                     |
| `indexId` | str, optional | The ID of the index from which to retrieve documents. If not provided, the default index ID will be used. |

```ts theme={null}

const documentIDs = ['document_id_123', 'document_id_456'];
const result = await Metal.getMany(documentIDs);

```

## `getQueries()`

#### Parameters

| Name      | Type | Description                                           |
| --------- | ---- | ----------------------------------------------------- |
| `indexId` | str  | The ID of the index from which to retrieve documents. |

```typescript theme={null}
const queries = await metal.getQueries("index-id");
```

## `deleteOne()`

Delete a single embedding document.

#### Parameters

| Param     | Type   | Description                                            |
| --------- | ------ | ------------------------------------------------------ |
| `id`      | string | The ID of the document to delte.                       |
| `indexId` | string | Optional. The ID of the index containing the document. |

```typescript theme={null}
const document = await metal.deleteOne("documentId-123");
```

## `deleteMany()`

Deletes multiple documents in an index based on their IDs.

#### Parameters

| Param     | Type      | Description                                            |
| --------- | --------- | ------------------------------------------------------ |
| `id`      | string\[] | The IDs of the documents to delete.                    |
| `indexId` | string    | Optional. The ID of the index containing the document. |

```typescript theme={null}
const document = await metal.deleteOne("documentId-123");
```

# Memory (Motorhead)

## App Setup

| Param      | Type   | Description                                                                                                                                                 |
| ---------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `apiKey`   | string | For Managed. The API Key for your Org. \[details]\(- [Create an API Key](/misc-get-keys) for [Authentication](/api-reference/introduction#authentication).  |
| `clientId` | string | For Managed.The Client ID for your Org. \[details]\(- [Create an API Key](/misc-get-keys) for [Authentication](/api-reference/introduction#authentication). |
| `baseUrl`  | string | For Self-hosted. The URL path to your motorhead instance                                                                                                    |

```typescript theme={null}
import { Motorhead } from "@getmetal/metal-sdk";

// Managed
const motorhead = new Motorhead({ apiKey: "apiKey", clientId: "clientId" });

// Self-hosted
const motorhead = new Motorhead({
  baseUrl: "https://motorhead.yourdomain.com",
});
```

## `addMemory()`

```ts theme={null}
interface Memory {
  messages: {
    content: string;
    role: "AI" | "Human";
  };
  context: string;
}
```

| Param       | Type      | Description                                                              |
| ----------- | --------- | ------------------------------------------------------------------------ |
| `sessionId` | string    | The ID of the session. If the session does not exist, it will create one |
| `memory`    | Memory\[] | A memory payload to update the session memory                            |

```ts theme={null}
const memoryPayload = {
  messages: [
    { role: "Human", content: "Who is the best vocalist of all time?" },
    { role: "AI", content: "Ozzy!" },
  ],
  context:
    "User ask what can he eat in Colombia. The AI responds arepas are really nice",
};

await motorhead.addMemory("session-id", memoryPayload);
```

## `getMemory()`

| Param       | Type   | Description                                  |
| ----------- | ------ | -------------------------------------------- |
| `sessionId` | string | The ID of the session to retrieve memory for |

```ts theme={null}
await motorhead.getMemory("session-id");
```

## `deleteMemory()`

| Param       | Type   | Description                                |
| ----------- | ------ | ------------------------------------------ |
| `sessionId` | string | The ID of the session to delete memory for |

```ts theme={null}
await motorhead.deleteMemory("session-id");
```

# Datasources

## `addDatasource()`

Add a new Datasource to your Metal instance.

#### Parameters

| Name             | Type    | Description                                                                                                                               |
| ---------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `name`           | string  | Name of the Datasource.                                                                                                                   |
| `sourcetype`     | string  | Type of the source. It can either be 'File' or 'Text'.                                                                                    |
| `autoExtract`    | boolean | Flag indicating whether auto-extraction is enabled.                                                                                       |
| `metadataFields` | array   | An array of fields specifying which attributes you wish to extract. Each object should have `name`, `type`, and `description` properties. |

#### Metadata Fields

| Name          | Type    | Description                                                         |
| ------------- | ------- | ------------------------------------------------------------------- |
| `name`        | string  | Name of the attribute you wish to extract.                          |
| `type`        | string  | Type of the attribute (e.g., "String", "Number").                   |
| `description` | string  | Brief description or example of the attribute.                      |
| `autoExtract` | boolean | Determines if attribute will be auto extracted from content or not. |

```ts theme={null}
const dataSourcePayload = {
    name: "my datasource",
    sourcetype: "File",
    autoExtract: true,
    metadataFields: [
      {
        name: "band",
        type: "String",
        description: "Which heavy metal band is represented by the iconic mascot Eddie?",
        autoExtract: true,
      }
    ]
};

const response = await metal.addDatasource(datasourcePayload);
```

## `updateDatasource()`

#### Parameters

| Name             | Type    | Description                                                     |
| ---------------- | ------- | --------------------------------------------------------------- |
| `datasourceId`   | string  | Identifier of the Datasource you want to update.                |
| `name`           | string  | Updated name of the Datasource.                                 |
| `sourcetype`     | string  | Updated type of the source. Either 'File' or 'Text'.            |
| `autoExtract`    | boolean | Updated flag indicating whether auto-extraction is enabled.     |
| `metadataFields` | array   | Updated fields specifying which attributes you wish to extract. |

```ts theme={null}

const updatedDatasourcePayload = {
    name: "updated_datasource",
    sourcetype: "Text",
    autoExtract: false,
    metadataFields: [
        {
            name: "song",
            type: "String",
            description: "Which was Iron Maiden's first single?"
        }
    ]
};

const response = await Metal.updateDatasource("existing_dataSourceId", updatedDataSourcePayload);

```

## `getDatasource()`

#### Parameters

| Name           | Type   | Description                                                          |
| -------------- | ------ | -------------------------------------------------------------------- |
| `dataSourceId` | string | Identifier of the Datasource you want to retrieve information about. |

```ts theme={null}
const datasourceId = "existing_datasourceId";
const response = await Metal.getDatasource(datasourceId);
```

## `getAllDatasources()`

#### Parameters

| Name    | Type   | Description                                                                                   |
| ------- | ------ | --------------------------------------------------------------------------------------------- |
| `limit` | number | (Optional) The maximum number of Datasources to return. Default is 10, with a maximum of 100. |
| `page`  | number | (Optional) The page number for pagination. Should be a positive integer up to 100.            |

```ts theme={null}

const limit = 10;
const page = 2;

const response = await Metal.getAllDatasources({ limit, page });
```

## `deleteDatasource()`

#### Parameters

| Name           | Type   | Description                                  |
| -------------- | ------ | -------------------------------------------- |
| `datasourceId` | string | The ID of the Datasource you want to delete. |

```ts theme={null}
const datasourceId = 'your_data_source_id';
const response = await Metal.deleteDatasource(datasourceId);
```

# Data Entities

## `addDataEntity()`

#### Parameters

| Name           | Type   | Description                                      |
| -------------- | ------ | ------------------------------------------------ |
| `datasourceId` | string | The ID of the datasource where the file belongs. |
| `filepath`     | string | The local file path of the file to upload.       |
| `metadata`     | object | Additional Metadata that isn't extracted.        |

### Example

```ts theme={null}

const datasourceId = 'your_datasource_id';
const filePath = './file_path.csv';

const results = await metal.addDataEntity({
  datasource: datasourceId,
  filepath,
  metadata: {
    band: 'Iron Maiden',
    song: 'The Trooper',
  },
});

```

## `getDataEntity()`

#### Parameters

| Name | Type   | Description                                     |
| ---- | ------ | ----------------------------------------------- |
| `id` | string | The ID of the data entity you want to retrieve. |

```ts theme={null}

const dataEntityId = 'your_dataentity_id';

const dataEntity = await metal.getDataEntity(dataEntityId);
```

## `getAllDataEntities()`

#### Parameters

| Name           | Type   | Description                                                                                     |
| -------------- | ------ | ----------------------------------------------------------------------------------------------- |
| `datasourceID` | string | The ID of the datasource for which you want to list the data entities.                          |
| `limit`        | number | (Optional) The maximum number of data entities to return. Default is 10, with a maximum of 100. |
| `page`         | number | (Optional) The page number for pagination. Should be a positive integer up to 100.              |

```ts theme={null}

const datasourceID = 'your_datasource_id';
const page = 1;
const limit = 10;

const dataEntities = await Metal.listDataEntities(datasourceID, page, limit);
```

## `deleteDataEntity()`

#### Parameters

| Name | Type   | Description                                   |
| ---- | ------ | --------------------------------------------- |
| `id` | string | The ID of the data entity you want to delete. |

### Example

```ts theme={null}

const dataEntityID = 'your_dataentity_id';

const result = await Metal.deleteDataEntity(dataEntityID);
```

***

Checkout the code on [Github](https://github.com/getmetal/metal-ts)


# Python SDK
Source: https://docs.getmetal.io/api-reference/python

Overview of the Python Metal SDK.

<Card title="Python SDK" icon="github" href="https://github.com/getmetal/metal-python">
  Visit the GitHub Repo
</Card>

<Note>
  You'll need to [Create an API Key](/misc-get-keys) to [Authentication](/api-reference/introduction#authentication) with Metal.
</Note>

## Installation

```bash theme={null}
pip3 install metal-sdk
```

# Retrieval

## App Setup

| Param       | Type   | Description                                                                     |
| ----------- | ------ | ------------------------------------------------------------------------------- |
| `api_key`   | string | The API Key for your Org. [Learn more.](/misc-get-keys)                         |
| `client_id` | string | The Client ID for your Org. [Learn more.](/misc-get-keys)                       |
| `index_id`  | string | The ID of the index you want to connect with. [Learn more.](/misc-create-index) |

```python theme={null}
from metal_sdk.metal import Metal

metal = Metal("api_key", "client_id", "index_id")
```

## Adding an App

| Param     | Type   | Description                                 |
| --------- | ------ | ------------------------------------------- |
| `name`    | string | Name of the App.                            |
| `indexes` | array  | An array of the index connected to the app. |

```python theme={null}
metal.add_app({"name": "My App", "indexes": ["index1"]})
```

## Get One App

| Param    | Type   | Description    |
| -------- | ------ | -------------- |
| `app_id` | string | ID of the App. |

```python theme={null}
metal.get_app("app_id")
```

## Get All Apps

```python theme={null}
metal.get_apps()
```

## Updating an App

| Param     | Type              | Description                                          |   |
| --------- | ----------------- | ---------------------------------------------------- | - |
| `app_id`  | string            | The unique identifier of the app you want to update. |   |
| `name`    | string (optional) | The updated name for the App.                        |   |
| `indexes` | array (optional)  | An updated array of the index connected to the app.  |   |

```python theme={null}

metal.update_app("app_id", {"name": "Updated Metal App", "indexes": ["index1"]})

```

## Adding an Index

### Payload

| Param        | Type   | Description                                                          |
| ------------ | ------ | -------------------------------------------------------------------- |
| `model`      | string | Identifier for the model being used, e.g., "text-embedding-ada-002". |
| `name`       | string | Name of the index.                                                   |
| `datasource` | string | Unique identifier for the Datasource.                                |
| `indexType`  | string | Type of index structure, e.g., "FLAT", "HNSW".                       |
| `dimensions` | number | Number of dimensions for the index. e.g.1536                         |
| `filters`    | array  | An array of filters specifying fields for advanced searching.        |

### Filters

| Param   | Type   | Description                                       |
| ------- | ------ | ------------------------------------------------- |
| `field` | string | Name of the attribute you wish to filter by.      |
| `type`  | string | Type of the attribute (e.g., "string", "number"). |

```python theme={null}
metal = Metal("api_key", "client_id")

payload = {
    "model": "text-embedding-ada-002",
    "name": "my index",
    "datasource": "<datasource_id>",
    "indexType": "FLAT",
    "dimensions": 1536,
    "filters": [
      {
        "field": "name",
        "type": "string"
      },
      {
        "field": "age",
        "type": "number"
      }
    ]
}

metal.add_index(payload)
```

## Get One Index

| Param      | Type   | Description                                              |
| ---------- | ------ | -------------------------------------------------------- |
| `index_id` | string | The unique identifier of the index you want to retrieve. |

```python theme={null}
metal.get_index("index_id")
```

## Indexing a Document

| Param      | Type   | Description                                      |
| ---------- | ------ | ------------------------------------------------ |
| `payload`  | dict   | Dictionary with index parameters                 |
| `index_id` | string | Optional index id where record will get indexed. |

#### Payload

| Param       | Type      | Description                                              |
| ----------- | --------- | -------------------------------------------------------- |
| `id`        | string    | Optional identifier for your embedding.                  |
| `embedding` | number\[] | Raw embeddings to be indexed.                            |
| `imageUrl`  | string    | A URL for an image to be vectorized and indexed.         |
| `text`      | string    | The text to be vectorized and indexed.                   |
| `metadata`  | object    | A flexible metdata object to be stored w/ the embedding. |

```python theme={null}
metal.index({ "text": "text-to-index" }, "index_id")
```

## Multi(Bulk) Indexing

You can index up to `100` records in a single request.

| Param     | Type             | Description         |
| --------- | ---------------- | ------------------- |
| `payload` | BulkIndexItem\[] | Array of bulk items |

#### BulkIndexItem

| Param       | Type      | Description                                              |
| ----------- | --------- | -------------------------------------------------------- |
| `id`        | string    | Optional identifier for your embedding.                  |
| `index`     | string    | Required - index id.                                     |
| `embedding` | number\[] | Raw embeddings to be indexed.                            |
| `imageUrl`  | string    | A URL for an image to be vectorized and indexed.         |
| `text`      | string    | The text to be vectorized and indexed.                   |
| `metadata`  | object    | A flexible metdata object to be stored w/ the embedding. |

```python theme={null}
metal.index_many([{ "text": "blacksabbath", "index": "index-id" }, { "text": "ironmaiden", "index": "index-id" }])
```

## Searching

`self, payload: SearchPayload = {}, index_id=None, ids_only=False, limit=1`

| Param      | Type    | Description                                          |
| ---------- | ------- | ---------------------------------------------------- |
| `payload`  | dict    | search payload.                                      |
| `index_id` | string  | Id of index to search.                               |
| `ids_only` | boolean | Optional return only the ids                         |
| `limit`    | number  | Number of documents returned by the API (default=1). |

#### Payload

| Param       | Type      | Description                                       |
| ----------- | --------- | ------------------------------------------------- |
| `embedding` | number\[] | Raw embeddings to be searched.                    |
| `imageUrl`  | string    | A URL for an image to be vectorized and searched. |
| `text`      | string    | The text to be vectorized and searched.           |
| `filters`   | dict\[]   | List of filters to match in search.               |

#### Filter

| Param | Type | Description                                         |
| ----- | ---- | --------------------------------------------------- |
| `and` | dict | And clause, all members have to be satisfied.       |
| `or`  | dict | Or clause, at least one member has to be satisfied. |

#### Filter Item

| Param      | Type             | Description                                      |
| ---------- | ---------------- | ------------------------------------------------ |
| `value`    | string \| number | Value to match.                                  |
| `field`    | string           | Field to filter.                                 |
| `operator` | string           | Possible values: `eq`, `gt`, `gte`, `lt`, `lte`. |

```python theme={null}
results = metal.search({ "text": "term-to-search", "filters": { "and": [{"field": "band", "value": "Black Sabbath", "operator":"eq"}]} }, index_id="indexID", limit=10)
```

## Get One Document

| Param      | Type   | Description                         |
| ---------- | ------ | ----------------------------------- |
| `id`       | string | The ID of the document to retrieve. |
| `index_id` | string | Optional. Id of index to search.    |

```python theme={null}
document = metal.get_one("document_id_123")
```

## Get Many Documents

| Param      | Type          | Description                                                                                               |
| ---------- | ------------- | --------------------------------------------------------------------------------------------------------- |
| `ids`      | list of str   | A list of document IDs to retrieve.                                                                       |
| `index_id` | str, optional | The ID of the index from which to retrieve documents. If not provided, the default index ID will be used. |

```python theme={null}
document = metal.get_many("[document_id_123, document_id_456]")
```

## Get Queries

| Param      | Type   | Description                      |
| ---------- | ------ | -------------------------------- |
| `index_id` | string | The ID of the Index to retrieve. |

```python theme={null}
queries = metal.get_queries("index_id")
```

## Delete One

| Param          | Type   | Description                                    |
| -------------- | ------ | ---------------------------------------------- |
| `embedding_id` | string | The ID of the document or embedding to delete. |
| `index_id`     | string | The ID of the index containing the document.   |

```python theme={null}
metal.delete_one("embedding_id", "index_id")
```

## Delete Many (Bulk)

| Param           | Type   | Description                                              |
| --------------- | ------ | -------------------------------------------------------- |
| `embedding_ids` | list   | A list of IDs for the documents or embeddings to delete. |
| `index_id`      | string | The ID of the index containing the documents.            |

```python theme={null}
embedding_ids_to_delete = ["id1", "id2", "id3"]
metal.delete_many(embedding_ids_to_delete, "index_id")
```

# Memory (Motorhead)

## App Setup

| Param       | Type   | Description                                                                                                                                                 |
| ----------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `api_key`   | string | For Managed. The API Key for your Org. \[details]\(- [Create an API Key](/misc-get-keys) for [Authentication](/api-reference/introduction#authentication).  |
| `client_id` | string | For Managed.The Client ID for your Org. \[details]\(- [Create an API Key](/misc-get-keys) for [Authentication](/api-reference/introduction#authentication). |
| `base_url`  | string | For Self-hosted. The URL path to your motorhead instance                                                                                                    |

```python theme={null}
from metal_sdk.motorhead import Motorhead

# Managed
motorhead = Motorhead({ "api_key": "api_key", "client_id": "client-id" })

# Self-hosted
motorhead = Motorhead({ "base_url": "https://motorhead.yourdomain.com" })
```

## Create Memory

```python theme={null}
class Message:
    content: str
    role: str  # "AI" or "Human"

class Memory:
    messages: list  # List of Message instances
    context: str
```

| Param        | Type      | Description                                                              |
| ------------ | --------- | ------------------------------------------------------------------------ |
| `session_id` | string    | The ID of the session. If the session does not exist, it will create one |
| `memory`     | Memory\[] | A memory payload to update the session memory                            |

```python theme={null}
memory_payload = {
  "messages": [
    {"role": "Human", "content": "Who is the best vocalist of all time?"},
    {"role": "AI", "content": "Ozzy!"},
  ],
  "context":
    "User ask what can he eat in Colombia. The AI responds arepas are really nice",
}


motorhead.add_memory("session-id", memory_payload)
```

## Get Memory

| Param        | Type   | Description                                  |
| ------------ | ------ | -------------------------------------------- |
| `session_id` | string | The ID of the session to retrieve memory for |

```python theme={null}
motorhead.get_memory("session-id")
```

## Delete Memory

| Param        | Type   | Description                                |
| ------------ | ------ | ------------------------------------------ |
| `session_id` | string | The ID of the session to delete memory for |

```python theme={null}
motorhead.delete_memory("session-id")
```

# Datasources

## Add a Datasource

| Param            | Type    | Description                                                                                                                               |
| ---------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `name`           | string  | Name of the Datasource.                                                                                                                   |
| `sourcetype`     | string  | Type of the source. It can either be 'File' or 'Text'.                                                                                    |
| `autoExtract`    | boolean | Flag indicating whether auto-extraction is enabled.                                                                                       |
| `metadataFields` | array   | An array of fields specifying which attributes you wish to extract. Each object should have `name`, `type`, and `description` properties. |

#### Metadata Fields

| Param         | Type   | Description                                       |
| ------------- | ------ | ------------------------------------------------- |
| `name`        | string | Name of the attribute you wish to extract.        |
| `type`        | string | Type of the attribute (e.g., "String", "Number"). |
| `description` | string | Brief description or example of the attribute.    |

```python theme={null}
from metal_sdk.metal import Metal

payload = {
    "name": "my datasource",
    "sourcetype": "File",
    "autoExtract": True,
    "metadataFields": [
      {
        "name": "band",
        "type": "string",
        "description": "Which heavy metal band is represented by the iconic mascot Eddie?"
      }
    ]
}

metal.add_datasource(payload)
```

## Update Datasource

### Parameters

| Param            | Type    | Description                                                     |
| ---------------- | ------- | --------------------------------------------------------------- |
| `dataSourceId`   | string  | Identifier of the Datasource you want to update.                |
| `name`           | string  | Updated name of the Datasource.                                 |
| `sourcetype`     | string  | Updated type of the source. Either 'File' or 'Text'.            |
| `autoExtract`    | boolean | Updated flag indicating whether auto-extraction is enabled.     |
| `metadataFields` | array   | Updated fields specifying which attributes you wish to extract. |

### Example

```python theme={null}
payload = {
    "name": "updated_datasource",
    "sourcetype": "Text",
    "autoExtract": False,
    "metadataFields": [
        {
            "name": "song",
            "type": "string",
            "description": "Which was Iron Maiden's first single?"
        }
    ]
}

metal.update_datasource("existing_datasourceId", payload)
```

## Get Datasource

### Parameters

| Param          | Type   | Description                                                          |
| -------------- | ------ | -------------------------------------------------------------------- |
| `dataSourceId` | string | Identifier of the Datasource you want to retrieve information about. |

### Example

```python theme={null}
metal.get_datasource("existing_datasourceId")
print(response)  # This will display detailed information about the Datasource
```

## List Datasources

### Parameters

| Param   | Type | Description                                                                                 |
| ------- | ---- | ------------------------------------------------------------------------------------------- |
| `limit` | int  | (Optional) The maximum number of Datasources to return. Default is 10 and a maximum of 100. |
| `page`  | int  | (Optional) The page number for pagination. Should be a positive integer up to 100.          |

```python theme={null}
metal.get_all_datasources(page=1, limit=10)
```

## Delete Datasource

### Parameters

| Param           | Type   | Description                                  |
| --------------- | ------ | -------------------------------------------- |
| `datasource_id` | string | The ID of the Datasource you want to delete. |

### Example

```python theme={null}
response = metal.delete_datasource("datasourceID")
print(response)  # Expected: "Datasource successfully deleted."
```

# Data Entities

## Add a Data entity

`.pdf`, `doc`, `.docx`, `.xlsx`, and `.csv` are accepted.

## Parameters

| Name           | Type   | Description                                      |
| -------------- | ------ | ------------------------------------------------ |
| `datasourceId` | string | The ID of the datasource where the file belongs. |
| `file_path`    | string | The local file path of the file to upload.       |
| `metadata`     | object | Additional Metadata that isn't extracted.        |

## Example

```python theme={null}
results = metal.add_data_entity("datasource_id", "./file_path.csv", metadata=metadata)
```

## Get Data Entity

### Parameters

| Param | Type   | Description                                     |
| ----- | ------ | ----------------------------------------------- |
| `id`  | string | The ID of the data entity you want to retrieve. |

```python theme={null}
metal.get_data_entity("data_entity_id")
```

## List Data Entities

### Parameters

| Param          | Type   | Description                                                                                   |
| -------------- | ------ | --------------------------------------------------------------------------------------------- |
| `datasourceID` | string | The ID of the datasource for which you want to list the data entities.                        |
| `limit`        | int    | (Optional) The maximum number of data entities to return. Default is 10 and a maximum of 100. |
| `page`         | int    | (Optional) The page number for pagination. Should be a positive integer up to 100.            |

```python theme={null}
metal.get_all_data_entities("datasource_id", page=1, limit=10)
```

## Delete Data Entity

### Parameters

| Param | Type   | Description                                   |
| ----- | ------ | --------------------------------------------- |
| `id`  | string | The ID of the data entity you want to delete. |

```python theme={null}
metal.delete_data_entity("data_entity_id")
```


# REST API
Source: https://docs.getmetal.io/api-reference/rest-api

Overview of the Metal API.

## Base URL

The Metal API is available at the following URL:

```
https://api.getmetal.io
```

<Card title="Get started with the API" icon="globe" href="/rest-api">
  View the API endpoints
</Card>


# Examples in Node
Source: https://docs.getmetal.io/examples/node

Discover implemented use cases of the Metal Platform using Node.

<CardGroup>
  <Card title="Chat App with Memory" icon="memory" href="https://github.com/getmetal/motorhead-redis-example">
    Motorhead and Redis LLM Chat example in Node.
  </Card>

  <Card title="Command Line Chatbot" icon="messages" href="https://github.com/getmetal/Metal/tree/main/examples/02-motorhead-cli-chatgpt">
    Build a CLI chatbot with Metal Memory for LLMs in Node.
  </Card>

  <Card title="'UFO' Chatbot" icon="ufo-beam" href="https://github.com/getmetal/Metal/tree/main/examples/05-ufo-chatbot-nodejs">
    Chatbot with the unclassified 'UAP' reports in Node.
  </Card>

  <Card title="Hello World Search" icon="magnifying-glass" href="https://replit.com/@getmetal/Metal-Simple-Search-node">
    Semantic search example with Metal Retrieval in Node.
  </Card>
</CardGroup>


# Examples
Source: https://docs.getmetal.io/examples/overview

Discover implemented use cases of the Metal Platform.

<CardGroup>
  <Card title="Chat App with Memory" icon="memory" href="https://github.com/getmetal/motorhead-redis-example">
    Motorhead and Redis LLM Chat example in Node.
  </Card>

  <Card title="Command Line Chatbot" icon="messages" href="https://github.com/getmetal/Metal/tree/main/examples/02-motorhead-cli-chatgpt">
    Build a CLI chatbot with Metal Memory for LLMs in Node.
  </Card>

  <Card title="'UFO' Chatbot" icon="ufo-beam" href="https://github.com/getmetal/Metal/tree/main/examples/05-ufo-chatbot-nodejs">
    Chatbot with the unclassified 'UAP' reports in Node.
  </Card>

  <Card title="Hello World Search" icon="magnifying-glass" href="https://replit.com/@getmetal/Metal-Simple-Search-node">
    Semantic search example with Metal Retrieval in Node.
  </Card>

  <Card title="Financial Analysis Chatbot" icon="file-spreadsheet" href="https://github.com/getmetal/Metal/blob/main/examples/04-financial-chatbot/financial_chatbot.ipynb">
    Deploy a chat interface to talk with spreadsheets and tabular data in Python.
  </Card>

  <Card title="Multi Tenant Search" icon="people-roof" href="https://github.com/getmetal/Metal/blob/main/examples/03-multitenancy/multitenancy_search.ipynb">
    Build semantic search with Multi tenancy in Python.
  </Card>

  <Card title="RAG Chatbot" icon="futbol" href="https://github.com/getmetal/Metal/blob/main/examples/06-wwc_chatbot-python/wwc.ipynb">
    Retrieval augmented chatbot with different data types in Python.
  </Card>

  <Card title="RAG Langchain Agent " icon="user-robot" href="https://github.com/getmetal/Metal/blob/main/examples/07-rag-agent-langchain/rag_agent_langchain.ipynb">
    Retrieval augmented agent powered by Langchain in Python.
  </Card>
</CardGroup>


# Examples in Python
Source: https://docs.getmetal.io/examples/python

Discover implemented use cases of the Metal Platform using Python.

<CardGroup>
  <Card title="Financial Analysis Chatbot" icon="file-spreadsheet" href="https://github.com/getmetal/Metal/blob/main/examples/04-financial-chatbot/financial_chatbot.ipynb">
    Deploy a chat interface to talk with spreadsheets and tabular data in Python.
  </Card>

  <Card title="Multi Tenant Search" icon="people-roof" href="https://github.com/getmetal/Metal/blob/main/examples/03-multitenancy/multitenancy_search.ipynb">
    Build semantic search with Multi tenancy in Python.
  </Card>

  <Card title="RAG Chatbot" icon="futbol" href="https://github.com/getmetal/Metal/blob/main/examples/06-wwc_chatbot-python/wwc.ipynb">
    Retrieval augmented chatbot with different data types in Python.
  </Card>

  <Card title="RAG Langchain Agent " icon="user-robot" href="https://github.com/getmetal/Metal/blob/main/examples/07-rag-agent-langchain/rag_agent_langchain.ipynb">
    Retrieval augmented agent powered by Langchain in Python.
  </Card>
</CardGroup>


# Quickstart
Source: https://docs.getmetal.io/getting-started/quickstart

Build your first project with Metal

### Start with our application walk-throughs:

<CardGroup>
  <Card title="Build a Chatbot" icon="message-bot" href="/guides/misc-create-chatbot">
    Build a chatbot in the Metal platform.
  </Card>

  <Card title="Build Semantic Search" icon="searchengin" href="/guides/misc-semantic-search">
    Build semantic search in the Metal platform.
  </Card>

  <Card title="Run Attribute Extraction" icon="magnet" href="/guides/misc-run-attribute-extraction">
    Run attribute extraction in the Metal platform.
  </Card>
</CardGroup>

### Or crack open your IDE with our Examples:

<CardGroup>
  <Card title="Python" icon="python" href="/examples/python">
    Explore examples of using Metal with the Python SDK.
  </Card>

  <Card title="Node" icon="js" href="/examples/node">
    Explore examples of using Metal with the Node SDK.
  </Card>
</CardGroup>

### Explore our API:

<Card title="API Endpoints" icon="globe" href="/rest-api">
  Get started with our REST API guide.
</Card>


# Chunking Configuration
Source: https://docs.getmetal.io/guides/misc-chunking-config

How to configure chunking for your indexes

## What is Chunking?

Chunking breaks documents into smaller sections when they are indexed.
This process not only enhances the performance of retrieval engines but also
allows for more efficient searches within concise text. Additionally, it overcomes
token size restrictions in language models. Given that these models have a set token
limit for each input, dividing the text ensures it stays within this constraint.

## How to Configure Chunking?

Create an Index and select the `Chunking` options:

<img alt="Chunk Configuration" />

Chunk configuration is composed of two parameters:

* Chunk Size: The maximum number of tokens per chunk.
* Overlap Size: The number of tokens that overlap between chunks. This is useful for
  ensuring that the context of the chunk is preserved. We suggest beginning with an overlap that's roughly 10% of the chunk size.


# Build a Chatbot
Source: https://docs.getmetal.io/guides/misc-create-chatbot

Build a custom chatbot on your data in just a few steps.

## 1. Create an App

Go to your organization [Dashboard](https://app.getmetal.io) and start by creating a new App. Give it a name and select type: Chatbot.

<img alt="chatbot-1" />

## 2. Connect to Datasource

A Datasource is a collection of data that will feed your app. You can add multiple files or Data Entities to a Datasource.
Connect your app to an existing Datasource or create a new one.

<img alt="chatbot-2" />

## 3. Add Data Entities

A Data Entity is a single file or entry in a Datasource. Accepted file formats are PDF, DOC, DOCX, XLSX, and CSV.

After you've added a Data Entity, Metal will create an Index, process the data into chunks, and generate the embeddings that will be used to match the user's input to the right response.

<img alt="chatbot-3" />

## 4. Deploy your Chatbot

Try out your chatbot with Metal's open source [Chatbot UI](https://github.com/getmetal/chatbot).
Start by cloning the repo and following the instructions in the README.
You'll need to add Open AI `API Key`, and Metal's `API key`, `Client Id` and `Index Id` to the `.env.local file`.

<img alt="chatbot-4" />


# Add an Index
Source: https://docs.getmetal.io/guides/misc-create-index

Indexes in Metal represent a particular ML based experience (eg Video Ranking).

## Adding a Metal Index (`indexId`)

Log into your [Metal Account](https://app.getmetal.io/login) to get started.<br />
First, we'll want to create an Index within the [Metal Dashboard](https://app.getmetal.io). Eg:

<img alt="Create Index" />

After creation, you should be able to access the Index Dashboard:

<img alt="Index Dashboard" />

Note: You can find the `indexId` in the settings of your Index Dashboard.

<img alt="Index Settings" />


# Generate an API Key
Source: https://docs.getmetal.io/guides/misc-get-keys

API Keys are used to authenticate requests to the Metal API and SDKs.

## Generate API Keys (`apiKey` + `clientId`)

Navigate to the [Org Settings page](https://app.getmetal.io/settings/organization) to generate an API Key:

<img alt="Org Settings" />

Immeditely upon creation, you'll receive an `apiKey`. Make sure to store this somewhere safe, as you won't be able to access it after navigating away.
After returning to the list page, you'll be able to see the `clientId`for each API Key.


# Analyze User Queries
Source: https://docs.getmetal.io/guides/misc-get-queries

Understand your users interactions

## Overview

Metal provides a way to analyze user queries and understand how users interact with your application.
This is done by accesing the `Analytics` tab in your [application dashboard.](https://app.getmetal.io)

<img alt="queries-1" />

You can also access the queries via the [Metal API](http://localhost:3000/rest-api/analytics/get-queries).

<CodeGroup>
  ```python Python theme={null}
  metal.get_queries("index_id")
  ```

  ```javascript Node theme={null}
  const queries = await metal.getQueries("index-id");
  ```
</CodeGroup>

The response will be an array of objects, each representing a query.

```json theme={null}
{
  "data": [
    {
      "t": "2023-08-30T11:34:35.128Z",
      "d": 0.176956892014,
      "q": "Who is the best drummer of all time?"
      },
    {
      "t": "2023-08-29T11:34:18.099Z",
      "d": 0.156819581985,
      "q": "Which band is represented by the iconic mascot Eddie?"
    }
  ]
}
```

Where:

* t: Timestamp of the query
* d: Cosine distance between the search query and the retrieved embedding. A smaller value indicates a closer match.
* q: Actual user query


# Index Archiving
Source: https://docs.getmetal.io/guides/misc-index-archiving

How to archive and unarchive indexes in Metal

Index archiving allows you to manage how much data you store in Metal by temporarily shutting down indexes that are not in use. This allows you to optimize the usage of your apps, helping you save costs and make the most of your resources.

### Using the API

To archive an index using the API, call the [update index endpoint](https://docs.getmetal.io/rest-api/apps/update-app) and set the status to `DEACTIVATING`.

<CodeGroup>
  ```python Python SDK theme={null}
  metal.update_index('index_id', {'status': 'DEACTIVATING'})
  ```

  ```ts Node SDK theme={null}
  metal.updateIndex('index_id', {status: 'DEACTIVATING'})
  ```
</CodeGroup>

To unarchive an index, set the status parameter to 'UNARCHIVED'.

<CodeGroup>
  ```python Python SDK theme={null}
  metal.update_index('index_id', {'status': 'UNARCHIVED'})
  ```

  ```ts Node SDK theme={null}
  metal.updateIndex('index_id', {status: 'UNARCHIVED'})
  ```
</CodeGroup>

### Using the UI

1. Go to the Metal dashboard and select the index you want to archive.
2. Click on the Settings tab and scroll down to the Archive section.
3. Click on the Archive button and confirm your action.

<img alt="ui-flow" />


# Create a Retrieval Pipeline
Source: https://docs.getmetal.io/guides/misc-retrieval-pipeline

Build a LLM retrieval pipeline with Metal (RAG)

## What is RAG (Retrieval-Augmented Generation)?

RAG combines LLMs with external collections of data, like the Datasources we'll create next. It works in two steps:

* Retrieval: The model taps into an external source (a Datasource) to find relevant data based on a query.
* Augmented Generation: With the data it found, the model crafts a more informed response.

## 1. Create a Datasource

A Datasource is the collection of files that need to be preprocessed to feed our application.
To start, go to your organization [Dashboard](https://app.getmetal.io), navigate to the Datasources tab, and click on the `Add Datasource` button to create a new Datasource.

<img alt="rag-1" />

## 2. Add an Index and Connect it to a Datasource

Indexes are where your preprocessed data will be transformed and made queryable for your application.
To set up an Index, go to the Indexes tab, and click on the `Add Index` button to create a new Index.
You can then connect your Index to the existing Datasource.

<img alt="rag-2" />

## 3. Add Files to your Datasource

Go to the Datasource tab, and click on the `Upload File` button to upload your files.
Accepted file formats are PDF, DOC, DOCX, XLSX, and CSV. This will now become `Data Entities`.

After you've added a file, Metal preprocess the data, create the chunks and generate the embeddings that will be used for retrieval.

<img alt="rag-3" />


# Run Attribute Extraction
Source: https://docs.getmetal.io/guides/misc-run-attribute-extraction

Identify and retrieve specific pieces of information from unstructured data and transform them into a structured format.

Identify and retrieve specific pieces of information from unstructured data and transform them into a structured format.
This is useful for extracting information from documents such as invoices, receipts, and contracts.
Begin by creating a Datasource and specifying the attributes to extract. Then upload a Data Entity (pdf, docx, or csv) and stand by for the extracted results.

## 1. Add a Datasource

Log into your [Metal Account](https://app.getmetal.io/login) to get started.<br />

Navigate to the [Datasource](https://app.getmetal.io/datasources) page and add a Datasource specifying the attributes to extract. Use the description box to guide the LLM.

<img alt="Add Datasource" />

## 2. Add a Data Entity

Next create a Data Entity by uploading a file (pdf, docx, or csv) and stand by for the extracted results. Note you can add multiple Data Entities to the same Datasource.

<img alt="Upload Dataentity" />

## 3. View the Results

Once the Data Entity has been processed, you can view the extracted results.

<img alt="Results" />


# Build Semantic Search
Source: https://docs.getmetal.io/guides/misc-semantic-search

Build a semantic search engine with Metal.

## 1. Create an App

Go to your organization [Dashboard](https://app.getmetal.io) and start by creating a new App. Give it a name and select type: Search.

<img alt="search-1" />

## 2. Connect to Datasource

A Datasource is a collection of data that will feed your app. You can add multiple files or Data Entities to a Datasource.
Connect your app to an existing Datasource or create a new one.

<img alt="search-2" />

## 3. Add Data Entities

A Data Entity is a single file or entry in a Datasource. Accepted file formats are PDF, DOC, DOCX, XLSX, and CSV.

After you've added a Data Entity, Metal will create an Index, process the data into chunks, and generate the embeddings that will be used to match the user's query to the most relevant documents.

<img alt="search-3" />

## 4. Query your Search Engine

Once the data is processed, you can query your app using the search bar in the UI or using search() via the API.
Metal will return the most relevant documents based on the user's query.

<img alt="search-4" />


# Welcome
Source: https://docs.getmetal.io/introduction/introduction

Metal is your production ready, fully-managed, LLM retrieval engine.

<CardGroup>
  <Card title="Quickstart" icon="play" href="get-started-intro">
    Kick off with our Quickstart guide.
  </Card>

  <Card title="What is Metal?" icon="rectangles-mixed" href="/learn-introduction">
    Learn about the Metal Platform.
  </Card>
</CardGroup>

## API & SDK Libraries

<Card title="API Endpoints" icon="globe" href="/rest-api">
  Get started with our REST API guide.
</Card>

<CardGroup>
  <Card title="Node.js SDK" icon="js" href="/api-reference/node">
    Integrate Metal into your Node server.
  </Card>

  <Card title="Python SDK" icon="python" href="/api-reference/python">
    Integrate Metal into your Python server.
  </Card>
</CardGroup>

## Explore

<CardGroup>
  <Card title="Overview" icon="rectangles-mixed" href="/learn-introduction">
    A high level overview of the Metal Platform.
  </Card>

  <Card title="Examples" icon="code" href="examples/overview">
    Explore our examples.
  </Card>

  <Card title="Guides" icon="bars-staggered" href="/guides">
    How to guides for Metal.
  </Card>

  <Card title="Platform" icon="sitemap" href="/platform">
    Learn about the Metal Platform.
  </Card>
</CardGroup>

## Get in touch

<CardGroup>
  <Card title="Star us on GitHub" icon="star" href="https://github.com/getmetal/Metal">
    Clone our repos and examples.
  </Card>

  <Card title="Join our Discord" icon="discord" href="https://discord.com/channels/1071958892134793306/1071958892617146401">
    Meet the community and chat with us.
  </Card>

  <Card title="Follow us on X" icon="twitter" href="https://twitter.com/Metal_io">
    Stay tuned for the latest updates.
  </Card>
</CardGroup>


# AI Extraction
Source: https://docs.getmetal.io/introduction/learn-extraction

Metal Attribute Extraction allows you to easily pull and organize information from unstructured data.

## Overview

Attribute Extraction is the process of identifying specific information within unstructured data and converting it into a structured format. It scans Datasources, isolates particular pieces of information, and presents them in a structured manner.

| Title                      | Description                                       | Demo                                                                                                                            |
| :------------------------- | :------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------------ |
| Content Organization       | Structure scattered data into defined formats.    | [Organize Data Formats](https://getmetal.io/posts/19-attribute-extraction)                                                      |
| Tabular Data Extraction    | Extract tabular data from PDFs and images.        | [Tabular Data Extraction](https://getmetal.io/posts/20-finance-attribute-extraction)                                            |
| Financial Analysis Chatbot | Analyze financial documents and answer questions. | [Financial Analysis Chatbot](https://github.com/getmetal/Metal/blob/main/examples/04-financial-chatbot/financial_chatbot.ipynb) |
| Compare Documents          | Compare documents and identify differences.       | [Comparing Insurance Policies](https://getmetal.io/posts/27-compare-documents)                                                  |

## Usage

### Add Datasource

Create a datasource by calling the [Add datasource endpoint](https://docs.getmetal.io/rest-api/datasources/create) to define the field attributes to extract. Use the description parameter to guide the LLM.

### Get Datasource

Get the datasource by calling the [Get datasource endpoint](https://docs.getmetal.io/rest-api/datasources/get-one). This will return the datasource with the corresponding id.

### File Uploading ([Add Data Entities](https://docs.getmetal.io/rest-api/dataentities/create))

We support the following file types for Attribute Extraction:

* `.pdf`
* `.csv`
* `.docx`
* `.xlsx`

Upon upload, these files run through the following pipeline:

1. File is converted into a text representation via OCR (if applicable)
2. The text runs through a series of metadata extractors + augmenters
3. Attributes are stored in our database and ready to use in your indexes.

## Definitions

## Glossary

| Term                                | Definition                                                                                                                                                        |
| ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Attribute                           | Specific pieces of information identified and extracted from the raw data.                                                                                        |
| Attribute Extraction template       | An outline of specified attributes and its descriptions.                                                                                                          |
| Data Entity                         | A unique entry in a Datasource. Eg: an uploaded file and the extracted attributes.                                                                                |
| Datasource                          | A collection of Data Entities addressing a specific data concern. This could be from an integration, grouping of files, etc.                                      |
| OCR (Optical Character Recognition) | A technology that recognizes and converts different types of documents, such as scanned paper documents, PDF files, or images, into editable and searchable text. |


# What is Metal?
Source: https://docs.getmetal.io/introduction/learn-introduction

Metal is an ETL and Retrieval Engine for LLM driven experiences

<img alt="Metal Overview" />

Metal provides developers with powerful tooling to easily build high powered data ingestion and data retrieval pipelines in order to get the most out of your data and LLMs.

* **Ingestion Pipeline:** Ingest data from any source, in any format, at any scale.
* **Retrieval Engine:** Query your data for usage with LLMs.

## 🔒 Enterprise Grade

Metal is built for production. Our system is designed to be highly available, scalable, and secure. LLMs are computationally expensive, so we've built our system to be highly performant and cost effective.


# AI Memory
Source: https://docs.getmetal.io/introduction/learn-memory

Metal Memory enables you to give your users LLM session history.

## Overview

LLM Memory is the process of giving your AI the ability to remember and learn over time. This is done by storing the history of your AI's interactions with your users, and then using that data to improve your AI's performance.

## Use Cases

| Title                | Description                                                                                                | Demo                                                                                                  |
| :------------------- | :--------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------- |
| Contextual Chat      | Give your users a contextual chat experience                                                               | [CLI Chat with Memory](https://github.com/getmetal/Metal/tree/main/examples/02-motorhead-cli-chatgpt) |
| Personalized Chat    | Give your users a personalized chat experience by managing multiple conversations and remembering messages | [Chat Memory Server](https://github.com/getmetal/motorhead-redis-example)                             |
| Conversational Agent | Enable memory for your conversational agent                                                                | [Conversational Agent with Memory](https://getmetal.io/posts/15-conversational-agent-with-memory)     |

## Usage

### Add Session Memory

Add memory to your session by calling the [Add Session Memory](/api-reference/memory/post-memory) endpoint. This will add the memory to your session, and then index it.

### Get Session Memory

Get memory from your session by calling the [Get Session Memory](/api-reference/memory/get-memory) endpoint. This will return the memory from your session.

## Definitions

| Term    | Definition                                                 |
| ------- | ---------------------------------------------------------- |
| Context | Auto-generated summary of previous historical interactions |
| Memory  | An array of interactions between the AI & Human            |
| Session | A single grouping of interactions                          |


# AI Retrieval
Source: https://docs.getmetal.io/introduction/learn-retrieval

Metal Retrieval allows you to utilize AI on your data. 

## Overview

LLM Retrieval is the process of making your data compatible with LLMs. This is done by embedding your data into a vector space, then indexed into a database. This allows you to run semantic searches on your data, which is the core of LLMs.

## Use Cases

| Title              | Description                                                            | Demo                                                                                                                            |
| :----------------- | :--------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------ |
| Semantic Search    | LLMs give your search the knowledge of semantic meaning                | [Multi Tenant search](https://github.com/getmetal/Metal/blob/main/examples/03-multitenancy/multitenancy_search.ipynb)           |
| Chatbots           | Utilize LLMs to create chatbots that can answer questions on your data | [RAG Chatbot](https://github.com/getmetal/Metal/blob/main/examples/06-wwc_chatbot-python/wwc.ipynb)                             |
| Question+Answering | Build Chat apps that answer questions on top of your unstructured data | [Chat App](https://github.com/getmetal/Metal/tree/main/examples/05-ufo-chatbot-nodejs)                                          |
| Tabular Analysis   | Analyze tabular data with LLMs                                         | [Financial Analysis Chatbot](https://github.com/getmetal/Metal/blob/main/examples/04-financial-chatbot/financial_chatbot.ipynb) |
| Clustering         | Uncover hidden trends within your unstructured data                    | [Clustering Tool](https://getmetal.io/posts/10-clustering-tutorial)                                                             |
| Image Search       | Search for images based on semantic meaning                            | [Image Retrieval with CLIP](https://getmetal.io/posts/18-beyond-pixels)                                                         |

## Usage

### Indexing

We provide APIs to easily push data into our system. We support the following "primitive" data types for ingestion:

* Image URLs (.jpg, .png, .gif, .bmp, .tiff)
* Text (string)
* Embeddings (number\[])

This data will then be pushed into our indexing pipeline to generate embeddings and store in a vector db. Checkout the [Index](/api-reference/documents/indexing) endpoint for more details.

### File Importing

We provide APIs to easily push larger collections of data into our system as well. We support the following file types for ingestion:

* `.csv`
* `.docx`
* `.pdf`
* `.pptx`
* `.txt`
* `.xlsx`

Upon upload, these files run through the following pipeline:

1. The file runs through a series of metadata extractors + augmenters
2. File is converted into a text representation via OCR (if applicable)
3. The text is split into overlapping 500 token chunks
4. These chunks are embedded based on the chosen embeddings model (ada, clip, etc)
5. The embeddings are indexed into a Vector database
6. Vector is indexed into our database

### Searching

Run semantic search out of the box with our API. We support the following search term types:

* Images (.jpg, .png, .gif, .bmp, .tiff)
* Text (string)
* Embeddings (number\[])

Filtered search is also supported. Check out the [Search](/api-reference/documents/search) endpoint for more details.

## Definitions

| Term      | Definition                                                                          |
| --------- | ----------------------------------------------------------------------------------- |
| Document  | A record that stores an embedding & metadata                                        |
| Embedding | A vector representation of your data.                                               |
| Index     | A database of your embeddings.                                                      |
| Indexing  | Pushing data (raw text, files, images, etc) into our system.                        |
| Search    | An operation to run semantic searches on your index.                                |
| Tuning    | A mechanism to improve the quality of your embeddings for your particular use case. |


# Analytics
Source: https://docs.getmetal.io/platform/analytics

Observability and analytics for your application

With Analytics you can get insights into your application's performance and usage.

<img alt="Analytics" />

Understand how many queries are being executed and their average cosine distance over time.


# Integrations
Source: https://docs.getmetal.io/platform/integrations

Integrate Metal into your existing stack.

<CardGroup>
  <Card
    title="LangChain"
    icon={
<svg style={{ height: '2.5em', minWidth: '2.5em', fontSize: '1.5em'  }}>
  <text x="0" y="20">🦜🔗</text>
  </svg>
}
    href="https://python.langchain.com/docs/modules/data_connection/retrievers/integrations/metal"
  >
    Build with Metal in the LangChain NLP framework.
  </Card>

  <Card
    title="LlamaIndex"
    icon={
<svg style={{ height: '2.5em', minWidth: '2.5em', fontSize: '1.5em'  }}>
  <text x="0" y="20"> 🦙</text>
  </svg>
}
    href="https://gpt-index.readthedocs.io/en/latest/examples/vector_stores/MetalIndexDemo.html"
  >
    Build with Metal in the LlamaIndex NLP framework.
  </Card>
</CardGroup>

## Building a Metal Integration

[Reach out to us](mailto:founders@getmetal.io) to get started.


# Limits & Plans
Source: https://docs.getmetal.io/platform/limits

Plan and request limits

## Product Limits

Metal has usage limits in place at each payment plan. These limits are applied on a per-org basis. If you exceed the usage limit, you'll receive a `422` response with the following body:

```json theme={null}
{
  "statusCode": 422,
  "message": "<feature> limit reached"
}
```

### Product Limits by Plan

| Limit      | Hobbyist | Developer | Enterprise |
| :--------- | :------- | :-------- | :--------- |
| Embeddings | 100k     | 1M        | custom     |
| Files      | 100      | 500       | custom     |
| Indexes    | 2        | 10        | custom     |
| Jobs       | 10       | 100       | custom     |
| Memories   | 1k       | 1k        | custom     |
| Users      | 1        | 3         | custom     |

## Rate Limits

Metal has rate limits in place to ensure the stability of the API. These limits are applied on a per-minute basis. If you exceed the rate limit, you'll receive a `429 Too Many Requests` response with the following body:

```json theme={null}
{
  "statusCode": 429,
  "error": "Too Many Requests",
  "message": "Rate limit exceeded, retry in 1 minute"
}
```

### Rate Limits by Plan

| Plan       | Limit   |
| :--------- | :------ |
| Hobbyist   | 100 RPM |
| Developer  | 300 RPM |
| Enterprise | 1k+ RPM |

## Index Archiving

After 30 days of inactivity in an index, the index will be automatically `ARCHIVED`. Archived indexes aren't able to be searched, in order to reactivate them they need to be updated using the Index Update API with `status: "UNARCHIVED"`.

### Index Statuses

`LIVE` : Index is ready to use

`ARCHIVED`: Archived 30 days of inactivity (no search, no indexing)

`UNARCHIVED`: Ready to reindex and migrate to Live

`REINDEXING` : In the process of reindexing

`DEACTIVATING`: In the process of archiving


# Logs
Source: https://docs.getmetal.io/platform/logs

Records of user queries

Logs detailing endpoint hits, search terms, dates, status, and cosine distances.

<img alt="Logs" />

Understand user behavior, system performance, and potential areas of improvement.


# Attribute Extraction
Source: https://docs.getmetal.io/product/attribute-extraction

Extracting attributes from documents and other unstructured data.

Metal provides a simple way to extract important values out of your documents. With Attribute Extraction, you can declare what fields you care about, and Metal will extract them as you upload documents to the platform.

# Run Attribute Extraction

<CardGroup>
  <Card title="UI Creation Flow" icon="print-magnifying-glass" href="/guides/misc-run-attribute-extraction">
    How-to Guide: Run Attribute Extraction.
  </Card>

  <Card title="API" icon="gears" href="/rest-api/datasources">
    Use the API to run attribute extraction on your data.
  </Card>
</CardGroup>

# Examples

<CardGroup>
  <Card title="Attribute Extraction in Financial Analysis" icon="radar" href="https://getmetal.io/posts/20-finance-attribute-extraction">
    Extracting attributes from financial documents.
  </Card>
</CardGroup>


# Chatbot
Source: https://docs.getmetal.io/product/chatbot

Build a custom chatbot on your data in just a few steps

Metal provides all the tools you need to create a custom chatbot using your data.
You can use this chatbot to talk to users or automate tasks within your organization.
With Metal, you can make sure your chatbot understands the current conversation and
remembers information from past chats.

# Build a Chatbot

<CardGroup>
  <Card title="UI Creation Flow" icon="message-bot" href="/guides/misc-create-chatbot">
    How-to Guide: Building a Chatbot using the UI.
  </Card>

  <Card title="API" icon="gears" href="/rest-api/datasources">
    Use the API to build your chatbot.
  </Card>
</CardGroup>

# Examples

<CardGroup>
  <Card title="Chat App with Memory" icon="memory" href="https://github.com/getmetal/motorhead-redis-example">
    Motorhead and Redis LLM Chat example in Node.
  </Card>

  <Card title="Financial Analysis Chatbot" icon="file-spreadsheet" href="https://github.com/getmetal/Metal/blob/main/examples/04-financial-chatbot/financial_chatbot.ipynb">
    Deploy a chat interface to talk with spreadsheets and tabular data in Python.
  </Card>

  <Card title="CLI Chatbot" icon="square-terminal" href="https://github.com/getmetal/Metal/tree/main/examples/02-motorhead-cli-chatgpt">
    Build a CLI chatbot with Metal Memory for LLMs in Node.
  </Card>

  <Card title="'UFO' Chatbot" icon="ufo-beam" href="https://github.com/getmetal/Metal/tree/main/examples/05-ufo-chatbot-nodejs">
    Chatbot with the unclassified 'UAP' reports in Node.
  </Card>
</CardGroup>


# Semantic Search
Source: https://docs.getmetal.io/product/semantic-search

Build semantic search on your data in just a few steps

Metal provides a simple way to build semantic search on your data.
You can use the UI or the API to build a search engine over your entire dataset, or refine your queries with filters to get specific results.

# Build Semantic Search

<CardGroup>
  <Card title="UI Creation Flow" icon="searchengin" href="/guides/misc-semantic-search">
    How-to Guide: Building Semantic Search using the UI.
  </Card>

  <Card title="API" icon="gears" href="/rest-api/documents/indexing">
    Use the API to build your search engine.
  </Card>
</CardGroup>

# Examples

<CardGroup>
  <Card title="Semantic Search" icon="magnifying-glass" href="https://replit.com/@getmetal/Metal-Simple-Search-node">
    Semantic search example with Metal Retrieval in Node.
  </Card>

  <Card title="Multi Tenant Search" icon="people-roof" href="https://github.com/getmetal/Metal/blob/main/examples/03-multitenancy/multitenancy_search.ipynb">
    Build semantic search with Multi tenancy in Python.
  </Card>
</CardGroup>


# Current API Key
Source: https://docs.getmetal.io/rest-api/account/get-current

GET https://api.getmetal.io/v1/keys/current
This endpoint retrieves the current API key information for the authenticated user.

### Auth Headers

<ParamField type="string">
  API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your organization.
</ParamField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/keys/current' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
  	"data": {
  		"id": "<key-id>"
  	}
  }
  ```
</ResponseExample>


# Current Organization
Source: https://docs.getmetal.io/rest-api/account/get-org

GET https://api.getmetal.io/v1/orgs/current
This endpoint retrieves the current organization's information.

### Auth Headers

<ParamField type="string">
  An API key for your organization.
</ParamField>

<ParamField type="string">
  A Client ID for your organization.
</ParamField>

### Response

<ResponseField name="data" type="Organization Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      ID of the organization.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the organization.
    </ResponseField>

    <ResponseField name="plan" type="string">
      The current plan the organization is on.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}

  curl --location --request GET 'https://api.getmetal.io/v1/orgs/current' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
    "data": {
      "id": "<org-id>",
      "name": "My Organization",
      "plan": "developer"
    }
  }
  ```
</ResponseExample>


# List Queries
Source: https://docs.getmetal.io/rest-api/analytics/get-queries

GET https://api.getmetal.io/v1/indexes/{id}/queries
This endpoint retrieves a list of queries related to a specific index, sorted by time and limited to the last 100 queries.

### Auth Headers

<ParamField type="string">
  An API key for your organization.
</ParamField>

<ParamField type="string">
  A Client ID for your organization.
</ParamField>

### Params

<ParamField type="string">
  The Index ID
</ParamField>

### Response

<ResponseField name="data" type="QueryData Array">
  <Expandable title="properties">
    <ResponseField name="m" type="object">
      <Expandable title="sub-properties">
        <ResponseField name="org" type="string">
          Organization ID
        </ResponseField>

        <ResponseField name="index" type="string">
          Index ID
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="t" type="string">
      Timestamp of the query
    </ResponseField>

    <ResponseField name="d" type="number">
      The cosine distance of the query
    </ResponseField>

    <ResponseField name="q" type="string">
      The query
    </ResponseField>
  </Expandable>
</ResponseField>

### Request Example

<RequestExample>
  ```bash theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/indexes/{id}/queries' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
    "data": [
      {
        "t": "2023-08-30T11:34:35.128Z",
        "d": 0.176956892014,
        "q": "Who is the best drummer of all time?"
        },
      {
        "t": "2023-08-29T11:34:18.099Z",
        "d": 0.156819581985,
        "q": "Which band is represented by the iconic mascot Eddie?"
      }
      // ... up to 100 items
    ]
  }
  ```
</ResponseExample>


# Add an App
Source: https://docs.getmetal.io/rest-api/apps/add-app

POST https://api.getmetal.io/v1/apps
This endpoint creates an app.

## Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

## Body

<ParamField type="string" placeholder="name of the app">
  Name of the App.
</ParamField>

<ParamField type="array" placeholder="['indexID']">
  An array containing the ID of the index you want to connect with the app. This array can have a maximum length of 1.
</ParamField>

## Response

<ResponseField name="data" type="App Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the app.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the app.
    </ResponseField>

    <ResponseField name="indexes" type="array">
      An array indicating the connected index for the app. Empty if there are is connected index.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  curl --location --request POST 'https://api.getmetal.io/v1/apps' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "name": "Metal App"
      "indexes": ["indexID"]
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
      "data": {
          "id": "6535126922af1b89d6d22ffa",
          "name": "Metal App",
          "indexes": ["indexID"]
      }
  }
  ```
</ResponseExample>


# Get an App
Source: https://docs.getmetal.io/rest-api/apps/get-app

GET https://api.getmetal.io/v1/apps/{app_id}
This endpoint retrieves the details of an app by its ID.

## Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

## Path Parameters

<ParamField type="string">
  The unique ID of the app you want to retrieve.
</ParamField>

## Response

<ResponseField name="data" type="App Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the app.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the app.
    </ResponseField>

    <ResponseField name="indexes" type="array">
      An array indicating the connected indexes for the app. Empty if there are no connected indexes.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/apps/{app_id}' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
      "data": {
          "id": "6535126922af1b89d6d22ffa",
          "name": "My App",
          "indexes": [
              "6525639932cbb1afa1bee7e9"
          ]
      }
  }
  ```
</ResponseExample>


# Get All Apps
Source: https://docs.getmetal.io/rest-api/apps/get-apps

GET https://api.getmetal.io/v1/apps
This endpoint retrieves a list of all apps for the specified organization

## Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

## Response

<ResponseField name="data" type="App Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the app.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the app.
    </ResponseField>

    <ResponseField name="indexes" type="array">
      An array indicating the connected indexes for the app. Empty if there are no connected indexes.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/apps' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
  	"data": [
          {
  		  "id": "6535126922af1b89d6d22ffa",
  		  "name": "my metal app",
  		  "indexes": []
  	    },
          //... other app objects
      ]
  }

  ```
</ResponseExample>


# Update an App
Source: https://docs.getmetal.io/rest-api/apps/update-app

PUT https://api.getmetal.io/v1/apps/{appId}
This endpoint updates an existing app based on the provided App ID.

## Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

## Path Parameters

<ParamField type="string">
  The unique identifier of the app you want to update.
</ParamField>

## Body

<ParamField type="string" placeholder="updated name of the app">
  The updated name for the App.
</ParamField>

<ParamField type="array" placeholder="['indexID']">
  An updated array of an index connected to the app.
</ParamField>

## Response

<ResponseField name="data" type="App Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the updated app.
    </ResponseField>

    <ResponseField name="name" type="string">
      Updated name of the app.
    </ResponseField>

    <ResponseField name="indexes" type="array">
      An array indicating the connected index for the updated app. Empty if there is no connected index.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  curl --location --request PUT 'https://api.getmetal.io/v1/apps/6535126922af1b89d6d22ffa' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "name": "Updated Metal App",
      "indexes": ["index1"]
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
      "data": {
          "id": "6535126922af1b89d6d22ffa",
          "name": "Updated Metal App",
          "indexes": ["index1"]
      }
  }
  ```
</ResponseExample>

Feel free to adjust as per the exact requirements and specifics of the API endpoint.


# Add a Data Entity
Source: https://docs.getmetal.io/rest-api/dataentities/create

POST https://api.getmetal.io/v1/data-entities
This endpoint creates a new Data Entity.

### Two-step process

This endpoint is a two-step process. First, you create the data entity's URL, and then you upload the file to the signed URL.

* Step 1: `POST https://api.getmetal.io/v1/data-entities`
* Step 2: `PUT` to the signed URL

## Step 1: Request URL for the File

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="ObjectID" placeholder="Datasource ObjectID">
  The datasource ID.
</ParamField>

<ParamField type="string" placeholder="Entity Name">
  The name of the data entity.
</ParamField>

<ParamField type="object">
  An object of metdatadata keys/values. The key is the field key and the value
  is the field value for this entity.
</ParamField>

<ParamField type="string" placeholder="Source Type">
  The type of the source (e.g., "PDF", "Text").
</ParamField>

### Response

<ResponseField name="data" type="Data Entity Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      The unique identifier of the data entity.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the data entity.
    </ResponseField>

    <ResponseField name="createdBy" type="string">
      The user ID who created this entity.
    </ResponseField>

    <ResponseField name="createdAt" type="string">
      Timestamp indicating when the data entity was created.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X POST "https://api.getmetal.io/v1/data-entities" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data '{
    "datasource": "datasourceID",
    "name": "Song Lyrics",
    "sourceType": "file",
    "metadata": {
      "author": "Rolling Stone"
    }
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
    "data": {
      "id": "dataEntityID",
      "datasource": "datasourceID",
      "name": "Song Lyrics",
      "extractedMetadata": [
        {
          "name": "band",
          "value": "Iron Maiden",
          "type": "string",
          "autoExtract": true
        }
      ],
      "sourcetype": "file",
      "status": "CREATED",
      "createdAt": "2023-08-29T17:00:55.002Z",
      "updatedAt": "1970-01-01T00:00:00Z",
      "metadata": {
        "author": "Rolling Stone",
        "band": "Iron Maiden"
      },
      "metadataFields": [
        {
          "name": "band",
          "type": "string",
          "description": "Which heavy metal band is represented by the iconic mascot Eddie?",
          "autoExtract": true
        },
        {
          "name": "author",
          "type": "string",
          "autoExtract": false
        }
      ],
      "url": "signed_upload_url",
      "createdBy": "userID"
    }
  }
  ```
</ResponseExample>

## Step 2: Upload the File to the Signed URL

```bash theme={null}
# Example Request
curl --location --request PUT '<preSignedUrl>' \
--header 'Content-Type: application/pdf' \
--upload-file '/path/to/your/example.pdf'
```


# Delete Data Entity
Source: https://docs.getmetal.io/rest-api/dataentities/delete

DELETE https://api.getmetal.io/v1/data-entities/{id}
This endpoint deletes a specific data entity by its ID.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The unique ID of the data entity you wish to delete.
</ParamField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X DELETE "https://api.getmetal.io/v1/data-entities/{id}" \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  204 No Content
  ```
</ResponseExample>


# List Data Entities
Source: https://docs.getmetal.io/rest-api/dataentities/get-all

GET https://api.getmetal.io/v1/datasources/{datasourceID}/data-entities
This endpoint retrieves a list of all data entities.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Query

<ParamField type="integer">
  The maximum number of data entities to return. Default is 10.
</ParamField>

<ParamField type="integer">
  The starting position for the query. Useful for pagination.
</ParamField>

### Response

<ResponseField name="data" type="array">
  <Expandable title="Data Entity Object">
    <ResponseField name="id" type="string">
      The unique identifier of the data entity.
    </ResponseField>

    <ResponseField name="datasource" type="string">
      The ID of the datasource this entity belongs to.
    </ResponseField>

    <ResponseField name="name" type="string">
      The name of the data entity.
    </ResponseField>

    <ResponseField name="extractedMetadata" type="array">
      An array of extracted metadata fields.
    </ResponseField>

    <ResponseField name="sourceType" type="string">
      The source type of the data entity.
    </ResponseField>

    <ResponseField name="status" type="string">
      The status of the data entity.
    </ResponseField>

    <ResponseField name="createdAt" type="string">
      The creation timestamp of the data entity.
    </ResponseField>

    <ResponseField name="updatedAt" type="string">
      The update timestamp of the data entity.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X GET "https://api.getmetal.io/v1/datasources/{datasourceID}/data-entities" \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
    "data": [
      {
        "id": "id1",
        "datasource": "datasourceID",
        "name": "record_deal.pdf",
        "extractedMetadata": [
          {
            "name": "band",
            "value": "Iron Maiden",
            "type": "string"
          }
        ],
        "sourceType": "File",
        "status": "Active",
        "createdAt": "2023-08-29T10:51:04.246Z",
        "updatedAt": "2023-08-29T10:51:04.246Z",
        "metadataFields": [
          {
            "name": "band",
            "type": "string",
            "description": "Which band is this record deal for?"
          }
        ],
        "createdBy": "userID"
      },
      {
        "id": "id2",
        "datasource": "datasource_id",
        "name": "concert_list.docx",
        "extractedMetadata": [
          {
            "name": "Venue",
            "value": "Glastonbury",
            "type": "string"
          }
        ],
        "sourceType": "File",
        "status": "Active",
        "createdAt": "2023-08-29T10:51:04.246Z",
        "updatedAt": "2023-08-29T10:51:04.246Z",
        "metadataFields": [
          {
            "name": "band",
            "type": "string",
            "description": "Where is the next concert at?"
          }
        ],
        "createdBy": "userID"
      }
    ]
  }
  ```
</ResponseExample>


# Get Data Entity
Source: https://docs.getmetal.io/rest-api/dataentities/get-one

GET https://api.getmetal.io/v1/data-entities/{id}
This endpoint retrieves a Data Entity by its ID

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The unique identifier of the data entity you wish to retrieve.
</ParamField>

### Response

<ResponseField name="data" type="Data Entity Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      The unique identifier of the data entity.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the data entity.
    </ResponseField>

    <ResponseField name="attributes" type="array">
      An array of attributes, each with a name, type, and value.
    </ResponseField>

    <ResponseField name="createdBy" type="string">
      The user ID who created this entity.
    </ResponseField>

    <ResponseField name="createdAt" type="string">
      Timestamp indicating when the data entity was created.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X GET "https://api.getmetal.io/v1/data-entities/{id}" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
    "data": {
      "id": "entityID",
      "datasource": "datasourceID",
      "name": "record_deal.pdf",
      "extractedMetadata": [
        {
          "name": "band",
          "type": "string",
          "value": "Iron Maiden"
        },
        {
  				"name": "recordLabel",
  				"value": "EMI",
  				"type": "string"
  			},
      ],
      "sourcetype": "file",
  		"status": "EXTRACTED",
      "createdAt": "2023-08-29T12:34:56.789Z",
      "ocrData": {
  			"updatedAt": "1970-01-01T00:00:00Z",
  			"engine": "unstructured",
  			"dataEntity": "",
  			"filetype": "pdf",
  			"filename": "record_deal.pdf",
  			"data": [
  				{
  					"type": "UncategorizedText",
  					"text": "SEP-3-2R1@ B3:11F FROM: T0:17825834718 P.2"
  				},
  				{
  					"type": "Title",
  					"text": "RECORD DEAL AGREEMENT"
  				},
          // ...
        ],
      },
      "metadataFields": [
  			{
  				"name": "Partners",
  				"type": "string",
  				"description": "Who are the partners involved in the deal?"
        },
      ],
      "url": "signed-url-to-file",
      "createdBy": "007"
  }
  }
  ```
</ResponseExample>


# Add a Datasource
Source: https://docs.getmetal.io/rest-api/datasources/create

POST https://api.getmetal.io/v1/datasources
This endpoint creates and stores a Datasource with the provided details.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="string" placeholder="Datasource Name">
  Name of the Datasource.
</ParamField>

<ParamField type="string" placeholder="Source Type">
  Type of the source. File or Text.
</ParamField>

<ParamField type="boolean">
  Flag indicating whether auto-extraction is enabled. If disabled, it will
  short-cirtuit to bypass extraction.
</ParamField>

<ParamField type="array">
  An array containing metadata fields. Each object should have `name`, `type`,
  `description`, and `autoExtract` properties.
</ParamField>

<RequestExample>
  ```bash Example Request theme={null}
  curl -X POST "https://api.getmetal.io/v1/datasources" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "name": "my_datasource",
      "metadataFields": [
        {
          "name": "band",
          "type": "string",
          "description": "The band that the uploaded file is about.",
          "autoExtract": true
        },
        {
          "name": "author",
          "type": "string",
          "autoExtract": false
        }
      ],
      "sourcetype": "file",
      "autoExtract": true
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "id": "datasourceID",
      "createdAt": "2023-08-29T10:51:04.246Z",
      "createdBy": "userId",
      "name": "my_datasource",
      "metadataFields": [
        // Extracted field, which we will detect on upload
        {
          "name": "band",
          "type": "string",
          "description": "Which heavy metal band is represented by the iconic mascot Eddie?",
          "autoExtract": true
        },
        // Normal field, which needs to be manually set on upload
        {
          "name": "author",
          "type": "string",
          "autoExtract": false
        }
      ],
      "sourcetype": "File",
      "autoExtract": true
    }
  }
  ```
</ResponseExample>


# Delete Datasource
Source: https://docs.getmetal.io/rest-api/datasources/delete

DELETE https://api.getmetal.io/v1/datasources/{id}
This endpoint deletes a Datasource by its ID.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The unique identifier of the Datasource you wish to delete.
</ParamField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X DELETE "https://api.getmetal.io/v1/datasources/{id}" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  204 No Content
  ```
</ResponseExample>


# List Datasources
Source: https://docs.getmetal.io/rest-api/datasources/get-all

GET https://api.getmetal.io/v1/datasources
This endpoint retrieves a list with all Datasources available for the authenticated org.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Query Parameters

<ParamField type="integer">
  Number of Datasources to retrieve per request (optional).
</ParamField>

<ParamField type="integer">
  Number to start pagination from (optional).
</ParamField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X GET "https://api.getmetal.io/v1/datasources" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
    {
      "data": [
        {
          "id": "datasourceID",
          "createdAt": "2023-08-22T21:49:26.564Z",
          "createdBy": "000000000000000000000000",
          "name": "my-data-source-1",
          "metadataFields": [
            {
              "name": "band",
              "type": "string",
              "description": "Which heavy metal band is represented by the iconic mascot Eddie?"
            }
          ],
          "sourcetype": "File",
          "autoExtract": true
        },
        {
          "id": "id2",
          "createdAt": "2023-08-22T21:49:26.564Z",
          "createdBy": "userID",
          "name": "my-data-source-2",
          "metadataFields": [
            {
              "name": "year",
              "type": "number",
              "description": "In what year was the debut album featuring Eddie released?"
            }
          ],
          "sourcetype": "File",
          "autoExtract": false
        }
      ],
    }

  ```
</ResponseExample>


# Get Datasource
Source: https://docs.getmetal.io/rest-api/datasources/get-one

GET https://api.getmetal.io/v1/datasources/{id}
This endpoint retrieves the details of a specific Datasource by its ID.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The unique identifier of the Datasource you wish to retrieve.
</ParamField>

### Response

<ResponseField name="data" type="Datasource Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      The unique identifier of the Datasource.
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the Datasource.
    </ResponseField>

    <ResponseField name="metadataFields" type="array">
      An array of metadata fields, each with a name, type, and description.
    </ResponseField>

    <ResponseField name="sourcetype" type="string">
      Type of the Datasource.
    </ResponseField>

    <ResponseField name="autoExtract" type="boolean">
      Flag indicating whether auto-extraction is enabled or not.
    </ResponseField>

    <ResponseField name="createdAt" type="string">
      Timestamp indicating when the Datasource was created.
    </ResponseField>

    <ResponseField name="createdBy" type="string">
      ID of the user or key that created the Datasource.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X GET "https://api.getmetal.io/v1/datasources/{id}" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "id": "datasourceID",
      "createdAt": "2023-08-29T11:22:26.172Z",
  		"createdBy": "userID",
      "name": "my_datasource",
      "metadataFields": [
        {
          "name": "band",
          "type": "string",
          "description": "Which heavy metal band is represented by the iconic mascot Eddie?"
        }
      ],
      "sourcetype": "Text",
      "autoExtract": false,
      "createdAt": "2023-08-29T10:51:04.246Z",
      "createdBy": "userId",
    }
  }
  ```
</ResponseExample>


# Update Datasource
Source: https://docs.getmetal.io/rest-api/datasources/update

PUT https://api.getmetal.io/v1/datasources/{id}
This endpoint updates the details of an existing Datasource by its ID.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The unique identifier of the Datasource you wish to update.
</ParamField>

### Body

<ParamField type="string">
  Updated name of the Datasource.
</ParamField>

<ParamField type="array">
  An array of updated metadata fields, each having a name, type, and description.
</ParamField>

<ParamField type="string">
  Updated type of the Datasource.
</ParamField>

<ParamField type="boolean">
  Updated flag indicating whether auto-extraction is enabled or not.
</ParamField>

<RequestExample>
  ```bash theme={null}
  # Example Request
  curl -X PUT "https://api.getmetal.io/v1/datasources/{id}" \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
    "name": "updated_datasource",
    "metadataFields": [
      {
        "name": "updated_field",
        "type": "string",
        "description": "Updated description"
      }
    ],
    "sourcetype": "updated_type",
    "autoExtract": false
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json theme={null}
  {
    "data": {
      "id": "datasourceID",
      "name": "updated_datasource",
      "metadataFields": [
        {
          "name": "updated_field",
          "type": "string",
          "description": "Updated description"
        }
      ],
      "sourcetype": "updated_type",
      "autoExtract": false,
      "updatedAt": "2023-03-17T17:21:13.163Z"
    }
  }
  ```
</ResponseExample>


# Add Documents (bulk)
Source: https://docs.getmetal.io/rest-api/documents/bulk

POST https://api.getmetal.io/v1/index/bulk
This endpoint generates and stores a Document(embedding) with the inputted data in bulk.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="array" placeholder="Data">
  data
</ParamField>

#### Data Object

<ParamField type="string" placeholder="Index ID">
  Index ID
</ParamField>

<ParamField type="string">
  Identifier for your embedding. Must be between 3 - 256 chars and consist of
  numbers, letters, or `-`. If not provided, a random ID will be generated.
</ParamField>

<ParamField type="string">
  Text to be embedded. Must be between 3 - 10,000 chars.
</ParamField>

<ParamField type="string">
  Image URL
</ParamField>

<ParamField type="number[]">
  Custom Embeddings
</ParamField>

<ParamField type="object">
  Additional metadata for the document.
</ParamField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request POST 'https://api.getmetal.io/v1/index/bulk' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "data": [
          {
              "index": "index-id",
              "text": "Ozzy is the best vocalist of all time",
              "id": "my-unique-id1",
              "metadata": {
                  "band": "Black Sabbath",
                  "year": 1970
              }
          },
          {
              "index": "index-id",
              "text": "Dave Mustain disagrees",
              "id": "my-unique-id2",
              "metadata": {
                  "band": "Megadeth",
                  "year": 1983

              }
          }
      ]
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
  	"data": {
  		"ids": [
  			"my-unique-id1",
  			"my-unique-id2"
  		]
  	}
  }
  ```
</ResponseExample>


# Delete Documents (bulk)
Source: https://docs.getmetal.io/rest-api/documents/delete-many

DELETE https://api.getmetal.io/v1/indexes/{indexId}/documents/bulk
This endpoint deletes multiple embedding documents.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="string[]">
  Array of document ids to delete
</ParamField>

### Response

<ResponseField name="data" type="object" />

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request DELETE 'https://api.getmetal.io/v1/indexes/{indexId}/v1/documents/bulk' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "ids": ["id1", "id2"],
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "ids": ["id1", "id2"]
    }
  }
  ```
</ResponseExample>


# Delete Document
Source: https://docs.getmetal.io/rest-api/documents/delete-one

DELETE https://api.getmetal.io/v1/indexes/{indexId}/documents/{documentId}
This endpoint deletes an embedding document.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The Document ID
</ParamField>

### Response

<ResponseField name="data" type="object" />

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request DELETE 'https://api.getmetal.io/v1/indexes/{indexId}/documents/{documentId}' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "id": "document-id"
    }
  }
  ```
</ResponseExample>


# List Documents
Source: https://docs.getmetal.io/rest-api/documents/get

GET https://api.getmetal.io/v1/indexes/{indexId}/documents
This endpoint gets a list of documents for a given index.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The Index ID
</ParamField>

### Query Params

<ParamField type="number">
  The number of documents to fetch. Defaults to 10.
</ParamField>

<ParamField type="number">
  The page number to fetch. Defaults to 1.
</ParamField>

### Response

<ResponseField name="data" type="Document Array">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the document
    </ResponseField>

    <ResponseField name="imageUrl" type="string">
      If set, Image URL for the document
    </ResponseField>

    <ResponseField name="text" type="string">
      If set, Text for the document
    </ResponseField>

    <ResponseField name="createdAt" type="string">
      The created at date of the document
    </ResponseField>

    <ResponseField name="metadata" type="object">
      The metadata of the document
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="lastSeenObjectId" type="String">
  The last object id seen in the result set. This can be used for deep pagination.
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/indexes/{indexId}/documents' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": [
      {
        "id": "1",
        "text": "My text 1",
        "createdAt": "2023-07-03T14:27:02.413Z",
        "metadata": {}
      },
      {
        "id": "2",
        "text": "My text 2",
        "createdAt": "2023-07-03T14:27:02.418Z",
        "metadata": {}
      }
    ],
    "lastSeenObjectId": "64d529f3bd43b75ce5e020f9"
  }
  ```
</ResponseExample>


# Get Document
Source: https://docs.getmetal.io/rest-api/documents/get-one

GET https://api.getmetal.io/v1/indexes/{indexId}/documents/{documentId}
This endpoint gets an embedding document.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The Document ID
</ParamField>

### Response

<ResponseField name="data" type="App Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the document
    </ResponseField>

    <ResponseField name="text" type="string">
      The embedded text
    </ResponseField>

    <ResponseField name="metadata" type="object">
      The metadata of your embedding
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/indexes/{indexId}/documents/{documentId}' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "id": "642b6f4za4b817d11769ae15",
      "text": "my embedded text",
      "createdAt": "2023-04-04T00:28:58.425Z",
      "metadata": {}
    }
  }
  ```
</ResponseExample>


# Add Document
Source: https://docs.getmetal.io/rest-api/documents/indexing

POST https://api.getmetal.io/v1/index
This endpoint generates and stores a Document(embedding) with the inputted data.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="string" placeholder="Index ID">
  Index ID
</ParamField>

<ParamField type="string">
  Identifier for your embedding. Must be between 3 - 256 chars and consist of
  numbers, letters, or `-`. If not provided, a random ID will be generated.
</ParamField>

<ParamField type="string">
  Text to be embedded. Must be between 3 - 10,000 chars.
</ParamField>

<ParamField type="string">
  Image URL
</ParamField>

<ParamField type="number[]">
  Custom Embeddings
</ParamField>

<ParamField type="object">
  Additional metadata for the document.
</ParamField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request POST 'https://api.getmetal.io/v1/index' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "index": "index-id",
      "text": "text that will be embedded",
      "id": "my-unique-id",
      "metadata": {
        "fieldA": "my custom value",
        "fieldB": "my other custom value"
      }
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {

    "data": {
      "id": "my-unique-id",
      "text": "text that will be embedded",
      "createdAt": "2023-03-17T17:21:13.163Z",
      "metadata": {
        "fieldA": "my custom value",
        "fieldB": "my other custom value"

        }
    }

  }
  ```
</ResponseExample>


# Search Documents
Source: https://docs.getmetal.io/rest-api/documents/search

POST https://api.getmetal.io/v1/search
Semantically search your embedded documents.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="string" placeholder="Index ID">
  Index ID
</ParamField>

<ParamField type="string">
  Text
</ParamField>

<ParamField type="string">
  Image URL
</ParamField>

<ParamField type="number[]">
  Custom Embeddings
</ParamField>

<ParamField type="string[]">
  Subset of documents ids to match
</ParamField>

<ParamField type="object">
  <Expandable title="properties">
    <ParamField type="object[]">
      <Expandable title="properties">
        <ParamField type="string">
          The name of the filtereable field
        </ParamField>

        <ParamField type="string or number">
          The field value to search for.
        </ParamField>

        <ParamField type="string">
          Possible values: `eq`, `gt`, `gte`, `lt`, `lte`
        </ParamField>
      </Expandable>
    </ParamField>

    <ParamField type="object[]">
      <Expandable title="properties">
        <ParamField type="string">
          The name of the filtereable field
        </ParamField>

        <ParamField type="string or number">
          The field value to search for.
        </ParamField>

        <ParamField type="string">
          Possible values: `eq`, `gt`, `gte`, `lt`, `lte`
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

### Query

<ParamField type="boolean">
  Return only a list of ids without any metadata.
</ParamField>

<ParamField type="number">
  The limit of possible reponses from your search. Max 100.
</ParamField>

### Response

<ResponseField name="data" type="Results Array">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the embedding
    </ResponseField>

    <ResponseField name="dist" type="string">
      Distance to the vector queried
    </ResponseField>

    <ResponseField name="metadata" type="object">
      Metadata of the embedding
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request POST 'https://api.getmetal.io/v1/search' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "index": "index-id",
      "text": "Generals gathered in their masses",
      "filters": {
          "and": [{
              "field": "year",
              "value": 1984,
              "operator": "lte"
          }],
          "or": [{
              "field": "name",
              "value": "Iron Maiden",
              "operator": "eq"
          }, {
              "field": "name",
              "value": "Helloween",
              "operator": "eq"
          }]
      }
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": [
      {
        "dist": "0.666",
        "id": "1",
        "text": "Eagle Fly Free",
        "metadata": {
          "name": "Helloween",
          "year": 1984
        },
        "createdAt":"2023-08-22T18:30:11.493Z"
      },
      {
        "id": "2",
        "dist": "0.999",
        "text": "The Trooper",
        "metadata": {
          "name": "Iron Maiden",
          "year": 1978
        },
        "createdAt": "2023-08-10T11:44:53.010Z"
        }
      }
    ]
  }
  ```
</ResponseExample>


# Update Document
Source: https://docs.getmetal.io/rest-api/documents/update

POST https://api.getmetal.io/v1/index
This endpoint updates and replaces an existing Document(embedding).

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="string" placeholder="Index ID">
  Index ID
</ParamField>

<ParamField type="string" placeholder="ID to update">
  Existing Document ID to update. Must be between 3 - 256 chars and consist of
  numbers, letters, or `-`.
</ParamField>

<ParamField type="string">
  Text to be embedded. Must be between 3 - 10,000 chars.
</ParamField>

<ParamField type="string">
  Image URL
</ParamField>

<ParamField type="number[]">
  Custom Embeddings
</ParamField>

<ParamField type="object">
  Metadata object.
</ParamField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request POST 'https://api.getmetal.io/v1/index' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "index": "index-id",
      "text": "text that will replace the existing document",
      "id": "id-to-update",
      "metadata": {
          "fieldA": "my custom value"
      }
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "id": "id-to-update",
    "text": "text that will replace the existing document",
    "createdAt": "2023-03-17T17:21:13.163Z",
    "metadata": {
      "fieldA": "my custom value"
    }

  }
  ```
</ResponseExample>


# Delete Index
Source: https://docs.getmetal.io/rest-api/indexes/delete

DELETE https://api.getmetal.io/v1/indexes/{indexId}
This endpoint deletes an index and all documents within that index.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The Index ID
</ParamField>

### Response

<ResponseField name="accepted" type="boolean">Whether the deletion was successful or not.</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request DELETE 'https://api.getmetal.io/v1/indexes/{indexId}' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "accepted": true
  }
  ```
</ResponseExample>


# List Indexes
Source: https://docs.getmetal.io/rest-api/indexes/get

GET https://api.getmetal.io/v1/indexes
This endpoint gets a list of indexes for an app.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The App ID
</ParamField>

### Response

<ResponseField name="data" type="Index Array">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the index
    </ResponseField>

    <ResponseField name="app" type="string">
      Id of the app of the index
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the index
    </ResponseField>

    <ResponseField name="model" type="string">
      Model used to generate the embeddings
    </ResponseField>

    <ResponseField name="datasource" type="string">
      The datasource id, if one is connected.
    </ResponseField>

    <ResponseField name="dimensions" type="number">
      Dimensions of the embeddings
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/indexes' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": [
      {
        "id": "1",
        "createdAt": "2023-08-23T22:13:31.539Z",
        "app": "<APP_ID>",
        "name": "Bruce's Index",
        "model": "text-embedding-ada-002",
        "dimensions": 1536,
        "datasource": "<DATASOURCE_ID>",
        "lastActivityAt": "2023-09-14T13:18:52.547Z",
        "lastSearchAt": "2023-09-14T13:18:52.547Z",
        "filters": [
          {
            "field": "name",
            "type": "string"
          },
          {
            "field": "age",
            "type": "number"
          }
        ],
        "chunkConfig": {
          "size": 24000
        }
      },
      {
        "id": "2",
        "createdAt": "2023-08-23T15:35:27.398Z",
        "app": "<APP_ID>",
        "name": "Dave's Index",
        "model": "clip",
        "dimensions": 512,
        "datasource": "<DATASOURCE_ID>",
        "filters": [
          {
            "field": "name",
            "type": "string"
          },
          {
            "field": "age",
            "type": "number"
          }
        ],
        "chunkConfig": {
          "size": 24000
        }
      }
    ]
  }
  ```
</ResponseExample>


# Get Index
Source: https://docs.getmetal.io/rest-api/indexes/get-one

GET https://api.getmetal.io/v1/indexes/{indexId}
This endpoint gets a single index.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The Index ID
</ParamField>

### Response

<ResponseField name="data" type="Index Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the index
    </ResponseField>

    <ResponseField name="app" type="string">
      Id of the app of the index
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the index
    </ResponseField>

    <ResponseField name="model" type="string">
      Model used to generate the embeddings
    </ResponseField>

    <ResponseField name="datasource" type="string">
      The datasource id, if one is connected.
    </ResponseField>

    <ResponseField name="dimensions" type="number">
      Dimensions of the embeddings
    </ResponseField>

    <ResponseField name="chunkConfig" type="object" placeholder="Chunking configuration for data entity uploads.">
      <Expandable title="properties">
        <ResponseField name="size" type="number">
          The token size of each chunk.
        </ResponseField>

        <ResponseField name="overlap" type="number">
          The token amount of overlap between chunks.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="tableChunkConfig" type="object" placeholder="Chunking configuration for data entity uploads that contain tables.">
      <Expandable title="properties">
        <ResponseField name="size" type="number">
          The token size of each chunk.
        </ResponseField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request GET 'https://api.getmetal.io/v1/indexes/{indexId}' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "id": "1",
      "createdAt": "2023-07-03T14:25:22.104Z",
      "app": "<APP_ID>",
      "name": "Danzig's Index",
      "model": "clip",
      "datasource": "<DATASOURCE_ID>",
      "dimensions": 512,
      "lastActivityAt": "2023-09-14T13:18:52.547Z",
      "lastSearchAt": "2023-09-14T13:18:52.547Z",
      "filters": [
        {
          "field": "name",
          "type": "string"
        },
        {
          "field": "age",
          "type": "number"
        }
      ],
      "counts": {
        "docs": 42,
        "searches": 666
      },
      "chunkConfig": {
        "size": 999
      },
      "tableChunkConfig": {
        "size": 4000
      },
    }
  }
  ```
</ResponseExample>


# Add Index
Source: https://docs.getmetal.io/rest-api/indexes/post

POST https://api.getmetal.io/v1/indexes
This endpoint creates an index for an app.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Body

<ParamField type="string" placeholder="Embedding model used on your data.">
  The Embedding Model to use. Can be either "text-embedding-ada-002", "clip", or "custom".
</ParamField>

<ParamField type="string" placeholder="name of the index">
  Name of the Index
</ParamField>

<ParamField type="string" placeholder="datasource id">
  Datasource ID to connect to the index.
</ParamField>

<ParamField type="string" placeholder="FLAT or HNSW">
  Type of the index. `FLAT` or `HNSW`
</ParamField>

<ParamField type="integer" placeholder="number of dimensions that the embeddings will have">
  This is only required if you are using a "custom" model.
</ParamField>

<ParamField type="object[]" placeholder="Metadata filters that the index can support for /search.">
  <Expandable title="properties">
    <ParamField type="string">
      The name of the filtereable field
    </ParamField>

    <ParamField type="string">
      The field type. Enum: `string` or `number`.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="object" placeholder="Chunking configuration for data entity uploads.">
  <Expandable title="properties">
    <ParamField type="number">
      The token size of each chunk.
    </ParamField>

    <ParamField type="number">
      The token amount of overlap between chunks.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="object" placeholder="Chunking configuration for data entity uploads that contain tables.">
  <Expandable title="properties">
    <ParamField type="number">
      The token size of each table chunk.
    </ParamField>
  </Expandable>
</ParamField>

### Response

<ResponseField name="data" type="Index Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the index
    </ResponseField>

    <ResponseField name="status" type="string">
      Status of the index
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the index
    </ResponseField>

    <ResponseField name="model" type="string">
      Model used to generate the embeddings
    </ResponseField>

    <ResponseField name="dimensions" type="number">
      Dimensions of the embeddings
    </ResponseField>

    <ResponseField name="chunkConfig" type="object" placeholder="Chunking configuration for data entity uploads.">
      <Expandable title="properties">
        <ResponseField name="size" type="number">
          The token size of each chunk.
        </ResponseField>

        <ResponseField name="overlap" type="number">
          The token amount of overlap between chunks.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="tableChunkConfig" type="object" placeholder="Chunking configuration for data entity uploads that contain tables.">
      <Expandable title="properties">
        <ResponseField name="size" type="number">
          The token size of each chunk.
        </ResponseField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request POST 'https://api.getmetal.io/v1/indexes' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "model": "text-embedding-ada-002",
      "name": "Ozzy Osbourne",
      "filters": [
        {
          "field": "name",
          "type": "string"
        },
        {
          "field": "age",
          "type": "number"
        }
      ]
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
  	"data": {
  		"id": "1",
  		"createdAt": "2023-08-23T22:13:31.539Z",
  		"status": "LIVE",
  		"name": "Ozzy Osbourne",
  		"model": "text-embedding-ada-002",
  		"dimensions": 1536,
  		"filters": [
  			{
  				"field": "name",
  				"type": "string"
  			},
  			{
  				"field": "age",
  				"type": "number"
  			}
  		],
      "chunkConfig": {
        "size": 500,
        "overlap": 20
      },
      "tableChunkConfig": {
        "size": 4000
      },
  		"counts": {
  			"docs": 0,
  		}
  	}
  }
  ```
</ResponseExample>


# Update Index
Source: https://docs.getmetal.io/rest-api/indexes/update

PUT https://api.getmetal.io/v1/indexes/{indexId}
This endpoint updates an index.

### Auth Headers

<ParamField type="string">
  An API key for your org.
</ParamField>

<ParamField type="string">
  A Client ID for your org.
</ParamField>

### Params

<ParamField type="string">
  The Index ID to update
</ParamField>

### Body

<ParamField type="string" placeholder="status of the index">
  Status to update the index. Can only be `DEACTIVATING` to start the archiving process
</ParamField>

<ParamField type="object" placeholder="Chunking configuration for data entity uploads.">
  <Expandable title="properties">
    <ParamField type="number">
      The token size of each chunk.
    </ParamField>

    <ParamField type="number">
      The token amount of overlap between chunks.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField type="object" placeholder="Chunking configuration for data entity uploads that contain tables.">
  <Expandable title="properties">
    <ParamField type="number">
      The token size of each table chunk.
    </ParamField>
  </Expandable>
</ParamField>

### Response

<ResponseField name="data" type="Index Object">
  <Expandable title="properties">
    <ResponseField name="id" type="string">
      Id of the index
    </ResponseField>

    <ResponseField name="status" type="string">
      Status of the index
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the index
    </ResponseField>

    <ResponseField name="model" type="string">
      Model used to generate the embeddings
    </ResponseField>

    <ResponseField name="dimensions" type="number">
      Dimensions of the embeddings
    </ResponseField>

    <ResponseField name="chunkConfig" type="object" placeholder="Chunking configuration for data entity uploads.">
      <Expandable title="properties">
        <ResponseField name="size" type="number">
          The token size of each chunk.
        </ResponseField>

        <ResponseField name="overlap" type="number">
          The token amount of overlap between chunks.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="tableChunkConfig" type="object" placeholder="Chunking configuration for data entity uploads that contain tables.">
      <Expandable title="properties">
        <ResponseField name="size" type="number">
          The token size of each chunk.
        </ResponseField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```bash Example Request theme={null}
  curl --location --request PUT 'https://api.getmetal.io/v1/indexes/test-index-id' \
  --header 'Content-Type: application/json' \
  --header 'x-metal-api-key: <api-key>' \
  --header 'x-metal-client-id: <client-id>' \
  --data-raw '{
      "status": "DEACTIVATING"
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "data": {
      "id": "1",
      "createdAt": "2023-08-23T22:13:31.539Z",
      "status": "DEACTIVATING",
      "name": "Ozzy Osbourne",
      "model": "text-embedding-ada-002",
      "dimensions": 1536,
      "filters": [
        {
          "field": "name",
          "type": "string"
        },
        {
          "field": "age",
          "type": "number"
        }
      ],
      "chunkConfig": {
        "size": 500,
        "overlap": 20
      },
      "tableChunkConfig": {
        "size": 4000
      },
      "counts": {
        "docs": 0,
      }
    }
  }
  ```
</ResponseExample>


# Introduction
Source: https://docs.getmetal.io/rest-api/introduction

Familiarize with resources, response codes, and authentication.

## Base URL

The Metal API is available at the following URL:

```
https://api.getmetal.io
```

## Authentication

Metal uses API keys to authenticate requests. You can view and manage your API keys in the [Metal Application](https://app.getmetal.io/settings/organization). There are two headers that you'll retrieve from each Key.

### API Key Headers

| Title     | Header              | Example Key     |
| :-------- | :------------------ | :-------------- |
| API Key   | `x-metal-api-key`   | `pk_1234567890` |
| Client ID | `x-metal-client-id` | `ci_1234567890` |

### Example Authenticated Request

```bash theme={null}
curl --location --request GET 'https://api.getmetal.io' \
--header 'Content-Type: application/json' \
--header 'x-metal-api-key: pk_1234567890' \
--header 'x-metal-client-id: ci_1234567890' \
```

## Response Codes

Metal uses conventional HTTP response codes to indicate the success or failure of an API request. In general, codes in the `2xx` range indicate success, codes in the `4xx` range indicate an error that failed given the information provided (e.g., a required parameter was omitted, a charge failed, etc.), and codes in the `5xx` range indicate an error with Metal's servers.

| Code | Description                                                             |
| :--- | :---------------------------------------------------------------------- |
| 200  | OK - Everything worked as expected.                                     |
| 400  | Bad Request - Often missing a required parameter.                       |
| 401  | Unauthorized - No valid API key provided.                               |
| 402  | Request Failed - Parameters were valid but request failed.              |
| 404  | Not Found - The requested item doesn't exist.                           |
| 422  | Usage Limit Exceeded - Hit a feature or usage limit based on your plan. |
| 429  | Rate Limit Exceeded - Too many requests hit the API too quickly.        |
| 5xx  | Server Errors - something went wrong on Metal's end.                    |

## Normal pagination

For certain endpoints, such as `/v1/indexes/:id/documents`, pagination can be achieved using the `page` and `limit` query parameters.

Both page and limit should be positive integers with a maximum value of 100. This constraint means that using this method, callers can retrieve a maximum of `10,000` (i.e., `100` \* `100`) records.

## Deep pagination

To fetch documents beyond the limitations of normal pagination, you should utilize the `lastObjectIdSeen` value returned. This ID enables the API to access data from a deeper point in the dataset. Below is an illustrative script to demonstrate this approach:

```js theme={null}
const fetchAllDocuments = () => {
  do {
    try {
      const response = await axios
        .get('https://api.getmetal.io/v1/indexes/<indexID>/documents', {
          headers: {
            'x-metal-api-key': '<your_key>',
            'x-metal-client-id': '<your_client_id>',
          },
          params: {
            lt: lastSeenObjectId, // this will be a querystring parameter `lt=${lastSeenObjectId}`
            limit: 100,
          },
        });

      documents = documents.concat(response.data.data);

      lastSeenObjectId = response.data.lastSeenObjectId;
    } catch (error) {
      console.error('Error fetching documents:', error);
      break;
    }
  } while (lastSeenObjectId);

  return documents;
}
```


# Chatbot Starter Kit
Source: https://docs.getmetal.io/tools/chatbot-starter-kit

Get started in minutes with our chatbot starter kit built with Next.js.

<Card title="Next.js Chatbot" icon="github" href="https://github.com/getmetal/chatbot">
  Visit the GitHub Repo
</Card>