Files

"Files" are used to upload documents that can be used with features like "Fine-tuning", and "Inference Batch API". You need to first create the file object using "/v1/files" and get the presigned upload url. Then upload the file through presigned url.

Functionality	Max File Size	File Extension	File Format Required
Fine-tuning	512 MB	.jsonl
Inference Batch API	100 MB	.jsonl

The total storage limit for all files uploaded by a single organization is capped at 100 GB.

For Fine-tuning, we only support data in the OpenAI fine-tuning format.

The file upload API returns a presigned upload URL, and the file needs to be uploaded using this presigned URL. The presigned URL is valid for 2 hours.

Step 1: Create a File Object

In this step, you create a file object, which initializes the upload process by providing essential metadata like the file name and type. The "purpose" parameter can be set to either "fine-tune" or "batch".

Please note that when passing the "file_name" parameter, you need to include the file extension. Otherwise, we will default to treating it as a JSONL file.

Example Request:

Replace {{API_TOKEN}} with your actual token.

Curl

curl --location 'https://api.netmind.ai/v1/files' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{API_TOKEN}}' \
--data '{"file_name":"text.jsonl","purpose":"fine-tune"}'

Python

import requests
import json

url = "https://api.netmind.ai/v1/files"

payload = json.dumps({
  "file_name": "text.jsonl",
  "purpose": "fine-tune"
})
headers = {
  'Content-Type': 'application/json',
  'Authorization': 'Bearer {{API_TOKEN}}'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Example Response:

{
    "id": "c67b7...298f3",
    "presigned_url": "https://FAKE_PRESIGNED_URL/..."
}

---

Step 2: Upload File Using Presigned URL

Example Request:

Replace {{DATA_FILE_PATH}} with your actual data file path.

Curl

curl --location --request PUT 'https://FAKE_PRESIGNED_URL/...' \
--data '@{{DATA_FILE_PATH}}'

Python

import requests
import os


presigned_url = "https://FAKE_PRESIGNED_URL/..."
file_name = "{{DATA_FILE_PATH}}"
file_size = os.path.getsize(file_name)


with open(file_name, "rb") as f:
    response = requests.put(presigned_url, data=f)
    print(response.status_code)
    print(response.text)

Example Response:

{}