Skip to main content

S3 Connector

The S3 connector enables AxonFlow agents to interact with Amazon S3 and S3-compatible object storage services (MinIO, Ceph, etc.).

Overview​

PropertyValue
Types3
EditionCommunity
Auth MethodsIAM credentials, Access Keys, Instance Profile
Capabilitiesquery, execute, presign, multipart, streaming

Use Cases​

  • Store and retrieve documents for RAG pipelines
  • Upload agent-generated files (reports, exports)
  • Access data lakes for analytics queries
  • Manage file attachments in workflows

Configuration​

Environment Variables​

# Required
MCP_s3_storage_REGION="us-east-1"
MCP_s3_storage_DEFAULT_BUCKET="my-bucket"

# Authentication (choose one method)
# Option 1: Access Keys
MCP_s3_storage_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
MCP_s3_storage_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

# Option 2: IAM Role (recommended for AWS deployments)
# No credentials needed - uses instance profile

# Optional
MCP_s3_storage_ENDPOINT="https://s3.us-east-1.amazonaws.com"  # Custom endpoint for S3-compatible services
MCP_s3_storage_FORCE_PATH_STYLE="true"  # Required for MinIO
MCP_s3_storage_TIMEOUT="30s"

Connector Config (Customer Portal)​

{
  "name": "s3-documents",
  "type": "s3",
  "options": {
    "region": "us-east-1",
    "default_bucket": "my-documents-bucket",
    "force_path_style": false
  },
  "credentials": {
    "access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
  }
}

Operations​

Query Operations​

List Objects​

curl -X POST https://your-axonflow.com/mcp/resources/query \
  -H "Content-Type: application/json" \
  -d '{
    "connector": "s3-documents",
    "statement": "list_objects",
    "parameters": {
      "bucket": "my-bucket",
      "prefix": "documents/",
      "max_keys": 100
    }
  }'

Response:

{
  "rows": [
    {
      "key": "documents/report.pdf",
      "size": 102400,
      "last_modified": "2025-12-07T10:30:00Z",
      "etag": "\"abc123...\""
    }
  ],
  "metadata": {
    "is_truncated": false,
    "key_count": 1
  }
}

Get Object​

curl -X POST https://your-axonflow.com/mcp/resources/query \
  -H "Content-Type: application/json" \
  -d '{
    "connector": "s3-documents",
    "statement": "get_object",
    "parameters": {
      "bucket": "my-bucket",
      "key": "documents/report.pdf"
    }
  }'

Head Object (Metadata Only)​

curl -X POST https://your-axonflow.com/mcp/resources/query \
  -d '{
    "connector": "s3-documents",
    "statement": "head_object",
    "parameters": {
      "key": "documents/report.pdf"
    }
  }'

Generate Presigned URL​

curl -X POST https://your-axonflow.com/mcp/resources/query \
  -d '{
    "connector": "s3-documents",
    "statement": "presign_get",
    "parameters": {
      "key": "documents/report.pdf",
      "expires_in": 3600
    }
  }'

Execute Operations​

Put Object​

curl -X POST https://your-axonflow.com/mcp/tools/execute \
  -d '{
    "connector": "s3-documents",
    "action": "put_object",
    "parameters": {
      "bucket": "my-bucket",
      "key": "uploads/new-file.txt",
      "body": "File content here",
      "content_type": "text/plain"
    }
  }'

Delete Object​

curl -X POST https://your-axonflow.com/mcp/tools/execute \
  -d '{
    "connector": "s3-documents",
    "action": "delete_object",
    "parameters": {
      "key": "uploads/old-file.txt"
    }
  }'

Delete Multiple Objects​

curl -X POST https://your-axonflow.com/mcp/tools/execute \
  -d '{
    "connector": "s3-documents",
    "action": "delete_objects",
    "parameters": {
      "keys": ["file1.txt", "file2.txt", "file3.txt"]
    }
  }'

Copy Object​

curl -X POST https://your-axonflow.com/mcp/tools/execute \
  -d '{
    "connector": "s3-documents",
    "action": "copy_object",
    "parameters": {
      "source_bucket": "source-bucket",
      "source_key": "original.pdf",
      "dest_bucket": "dest-bucket",
      "dest_key": "copy.pdf"
    }
  }'

Create Bucket​

curl -X POST https://your-axonflow.com/mcp/tools/execute \
  -d '{
    "connector": "s3-documents",
    "action": "create_bucket",
    "parameters": {
      "bucket": "new-bucket-name",
      "region": "us-east-1"
    }
  }'

S3-Compatible Services​

MinIO​

MCP_s3_minio_ENDPOINT="http://localhost:9000"
MCP_s3_minio_ACCESS_KEY_ID="minioadmin"
MCP_s3_minio_SECRET_ACCESS_KEY="minioadmin"
MCP_s3_minio_FORCE_PATH_STYLE="true"
MCP_s3_minio_REGION="us-east-1"  # Required even for MinIO

DigitalOcean Spaces​

MCP_s3_spaces_ENDPOINT="https://nyc3.digitaloceanspaces.com"
MCP_s3_spaces_REGION="nyc3"
MCP_s3_spaces_ACCESS_KEY_ID="your-spaces-key"
MCP_s3_spaces_SECRET_ACCESS_KEY="your-spaces-secret"

Cloudflare R2​

MCP_s3_r2_ENDPOINT="https://your-account-id.r2.cloudflarestorage.com"
MCP_s3_r2_ACCESS_KEY_ID="your-r2-access-key"
MCP_s3_r2_SECRET_ACCESS_KEY="your-r2-secret-key"
MCP_s3_r2_REGION="auto"

Best Practices​

Security​

  1. Use IAM roles instead of access keys in AWS deployments
  2. Scope bucket access - don't give s3:* permissions
  3. Enable bucket versioning for critical data
  4. Use presigned URLs for client-side uploads/downloads

Performance​

  1. Set appropriate timeouts for large file operations
  2. Use multipart upload for files > 100MB (automatic)
  3. Leverage prefix partitioning for high-throughput workloads

Example IAM Policy​

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

Troubleshooting​

Access Denied​

  • Verify IAM permissions include required actions
  • Check bucket policy allows access from your VPC/IP
  • Ensure region is correct

Slow Operations​

  • Check network connectivity to S3 endpoint
  • Use VPC endpoints for AWS deployments
  • Increase timeout for large files

Connection Refused (S3-Compatible)​

  • Verify endpoint URL is correct
  • Check force_path_style setting
  • Ensure service is running and accessible