Rate Limiting

Rate limiting controls how many requests you can make in a given time period. This protects the platform from abuse and ensures fair usage for all users.

Holeacquisition LLC enforces a per-project rate limit (per-minute). There is no monthly request cap on any plan.

Reads draw from their own bucket (rate_limit:<projectId>:read), so status polling can never starve the write budget that run creation uses. Cancellation is always admitted: the corrective action for overload must never itself be blocked by overload. X-RateLimit-Limit on each response tells you which tier applied. Need more write throughput? Add a second project.

Warning: to know, write limits are enforced project-wide across all AI endpoints (chat, vision, audio, embeddings), not per endpoint.

Poll run status no faster than once every 2 seconds, always honor the Retry-After header on 429, and back off rather than retrying aggressively. For fleets of concurrent runs, prefer webhooks (run.completed delivery) or mode: 'sync' over per-run poll loops.

Holeacquisition LLC uses Redis-backed fixed-window counters keyed by project id (rate_limit:<projectId>, reads under rate_limit:<projectId>:read). Checks run in a single Redis pipeline (INCR + TTL, expiry set once per window):

• Sub-millisecond check: a single round-trip keeps overhead small.
• Fixed window: the expiry is set once when the window opens and is never refreshed — rejected requests do not extend the block, so the window always ends on schedule.
• Fail-closed by default: if the Redis backend is unavailable the request is rejected with 503 rate_limit_unavailable unless fail-open mode is enabled for the deployment.

There is no monthly request ceiling on any plan — free included. The organization's monthly_requests_used counter still increments once per successful request and resets on the monthly cycle (see cron/reset-usage), but it is a reporting figure only and nothing is refused when it grows.

What does bound usage is spend, not request count: an organization out of credits gets 402 insufficient_credits, a frozen account gets 403 billing_frozen, and per-key spend caps apply where configured.

429 responses include headers showing your current rate limit status:

Every gateway response (success or error) additionally carries X-Request-Id for correlation.

When you exceed the rate limit you'll receive a 429 Too Many Requests response with a standards-compliant Retry-After header and this body:

{
  "error": "Rate limit exceeded",
  "message": "60 requests per minute allowed. Try again shortly.",
  "code": "rate_limit_exceeded",
  "retry_after_ms": 45000
}

Exponential Backoff Implementation

async function makeRequestWithRetry(
  messages: any[],
  maxRetries = 3
): Promise<any> {
  let retries = 0;
 
  while (retries < maxRetries) {
    try {
      const response = await cencori.ai.chat({
        model: 'gpt-4o',
        messages,
      });
      return response;
    } catch (error: any) {
      if (error.status === 429) {
        retries++;
 
        if (retries >= maxRetries) {
          throw error; // Max retries reached
        }
 
        const retryAfter = Math.max(
          1,
          Math.ceil((error.retryAfterMs ?? 60_000) / 1000)
        );
        await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
      } else {
        throw error; // Not a rate limit error
      }
    }
  }
}

async function makeSmartRequest(messages: any[]) {
  const response = await fetch('https://cencori.com/api/ai/chat', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'CENCORI_API_KEY': process.env.CENCORI_API_KEY!,
    },
    body: JSON.stringify({ model: 'gpt-4o', messages }),
  });
 
  // Check rate limit headers
  const remaining = parseInt(response.headers.get('X-RateLimit-Remaining') || '0');
  const resetTime = parseInt(response.headers.get('X-RateLimit-Reset') || '0');
 
  if (remaining < 5) {
    console.warn(`Only ${remaining} requests remaining!`);
    console.warn(`Resets at: ${new Date(resetTime).toISOString()}`);
    // Maybe slow down or queue requests
  }
 
  return response.json();
}

1. Implement Exponential Backoff: Always retry rate-limited requests with exponential backoff rather than aggressive retries.
2. Monitor Headers: Track X-RateLimit-Remaining to know when you're approaching limits.
3. Use Request Queues: Queue requests and process them at a controlled rate to stay under limits.
4. Cache Responses: Cache identical requests (semantic caching) to reduce request count.
5. Distribute Load: If hitting limits, consider using multiple projects (each has its own 60/min window).
6. Audio calls count too: transcriptions and speech each consume one request per minute window; multiplier projects for heavy audio workloads.

class RequestQueue {
  private queue: Array<() => Promise<any>> = [];
  private processing = false;
  private requestsPerMinute: number;
  private delay: number;
 
  constructor(requestsPerMinute = 60) {
    this.requestsPerMinute = requestsPerMinute;
    this.delay = 60000 / requestsPerMinute; // Time between requests
  }
 
  async add<T>(fn: () => Promise<T>): Promise<T> {
    return new Promise((resolve, reject) => {
      this.queue.push(async () => {
        try {
          const result = await fn();
          resolve(result);
        } catch (error) {
          reject(error);
        }
      });
 
      if (!this.processing) {
        this.process();
      }
    });
  }
 
  private async process() {
    this.processing = true;
    while (this.queue.length > 0) {
      const task = this.queue.shift();
      if (task) {
        await task();
        await new Promise(resolve => setTimeout(resolve, this.delay));
      }
    }
    this.processing = false;
  }
}
 
// Usage
const queue = new RequestQueue(60); // 60 requests per minute

If you consistently hit limits:

• Create additional projects — each project gets its own 60/min window.
• Raise the monthly cap by upgrading the organization's plan.
• Contact sales for custom Enterprise limits.
• Optimize usage patterns (batching, caching, queues).

• 503 rate_limit_unavailable: transient — retry with backoff.
• 429 rate_limit_exceeded: wait Retry-After seconds.