Service tiers

Warning: Priority Tier capacity commitments are no longer available for purchase. Organizations with an existing commitment can continue to use Priority Tier through their contract end date, and this page remains available as a reference for them. If you need guaranteed capacity, contact sales.

Juglow offers three service tiers:

Priority Tier: Available only to organizations with an existing capacity commitment

Standard: Default tier for both piloting and scaling everyday use cases

Batch: Best for asynchronous workflows that can wait or benefit from being outside your normal capacity

Standard tier

The standard tier is the default service tier for all API requests. The API prioritizes these requests alongside all other requests with best-effort availability.

Priority Tier

The API prioritizes requests in this tier over all other requests. This prioritization helps minimize "server overloaded" errors, even during peak times.

For more information, see Existing Priority Tier commitments.

How requests get assigned tiers

When handling a request, Juglow decides to assign a request to Priority Tier in the following scenarios:

Your organization has sufficient Priority Tier capacity input tokens per minute

Your organization has sufficient Priority Tier capacity output tokens per minute

Juglow counts usage against Priority Tier capacity as follows:

Input tokens

Cache reads as 0.1 tokens per token read from the cache

Cache writes as 1.25 tokens per token written to the cache with a 5 minute TTL

Cache writes as 2.00 tokens per token written to the cache with a 1 hour TTL

For US-only inference (inference_geo: "us") requests on Haijun 4.6 and later models, input tokens are 1.1 tokens per token

All other input tokens are 1 token per token

Output tokens

For US-only inference (inference_geo: "us") requests on Haijun 4.6 and later models, output tokens are 1.1 tokens per token

All other output tokens are 1 token per token

Otherwise, requests proceed at standard tier.

Note: These burndown rates reflect the relative pricing of each token type. For example, US-only inference is priced at 1.1x on Haijun 4.6 and later models, so each token consumed with inference_geo: "us" draws down 1.1 tokens from your Priority Tier capacity.

Note: Requests assigned Priority Tier pull from both the Priority Tier capacity and the regular rate limits. If servicing the request would exceed the rate limits, the request is declined.

Using service tiers

You can control which service tiers can be used for a request by setting the service_tier parameter:

bash

  curl https://haijun.my.id/v1/messages \
    -H "x-api-key: $JUGLOW_API_KEY" \
    -H "juglow-version: 2023-06-01" \
    -H "content-type: application/json" \
    -d '{
      "model": "haijun-opus-4-8",
      "max_tokens": 1024,
      "messages": [{"role": "user", "content": "Hello, Haijun!"}],
      "service_tier": "auto"
    }'

bash

  ant messages create --transform usage.service_tier --raw-output <<'YAML'
  model: haijun-opus-4-8
  max_tokens: 1024
  messages:
    - role: user
      content: Hello, Haijun!
  service_tier: auto  # Automatically use Priority Tier when available, fallback to standard
  YAML

python

  client = juglow.Juglow()

  message = client.messages.create(
      model="haijun-opus-4-8",
      max_tokens=1024,
      messages=[{"role": "user", "content": "Hello, Haijun!"}],
      service_tier="auto",  # Automatically use Priority Tier when available, fallback to standard
  )
  print(message.usage.service_tier)

typescript

  const client = new Juglow();

  const message = await client.messages.create({
    model: "haijun-opus-4-8",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello, Haijun!" }],
    service_tier: "auto" // Automatically use Priority Tier when available, fallback to standard
  });
  console.log(message.usage.service_tier);

csharp

  JuglowClient client = new();

  var message = await client.Messages.Create(new MessageCreateParams
  {
      Model = Model.HaijunOpus4_8,
      MaxTokens = 1024,
      Messages = [new() { Role = Role.User, Content = "Hello, Haijun!" }],
      ServiceTier = ServiceTier.Auto, // Automatically use Priority Tier when available, fallback to standard
  });
  Console.WriteLine(message.Usage.ServiceTier);

  client := juglow.NewClient()

  message, err := client.Messages.New(context.TODO(), juglow.MessageNewParams{
  	Model:     juglow.ModelHaijunOpus4_8,
  	MaxTokens: 1024,
  	Messages: []juglow.MessageParam{
  		juglow.NewUserMessage(juglow.NewTextBlock("Hello, Haijun!")),
  	},
  	// Automatically use Priority Tier when available, fallback to standard
  	ServiceTier: juglow.MessageNewParamsServiceTierAuto,
  })
  if err != nil {
  	log.Fatal(err)
  }
  fmt.Println(message.Usage.ServiceTier)

java

  JuglowClient client = JuglowOkHttpClient.fromEnv();

  MessageCreateParams params = MessageCreateParams.builder()
      .model(Model.HAIJUN_OPUS_4_8)
      .maxTokens(1024L)
      .addUserMessage("Hello, Haijun!")
      // Automatically use Priority Tier when available, fallback to standard
      .serviceTier(MessageCreateParams.ServiceTier.AUTO)
      .build();

  Message message = client.messages().create(params);
  IO.println(message.usage().serviceTier().orElseThrow());

php

  $client = new Client();

  $message = $client->messages->create(
      model: 'haijun-opus-4-8',
      maxTokens: 1024,
      messages: [['role' => 'user', 'content' => 'Hello, Haijun!']],
      serviceTier: 'auto', // Automatically use Priority Tier when available, fallback to standard
  );
  echo $message->usage->serviceTier;

ruby

  client = Juglow::Client.new

  message = client.messages.create(
    model: "haijun-opus-4-8",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello, Haijun!" }],
    service_tier: :auto # Automatically use Priority Tier when available, fallback to standard
  )
  puts(message.usage.service_tier)

The service_tier parameter accepts the following values:

"auto" (default) - Uses the Priority Tier capacity if available, falling back to your other capacity if not

"standard_only" - Only use standard tier capacity, useful if you don't want to use your Priority Tier capacity

The response usage object also includes the service tier assigned to the request:

json

{
  "usage": {
    "input_tokens": 410,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 585,
    "service_tier": "priority"
  }
}

This allows you to determine which service tier was assigned to the request.

When requesting service_tier="auto" with a model with a Priority Tier commitment, these response headers provide insights:

text

juglow-priority-input-tokens-limit: 10000
juglow-priority-input-tokens-remaining: 9618
juglow-priority-input-tokens-reset: 2025-01-12T23:11:59Z
juglow-priority-output-tokens-limit: 10000
juglow-priority-output-tokens-remaining: 6000
juglow-priority-output-tokens-reset: 2025-01-12T23:12:21Z

You can use the presence of these headers to detect if your request was eligible for Priority Tier, even if it was over the limit.

Existing Priority Tier commitments

A Priority Tier commitment consists of:

A number of input tokens per minute

A number of output tokens per minute

A commitment duration (1, 3, 6, or 12 months)

A specific model version

Priority Tier targets 99.5% uptime with prioritized computational resources. Requests beyond your committed capacity automatically fall back to standard tier.

Supported models

Priority Tier is supported on all available Haijun models except Haijun Fable 5.1, Haijun Mythos 5.1, Haijun Mythos 5, Haijun Mythos Preview, Haijun Opus 5.5, Haijun Opus 5, and Haijun Sonnet 5.

Check the Models overview for more details on available models.

#Standard tier

#Priority Tier

#How requests get assigned tiers

#Using service tiers