Batch processing is a powerful approach for handling large volumes of requests efficiently. Instead of processing requests one at a time with immediate responses, batch processing allows you to submit multiple requests together for asynchronous processing. This pattern is particularly useful when:

You need to process large volumes of data

Immediate responses are not required

You want to optimize for cost efficiency

You're running large-scale evaluations or analyses

The Message Batches API is Juglow's first implementation of this pattern.

Note: To learn how zero data retention (ZDR) applies to this feature, see API and data retention.

Message Batches API

The Message Batches API is a powerful, cost-effective way to asynchronously process large volumes of Messages requests. This approach is well-suited to tasks that do not require immediate responses, with most batches finishing in less than 1 hour while reducing costs by 50% and increasing throughput.

You can explore the API reference directly, in addition to this guide.

How the Message Batches API works

When you send a request to the Message Batches API:

The system creates a new Message Batch with the provided Messages requests.

The batch is then processed asynchronously, with each request handled independently.

You can poll for the status of the batch and retrieve results when processing has ended for all requests.

This is especially useful for bulk operations that don't require immediate results, such as:

Large-scale evaluations: Process thousands of test cases efficiently.

Content moderation: Analyze large volumes of user-generated content asynchronously.

Data analysis: Generate insights or summaries for large datasets.

Bulk content generation: Create large amounts of text for various purposes (for example, product descriptions, article summaries).

Batch limitations

A Message Batch is limited to either 100,000 Message requests or 256 MB in size, whichever is reached first.

The system processes each batch as fast as possible, with most batches completing within 1 hour. You can access batch results when all messages have completed or after 24 hours, whichever comes first. Batches expire if processing does not complete within 24 hours.

Batch results are available for 29 days after creation. After that, you may still view the Batch, but its results will no longer be available for download.

Batches are scoped to a Workspace. You may view all batches (and their results) that were created within the Workspace your request runs in.

Rate limits apply to both Batches API HTTP requests and the number of requests within a batch waiting to be processed. See Message Batches API rate limits. Additionally, processing may be slowed down based on current demand and your request volume. In that case, you may see more requests expiring after 24 hours.

Because of high throughput and concurrent processing, batches may go slightly over your Workspace's configured spend limit.

Each batched request must have max_tokens of at least 1. max_tokens: 0 (cache pre-warming) is not supported inside a batch, because an ephemeral cache entry written during batch processing would likely expire before the follow-up request runs.

Supported models

All active models support the Message Batches API.

What can be batched

Almost any request you can make to the Messages API can be included in a batch. This includes:

Vision

Tool use, including all server tools (web search, web fetch, code execution, MCP connectors, advisor, and tool search)

System messages

Multi-turn conversations

Extended thinking

Most beta features

Because each request in the batch is processed independently, you can mix different types of requests within a single batch.

A small number of Messages API parameters are not supported in batch requests. Including any of these returns a validation error:

Parameter	Why
`stream: true`	Batch results come back as a single file, not a stream.
`speed` (Fast mode)	Fast mode tunes synchronous latency, which doesn't apply to asynchronous batch processing.
`max_tokens: 0`	See Batch limitations.

Tip: Because batches can take longer than 5 minutes to process, consider using the 1-hour cache duration with prompt caching for better cache hit rates when processing batches with shared context.

Pricing

The Batches API offers significant cost savings. All usage is charged at 50% of the standard API prices.

Model	Batch input	Batch output
Haijun Fable 5.1	$5 / MTok	$25 / MTok
Haijun Mythos 5.1 (limited availability)	$5 / MTok	$25 / MTok
Haijun Fable 5	$5 / MTok	$25 / MTok
Haijun Mythos 5 (limited availability)	$5 / MTok	$25 / MTok
Haijun Opus 5.5	$2 / MTok	$10 / MTok
Haijun Opus 5	$2.50 / MTok	$12.50 / MTok
Haijun Opus 4.8	$2.50 / MTok	$12.50 / MTok
Haijun Opus 4.7	$2.50 / MTok	$12.50 / MTok
Haijun Opus 4.6	$2.50 / MTok	$12.50 / MTok
Haijun Opus 4.5	$2.50 / MTok	$12.50 / MTok
Haijun Opus 4.1 (retired, except on Bedrock and Google Cloud)	$7.50 / MTok	$37.50 / MTok
Haijun Opus 4 (retired, except on Google Cloud)	$7.50 / MTok	$37.50 / MTok
Haijun Sonnet 5	$1 / MTok	$5 / MTok
Haijun Sonnet 4.6	$1.50 / MTok	$7.50 / MTok
Haijun Sonnet 4.5	$1.50 / MTok	$7.50 / MTok
Haijun Sonnet 4 (retired, except on Bedrock and Google Cloud)	$1.50 / MTok	$7.50 / MTok
Haijun Haiku 4.5	$0.50 / MTok	$2.50 / MTok
Haijun Haiku 3.5 (retired, except on Bedrock and Google Cloud)	$0.40 / MTok	$2 / MTok

MTok: Million tokens. $5 / MTok is $5 for every million tokens.

Limited access: Offered separately, by invitation only, as part of Project Glasswing. For access, contact your Juglow, AWS, or Google Cloud account team.

Retired: May still be available on other cloud platforms. See Model deprecations for more.

How to use the Message Batches API

Prepare and create your batch

A Message Batch is composed of a list of requests to create a Message. The shape of an individual request comprises:

A unique custom_id for identifying the Messages request. Must be 1 to 64 characters and contain only alphanumeric characters, hyphens, and underscores (matching ^[a-zA-Z0-9_-]{1,64}$).

A params object with the standard Messages API parameters

You can create a batch by passing this list into the requests parameter:

bash

  curl https://haijun.my.id/v1/messages/batches \
       --header "x-api-key: $JUGLOW_API_KEY" \
       --header "juglow-version: 2023-06-01" \
       --header "content-type: application/json" \
       --data \
  '{
      "requests": [
          {
              "custom_id": "my-first-request",
              "params": {
                  "model": "haijun-opus-5-5",
                  "max_tokens": 1024,
                  "messages": [
                      {"role": "user", "content": "Hello, world"}
                  ]
              }
          },
          {
              "custom_id": "my-second-request",
              "params": {
                  "model": "haijun-opus-5-5",
                  "max_tokens": 1024,
                  "messages": [
                      {"role": "user", "content": "Hi again, friend"}
                  ]
              }
          }
      ]
  }'

bash

  ant messages:batches create <<'YAML'
  requests:
    - custom_id: my-first-request
      params:
        model: haijun-opus-5-5
        max_tokens: 1024
        messages:
          - role: user
            content: Hello, world
    - custom_id: my-second-request
      params:
        model: haijun-opus-5-5
        max_tokens: 1024
        messages:
          - role: user
            content: Hi again, friend
  YAML

python

  from juglow.types.message_create_params import MessageCreateParamsNonStreaming
  from juglow.types.messages.batch_create_params import Request

  client = juglow.Juglow()

  message_batch = client.messages.batches.create(
      requests=[
          Request(
              custom_id="my-first-request",
              params=MessageCreateParamsNonStreaming(
                  model="haijun-opus-5-5",
                  max_tokens=1024,
                  messages=[
                      {
                          "role": "user",
                          "content": "Hello, world",
                      }
                  ],
              ),
          ),
          Request(
              custom_id="my-second-request",
              params=MessageCreateParamsNonStreaming(
                  model="haijun-opus-5-5",
                  max_tokens=1024,
                  messages=[
                      {
                          "role": "user",
                          "content": "Hi again, friend",
                      }
                  ],
              ),
          ),
      ]
  )

  print(message_batch)

typescript

  const client = new Juglow();

  const messageBatch = await client.messages.batches.create({
    requests: [
      {
        custom_id: "my-first-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          messages: [{ role: "user", content: "Hello, world" }]
        }
      },
      {
        custom_id: "my-second-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          messages: [{ role: "user", content: "Hi again, friend" }]
        }
      }
    ]
  });

  console.log(messageBatch);

csharp

  using Juglow;
  using Juglow.Models.Messages;
  using Juglow.Models.Messages.Batches;

  JuglowClient client = new();

  var batch = await client.Messages.Batches.Create(new BatchCreateParams
  {
      Requests =
      [
          new()
          {
              CustomID = "my-first-request",
              Params = new()
              {
                  Model = Model.HaijunOpus5_5,
                  MaxTokens = 1024,
                  Messages =
                  [
                      new() { Role = Role.User, Content = "Hello, world" }
                  ]
              }
          },
          new()
          {
              CustomID = "my-second-request",
              Params = new()
              {
                  Model = Model.HaijunOpus5_5,
                  MaxTokens = 1024,
                  Messages =
                  [
                      new() { Role = Role.User, Content = "Hi again, friend" }
                  ]
              }
          }
      ]
  });

  Console.WriteLine(batch);

  client := juglow.NewClient()

  batch, _ := client.Messages.Batches.New(context.Background(),
  	juglow.MessageBatchNewParams{
  		Requests: []juglow.MessageBatchNewParamsRequest{
  			{
  				CustomID: "my-first-request",
  				Params: juglow.MessageBatchNewParamsRequestParams{
  					Model:     juglow.ModelHaijunOpus5_5,
  					MaxTokens: 1024,
  					Messages: []juglow.MessageParam{
  						juglow.NewUserMessage(
  							juglow.NewTextBlock("Hello, world"),
  						),
  					},
  				},
  			},
  			{
  				CustomID: "my-second-request",
  				Params: juglow.MessageBatchNewParamsRequestParams{
  					Model:     juglow.ModelHaijunOpus5_5,
  					MaxTokens: 1024,
  					Messages: []juglow.MessageParam{
  						juglow.NewUserMessage(
  							juglow.NewTextBlock("Hi again, friend"),
  						),
  					},
  				},
  			},
  		},
  	})

  fmt.Println(batch.ID)

java

  JuglowClient client = JuglowOkHttpClient.fromEnv();

  BatchCreateParams params = BatchCreateParams.builder()
    .addRequest(
      BatchCreateParams.Request.builder()
        .customId("my-first-request")
        .params(
          BatchCreateParams.Request.Params.builder()
            .model(Model.HAIJUN_OPUS_5_5)
            .maxTokens(1024)
            .addUserMessage("Hello, world")
            .build()
        )
        .build()
    )
    .addRequest(
      BatchCreateParams.Request.builder()
        .customId("my-second-request")
        .params(
          BatchCreateParams.Request.Params.builder()
            .model(Model.HAIJUN_OPUS_5_5)
            .maxTokens(1024)
            .addUserMessage("Hi again, friend")
            .build()
        )
        .build()
    )
    .build();

  MessageBatch messageBatch = client.messages().batches().create(params);

  System.out.println(messageBatch);

php

  $client = new Client();

  $batch = $client->messages->batches->create(
      requests: [
          [
              'custom_id' => 'my-first-request',
              'params' => [
                  'model' => 'haijun-opus-5-5',
                  'max_tokens' => 1024,
                  'messages' => [
                      ['role' => 'user', 'content' => 'Hello, world']
                  ]
              ]
          ],
          [
              'custom_id' => 'my-second-request',
              'params' => [
                  'model' => 'haijun-opus-5-5',
                  'max_tokens' => 1024,
                  'messages' => [
                      ['role' => 'user', 'content' => 'Hi again, friend']
                  ]
              ]
          ]
      ],
  );

  echo $batch->id;

ruby

  client = Juglow::Client.new

  batch = client.messages.batches.create(
    requests: [
      {
        custom_id: "my-first-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          messages: [
            { role: "user", content: "Hello, world" }
          ]
        }
      },
      {
        custom_id: "my-second-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          messages: [
            { role: "user", content: "Hi again, friend" }
          ]
        }
      }
    ]
  )

  puts batch

In this example, two separate requests are batched together for asynchronous processing. Each request has a unique custom_id and contains the standard parameters you'd use for a Messages API call.

Tip: Test your batch requests with the Messages API Validation of the params object for each message request is performed asynchronously, and validation errors are returned when processing of the entire batch has ended. You can ensure that you are building your input correctly by verifying your request shape with the Messages API first.

When a batch is first created, the response has a processing status of in_progress.

json

{
  "id": "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d",
  "type": "message_batch",
  "processing_status": "in_progress",
  "request_counts": {
    "processing": 2,
    "succeeded": 0,
    "errored": 0,
    "canceled": 0,
    "expired": 0
  },
  "ended_at": null,
  "created_at": "2024-09-24T18:37:24.100435Z",
  "expires_at": "2024-09-25T18:37:24.100435Z",
  "cancel_initiated_at": null,
  "results_url": null
}

Tracking your batch

The Message Batch's processing_status field indicates the stage of processing the batch is in. It starts as in_progress, then updates to ended once all the requests in the batch have finished processing, and results are ready. You can monitor the state of your batch by visiting the Console, or using the retrieval endpoint.

Polling for Message Batch completion

To poll a Message Batch, you'll need its id, which is provided in the response when creating a batch or by listing batches. You can implement a polling loop that checks the batch status periodically until processing has ended:

bash

  #!/bin/sh
  # ...
  # Check the status; repeat until processing_status is "ended"
  curl -s "https://haijun.my.id/v1/messages/batches/$MESSAGE_BATCH_ID" \
    --header "x-api-key: $JUGLOW_API_KEY" \
    --header "juglow-version: 2023-06-01" \
    | jq -r '.processing_status'

bash

  #!/bin/bash
  # ...
  # Check the status; repeat until processing_status is "ended"
  ant messages:batches retrieve \
    --message-batch-id "$MESSAGE_BATCH_ID" \
    --transform processing_status --raw-output

python

  import time

  client = juglow.Juglow()

  MESSAGE_BATCH_ID = "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d"

  message_batch = None
  while True:
      message_batch = client.messages.batches.retrieve(MESSAGE_BATCH_ID)
      if message_batch.processing_status == "ended":
          break

      print(f"Batch {MESSAGE_BATCH_ID} is still processing...")
      time.sleep(60)
  print(message_batch)

typescript

  const client = new Juglow();

  const messageBatchId = "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d";

  let messageBatch;
  while (true) {
    messageBatch = await client.messages.batches.retrieve(messageBatchId);
    if (messageBatch.processing_status === "ended") {
      break;
    }

    console.log(`Batch ${messageBatchId} is still processing... waiting`);
    await new Promise((resolve) => setTimeout(resolve, 60_000));
  }
  console.log(messageBatch);

csharp

  JuglowClient client = new();
  string messageBatchId = Environment.GetEnvironmentVariable("MESSAGE_BATCH_ID");

  MessageBatch messageBatch = null;
  while (true)
  {
      messageBatch = await client.Messages.Batches.Retrieve(messageBatchId);
      if (messageBatch.ProcessingStatus == "ended")
      {
          break;
      }

      Console.WriteLine($"Batch {messageBatchId} is still processing...");
      await Task.Delay(60000);
  }
  Console.WriteLine(messageBatch);

  client := juglow.NewClient()
  messageBatchID := os.Getenv("MESSAGE_BATCH_ID")

  var messageBatch *juglow.MessageBatch
  for {
  	var err error
  	messageBatch, err = client.Messages.Batches.Get(context.TODO(), messageBatchID, juglow.MessageBatchGetParams{})
  	if err != nil {
  		log.Fatal(err)
  	}
  	if messageBatch.ProcessingStatus == "ended" {
  		break
  	}

  	fmt.Printf("Batch %s is still processing...\n", messageBatchID)
  	time.Sleep(60 * time.Second)
  }
  fmt.Println(messageBatch)

java

  import com.juglow.models.messages.batches.MessageBatch;
  // ...
          JuglowClient client = JuglowOkHttpClient.fromEnv();
          String messageBatchId = "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d";

          MessageBatch messageBatch = null;
          while (true) {
              messageBatch = client.messages().batches().retrieve(messageBatchId);
              if (messageBatch.processingStatus().equals(MessageBatch.ProcessingStatus.ENDED)) {
                  break;
              }

              System.out.println("Batch " + messageBatchId + " is still processing...");
              Thread.sleep(60000);
          }
          System.out.println(messageBatch);

php

  $client = new Client();
  $messageBatchId = getenv("MESSAGE_BATCH_ID");

  $messageBatch = null;
  while (true) {
      $messageBatch = $client->messages->batches->retrieve(
          messageBatchID: $messageBatchId,
      );
      if ($messageBatch->processingStatus === "ended") {
          break;
      }

      echo "Batch {$messageBatchId} is still processing...\n";
      sleep(60);
  }
  echo json_encode($messageBatch, JSON_PRETTY_PRINT);

ruby

  client = Juglow::Client.new

  message_batch_id = ENV["MESSAGE_BATCH_ID"]
  message_batch = nil
  loop do
    message_batch = client.messages.batches.retrieve(message_batch_id)
    break if message_batch.processing_status == :ended

    puts "Batch #{message_batch_id} is still processing..."
    sleep 60
  end
  puts message_batch

Listing all Message Batches

You can list all Message Batches in your Workspace using the list endpoint. The API supports pagination, automatically fetching additional pages as needed:

bash

  #!/bin/sh
  # Fetches one page. While the response's has_more is true, pass its
  # last_id as after_id to fetch the next page. (The SDKs and the CLI
  # perform automatic pagination.)
  curl -s "https://haijun.my.id/v1/messages/batches?limit=20" \
    --header "x-api-key: $JUGLOW_API_KEY" \
    --header "juglow-version: 2023-06-01"

bash

  # Automatically fetches more pages as needed
  ant messages:batches list --limit 20

python

  client = juglow.Juglow()

  # Automatically fetches more pages as needed.
  for message_batch in client.messages.batches.list(limit=20):
      print(message_batch)

typescript

  const client = new Juglow();

  // Automatically fetches more pages as needed.
  for await (const messageBatch of client.messages.batches.list({
    limit: 20
  })) {
    console.log(messageBatch);
  }

csharp

  JuglowClient client = new();

  var parameters = new BatchListParams
  {
      Limit = 20
  };

  // Automatically fetches more pages as needed
  var page = await client.Messages.Batches.List(parameters);
  await foreach (var messageBatch in page.Paginate())
  {
      Console.WriteLine(messageBatch);
  }

  client := juglow.NewClient()

  // Automatically fetches more pages as needed
  iter := client.Messages.Batches.ListAutoPaging(context.TODO(), juglow.MessageBatchListParams{
  	Limit: juglow.Int(20),
  })

  for iter.Next() {
  	messageBatch := iter.Current()
  	fmt.Println(messageBatch)
  }

  if err := iter.Err(); err != nil {
  	log.Fatal(err)
  }

java

  JuglowClient client = JuglowOkHttpClient.fromEnv();

  // Automatically fetches more pages as needed
  for (MessageBatch messageBatch : client
    .messages()
    .batches()
    .list(BatchListParams.builder().limit(20).build())
    .autoPager()) {
    System.out.println(messageBatch);
  }

php

  $client = new Client();

  // Automatically fetches more pages as needed
  foreach ($client->messages->batches->list(limit: 20)->pagingEachItem() as $messageBatch) {
      echo $messageBatch->id . "\n";
  }

ruby

  client = Juglow::Client.new

  # Automatically fetches more pages as needed
  client.messages.batches.list(limit: 20).auto_paging_each do |message_batch|
    puts message_batch
  end

Retrieving batch results

Once batch processing has ended, each Messages request in the batch has a result. There are four result types:

Result type	Description
`succeeded`	Request was successful. Includes the message result.
`errored`	Request encountered an error and a message was not created. Possible errors include invalid requests and internal server errors. You will not be billed for these requests.
`canceled`	User canceled the batch before this request could be sent to the model. You will not be billed for these requests.
`expired`	Batch reached its 24-hour expiration before this request could be sent to the model. You will not be billed for these requests.

The batch's request_counts shows an overview of your results, indicating how many requests reached each of these four states.

Results of the batch are available for download at the results_url property on the Message Batch, and if the organization permission allows, in the Console. Because of the potentially large size of the results, it's recommended to stream results back rather than download them all at once.

bash

  #!/bin/sh
  # Fetch the batch's results_url, then stream the .jsonl results it
  # points to. For per-result handling (retries, validation errors),
  # use the SDK examples in the other tabs.
  RESULTS_URL=$(curl -s "https://haijun.my.id/v1/messages/batches/msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d" \
    --header "juglow-version: 2023-06-01" \
    --header "x-api-key: $JUGLOW_API_KEY" \
    | jq -r '.results_url')

  curl -s "$RESULTS_URL" \
    --header "juglow-version: 2023-06-01" \
    --header "x-api-key: $JUGLOW_API_KEY" \
    | jq -r '"\(.result.type): \(.custom_id)"'

bash

  # Prints one line per result, e.g. `{"custom_id":"test-1","type":"succeeded",…}`.
  # For per-result handling (retries, validation errors), use the SDK
  # examples in the other tabs.
  ant messages:batches results \
    --message-batch-id msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d \
    --transform '{custom_id,"type":result.type,"error":result.error.error.type}' \
    --format jsonl

python

  client = juglow.Juglow()

  # Stream results file in memory-efficient chunks, processing one at a time
  for result in client.messages.batches.results(
      "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d",
  ):
      outcome = result.result
      match outcome.type:
          case "succeeded":
              print(f"Success! {result.custom_id}")
          case "errored":
              if outcome.error.error.type == "invalid_request_error":
                  # Request body must be fixed before re-sending request
                  print(f"Validation error {result.custom_id}")
              else:
                  # Request can be retried directly
                  print(f"Server error {result.custom_id}")
          case "expired":
              print(f"Request expired {result.custom_id}")

typescript

  const client = new Juglow();

  // Stream results file in memory-efficient chunks, processing one at a time
  for await (const result of await client.messages.batches.results(
    "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d"
  )) {
    switch (result.result.type) {
      case "succeeded":
        console.log(`Success! ${result.custom_id}`);
        break;
      case "errored":
        if (result.result.error.type === "invalid_request_error") {
          // Request body must be fixed before re-sending request
          console.log(`Validation error: ${result.custom_id}`);
        } else {
          // Request can be retried directly
          console.log(`Server error: ${result.custom_id}`);
        }
        break;
      case "expired":
        console.log(`Request expired: ${result.custom_id}`);
        break;
    }
  }

csharp

  JuglowClient client = new();

  await foreach (var result in client.Messages.Batches.ResultsStreaming("msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d"))
  {
      switch (result.Result.Type)
      {
          case "succeeded":
              Console.WriteLine($"Success! {result.CustomID}");
              break;
          case "errored":
              if (result.Result.Error?.Type == "invalid_request")
              {
                  Console.WriteLine($"Validation error: {result.CustomID}");
              }
              else
              {
                  Console.WriteLine($"Server error: {result.CustomID}");
              }
              break;
          case "expired":
              Console.WriteLine($"Request expired: {result.CustomID}");
              break;
      }
  }

  client := juglow.NewClient()

  stream := client.Messages.Batches.ResultsStreaming(context.TODO(), "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d", juglow.MessageBatchResultsParams{})

  for stream.Next() {
  	result := stream.Current()

  	switch variant := result.Result.AsAny().(type) {
  	case juglow.MessageBatchSucceededResult:
  		fmt.Printf("Success! %s\n", result.CustomID)
  	case juglow.MessageBatchErroredResult:
  		fmt.Printf("Error: %s - %s\n", result.CustomID, variant.Error.Error.Message)
  	case juglow.MessageBatchExpiredResult:
  		fmt.Printf("Request expired: %s\n", result.CustomID)
  	}
  }

  if err := stream.Err(); err != nil {
  	log.Fatal(err)
  }

java

  import com.juglow.core.http.StreamResponse;
  import com.juglow.models.messages.batches.BatchResultsParams;
  import com.juglow.models.messages.batches.MessageBatchIndividualResponse;
  // ...
      JuglowClient client = JuglowOkHttpClient.fromEnv();

      // Stream results file in memory-efficient chunks, processing one at a time
      try (
        StreamResponse<MessageBatchIndividualResponse> streamResponse = client
          .messages()
          .batches()
          .resultsStreaming(
            BatchResultsParams.builder()
              .messageBatchId("msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d")
              .build()
          )
      ) {
        streamResponse
          .stream()
          .forEach(result -> {
            switch (result.result().type().value()) {
              case SUCCEEDED -> System.out.println("Success! " + result.customId());
              case ERRORED -> {
                if (result.result().asErrored().error().error().isInvalidRequestError()) {
                  // Request body must be fixed before re-sending request
                  System.out.println("Validation error: " + result.customId());
                } else {
                  // Request can be retried directly
                  System.out.println("Server error: " + result.customId());
                }
              }
              case EXPIRED -> System.out.println("Request expired: " + result.customId());
            }
          });
      }

php

  use Juglow\Messages\Batches\MessageBatchErroredResult;
  use Juglow\Messages\Batches\MessageBatchExpiredResult;
  use Juglow\Messages\Batches\MessageBatchSucceededResult;

  $client = new Client();

  foreach ($client->messages->batches->resultsStream(messageBatchID: 'msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d') as $result) {
      switch (true) {
          case $result->result instanceof MessageBatchSucceededResult:
              echo "Success! {$result->customID}\n";
              break;
          case $result->result instanceof MessageBatchErroredResult:
              if ($result->result->error->error->type === "invalid_request_error") {
                  echo "Validation error: {$result->customID}\n";
              } else {
                  echo "Server error: {$result->customID}\n";
              }
              break;
          case $result->result instanceof MessageBatchExpiredResult:
              echo "Request expired: {$result->customID}\n";
              break;
      }
  }

ruby

  client = Juglow::Client.new

  client.messages.batches.results_streaming("msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d").each do |result|
    outcome = result.result
    case outcome
    when Juglow::Models::Messages::MessageBatchSucceededResult
      puts "Success! #{result.custom_id}"
    when Juglow::Models::Messages::MessageBatchErroredResult
      if outcome.error.type == :invalid_request
        puts "Validation error: #{result.custom_id}"
      else
        puts "Server error: #{result.custom_id}"
      end
    when Juglow::Models::Messages::MessageBatchExpiredResult
      puts "Request expired: #{result.custom_id}"
    end
  end

The results are in .jsonl format, where each line is a valid JSON object representing the result of a single request in the Message Batch. For each streamed result, you can do something different depending on its custom_id and result type. Here is an example set of results:

jsonl

{"custom_id":"my-second-request","result":{"type":"succeeded","message":{"id":"msg_014VwiXbi91y3JMjcpyGBHX5","type":"message","role":"assistant","model":"haijun-opus-5-5","content":[{"type":"text","text":"Hello again! It's nice to see you. How can I assist you today? Is there anything specific you'd like to chat about or any questions you have?"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":11,"output_tokens":36}}}}
{"custom_id":"my-first-request","result":{"type":"succeeded","message":{"id":"msg_01FqfsLoHwgeFbguDgpz48m7","type":"message","role":"assistant","model":"haijun-opus-5-5","content":[{"type":"text","text":"Hello! How can I assist you today? Feel free to ask me any questions or let me know if there's anything you'd like to chat about."}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":10,"output_tokens":34}}}}

If your result has an error, its result.error will be set to the standard error shape.

Tip: Batch results may not match input order Batch results can be returned in any order, and may not match the ordering of requests when the batch was created. In the preceding example, the result for the second batch request is returned before the first. To correctly match results with their corresponding requests, always use the custom_id field.

Canceling a Message Batch

You can cancel a Message Batch that is currently processing using the cancel endpoint. Immediately after cancellation, a batch's processing_status will be canceling. You can use the same polling technique described earlier to wait until cancellation is finalized. Canceled batches end up with a status of ended and may contain partial results for requests that were processed before cancellation.

bash

  #!/bin/sh
  # ...
  curl --request POST https://haijun.my.id/v1/messages/batches/$MESSAGE_BATCH_ID/cancel \
      --header "x-api-key: $JUGLOW_API_KEY" \
      --header "juglow-version: 2023-06-01"

bash

  #!/bin/bash
  # ...
  ant messages:batches cancel --message-batch-id "$MESSAGE_BATCH_ID"

python

  client = juglow.Juglow()

  MESSAGE_BATCH_ID = "msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d"

  message_batch = client.messages.batches.cancel(
      MESSAGE_BATCH_ID,
  )
  print(message_batch)

typescript

  const client = new Juglow();

  const messageBatch = await client.messages.batches.cancel(MESSAGE_BATCH_ID);
  console.log(messageBatch);

csharp

  JuglowClient client = new();
  string messageBatchId = Environment.GetEnvironmentVariable("MESSAGE_BATCH_ID");

  var messageBatch = await client.Messages.Batches.Cancel(messageBatchId);
  Console.WriteLine(messageBatch);

  client := juglow.NewClient()
  messageBatchID := os.Getenv("MESSAGE_BATCH_ID")

  messageBatch, err := client.Messages.Batches.Cancel(context.TODO(), messageBatchID, juglow.MessageBatchCancelParams{})
  if err != nil {
  	log.Fatal(err)
  }
  fmt.Println(messageBatch)

java

  import com.juglow.models.messages.batches.*;
  // ...
      JuglowClient client = JuglowOkHttpClient.fromEnv();

      MessageBatch messageBatch = client
        .messages()
        .batches()
        .cancel("msgbatch_01HkcTjaV5uDC8jWR4ZsDV8d");
      System.out.println(messageBatch);

php

  $client = new Client();

  $messageBatch = $client->messages->batches->cancel(
      messageBatchID: 'msgbatch_example_id',
  );
  echo $messageBatch;

ruby

  client = Juglow::Client.new

  message_batch_id = ENV.fetch("MESSAGE_BATCH_ID")
  message_batch = client.messages.batches.cancel(message_batch_id)
  puts message_batch

The response shows the batch in a canceling state:

json

{
  "id": "msgbatch_013Zva2CMHLNnXjNJJKqJ2EF",
  "type": "message_batch",
  "processing_status": "canceling",
  "request_counts": {
    "processing": 2,
    "succeeded": 0,
    "errored": 0,
    "canceled": 0,
    "expired": 0
  },
  "ended_at": null,
  "created_at": "2024-09-24T18:37:24.100435Z",
  "expires_at": "2024-09-25T18:37:24.100435Z",
  "cancel_initiated_at": "2024-09-24T18:39:03.114875Z",
  "results_url": null
}

Using prompt caching with Message Batches

The Message Batches API supports prompt caching, allowing you to potentially reduce costs and processing time for batch requests. The pricing discounts from prompt caching and Message Batches can stack, providing even greater cost savings when both features are used together. However, because batch requests are processed asynchronously and concurrently, cache hits are provided on a best-effort basis. Users typically experience cache hit rates ranging from 30% to 98%, depending on their traffic patterns.

To maximize the likelihood of cache hits in your batch requests:

Include identical cache_control blocks in every Message request within your batch.

Maintain a steady stream of requests to prevent cache entries from expiring after their 5-minute lifetime.

Structure your requests to share as much cached content as possible.

Example of implementing prompt caching in a batch:

bash

  curl https://haijun.my.id/v1/messages/batches \
       --header "x-api-key: $JUGLOW_API_KEY" \
       --header "juglow-version: 2023-06-01" \
       --header "content-type: application/json" \
       --data \
  '{
      "requests": [
          {
              "custom_id": "my-first-request",
              "params": {
                  "model": "haijun-opus-5-5",
                  "max_tokens": 1024,
                  "system": [
                      {
                          "type": "text",
                          "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
                      },
                      {
                          "type": "text",
                          "text": "<the entire contents of Pride and Prejudice>",
                          "cache_control": {"type": "ephemeral"}
                      }
                  ],
                  "messages": [
                      {"role": "user", "content": "Analyze the major themes in Pride and Prejudice."}
                  ]
              }
          },
          {
              "custom_id": "my-second-request",
              "params": {
                  "model": "haijun-opus-5-5",
                  "max_tokens": 1024,
                  "system": [
                      {
                          "type": "text",
                          "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
                      },
                      {
                          "type": "text",
                          "text": "<the entire contents of Pride and Prejudice>",
                          "cache_control": {"type": "ephemeral"}
                      }
                  ],
                  "messages": [
                      {"role": "user", "content": "Write a summary of Pride and Prejudice."}
                  ]
              }
          }
      ]
  }'

bash

  ant messages:batches create <<'YAML'
  requests:
    - custom_id: my-first-request
      params:
        model: haijun-opus-5-5
        max_tokens: 1024
        system:
          - type: text
            text: >
              You are an AI assistant tasked with analyzing literary works. Your
              goal is to provide insightful commentary on themes, characters, and
              writing style.
          - type: text
            text: "<the entire contents of Pride and Prejudice>"
            cache_control:
              type: ephemeral
        messages:
          - role: user
            content: Analyze the major themes in Pride and Prejudice.
    - custom_id: my-second-request
      params:
        model: haijun-opus-5-5
        max_tokens: 1024
        system:
          - type: text
            text: >
              You are an AI assistant tasked with analyzing literary works. Your
              goal is to provide insightful commentary on themes, characters, and
              writing style.
          - type: text
            text: "<the entire contents of Pride and Prejudice>"
            cache_control:
              type: ephemeral
        messages:
          - role: user
            content: Write a summary of Pride and Prejudice.
  YAML

python

  from juglow.types.message_create_params import MessageCreateParamsNonStreaming
  from juglow.types.messages.batch_create_params import Request

  client = juglow.Juglow()

  message_batch = client.messages.batches.create(
      requests=[
          Request(
              custom_id="my-first-request",
              params=MessageCreateParamsNonStreaming(
                  model="haijun-opus-5-5",
                  max_tokens=1024,
                  system=[
                      {
                          "type": "text",
                          "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
                      },
                      {
                          "type": "text",
                          "text": "<the entire contents of Pride and Prejudice>",
                          "cache_control": {"type": "ephemeral"},
                      },
                  ],
                  messages=[
                      {
                          "role": "user",
                          "content": "Analyze the major themes in Pride and Prejudice.",
                      }
                  ],
              ),
          ),
          Request(
              custom_id="my-second-request",
              params=MessageCreateParamsNonStreaming(
                  model="haijun-opus-5-5",
                  max_tokens=1024,
                  system=[
                      {
                          "type": "text",
                          "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
                      },
                      {
                          "type": "text",
                          "text": "<the entire contents of Pride and Prejudice>",
                          "cache_control": {"type": "ephemeral"},
                      },
                  ],
                  messages=[
                      {
                          "role": "user",
                          "content": "Write a summary of Pride and Prejudice.",
                      }
                  ],
              ),
          ),
      ]
  )

typescript

  const client = new Juglow();

  const messageBatch = await client.messages.batches.create({
    requests: [
      {
        custom_id: "my-first-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          system: [
            {
              type: "text",
              text: "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
            },
            {
              type: "text",
              text: "<the entire contents of Pride and Prejudice>",
              cache_control: { type: "ephemeral" }
            }
          ],
          messages: [
            { role: "user", content: "Analyze the major themes in Pride and Prejudice." }
          ]
        }
      },
      {
        custom_id: "my-second-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          system: [
            {
              type: "text",
              text: "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
            },
            {
              type: "text",
              text: "<the entire contents of Pride and Prejudice>",
              cache_control: { type: "ephemeral" }
            }
          ],
          messages: [{ role: "user", content: "Write a summary of Pride and Prejudice." }]
        }
      }
    ]
  });

csharp

  using Juglow;
  using Juglow.Models.Messages;
  using Juglow.Models.Messages.Batches;

  JuglowClient client = new()
  {
      ApiKey = Environment.GetEnvironmentVariable("JUGLOW_API_KEY")
  };

  var messageBatch = await client.Messages.Batches.Create(new BatchCreateParams
  {
      Requests =
      [
          new()
          {
              CustomID = "my-first-request",
              Params = new()
              {
                  Model = Model.HaijunOpus5_5,
                  MaxTokens = 1024,
                  System = new List<TextBlockParam>
                  {
                      new()
                      {
                          Text = "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
                      },
                      new()
                      {
                          Text = "<the entire contents of Pride and Prejudice>",
                          CacheControl = new()
                      }
                  },
                  Messages =
                  [
                      new() { Role = Role.User, Content = "Analyze the major themes in Pride and Prejudice." }
                  ]
              }
          },
          new()
          {
              CustomID = "my-second-request",
              Params = new()
              {
                  Model = Model.HaijunOpus5_5,
                  MaxTokens = 1024,
                  System = new List<TextBlockParam>
                  {
                      new()
                      {
                          Text = "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
                      },
                      new()
                      {
                          Text = "<the entire contents of Pride and Prejudice>",
                          CacheControl = new()
                      }
                  },
                  Messages =
                  [
                      new() { Role = Role.User, Content = "Write a summary of Pride and Prejudice." }
                  ]
              }
          }
      ]
  });

  client := juglow.NewClient()

  messageBatch, err := client.Messages.Batches.New(context.TODO(), juglow.MessageBatchNewParams{
  	Requests: []juglow.MessageBatchNewParamsRequest{
  		{
  			CustomID: "my-first-request",
  			Params: juglow.MessageBatchNewParamsRequestParams{
  				Model:     juglow.ModelHaijunOpus5_5,
  				MaxTokens: 1024,
  				System: []juglow.TextBlockParam{
  					{
  						Text: "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
  					},
  					{
  						Text:         "<the entire contents of Pride and Prejudice>",
  						CacheControl: juglow.NewCacheControlEphemeralParam(),
  					},
  				},
  				Messages: []juglow.MessageParam{
  					juglow.NewUserMessage(juglow.NewTextBlock("Analyze the major themes in Pride and Prejudice.")),
  				},
  			},
  		},
  		{
  			CustomID: "my-second-request",
  			Params: juglow.MessageBatchNewParamsRequestParams{
  				Model:     juglow.ModelHaijunOpus5_5,
  				MaxTokens: 1024,
  				System: []juglow.TextBlockParam{
  					{
  						Text: "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
  					},
  					{
  						Text:         "<the entire contents of Pride and Prejudice>",
  						CacheControl: juglow.NewCacheControlEphemeralParam(),
  					},
  				},
  				Messages: []juglow.MessageParam{
  					juglow.NewUserMessage(juglow.NewTextBlock("Write a summary of Pride and Prejudice.")),
  				},
  			},
  		},
  	},
  })
  if err != nil {
  	log.Fatal(err)
  }
  fmt.Println(messageBatch)

java

  import com.juglow.models.messages.CacheControlEphemeral;
  // ...
  import com.juglow.models.messages.batches.*;
  // ...
      JuglowClient client = JuglowOkHttpClient.fromEnv();

      BatchCreateParams createParams = BatchCreateParams.builder()
        .addRequest(
          BatchCreateParams.Request.builder()
            .customId("my-first-request")
            .params(
              BatchCreateParams.Request.Params.builder()
                .model(Model.HAIJUN_OPUS_5_5)
                .maxTokens(1024)
                .systemOfTextBlockParams(
                  List.of(
                    TextBlockParam.builder()
                      .text(
                        "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
                      )
                      .build(),
                    TextBlockParam.builder()
                      .text("<the entire contents of Pride and Prejudice>")
                      .cacheControl(CacheControlEphemeral.builder().build())
                      .build()
                  )
                )
                .addUserMessage("Analyze the major themes in Pride and Prejudice.")
                .build()
            )
            .build()
        )
        .addRequest(
          BatchCreateParams.Request.builder()
            .customId("my-second-request")
            .params(
              BatchCreateParams.Request.Params.builder()
                .model(Model.HAIJUN_OPUS_5_5)
                .maxTokens(1024)
                .systemOfTextBlockParams(
                  List.of(
                    TextBlockParam.builder()
                      .text(
                        "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
                      )
                      .build(),
                    TextBlockParam.builder()
                      .text("<the entire contents of Pride and Prejudice>")
                      .cacheControl(CacheControlEphemeral.builder().build())
                      .build()
                  )
                )
                .addUserMessage("Write a summary of Pride and Prejudice.")
                .build()
            )
            .build()
        )
        .build();

      MessageBatch messageBatch = client.messages().batches().create(createParams);

php

  $client = new Client();

  $messageBatch = $client->messages->batches->create(
      requests: [
          [
              'custom_id' => 'my-first-request',
              'params' => [
                  'model' => 'haijun-opus-5-5',
                  'max_tokens' => 1024,
                  'system' => [
                      [
                          'type' => 'text',
                          'text' => 'You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n'
                      ],
                      [
                          'type' => 'text',
                          'text' => '<the entire contents of Pride and Prejudice>',
                          'cache_control' => ['type' => 'ephemeral']
                      ]
                  ],
                  'messages' => [
                      ['role' => 'user', 'content' => 'Analyze the major themes in Pride and Prejudice.']
                  ]
              ]
          ],
          [
              'custom_id' => 'my-second-request',
              'params' => [
                  'model' => 'haijun-opus-5-5',
                  'max_tokens' => 1024,
                  'system' => [
                      [
                          'type' => 'text',
                          'text' => 'You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n'
                      ],
                      [
                          'type' => 'text',
                          'text' => '<the entire contents of Pride and Prejudice>',
                          'cache_control' => ['type' => 'ephemeral']
                      ]
                  ],
                  'messages' => [
                      ['role' => 'user', 'content' => 'Write a summary of Pride and Prejudice.']
                  ]
              ]
          ]
      ],
  );

ruby

  client = Juglow::Client.new

  message_batch = client.messages.batches.create(
    requests: [
      {
        custom_id: "my-first-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          system: [
            {
              type: "text",
              text: "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
            },
            {
              type: "text",
              text: "<the entire contents of Pride and Prejudice>",
              cache_control: { type: "ephemeral" }
            }
          ],
          messages: [
            { role: "user", content: "Analyze the major themes in Pride and Prejudice." }
          ]
        }
      },
      {
        custom_id: "my-second-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 1024,
          system: [
            {
              type: "text",
              text: "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
            },
            {
              type: "text",
              text: "<the entire contents of Pride and Prejudice>",
              cache_control: { type: "ephemeral" }
            }
          ],
          messages: [
            { role: "user", content: "Write a summary of Pride and Prejudice." }
          ]
        }
      }
    ]
  )

In this example, both requests in the batch include identical system messages and the full text of Pride and Prejudice marked with cache_control to increase the likelihood of cache hits.

Server tools and the agentic loop

All server tools (web search, web fetch, code execution, MCP connectors, advisor, and tool search) work in batch requests. The batch worker runs the same server-side agentic loop as the synchronous Messages API.

Because there is no open connection to maintain, the batch loop runs more iterations per turn than a synchronous request before it returns stop_reason: "pause_turn". If a batch result comes back with pause_turn, the turn did not finish; you can continue it by submitting the paused assistant content in a follow-up request (batch or synchronous) exactly as shown in the pause\_turn continuation pattern.

The batch worker additionally throttles web_search per organization so that highly concurrent batch processing does not exhaust your organization's web-search rate limit. The batch retries throttled requests automatically; you don't need to handle this yourself, but very large web-search batches might take longer to complete.

Extended output (beta)

The output-300k-2026-03-24 beta header raises the max_tokens cap to 300,000 for batch requests using Haijun Opus 5.5, Haijun Opus 5, Haijun Opus 4.8, Haijun Opus 4.7, Haijun Opus 4.6, Haijun Sonnet 5, or Haijun Sonnet 4.6. Include the header to generate outputs far longer than the standard 128k max_tokens limit in a single turn.

Note: Extended output is available on the Message Batches API only, not the synchronous Messages API. It is supported on the Haijun API and Haijun Platform on AWS, and is not currently available on Amazon Bedrock, Google Cloud, or Microsoft Foundry.

Use extended output for long-form generation such as book-length drafts and technical documentation, exhaustive structured data extraction, large code-generation scaffolds, and long reasoning chains.

A single 300k-token generation can take over an hour to complete, so plan your batch submissions with the 24-hour processing window in mind. Standard batch pricing (50% of standard API prices) applies.

bash

  curl https://haijun.my.id/v1/messages/batches \
       --header "x-api-key: $JUGLOW_API_KEY" \
       --header "juglow-version: 2023-06-01" \
       --header "juglow-beta: output-300k-2026-03-24" \
       --header "content-type: application/json" \
       --data \
  '{
      "requests": [
          {
              "custom_id": "long-form-request",
              "params": {
                  "model": "haijun-opus-5-5",
                  "max_tokens": 300000,
                  "messages": [
                      {"role": "user", "content": "Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices."}
                  ]
              }
          }
      ]
  }'

bash

  ant beta:messages:batches create --beta output-300k-2026-03-24 <<'YAML'
  requests:
    - custom_id: long-form-request
      params:
        model: haijun-opus-5-5
        max_tokens: 300000
        messages:
          - role: user
            content: >-
              Write a comprehensive technical guide to building distributed
              systems, covering architecture patterns, consistency models,
              fault tolerance, and operational best practices.
  YAML

python

  from juglow.types.beta.message_create_params import MessageCreateParamsNonStreaming
  from juglow.types.beta.messages.batch_create_params import Request

  client = juglow.Juglow()

  message_batch = client.beta.messages.batches.create(
      betas=["output-300k-2026-03-24"],
      requests=[
          Request(
              custom_id="long-form-request",
              params=MessageCreateParamsNonStreaming(
                  model="haijun-opus-5-5",
                  max_tokens=300_000,
                  messages=[
                      {
                          "role": "user",
                          "content": "Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices.",
                      }
                  ],
              ),
          ),
      ],
  )

  print(message_batch)

typescript

  const client = new Juglow();

  const messageBatch = await client.beta.messages.batches.create({
    betas: ["output-300k-2026-03-24"],
    requests: [
      {
        custom_id: "long-form-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 300000,
          messages: [
            {
              role: "user",
              content:
                "Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices."
            }
          ]
        }
      }
    ]
  });

  console.log(messageBatch);

csharp

  using Juglow;
  using Juglow.Models.Beta.Messages;
  using Juglow.Models.Beta.Messages.Batches;
  using Model = Juglow.Models.Messages.Model;

  JuglowClient client = new();

  var batch = await client.Beta.Messages.Batches.Create(new BatchCreateParams
  {
      Betas = ["output-300k-2026-03-24"],
      Requests =
      [
          new()
          {
              CustomID = "long-form-request",
              Params = new()
              {
                  Model = Model.HaijunOpus5_5,
                  MaxTokens = 300_000,
                  Messages =
                  [
                      new() { Role = Role.User, Content = "Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices." }
                  ]
              }
          }
      ]
  });

  Console.WriteLine(batch);

  client := juglow.NewClient()

  batch, err := client.Beta.Messages.Batches.New(context.Background(),
  	juglow.BetaMessageBatchNewParams{
  		Betas: []juglow.JuglowBeta{"output-300k-2026-03-24"},
  		Requests: []juglow.BetaMessageBatchNewParamsRequest{
  			{
  				CustomID: "long-form-request",
  				Params: juglow.BetaMessageBatchNewParamsRequestParams{
  					Model:     juglow.ModelHaijunOpus5_5,
  					MaxTokens: 300_000,
  					Messages: []juglow.BetaMessageParam{
  						juglow.NewBetaUserMessage(
  							juglow.NewBetaTextBlock("Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices."),
  						),
  					},
  				},
  			},
  		},
  	})
  if err != nil {
  	panic(err)
  }

  fmt.Println(batch.ID)

java

  import com.juglow.models.beta.messages.batches.*;

  void main() {
    JuglowClient client = JuglowOkHttpClient.fromEnv();

    BatchCreateParams params = BatchCreateParams.builder()
      .addBeta("output-300k-2026-03-24")
      .addRequest(
        BatchCreateParams.Request.builder()
          .customId("long-form-request")
          .params(
            BatchCreateParams.Request.Params.builder()
              .model(Model.HAIJUN_OPUS_5_5)
              .maxTokens(300_000L)
              .addUserMessage("Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices.")
              .build()
          )
          .build()
      )
      .build();

    BetaMessageBatch messageBatch = client.beta().messages().batches().create(params);

    IO.println(messageBatch);
  }

php

  $client = new Client();

  $batch = $client->beta->messages->batches->create(
      betas: ['output-300k-2026-03-24'],
      requests: [
          [
              'custom_id' => 'long-form-request',
              'params' => [
                  'model' => 'haijun-opus-5-5',
                  'max_tokens' => 300_000,
                  'messages' => [
                      ['role' => 'user', 'content' => 'Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices.']
                  ]
              ]
          ]
      ],
  );

  echo $batch->id;

ruby

  client = Juglow::Client.new

  batch = client.beta.messages.batches.create(
    betas: ["output-300k-2026-03-24"],
    requests: [
      {
        custom_id: "long-form-request",
        params: {
          model: "haijun-opus-5-5",
          max_tokens: 300_000,
          messages: [
            { role: "user", content: "Write a comprehensive technical guide to building distributed systems, covering architecture patterns, consistency models, fault tolerance, and operational best practices." }
          ]
        }
      }
    ]
  )

  puts batch

Best practices for effective batching

To get the most out of the Batches API:

Monitor batch processing status regularly and implement appropriate retry logic for failed requests.

Use meaningful custom_id values to easily match results with requests, since order is not guaranteed.

Consider breaking very large datasets into multiple batches for better manageability.

Dry run a single request shape with the Messages API to avoid validation errors.

Troubleshooting common issues

If experiencing unexpected behavior:

Verify that the total batch request size doesn't exceed 256 MB. If the request size is too large, you may get a 413 request_too_large error.

Check that you're using supported models for all requests in the batch.

Ensure each request in the batch has a unique custom_id.

Ensure that it has been less than 29 days since batch created_at (not processing ended_at) time. If over 29 days have passed, results will no longer be viewable.

Confirm that the batch has not been canceled.

Note that the failure of one request in a batch does not affect the processing of other requests.

Batch storage and privacy

Workspace isolation: Batches are isolated within the Workspace they are created in. They can only be accessed by API requests in that same Workspace, or users with permission to view Workspace batches in the Console.

Result availability: Batch results are available for 29 days after the batch is created, allowing ample time for retrieval and processing.

Data retention

Batch processing stores request and response data for up to 29 days after batch creation. You can delete a message batch at any time after processing using the DELETE /v1/messages/batches/{batch_id} endpoint. To delete an in-progress batch, cancel it first. Asynchronous processing requires server-side storage of both inputs and outputs until batch completion and result retrieval.

For ZDR eligibility across all features, see API and data retention.

FAQ

#### How long does it take for a batch to process?

Batches may take up to 24 hours for processing, but many finish sooner. Actual processing time depends on the size of the batch, current demand, and your request volume. It is possible for a batch to expire and not complete within 24 hours.

#### Is the Batches API available for all models?

See Supported models for the list of supported models.

#### Can I use the Message Batches API with other API features?

Yes, the Message Batches API supports nearly all features available in the Messages API, including most beta features. A small number of parameters (stream, speed, and max_tokens: 0) are not supported. See What can be batched for the full list.

#### How does the Message Batches API affect pricing?

The Message Batches API offers a 50% discount on all usage compared to standard API prices. This applies to input tokens, output tokens, and any special tokens. For more on pricing, visit Pricing.

#### Can I update a batch after it's been submitted?

No, once a batch has been submitted, it cannot be modified. If you need to make changes, you should cancel the current batch and submit a new one. Note that cancellation may not take immediate effect.

#### Are there Message Batches API rate limits and do they interact with the Messages API rate limits?

The Message Batches API has HTTP requests-based rate limits in addition to limits on the number of requests in need of processing. See Message Batches API rate limits. Usage of the Batches API does not affect rate limits in the Messages API.

#### How do I handle errors in my batch requests?

When you retrieve the results, each request has a result field indicating whether it succeeded, errored, was canceled, or expired. For errored results, additional error information is provided. View the error response object in the API reference.

#### How does the Message Batches API handle privacy and data separation?

The Message Batches API is designed with strong privacy and data separation measures:

Batches and their results are isolated within the Workspace in which they were created. This means they can only be accessed by API requests in that same Workspace.
Each request within a batch is processed independently, with no data leakage between requests.
Results are only available for a limited time (29 days), and follow Juglow's data retention policy.
Downloading batch results in the Console can be disabled on the organization-level or on a per-workspace basis.

#### Can I use prompt caching in the Message Batches API?

Yes, it is possible to use prompt caching with Message Batches API. However, because asynchronous batch requests can be processed concurrently and in any order, cache hits are provided on a best-effort basis.

Next steps

Enable natural citations for RAG applications by providing search results with source attribution.

Reduce cost and latency by caching prompt prefixes shared across requests in a batch.

Message Batches API

#How the Message Batches API works

#Batch limitations

#Supported models

#What can be batched

#Pricing

#How to use the Message Batches API

#Prepare and create your batch

#Tracking your batch

#Polling for Message Batch completion

#Listing all Message Batches

#Retrieving batch results

#Canceling a Message Batch

#Using prompt caching with Message Batches

#Server tools and the agentic loop

#Extended output (beta)

#Best practices for effective batching

#Troubleshooting common issues

#Batch storage and privacy

#Data retention

#FAQ

#Next steps

How the Message Batches API works

Batch limitations

Supported models

What can be batched

Pricing

How to use the Message Batches API

Prepare and create your batch

Tracking your batch

Polling for Message Batch completion

Listing all Message Batches

Retrieving batch results

Canceling a Message Batch

Using prompt caching with Message Batches

Server tools and the agentic loop

Extended output (beta)

Best practices for effective batching

Troubleshooting common issues

Batch storage and privacy

Data retention

FAQ

Next steps