The Haijun API features citation support that enables Haijun to provide detailed citations when answering questions about documents. Citations are a valuable affordance in many LLM powered applications to help users track and verify the sources of information in responses.
Citations are supported on:
haijun-3-5-haiku-20241022
The citations feature is an alternative to prompt-based citation techniques. Using this featue has the following advantages:
a>.
Setup
PDF document citation → page location format
Custom content document citation → content block location format
We will explore working with each of these in the examples below.
wrap" style="padding-left:4ch;text-indent:-4ch"> raw_response = {"content": []}
print("\n" + "=" * 80 + "\nRaw response:\n" + "=" * 80)
for content in response.content:
if content.type == "text":
block = {"type": "text", "text": content.text}
if hasattr(content, "citations") and content.citations:
block["citations"] = []
for citation in content.citations:
citation_dict = {
"type": citation.type,
"cited_text": citation.cited_text,
"document_title": citation.document_title,
}
if citation.type == "page_location":
citation_dict.update(
{
"start_page_number": citation.start_page_number,
"end_page_number": citation.end_page_number,
}
)
block["citations"].append(citation_dict)
raw_response["content"].append(block)
return json.dumps(raw_response, indent=2)
print(visualize_raw_response(response))
================================================================================ Raw response: ================================================================================ { "content": [ { "type": "text", "text": "Based on the documentation, I can explain why you don't see tracking yet: " }, { "type": "text", "text": "You'll receive an email with your tracking number once your order ships. If you don't receive a tracking number within 48 hours of your order confirmation, please contact our customer support team for assistance.", "citations": [ { "type": "char_location", "cited_text": "Once your order ships, you'll receive an email with a tracking number. ", "document_title": "Order Tracking Information" }, { "type": "char_location", "cited_text": "If you haven't received a tracking number within 48 hours of your order confirmation, please contact our customer support team.", "document_title": "Order Tracking Information" } ] }, { "type": "text", "text": "\n\nSince you just checked out, your order likely hasn't shipped yet. Once it ships, you'll receive the tracking information via email." } ] } Visualizing Citations By leveraging the citation data, we can create UIs that:
"text": "What is the main idea of Constitutional AI?"},
],
}
],
)
print(visualize_raw_response(pdf_response))
print(visualize_citations(pdf_response))
================================================================================ Raw response: ================================================================================ { "content": [ { "type": "text", "text": "Based on the paper, here are the key aspects of Constitutional AI:\n\n" }, { "type": "text", "text": "Constitutional AI is a method for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, hence the name \"Constitutional AI\".", "citations": [ { "type": "page_location", "cited_text": "We experiment with methods for training a harmless AI assistant through self\u0002improvement, without any human labels identifying harmful outputs. The only human\r\noversight is provided through a list of rules or principles, and so we refer to the method as\r\n\u2018Constitutional AI\u2019. ", "document_title": "Constitutional AI Paper", "start_page_number": 1, "end_page_number": 2 } ] }, { "type": "text", "text": "\n\nThe process involves two main phases:\n\n1. Supervised Learning Phase:\n" }, { "type": "text", "text": "In this phase, they sample from an initial model, generate self-critiques and revisions, and then finetune the original model on revised responses.", "citations": [ { "type": "page_location", "cited_text": "In the supervised phase we sample from an initial model, then generate\r\nself-critiques and revisions, and then finetune the original model on revised responses. ", "document_title": "Constitutional AI Paper", "start_page_number": 1, "end_page_number": 2 } ] }, { "type": "text", "text": "\n\n2. Reinforcement Learning Phase:\n" }, { "type": "text", "text": "In this phase, they:\n- Sample from the finetuned model\n- Use a model to evaluate which of two samples is better\n- Train a preference model from this dataset of AI preferences\n- Use \"RL from AI Feedback\" (RLAIF)", "citations": [ { "type": "page_location", "cited_text": "In\r\nthe RL phase, we sample from the finetuned model, use a model to evaluate which of the\r\ntwo samples is better, and then train a preference model from this dataset of AI prefer\u0002ences. We then train with RL using the preference model as the reward signal, i.e. we\r\nuse \u2018RL from AI Feedback\u2019 (RLAIF). ", "document_title": "Constitutional AI Paper", "start_page_number": 1, "end_page_number": 2 } ] }, { "type": "text", "text": "\n\nThe key outcomes are:\n\n" }, { "type": "text", "text": "- They are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them\n- Both the SL and RL methods can leverage chain-of-thought style reasoning to improve human-judged performance and transparency of AI decision making\n- These methods make it possible to control AI behavior more precisely and with far fewer human labels", "citations": [ { "type": "page_location", "cited_text": "As a result we are able to train a harmless but non\u0002evasive AI assistant that engages with harmful queries by explaining its objections to them.\r\nBoth the SL and RL methods can leverage chain-of-thought style reasoning to improve the\r\nhuman-judged performance and transparency of AI decision making. These methods make\r\nit possible to control AI behavior more precisely and with far fewer human labels.\r\n", "document_title": "Constitutional AI Paper", "start_page_number": 1, "end_page_number": 2 } ] }, { "type": "text", "text": "\n\n" }, { "type": "text", "text": "The ultimate goal is not to completely remove human supervision, but rather to make it more efficient, transparent and targeted. While this work reduces reliance on human supervision for harmlessness, they still relied on human supervision in the form of helpfulness labels. The researchers expect it is possible to achieve helpfulness and instruction-following without human feedback, starting from only a pretrained LM and extensive prompting, but leave this for future work.", "citations": [ { "type": "page_location", "cited_text": "By removing human feedback labels for harmlessness, we have moved further away from reliance on human\r\nsupervision, and closer to the possibility of a self-supervised approach to alignment. However, in this work\r\nwe still relied on human supervision in the form of helpfulness labels. We expect it is possible to achieve help\u0002fulness and instruction-following without human feedback, starting from only a pretrained LM and extensive\r\nprompting, but we leave this for future work.\r\nOur ultimate goal is not to remove human supervision entirely, but to make it more efficient, transparent, and\r\ntargeted. ", "document_title": "Constitutional AI Paper", "start_page_number": 15, "end_page_number": 16 } ] } ] } ================================================================================ Formatted response: ================================================================================ Based on the paper, here are the key aspects of Constitutional AI: Constitutional AI is a method for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, hence the name "Constitutional AI". [1] The process involves two main phases: 1. Supervised Learning Phase: In this phase, they sample from an initial model, generate self-critiques and revisions, and then finetune the original model on revised responses. [2] 2. Reinforcement Learning Phase: In this phase, they: - Sample from the finetuned model - Use a model to evaluate which of two samples is better - Train a preference model from this dataset of AI preferences - Use "RL from AI Feedback" (RLAIF) [3] The key outcomes are: - They are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them - Both the SL and RL methods can leverage chain-of-thought style reasoning to improve human-judged performance and transparency of AI decision making - These methods make it possible to control AI behavior more precisely and with far fewer human labels [4] The ultimate goal is not to completely remove human supervision, but rather to make it more efficient, transparent and targeted. While this work reduces reliance on human supervision for harmlessness, they still relied on human supervision in the form of helpfulness labels. The researchers expect it is possible to achieve helpfulness and instruction-following without human feedback, starting from only a pretrained LM and extensive prompting, but leave this for future work. [5] [1] "We experiment with methods for training a harmless AI assistant through selfimprovement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as ‘Constitutional AI’." found in "Constitutional AI Paper" [2] "In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses." found in "Constitutional AI Paper" [3] "In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use ‘RL from AI Feedback’ (RLAIF)." found in "Constitutional AI Paper" [4] "As a result we are able to train a harmless but nonevasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels." found in "Constitutional AI Paper" [5] "By removing human feedback labels for harmlessness, we have moved further away from reliance on human supervision, and closer to the possibility of a self-supervised approach to alignment. However, in this work we still relied on human supervision in the form of helpfulness labels. We expect it is possible to achieve helpfulness and instruction-following without human feedback, starting from only a pretrained LM and extensive prompting, but we leave this for future work. Our ultimate goal is not to remove human supervision entirely, but to make it more efficient, transparent, and targeted." found in "Constitutional AI Paper" Custom Content Documents While plain text documents are automatically chunked into sentences, custom content documents give you complete control over citation granularity. This API shape allows you to: