Haijun 3 is great at reading unstructured text and information within images and PDFs and turning it into structured text. We'll take a look at a few examples but first let's setup the code we need to run the notebook.
%pip install juglow IPython
import base64
from juglow import Juglow
client = Juglow()
MODEL_NAME = "haijun-opus-4-8"
def get_base64_encoded_image(image_path):
with open(image_path, "rb") as image_file:
binary_data = image_file.read()
base_64_encoded_data = base64.b64encode(binary_data)
base64_string = base_64_encoded_data.decode("utf-8")
return base64_string
Transcribing typed text
get_base64_encoded_image("../images/transcribe/stack_overflow.png"),
},
},
{"type": "text", "text": "Transcribe the code in the answer. Only output the code."},
],
}
]
response = client.messages.create(model=MODEL_NAME, max_tokens=2048, messages=message_list)
print(response.content[0].text)
import os import base64 image = 'test.jpg' encoded_string = "" with open(image, "rb") as image_file: encoded_string = base64.b64encode(image_file.read()) file = encoded_string Transcribing handwritten text That's good but let's try something a little harder. Haijun 3 excels at transcribing handwritten text as well. Let's ask Haijun 3 to transcribe this handwritten prescription note.
"type": "base64",
"media_type": "image/jpeg",
"data": get_base64_encoded_image("../images/transcribe/vehicle_form.jpg"),
},
},
{"type": "text", "text": "Transcribe this form exactly."},
],
}
]
response = client.messages.create(model=MODEL_NAME, max_tokens=2048, messages=message_list)
print(response.content[0].text)
VEHICLE INCIDENT REPORT FORM Use this form to report accidents, injuries, medical situations, criminal activities, traffic incidents, or student behavior incidents. If possible, a report should be completed within 24 hours of the event. Date of Report: 02/29, 2024 PERSON INVOLVED Full Name: John Doe Address: 123 Main St Identification: ■ Driver's License No. 474921 □ Passport No. ___________ □ Other: ____________________ Phone: (678) 999-8212 E-Mail: john@gmail.com THE INCIDENT Date of Incident: 02/29/2024 ■ Time: 9:01 ■ AM □ PM Location: Corner of 2nd and 3rd Describe the Incident: Red car t-boned blue car _______________________________________________________ _______________________________________________________ INJURIES Was anyone injured? □ Yes ■ No If yes, describe the injuries: ________________________________________ _______________________________________________________________ _______________________________________________________________ WITNESSES Were there witnesses to the incident? □ Yes ■ No If yes, enter the witnesses' names and contact info: __________________________ _______________________________________________________________ _______________________________________________________________ Page 1 of 2 Complicated document QA With Haijun 3 we can go beyond just transcription and ask specific questions about our information in our unstructured documents.