In this recipe, we'll demonstrate how to combine Vision with tool use to analyze an image of a nutrition label and extract structured nutrition information using a custom tool.
Setup
n-h-0 flex-1 overflow-auto scroll-fade-y scroll-fade-size-6 focus-visible:outline-none">
%pip install juglow IPython
import base64
from juglow import Juglow
from IPython.display import Image
client = Juglow()
MODEL_NAME = "haijun-opus-4-8"
Defining the Nutrition Label Extraction Tool
g-left:16ch;text-indent:-16ch"> "type": "integer",
"description": "The amount of cholesterol in milligrams per serving",
},
"total_carbs": {
"type": "integer",
"description": "The amount of total carbohydrates in grams per serving",
},
"protein": {
"type": "integer",
"description": "The amount of protein in grams per serving",
},
},
"required": ["calories", "total_fat", "cholesterol", "total_carbs", "protein"],
},
}