Adding Tool Calling to an OCI Generative AI Python Application

One of my more interesting OCI Generative AI experiments was function calling. OCI's generic chat request exposes this directly: GenericChatRequest accepts tools and tool_choice, and an assistant response can contain tool_calls.

This turns a text-only model call into an agent-style loop, while the application—not the LLM—retains execution authority.

1. The Complete Control Flow

User:
"What columns are in SALES_ORDER?"

       ↓

OCI request contains:
- user message
- tool definitions
- automatic tool selection

       ↓

Assistant response:
lookup_schema(table_name="SALES_ORDER")

       ↓

Python validates and executes the function

       ↓

Python returns a ToolMessage

       ↓

OCI generates the final explanation

2. Define a Narrow Tool Schema

from oci.generative_ai_inference.models import (
    FunctionDefinition,
    ToolDefinition,
)

lookup_schema_tool = ToolDefinition(
    function=FunctionDefinition(
        name="lookup_schema",
        description=(
            "Return column metadata for one Oracle table. "
            "Use this when the user asks about columns "
            "or data types."
        ),
        parameters={
            "type": "object",
            "properties": {
                "table_name": {
                    "type": "string",
                    "description": "Oracle table name"
                }
            },
            "required": ["table_name"],
            "additionalProperties": False,
        },
    )
)

The tool description is part of routing. Precise names and descriptions make model decisions easier to understand and test.

3. Allow Automatic Tool Selection

from oci.generative_ai_inference.models import (
    GenericChatRequest,
    ToolChoiceAuto,
    UserMessage,
    TextContent,
)

request = GenericChatRequest(
    messages=[
        UserMessage(
            content=[
                TextContent(
                    text="What columns are in SALES_ORDER?"
                )
            ]
        )
    ],
    tools=[lookup_schema_tool],
    tool_choice=ToolChoiceAuto(),
    temperature=0.1,
    max_tokens=800,
)

The model can answer directly when it does not need a function, or request a tool when live information is required.

4. Send the First Request

details = ChatDetails(
    compartment_id=COMPARTMENT_ID,
    serving_mode=OnDemandServingMode(
        model_id=MODEL_ID
    ),
    chat_request=request,
)

first_response = client.chat(details)

assistant_msg = (
    first_response.data.chat_response
    .choices[0]
    .message
)

5. Inspect the Proposed Call

if not assistant_msg.tool_calls:
    answer = assistant_msg.content[0].text
    print(answer)
else:
    for call in assistant_msg.tool_calls:
        print(call.id)
        print(call.function.name)
        print(call.function.arguments)

This is the key trust boundary: the model proposes a call; Python decides whether it is permitted.

6. Use an Explicit Tool Registry

TOOLS = {
    "lookup_schema": lookup_schema,
    "run_readonly_query": run_readonly_query,
}

I do not use eval() or dynamically import a model-supplied function name. The name must exist in a controlled registry.

7. Validate Arguments Before Execution

import json

def execute_tool(tool_call):
    name = tool_call.function.name

    if name not in TOOLS:
        raise ValueError(f"Tool not allowed: {name}")

    args = json.loads(tool_call.function.arguments)

    if name == "lookup_schema":
        table_name = args["table_name"].upper()

        if not table_name.replace("_", "").isalnum():
            raise ValueError("Invalid table name")

        args["table_name"] = table_name

    return TOOLS[name](**args)

For database functions I would add schema allow-lists, read-only credentials, row limits, SQL validation, execution timeouts, and audit logging.

8. Example Read-Only Oracle Tool

def lookup_schema(table_name):
    sql = """
        SELECT column_name,
               data_type,
               nullable
        FROM user_tab_columns
        WHERE table_name = :table_name
        ORDER BY column_id
    """

    with connection.cursor() as cur:
        cur.execute(sql, table_name=table_name)

        return [
            {
                "column_name": row[0],
                "data_type": row[1],
                "nullable": row[2],
            }
            for row in cur
        ]

The tool returns structured data. It does not need to compose the final English response.

9. Return the Tool Result with ToolMessage

from oci.generative_ai_inference.models import ToolMessage

tool_call = assistant_msg.tool_calls[0]
tool_result = execute_tool(tool_call)

tool_message = ToolMessage(
    tool_call_id=tool_call.id,
    content=[
        TextContent(
            text=json.dumps(tool_result)
        )
    ],
)

messages = [
    request.messages[0],
    assistant_msg,
    tool_message,
]

The tool_call_id ties the result to the function request generated by the model.

10. Ask OCI to Continue the Conversation

second_request = GenericChatRequest(
    messages=messages,
    tools=[lookup_schema_tool],
    tool_choice=ToolChoiceAuto(),
    temperature=0.1,
    max_tokens=800,
)

second_details = ChatDetails(
    compartment_id=COMPARTMENT_ID,
    serving_mode=OnDemandServingMode(
        model_id=MODEL_ID
    ),
    chat_request=second_request,
)

second_response = client.chat(second_details)

final_msg = (
    second_response.data.chat_response
    .choices[0]
    .message
)

print(final_msg.content[0].text)

11. Reasoning and Authority Are Different

Model can decide:
"I need schema information."

Application decides:
"lookup_schema is an approved tool."
"These arguments are valid."
"This credential is read-only."
"This execution is logged."
"Now the call may run."