Skip to main content

​
MaxTokens

A dataclass to be used on spawning an agent or decorating an agentic function to control the maximum number of output tokens generated by the underlying model. 
@dataclass(slots=True, frozen=True)
class MaxTokens:
    """Control the maximum number of tokens an agent can generate"""

    per_invocation: int | None = None
    per_round: int | None = None
    rounds: int | None = None
Parameters
​
per_invocation
int | None
The maximum number of output tokens generated over all rounds of inference in a single invocation.Defaults to None meaning unlimited.
​
per_round
int | None
The maximum number of output tokens generated in a single round of inference in a single invocation.Defaults to None meaning unlimited.
​
rounds
int | None
The maximum number of rounds of inference in a single invocation.Defaults to None meaning unlimited.

​
Usage

A dataclass outlining the token usage of the underlying model of an agent or agentic function. 
@dataclass(slots=True, frozen=True)
class Usage:
    """
    Token usage statistics.

    NOTE: total_tokens is not the same as input_tokens + output_tokens,
    it is just the total number of tokens involved in the belonging timeframe.
    """

    input_tokens: int
    output_tokens: int
    total_tokens: int
Parameters
​
input_tokens
int
The number of input tokens consumed by the model.
​
output_tokens
int
The number of output tokens consequently generated by the model.
​
total_tokens
int
The total number of tokens processed, not double-counting generated then re-consumed tokens.

​
total_usage

A function to obtain the total token usage of an agent or agentic function accross all invocations. 
def total_usage(ag: AgenticFunction | Agent) -> Usage:
Parameters
​
ag
AgenticFunction | Agent
The number of input tokens consumed by the model.
​
Returns
Usage
The token usage accross all invocations of the agent or agentic function.

​
last_usage

A function to obtain the token usage of an agent or agentic function for the last invocation. 
def last_usage(ag: AgenticFunction | Agent) -> Usage:
Parameters
​
ag
AgenticFunction | Agent
The number of input tokens consumed by the model.
​
Returns
Usage
The token usage of the last invocation of the agent or agentic function.