Tool Calling

Basic Tool Calling

Defining Tools

Define tools (functions) that the AI can call:

import { AnimusClient } from 'animus-client';

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a specific location",
      parameters: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "The city and state, e.g. San Francisco, CA"
          },
          unit: {
            type: "string",
            enum: ["celsius", "fahrenheit"],
            description: "The temperature unit to use"
          }
        },
        required: ["location"]
      }
    }
  },
  {
    type: "function",
    function: {
      name: "calculate_tip",
      description: "Calculate tip amount based on bill total and tip percentage",
      parameters: {
        type: "object",
        properties: {
          bill_amount: {
            type: "number",
            description: "The total bill amount"
          },
          tip_percentage: {
            type: "number",
            description: "The tip percentage (e.g., 15 for 15%)"
          }
        },
        required: ["bill_amount", "tip_percentage"]
      }
    }
  }
];

const client = new AnimusClient({
  tokenProviderUrl: 'https://your-backend.com/api/get-animus-token',
  chat: {
    model: 'vivian-llama3.1-70b-1.0-fp8',
    systemMessage: 'You are a helpful assistant with access to various tools.'
  }
});

Implementing Tool Functions

Create the actual functions that will be called:

// Tool function implementations
async function getWeather(location: string, unit: string = 'fahrenheit') {
  // Simulate API call to weather service
  const weatherData = {
    location,
    temperature: unit === 'celsius' ? 22 : 72,
    condition: 'Sunny',
    humidity: 45,
    unit
  };
  
  return {
    location: weatherData.location,
    temperature: `${weatherData.temperature}°${unit === 'celsius' ? 'C' : 'F'}`,
    condition: weatherData.condition,
    humidity: `${weatherData.humidity}%`
  };
}

function calculateTip(billAmount: number, tipPercentage: number) {
  const tipAmount = (billAmount * tipPercentage) / 100;
  const total = billAmount + tipAmount;
  
  return {
    bill_amount: billAmount,
    tip_percentage: tipPercentage,
    tip_amount: tipAmount.toFixed(2),
    total_amount: total.toFixed(2)
  };
}

// Tool registry for easy lookup
const toolFunctions = {
  get_weather: getWeather,
  calculate_tip: calculateTip
};

Making Tool-Enabled Requests

Send requests with tools available:

async function handleToolCalling() {
  try {
    const response = await client.chat.completions({
      messages: [
        { role: 'user', content: 'What\'s the weather like in New York, NY?' }
      ],
      tools: tools,
      tool_choice: 'auto' // Let the model decide when to use tools
    });

    const message = response.choices[0].message;

    // Check if the model wants to call a tool
    if (message.tool_calls && message.tool_calls.length > 0) {
      console.log('AI wants to call tools:', message.tool_calls);
      
      // Process each tool call
      const toolResults = await Promise.all(
        message.tool_calls.map(async (toolCall) => {
          const functionName = toolCall.function.name;
          const functionArgs = JSON.parse(toolCall.function.arguments);
          
          console.log(`Calling ${functionName} with args:`, functionArgs);
          
          // Call the actual function
          const result = await toolFunctions[functionName](...Object.values(functionArgs));
          
          return {
            tool_call_id: toolCall.id,
            role: 'tool' as const,
            content: JSON.stringify(result)
          };
        })
      );

      // Send tool results back to the model
      const finalResponse = await client.chat.completions({
        messages: [
          { role: 'user', content: 'What\'s the weather like in New York, NY?' },
          message, // The assistant's message with tool calls
          ...toolResults // The tool results
        ],
        tools: tools
      });

      console.log('Final response:', finalResponse.choices[0].message.content);
    } else {
      console.log('No tools called:', message.content);
    }

  } catch (error) {
    console.error('Tool calling error:', error);
  }
}

Advanced Tool Calling

Complex Tool Definitions

Define more sophisticated tools with nested parameters:

const advancedTools = [
  {
    type: "function",
    function: {
      name: "search_database",
      description: "Search a database with complex filters and sorting",
      parameters: {
        type: "object",
        properties: {
          query: {
            type: "string",
            description: "The search query"
          },
          filters: {
            type: "object",
            properties: {
              category: {
                type: "string",
                enum: ["products", "users", "orders"]
              },
              date_range: {
                type: "object",
                properties: {
                  start: { type: "string", format: "date" },
                  end: { type: "string", format: "date" }
                }
              },
              price_range: {
                type: "object",
                properties: {
                  min: { type: "number" },
                  max: { type: "number" }
                }
              }
            }
          },
          sort: {
            type: "object",
            properties: {
              field: { type: "string" },
              order: { type: "string", enum: ["asc", "desc"] }
            }
          },
          limit: {
            type: "number",
            minimum: 1,
            maximum: 100,
            default: 10
          }
        },
        required: ["query"]
      }
    }
  }
];

Tool Choice Strategies

Control when and which tools are used:

// Force the model to use a specific tool
const response = await client.chat.completions({
  messages: [{ role: 'user', content: 'Calculate a 20% tip on $85' }],
  tools: tools,
  tool_choice: {
    type: "function",
    function: { name: "calculate_tip" }
  }
});

// Prevent tool usage (force text response)
const textOnlyResponse = await client.chat.completions({
  messages: [{ role: 'user', content: 'What\'s the weather like today?' }],
  tools: tools,
  tool_choice: 'none'
});

// Let the model decide (default)
const autoResponse = await client.chat.completions({
  messages: [{ role: 'user', content: 'Help me with something' }],
  tools: tools,
  tool_choice: 'auto'
});

Error Handling in Tool Calls

Implement robust error handling for tool execution:

async function executeToolCall(toolCall: any) {
  try {
    const functionName = toolCall.function.name;
    const functionArgs = JSON.parse(toolCall.function.arguments);
    
    // Validate function exists
    if (!toolFunctions[functionName]) {
      throw new Error(`Unknown function: ${functionName}`);
    }
    
    // Validate arguments
    const result = await toolFunctions[functionName](...Object.values(functionArgs));
    
    return {
      tool_call_id: toolCall.id,
      role: 'tool' as const,
      content: JSON.stringify({
        success: true,
        data: result
      })
    };
    
  } catch (error) {
    console.error(`Tool execution error for ${toolCall.function.name}:`, error);
    
    return {
      tool_call_id: toolCall.id,
      role: 'tool' as const,
      content: JSON.stringify({
        success: false,
        error: error.message || 'Tool execution failed'
      })
    };
  }
}

Tool Calling with Chat History

Maintaining Context

When using chat history with tool calling, include all messages:

class ToolEnabledChat {
  private client: AnimusClient;
  private tools: any[];
  private conversation: any[] = [];

  constructor(client: AnimusClient, tools: any[]) {
    this.client = client;
    this.tools = tools;
  }

  async sendMessage(userMessage: string) {
    // Add user message to conversation
    this.conversation.push({
      role: 'user',
      content: userMessage
    });

    try {
      const response = await this.client.chat.completions({
        messages: this.conversation,
        tools: this.tools,
        tool_choice: 'auto'
      });

      const assistantMessage = response.choices[0].message;
      this.conversation.push(assistantMessage);

      // Handle tool calls if present
      if (assistantMessage.tool_calls && assistantMessage.tool_calls.length > 0) {
        const toolResults = await Promise.all(
          assistantMessage.tool_calls.map(toolCall => this.executeToolCall(toolCall))
        );

        // Add tool results to conversation
        this.conversation.push(...toolResults);

        // Get final response after tool execution
        const finalResponse = await this.client.chat.completions({
          messages: this.conversation,
          tools: this.tools
        });

        const finalMessage = finalResponse.choices[0].message;
        this.conversation.push(finalMessage);

        return finalMessage.content;
      }

      return assistantMessage.content;

    } catch (error) {
      console.error('Chat error:', error);
      throw error;
    }
  }

  async executeToolCall(toolCall: any) {
    // Implementation from previous example
    return executeToolCall(toolCall);
  }

  getConversationHistory() {
    return [...this.conversation];
  }

  clearHistory() {
    this.conversation = [];
  }
}

// Usage
const toolChat = new ToolEnabledChat(client, tools);

const response1 = await toolChat.sendMessage("What's the weather in Boston?");
console.log(response1);

const response2 = await toolChat.sendMessage("What about in Los Angeles?");
console.log(response2);

Real-World Tool Examples

API Integration Tool

const apiTools = [
  {
    type: "function",
    function: {
      name: "fetch_user_data",
      description: "Fetch user data from the API",
      parameters: {
        type: "object",
        properties: {
          user_id: {
            type: "string",
            description: "The user ID to fetch data for"
          },
          include_preferences: {
            type: "boolean",
            description: "Whether to include user preferences",
            default: false
          }
        },
        required: ["user_id"]
      }
    }
  }
];

async function fetchUserData(userId: string, includePreferences: boolean = false) {
  try {
    const response = await fetch(`/api/users/${userId}?preferences=${includePreferences}`, {
      headers: {
        'Authorization': 'Bearer ' + await getAuthToken()
      }
    });

    if (!response.ok) {
      throw new Error(`API request failed: ${response.status}`);
    }

    return await response.json();
  } catch (error) {
    throw new Error(`Failed to fetch user data: ${error.message}`);
  }
}

Database Query Tool

const databaseTools = [
  {
    type: "function",
    function: {
      name: "query_orders",
      description: "Query customer orders with filters",
      parameters: {
        type: "object",
        properties: {
          customer_id: { type: "string" },
          status: {
            type: "string",
            enum: ["pending", "processing", "shipped", "delivered", "cancelled"]
          },
          date_from: { type: "string", format: "date" },
          date_to: { type: "string", format: "date" },
          limit: { type: "number", minimum: 1, maximum: 50, default: 10 }
        }
      }
    }
  }
];

async function queryOrders(filters: any) {
  // Simulate database query
  const query = buildSQLQuery(filters);
  const results = await database.query(query);
  
  return {
    total_count: results.length,
    orders: results.map(order => ({
      id: order.id,
      status: order.status,
      total: order.total,
      created_at: order.created_at
    }))
  };
}

File System Tool

const fileSystemTools = [
  {
    type: "function",
    function: {
      name: "read_file",
      description: "Read contents of a file",
      parameters: {
        type: "object",
        properties: {
          file_path: {
            type: "string",
            description: "Path to the file to read"
          },
          encoding: {
            type: "string",
            enum: ["utf8", "base64"],
            default: "utf8"
          }
        },
        required: ["file_path"]
      }
    }
  },
  {
    type: "function",
    function: {
      name: "list_directory",
      description: "List contents of a directory",
      parameters: {
        type: "object",
        properties: {
          directory_path: {
            type: "string",
            description: "Path to the directory"
          },
          include_hidden: {
            type: "boolean",
            default: false
          }
        },
        required: ["directory_path"]
      }
    }
  }
];

Tool Calling with AutoTurn

Combine tool calling with conversational turns:

const client = new AnimusClient({
  tokenProviderUrl: 'https://your-backend.com/api/get-animus-token',
  chat: {
    model: 'vivian-llama3.1-70b-1.0-fp8',
    systemMessage: 'You are a helpful assistant with access to various tools.',
    autoTurn: true // Enable conversational turns
  }
});

// Tool calling works seamlessly with auto-turn
client.on('messageComplete', (data) => {
  console.log(`${data.messageType} message: ${data.content}`);
  displayMessage(data.content);
});

// Send message that might trigger both tool calling and auto-turn
client.chat.send("Hey, I'm planning a road trip next week. Can you check the weather for San Francisco, Portland, and Seattle?");

Best Practices

Tool Design

// Good: Clear, specific tool description
{
  name: "get_stock_price",
  description: "Get the current stock price for a specific company by ticker symbol",
  parameters: {
    type: "object",
    properties: {
      ticker: {
        type: "string",
        description: "Stock ticker symbol (e.g., AAPL, GOOGL)",
        pattern: "^[A-Z]{1,5}$"
      }
    },
    required: ["ticker"]
  }
}

// Bad: Vague description, unclear parameters
{
  name: "get_data",
  description: "Gets some data",
  parameters: {
    type: "object",
    properties: {
      input: { type: "string" }
    }
  }
}

Error Handling

// Comprehensive error handling
async function safeToolExecution(toolCall: any) {
  try {
    // Validate input
    if (!toolCall.function.name) {
      throw new Error('Missing function name');
    }

    // Parse arguments safely
    let args;
    try {
      args = JSON.parse(toolCall.function.arguments);
    } catch (e) {
      throw new Error('Invalid function arguments JSON');
    }

    // Execute with timeout
    const result = await Promise.race([
      toolFunctions[toolCall.function.name](args),
      new Promise((_, reject) => 
        setTimeout(() => reject(new Error('Tool execution timeout')), 30000)
      )
    ]);

    return {
      tool_call_id: toolCall.id,
      role: 'tool' as const,
      content: JSON.stringify({ success: true, data: result })
    };

  } catch (error) {
    return {
      tool_call_id: toolCall.id,
      role: 'tool' as const,
      content: JSON.stringify({ 
        success: false, 
        error: error.message,
        timestamp: new Date().toISOString()
      })
    };
  }
}

Security Considerations

// Validate and sanitize tool inputs
function validateToolInput(functionName: string, args: any) {
  const allowedFunctions = ['get_weather', 'calculate_tip', 'search_database'];
  
  if (!allowedFunctions.includes(functionName)) {
    throw new Error(`Function ${functionName} is not allowed`);
  }

  // Sanitize string inputs
  if (typeof args === 'object') {
    for (const [key, value] of Object.entries(args)) {
      if (typeof value === 'string') {
        args[key] = sanitizeString(value);
      }
    }
  }

  return args;
}

function sanitizeString(input: string): string {
  // Remove potentially dangerous characters
  return input.replace(/[<>\"'&]/g, '');
}

Next Steps

Image Generation

Combine tool calling with automatic image generation

Event System

Handle tool calling events in your application

Auto-Turn Conversations

Natural conversation flow with tool calling

Authentication Setup

Secure your tool calling implementation

Get Started

SDK Features

Advanced SDK

REST API Integration

Models

Basic Tool Calling

Defining Tools

Implementing Tool Functions

Making Tool-Enabled Requests

Advanced Tool Calling

Complex Tool Definitions

Tool Choice Strategies

Error Handling in Tool Calls

Tool Calling with Chat History

Maintaining Context

Real-World Tool Examples

API Integration Tool

Database Query Tool

File System Tool

Tool Calling with AutoTurn

Best Practices

Tool Design

Error Handling

Security Considerations

Next Steps

Image Generation

Event System

Auto-Turn Conversations

Authentication Setup

Get Started

SDK Features

Advanced SDK

REST API Integration

Models

​Basic Tool Calling

​Defining Tools

​Implementing Tool Functions

​Making Tool-Enabled Requests

​Advanced Tool Calling

​Complex Tool Definitions

​Tool Choice Strategies

​Error Handling in Tool Calls

​Tool Calling with Chat History

​Maintaining Context

​Real-World Tool Examples

​API Integration Tool

​Database Query Tool

​File System Tool

​Tool Calling with AutoTurn

​Best Practices

​Tool Design

​Error Handling

​Security Considerations

​Next Steps

Image Generation

Event System

Auto-Turn Conversations

Authentication Setup

Basic Tool Calling

Defining Tools

Implementing Tool Functions

Making Tool-Enabled Requests

Advanced Tool Calling

Complex Tool Definitions

Tool Choice Strategies

Error Handling in Tool Calls

Tool Calling with Chat History

Maintaining Context

Real-World Tool Examples

API Integration Tool

Database Query Tool

File System Tool

Tool Calling with AutoTurn

Best Practices

Tool Design

Error Handling

Security Considerations

Next Steps