Honeyhive integration

Add datapoints to dataset

Tool to add datapoints to a dataset. Use when you need to append multiple entries with specified input, ground truth, and history mappings.

Compare Experiment Runs

Tool to retrieve experiment comparison between two evaluation runs. Use when you need to analyze the differences in metrics, datapoints, and events between two runs.

Compare Runs Events

Tool to compare events between two experiment runs side-by-side. Use when analyzing differences in model behavior, performance metrics, or outputs between evaluation runs. Returns matched event pairs with their respective data from both runs for comparison.

Batch Create Datapoints

Tool to create multiple datapoints in a single batch operation. Use when you need to bulk-import events into a dataset or create many datapoints at once. Supports filtering by date range, event IDs, or custom criteria. Efficient for migrating large numbers of events to evaluation datasets.

Create Batch Model Events

Tool to create multiple model events in a single request. Use when you need to log a batch of event interactions to HoneyHive.

Create Batch Tool Events

Tool to log a batch of external API calls as tool events. Use when you need to record multiple tool events in one request—use after gathering all event data.

Create Configuration

Creates a new configuration in HoneyHive for managing LLM or pipeline settings. Use this to define reusable configurations with specific models, prompts, and parameters that can be deployed across different environments (dev, staging, prod). Configurations enable version control and environment-spe…

Create Datapoint

Tool to create a new datapoint with input-output pairs. Use when you need to add a single datapoint with inputs, ground truth, conversation history, and metadata.

Create Dataset

Tool to create a dataset. Use when you need to initialize a new dataset within a project.

Create Event

Tool to create a new event in HoneyHive to track execution of different parts of your application. Use when you need to log a model call, tool execution, or chain step. Events can be grouped into sessions and nested hierarchically using parent_id and children_ids.

Create Metric

Tool to create a new metric in HoneyHive. Use when you need to define how to evaluate model outputs, whether through code (PYTHON), AI evaluation (LLM), human review (HUMAN), or combining multiple metrics (COMPOSITE). Important: LLM metrics require both model_provider and model_name to be specified.

Create Model Event

Tool to create a new model event to log LLM call data. Use when you need to track a single model interaction including messages, responses, usage, and metadata.