Experimentation And Discovery

Experiments is what we consider to be Libretto's killer feature - a tool that allows you to automatically explore dozens, or even hundreds, of prompt variations within minutes. Alongside our evaluation tools, this makes it simple to boost prompt accuracy without any of the usual guesswork.

Experiment Types

We offer multiple types of Experiments, and we're constantly adding new methods - from both internal and cutting-edge external research - to discover improved Variants.

Few Shots

A Few-Shot Experiment will inject a mix of test cases into your prompt to determine which ones perform best. This better instructs LLMs how to format its response in your desired format and often gives it a better sense of what kind of content output you want.

Magic Words

Magic word variations strategically add specific words or phrases into your prompts. These act as cues to the LLM to reframe their response with more relevant and high-quality. results.

Model Version

Testing your prompt on different models is not only tedious, but with the speed and veracity of AI development, it's incredibly difficult to stay on top of newly released models. This Experiment lets a user easily compare their prompt across models, with their desired metric in mind.