Create a New Workshop and Add Prompts and Test Cases

Workshops allow you to test a set of prompt variants against a set of test cases. The prompt variants might include different instructions in the body or use different models or model parameters.

The test cases will test different variable values for all the prompts in the workshop. All prompts used in a workshop must include the same set of variables.

When the tests are run, a test result will be created for each test case for each prompt. You can then compare the results to determine which prompts are performing correctly, then iterate on the prompts and run the tests again to repeat the process and optimize a prompt to use in production.

To create a new workshop, click on the "Workshops" tab at the top of your screen when logged in.

This will take you to the workshops index which will show you all the workshops in your team account. To create a new workshop, click the "New Workshop" button.

Enter a name for your workshop and then add any prompts you'd like to test in this workshop. The prompts you test must take the same set of variables.

Click the "Create Workshop" button to save the workshop.

Test Cases

Add a test case by clicking the "Add a New Test Case" button. Fill in the values you'd like to use to test against the prompts included in your workshop.

After clicking the "Create Test Case" button you will be redirected back to the workshop. To run the tests, click the "Run Tests" button. The tests will run in the background and populate on the Test Case Results view when they have completed.