Show HN: Reprompt – Collaborative Prompt Testing (opens in new tab)

We recently developed an AI app, but we struggled to write good prompts for it. Although GPT3.5 is affordable, we found it challenging to provide effective instructions. We often made changes to our prompts that unintentionally led to poor performance.

Testing prompts is the solution, but existing tools make this task difficult. You must generate an output, review it, delete it, and generate again, which is time-consuming. After only a few rounds of testing, you may think that you have a bulletproof prompt without realizing its flaws.

To address this issue, we developed Reprompt, which enables you to generate multiple outputs simultaneously. This feature allows you to test more data and find inconsistencies more easily. Instead of just testing a few times, you can test dozens of times.

Moreover, Reprompt automatically analyzes all the outputs, eliminating the need to review each response individually. You only need to specify the expected output, and Reprompt will analyze all the responses and provide a score. This score enables you to determine whether your prompt is improving or not.

Lastly, Reprompt lets you share prompts with your team in an easy way so that you can all work together to improve them.

Give it a try and let us know your thoughts. Thank you!