How to use GenAI to ask better questions & improve results

GenAI has forced its way into many peoples’ minds over the past few years, partly due to its incredible ability to answer natural language questions, ease of use and quality of replies. However, it’s still a tool that’s limited by the person using it and needs care in use. I’ve chosen a light-hearted example to examine how simply improving prompt engineering can yield better results.

Impact of using GenAI to ask the right questions

It’s five years now since lockdown hit and it seemed quizzes became a key (and sometimes only) part of peoples’ social lives. I remember being part of at least three weekly quizzes during those spring lockdown months as everyone stepped up their efforts to see each other on their laptop screens and scrabbled around to find someone with a good enough Zoom licence to allow longer than 20 minutes per call. They were a great way to have fun while staying isolated, but there was always the dread when it was your week to write the quiz and you had to research questions that weren’t too easy or too difficult. How much better would it have been if we had the likes of ChatGPT and CoPilot for our quarantine quizzes? Just how good are these exciting new tools at writing a perfect pub quiz?

Discussions about what makes a perfect quiz could take as long as a quiz itself, so for the purposes of this blog, I’ll stick to a simple and achievable definition. Across the 20 questions I’m going to ask for, there should be a mix of subjects and a range of difficulties. For subject range, I’ll categorise each question into its closest Trivial Pursuit category (Geography, Entertainment, History, Art & Literature, Science & Nature, and Sports & Leisure). I’ll also classify each question into a difficulty category based on how hard it is for a pub quiz environment– while not an exact science, it’ll be a good estimate of how hard the quiz is.

“Can you write me a pub quiz of twenty questions on a range of general knowledge topics please?”

Firstly, I ask ChatGPT to write me a twenty-question quiz. It came up with the following range of questions:

A good first attempt, but not particularly varied in category of question or difficulty. A wrong answer is also a blot for this first attempt. Many questions are geography based, three of which are linked to Japan. I’ve also had a lot of these in pub quizzes I’ve attended in the last year so I would suspect that this is drawn from a relatively small pool of questions for a basic request.

A perfect pub quiz needs more variation than this.

“Can you make it a bit more difficult and split into these 6 topics: Science & Nature; Arts & Literature; Geography; History; Entertainment; Sports & Games”

This is a much harder quiz, perhaps with too many tricky questions to make an engaging and entertaining evening in a pub. Some of the answers are wrong, or at least contestable enough based on different online sources to be the sort that a diligent question-master would want to avoid. It also hasn’t stuck to the original twenty question prompt. Let’s have one more try:

“Some of these are a bit too tricky. I’d like the quiz to be entertaining and engaging without being too easy or difficult. Can you try another set of twenty questions please, still split by the same categories?”

This is much better and a great framework for a quiz. While this seems like a trivial (pun-intended) and light-hearted exercise, it acts as a great example of GenAI and how best to use it.

Outcomes that businesses incorporating GenAI can expect

AI is a tool that can save huge amounts of time and effort and quickly expand the potential of its user. However, it’s still a tool that needs human input to get the best results. There’s a risk of getting the same results as everyone else, where sometimes more nuance and differentiation is needed. Every business offers a different proposition every customer presents a different need and it’s important to pick up on that subtlety. Effective and intelligent prompt engineering gives back much more effective and intelligent answers.

The thing that makes a great quiz evening is not the presence of questions, but how they’re delivered and how entertaining and how varied they are. GenAI rids us of the tedious work of researching the questions, but it still needs a careful, experienced hand to optimise the solution it delivers.

How CACI can help

If GenAI has created more headache than help for you or your business, CACI can support your understanding of it and ensure it is used in the most effective way. To learn more about how we can support you, contact us today.