Someone tell ChatGPT to stop hallucinating.

Let's talk about this phenomena found in LLMs.

 

Edition #5. Approximate read time: 10 minutes

Hey there!

Welcome to Edition 5!

I’m calling this edition You May Be Hallucinating (or not). Here’s what’s in store:

  • The Writing: Large language models (such as ChatGPT) love to make things up. Here, I discuss the impacts of these hallucinations and how you can go about minimizing them.

  • AI Tools: I’m going to repurpose this section. For this edition, I decided to leave it out, but anticipate it to make a comeback in a future edition in a different manner.

  • Midjourney Images: As always, I prompted Midjourney a few times to get some sweet images relating to this edition.

Oh, and there’s something part way through that I think you’ll find interesting - it really puts this concept of hallucination to the test.

Grab your cup of coffee, it’s time to dig in.

 

 New here? Grab a cup of coffee - we’re talking AI here. The content that’s in this newsletter is curated towards the integration and applications of AI in our lives, so if this sounds like your kind of thing, hit that “subscribe” button below!

 

Want to advertise in this newsletter?

If you are looking for an opportunity to advertise your product, course, service, or whatever and it’s in the domain of artificial intelligence (or coffee!), please visit the advertising page (link) or respond to this email to inquire further about advertising/sponsorship opportunities.

 

Large language models (LLMs) such as ChatGPT seem to be incredibly smart. In fact, ChatGPT has passed a collection of difficult exams, which include the bar and top-tier medical exams (link).

Despite the vast knowledge and articulate responses these types of models have, LLMs are notorious for hallucinating, which is a phenomena where it responds with false or nonsensical information.

Whether if you’re in academia, marketing, or even software engineering, you most likely will come across this phenomena if you haven’t yet when messing around with generative AI applications. Let’s talk about it further about it:

What is a hallucination?

A hallucination is when the model thinks that the output it gives is correct but is false and/or not coherent. One of the big reasons why ChatGPT took off back in early 2023 was due to the “knowledge” it had.

I decided to test ChatGPT’s knowledge on one of the bands I listen to, Gloryhammer. I fed it the opening lyrics to one of the songs minus the character name to see if it could guess the lyrics correctly:

I fed the same prompt to Claude 2:

They’re both factually wrong. ChatGPT correlates being the character of this song to Macbeth. According to Claude 2, Prince Hubert is a character in Camelot. After some googling, I don’t think Hubert is in Camelot!

Hallucinating poses a risk for credibility for not only the models, but for the company who makes the model from the public perspective.

Why is hallucination bad? Credibility, for one.

Imagine a scenario where a barista tells you multiple times they don't have sugar, when it's clearly available over on the condiment stand next to the register. You may question their reliability since it’s clearly available.

Over time, repeated false claims, such as this one, undermine trust and ultimately build up to a point to where the credibility of the coffee shop is damaged.

The same can be said about users of the LLM. When a LLM hallucinates over multiple iterations of either a single task or a multitude of tasks, credibility of the LLM is damaged due to the pattern of false information. This compounds over time, which in turn could damage the reputation of the company or the technology because it can be perceived as untrustworthy and unreliable.

Hallucinations have real-world impacts. For instance, a lawyer was recently sanctioned (link) by a judge because they cited non-existent court cases that GPT generated. In addition, a paper by Hussam Alkaissi and Samy I McFarlane (link) was published talking about hallucinations and how they impact science writing.

Now that I know about this, how can I prevent ChatGPT from hallucinating?

Preventing hallucination entirely is difficult and something that you can avoid, but there are measures you can put into place to minimize the probability of having the LLM hallucinate:

  1. Context is your friend. By providing the LLM as much context to the problem at hand as possible, it doesn’t have to make assumptions when generating the output.

  2. Use example outputs, if applicable. For instance, if you want the output to be formatted a certain way, prompt the LLM with the formatting style.

  3. Be specific with your prompts. I discuss in Edition 1 (link) how to craft “the perfect prompt”. Here’s a snippet from the edition that’s relevant to the discussion:

Let’s break down what I like to call “the 3 components of prompt crafting”, which are clarity, completeness, and specificity. In an ideal scenario, you’re going to end up [using all 3]…

… For instance, the prompt “Discuss how Python handles memory management”, leaves out the “completeness” component of the trio, as we didn’t define the target audience, which can lead to a vague output.

Edition 1 of Bytes and Brew

Just remember to always verify if the output is factually correct to avoid running into conflicts. Also keep in mind that this isn’t limited to ChatGPT - this is an issue that spans across multiple LLM’s.

Happy prompting!

Enjoy this edition so far? Here’s 2 things you can do:
1) Respond to this email and let me know your thoughts. I aim to respond to all emails.
2) Share Bytes and Brew with a friend.
Click the button below to share your referral code!

 

Prompt

“A representation of the brain emitting or immersed in vibrant, positive colors, symbolizing diversity, creativity, and positivity of thoughts and ideas.”

 

 

Prompt

Depict a cutaway shot of a sailing ship navigating through a sea of stars, nebulae, or abstract patterns, symbolizing exploration and discovery in the vast ocean of the mind. Sticker style.”

 

Currently, ChatGPT has an interesting… feature? If you prompt 1000 “a” characters with spaces between it, it’ll spit out some extremely random text with no context and just kinda… do it’s thing? Great example of hallucination!

 

Prompt

“Portray a figure gracefully dancing amid a swirl of colors and shapes, illustrating the beauty and fluidity of dreamlike visions”

 

I truly enjoy hearing from my audience - don’t hesitate to reach out to me and let me know what you thought of this edition!

 

Did you enjoy this edition?

Click/tap on your response. Any feedback is super helpful!

Login or Subscribe to participate in polls.

 

☕️ See you next edition 😃 

The next edition will be delivered to your inbox on Monday, August 21st

Join the conversation

or to participate.