How much information does ChatGPT get right? This week I tested ChatGPT using the prompt “In 250 words, what are the key differences between books and video games?” and while it wasn’t necessarily wrong in what it said, it left out a lot of important information. This was a learning experience for me because I haven’t played with ChatGPT that much. I’m very anti-genAI so it didn’t feel the best using it but I understand that it can be helpful to others. I had a hard time finding a prompt where it gave me enough information to work with.
A few years ago, it was easy to know when ChatGPT was wrong but it has been given more information and has been able to learn as users use it. The first few prompts I fed it, it came back with no factual errors so I had to rephrase or use a different subject. The prompt I landed on was a little bit vague and it didn’t take into account other forms of media like audiobooks or ebooks or what is considered a video game. For the most part, it was correct on the key differences between both mediums. They are both tools for storytelling and they have different ways that they engage with their audience.
I tested the program without having an account and I noticed it did give more basic answers, they weren’t as thorough and didn’t give any sources. Once I signed up for an account though, it gave more information on where they pulled their sources from (at the bottom by the “copy text” button, new buttons appeared, one showing sources).
Overall, I still don’t trust ChatGPT to get my information, it’s a helpful tool or source if I’m looking for something but don’t necessarily know how to word it. After asking it questions about popular books and TV series, it was able to tell me a lot of information about them which makes me think it was trained on that media which is kind of scary to think about.