The User Experience Challenges of Large Language Models like ChatGPT
User Experience (UX) has dramatically influenced the evolution of interfaces, enabling users to interact with technology efficiently and naturally. From the advent of graphical user interfaces (GUIs) and touchscreens to voice assistants and gesture controls, UX-driven innovation has led to progressively more accessible and user-friendly interactions. We are facing a new paradigm shift with the introduction of AI tools like ChatGPT, which requires a UX examination of how we interact with these new tools.
Large language models are powered by advanced artificial intelligence algorithms that have revolutionized our interactions with technology. However, despite their remarkable capabilities, users often need help leveraging these models. This article explores four user experience challenges commonly faced by users when interacting with large language models like ChatGPT and provides insights on overcoming these obstacles.
Understanding the limitations of AI tools
One UX challenge users encounter when working with large language models is the need for more understanding of the tool's limitations. Although ChatGPT and other tools like it, such as Google’s Bard, have been trained on a vast dataset, they are not infallible and can sometimes produce inaccurate or nonsensical responses.
To address this UX challenge, there are two paths, the first being making the tools more accurate and avoiding mistakes to begin with. We are seeing the first steps of integrating plugins to ChatGPT, which gives it superpowers so it can search the entirety of the web to find up-to-date answers or use tools like Wolfram Alpha to provide more detailed results. Plugins will significantly improve the accuracy of these tools and allow them to search the web and provide a referenced response that can give users more trust in the accuracy of AI’s responses. This may be the simplest example of plugins providing more accurate and valuable functionality. How AI will integrate with existing tools has significant implications as the technology matures.
The second path recognizes that the tools are fallible, and even when they can use additional plugins to assist, they may still be inaccurate. How can we design these chat interfaces in a way that helps the users understand their limitations? This is a complex question, as slapping a warning or information banner warning users of inaccuracy does not feel like an eloquent solution. Could we instead provide feedback to the user when the AI output is likely to contain errors? One idea of how a feature like this could work is when a user inputs a prompt, the AI would run a specialized analysis of the prompt and compare it to the types of prompts that do not often work well. This would make the feedback to the user more intelligent and help the users learn the tool's limitations.
Crafting effective prompts
Is talking with an AI chat tool the same as talking with another human? In some ways, it can be but in others, not so much. These AI chatbots work by predicting the next most likely word based on the prompt. Because of this, we are seeing a trend toward prompt engineering that uses specific strategies to get your desired result. This is not how people communicate with other people. However, a new user might need to understand that how you craft your prompt to the AI is directly connected to the result you get.
Can we design these AI tools to help users create effective prompts within a chat interface, enabling them to maximize the potential of tools like ChatGPT? One strategy that could be investigated would be to analyze the user's prompts and predict what they are trying to accomplish, then suggest adding additional input to the prompt. One example is if the user wants to write an article, the chatbot could suggest modifying the prompt by adding a prompt modifier that asks the AI to ask follow-up questions to better refine the details of the article. This can help ChatGPT gather additional information about the user's desired outcome leading to more accurate and relevant responses. I envision this working similarly to a tool like Grammarly that helps users write more effectively by suggesting tone, spelling, and grammar improvements.
By suggesting common prompt engineering strategies, users can better understand how to communicate with tools like ChatGPT, ultimately leading to a more productive and satisfying user experience.
Overcoming preconceived notions about AI capabilities
AI is exploding faster than anyone predicted. The writing on the wall is they will change the world. However, humans have been interacting with computers in a specific way for many decades and have built mental models of how they work. This causes us to approach these advanced AI tools with preconceived notions. Some users may underestimate the capabilities of tools AI chatbot tools due to these preconceived notions about what a computer chat is capable of and the quickly changing abilities. How can we help users take full advantage of these tools by better-informing users of what they can do?
Luckily these chat tools are built in a way that mimics how we communicate with other people. We have been using tools like SMS text messaging, AOL, Sykpe, and Slack to communicate with people since the dawn of the internet. We don’t expect an AI chatbot that understands what we are asking. We have been trained that computers only accept a narrow range of inputs, and we must stay within those bounds to use them effectively.
However, because it is a chat interface, there are very few affordances on how to use the tool effectively, and it leaves users wondering if they can ask it for “XYZ” or if it can do “ABC.” Today, the answer to that challenge is for the user to submit their prompt and see what they get. Through trial and error, the user discovers what works and what doesn’t. Can we flip that around so the users can train the tool to provide better results?
Considering how people communicate their value to others, it typically comes in a very organic way. When people meet someone new, they often start with small talk until there is some more profound connection point. From there, a relationship is established, and they slowly learn about each other, eventually allowing for value exchange. Is this what we want from AI tools? In most cases, no, we are looking for help with specific tasks, and understanding what the tool can do quickly is a pillar of usability.
AI in the form of chatbots is a very natural model for general interaction but may not be suitable for productivity in specific tasks. There are opportunities to give a more robust user interface that allows AI tools like ChatGPT to perform tasks in the task-specific context. A straightforward example would be incorporating AI into tools like Photoshop so that users can quickly replace the tree with a bicycle within the context of a photo manipulation tool. Adobe is already making strides with their Firefly product and integrating it with their existing tools like Photoshop. This leads us to the final challenge in this article with AI tools.
Integrating ChatGPT into existing workflows
Many people have tried ChatGPT, but a minority have found ways to integrate ChatGPT into existing workflows. Plugin support will be a game-changing feature for integrating ChatGPT into people's workflows. However, are there ways to help users understand how they can leverage tools like ChatGPT?
To start incorporating these powerful tools, users need to identify tasks within their workflows that could benefit from ChatGPT's capabilities, such as brainstorming, content creation, or fact-checking. Only when ChatGPT gets incorporated into existing products as an underlying technology will people be able to incorporate them into their day-to-day work. However, once tools get built with embedded AI assistants, we may start seeing users connect the dots on how to leverage them. Githubs AI Co-Pilot is a great example of this starting to happen. Co-Pilot enables programmers to get suggestions and assistance from AI while writing code.
As adoption grows, I predict less need for specific applications to provide context. In the early days of the iPhone, we saw a boom in skeuomorphic design trends that helped users bridge the gap to the new paradigm of using a phone to replace many tasks. Today's apps with AI integration will bridge the gaps in how we interface with AI tools. As we become more comfortable using a ChatGPT-like interface, it could revolutionize how we interact with technology.