ChatGPT Language Capabilities: The Breakdown

A few years ago, the introduction of ChatGPT to the world raised a lot of questions that were once thought of as fantastical, but are now becoming more and more of a reality in the translation community: Will AI Replace Human Translators?

As a large language model developed by OpenAI, ChatGPT has gained significant attention for its ability to engage in human-like conversations and generate creative text formats. But how proficient is it in handling different languages? Let's explore its current language capabilities.

How Many Languages Does ChatGPT Support?

Since ChatGPT is a constantly learning piece of technology, it can be trained to understand and generate text in numerous languages. It can also detect native languages, allowing it to understand and provide responses automatically in a wide variety of languages. This makes ChatGPT accessible and useful to a global audience.

ChatGPT currently supports over 80 languages. This includes popular languages like Chinese, Spanish, English, and many others. However as the model gathers more and more data from users, this might change in the future. As an additional note, while ChatGPT can support these languages, it doesn’t mean that the model can do it well. Accuracy will still be subject to how much information ChatGPT has.

List of Languages Supported by ChatGPT

Testing ChatGPT in Spanish, German, and Filipino

To check ChatGPT’s current capabilities, we put it through its paces with multiple languages and see how it stacks up against native speakers. We considered a few things to test the tool but arrived at two inherently human communication abilities: telling stories and debating a topic.  We’re also using ChatGPT’s 3.5 version, as it’s more accessible and free for use by the typical user.

Spanish

For Spanish, we started first with a story, just a personal anecdote on how generosity and sharing have helped them in life.

This is what ChatGPT came up with.

The story’s about a college student who spent most of their money on their school needs but now lacks the ability to buy more essential needs like food and transportation. A classmate eventually became their savior. It’s a straightforward plot. Grammar and punctuation get a pass, and ChatGPT stayed on topic, but it lacked the “personal touches” or “special details” that make it more believable or relatable, such as the classmate’s name, or how they became friends. ChatGPT also did not describe any human emotions or feelings about receiving such generosity from other people, which is what people would typically explain when sharing such experiences.

As for the debate, we asked ChatGPT to discuss a political topic: giving amnesty to the Catalan separatists in Spain, and to present both sides: support and opposition. We went even more specific, asking the tool to debate in favor of amnesty.

This is the result.

To start, ChatGPT didn't present both sides nor did it give a brief on what the issue is about. It presented only key reasons to support the amnesty deal. One of its reasons actually sounds in favor of the opposition, as the amnesty would promote reconciliation by “reintegrating” the separatists into Spanish society, which defeats the separatists’ main objective— independence of the Catalan region.

Overall, ChatGPT cannot provide clear-cut, coherent points when debating in Spanish. It doesn’t provide any context to the issue, and while it does stick to our prompt, it fails to give a proper look into both sides of the topic. At the very least, ChatGPT is not in favor of civil disobedience and provides points that favor a more peaceful reconciliation. 

German

As for German, we went with a different approach to storytelling, asking a very simple prompt: tell us the story of the first time you met your baby sister.

Here’s the result:

It appears that ChatGPT has some semblance of “awareness”, in the sense that it cannot tell this particular experience because as a tool, it has nothing on its own that it can provide. Instead, it offers to tell us a fictional story with the same theme, which tells us that ChatGPT understands its limitations and capabilities.

When it comes to debating in German, we asked it to present arguments for and against the ideologies of the Alternative for Germany (AfD) party.

Surprisingly, ChatGPT gave reasonable, neutral answers to this query.

While it gave 4 arguments against AfD, and only 3 arguments for AfD, ChatGPT didn't include subjective political opinions or personal beliefs which might have been provided by other users. The way ChatGPT structured various ideas may help those who have very little idea about the topic, assisting them to start forming their own opinions and to research more since it was able to point out key points for both sides.

Filipino

For Filipino, we returned to testing its storytelling abilities based on a specific kind of storytelling known in the Filipino language: a talinhaga, or a parable, as it’s more commonly known in English. We provided a lesson to learn: how to become generous to other people.

ChatGPT gives us this story:

While it does provide a more creative slant to the tale, such as making the protagonist a bird called “Pipit” who helps a family of fireflies by sharing with them fruits and other types of food, there are some inaccuracies to the tale, such as fireflies only being able to eat insects as food. The sentence structure is also awkward in some places, as there is a section where Pipit is “having its wings flown” as opposed to just “flying”. ChatGPT also sticks to the theme of the prompt, such as explaining the meaning of the parable at the end, though like in the Spanish test, it lacks “relatable” and “believable” touches to the story.

As for its debating skills, we asked to give arguments for and against a change in the Philippines’ current governmental legislative body from a constitutional republic to a parliamentary one.

This is how it answers the query:

As you can see, while it does deliver its points in Filipino, ChatGPT fails to translate the words in bold to their Filipino counterparts. While this won’t be an issue for locals as they are largely bilingual in both Filipino and English, the expectation is that ChatGPT will be able to provide answers in as “pure” Filipino as possible, especially with a prompt that lacks code-switching.

Conclusion

In total, ChatGPT's language capabilities depend on the information it can access, despite its wide range of support. Topics that are widely available online give more accurate answers, though it can still be a case of hit-or-miss, depending on how well the prompt is given, the industry ChatGPT is operating in, and its reasons for generating content, which is all provided by the user.

However, its potential for communication and education is undeniable, especially as it continues to evolve through human participation and touch. Stay up-to-date with the latest language news and trends by following Tomedes' blog.