About ChatGPT

At the end of 2022, the US company OpenAI released the chatbot ChatGPT and since then the system has been available to all internet users free of charge. ChatGPT is a chatbot, i.e. software that simulates a human conversation. There is a simple input window on the website, just like a search engine. Anyone who registers on the site can type in any question free of charge.
The special thing about ChatGPT is that the system generates texts in just a few seconds that at first glance can hardly be distinguished from those written by real people. The bot writes poems, essays, news texts, song lines and even program code.

This chatbot is based on the GPT-3.5 language model (GPT stands for Generative Pre-trained Transformer), an improved version of GPT-3, which also comes from OpenAI. GPT is based on Transformers, a machine learning model introduced by Google Brain, and was trained through self-supervised learning. Specifically, the task during the training was to predict the next word of a text excerpt. This text can come from a variety of sources, such as online forums, social media, newspaper articles, books and spoken language. This step is also referredto as pre-training, as a model has been generated that cannot yet fulfill the desired task, but the model parameters obtained represent favorable starting conditions for further fine-tuning.

It is important to note that the knowledge of this chatbot is limited to any internet sources and other sources until 2021.

More about ChatGPT here on SRF Tagesgespräch.

About GPT-3 or GPT-3.5

Companies and media reporting on ChatGPT often do not make a clear distinction between the GPT-3 or GPT-3.5 language model and the publicly known ChatGPT. However, it is important to understand that ChatGPT is the chatbot from OpenAI, which can be used by all registered users free of charge or with a paid Pro version.

GPT-3 or GPT-3.5 is the language model that is in the chatbot, so to speak, and is responsible for the fact that OpenAI’s chatbot has such a human form of expression.

It can therefore be said that ChatGPT is a possible application of the GPT-3 or GPT-3.5 language model.

How does GPT-3 work?

When asked how the language model works, the chatbot from OpenAI answers as follows:

GPT-3 works on the basis of deep neural networks that are trained on large volumes of text. It uses a special type of neural network known as a “transformer” to generate texts.

First, GPT-3 is trained with an enormous amount of text taken from books, articles, websites and other sources. The model analyzes these texts in order to recognize and understand patterns and connections in the language.

When you enter a text or make a request, GPT-3 applies the knowledge it has learned to generate an answer or text. Words and sentences are selected on the basis of patterns and contexts learned during training.

The model also uses a technique called “language modeling” to predict the probability of each next word in a text. This helps him to generate coherent and meaningful sentences.

In addition, GPT-3 can also draw on previous inputs or responses to provide context-related answers. In this way, it can hold conversations or perform tasks based on previous requests.

Overall, GPT-3 combines the ability to recognize and understand complex speech patterns with a huge dataset of texts to generate natural language texts and responses.

Who trains GPT bots?

As described above by OpenAI’s chatbot “ChatGPT”, the language model GPT-3 must first be trained. And this is precisely the crucial difference to ChatGPT. ChatGPT is, so to speak, a ready-made chatbot that has been trained by OpenAI. GPT-3, i.e. the language model, initially comes untrained and must then be trained as part of the development of the respective application, such as the chatbot.

At this point, it should be noted that OpenAI has trained its ChatGPT itself. Companies and consumers can therefore only use this bot as it has been trained. However, if we are talking about the model, i.e. GPT-3, companies can carry out the training themselves and thus influence the answers.

Is GPT a black box?

ChatGPT is often referred to as a black box. The reason for this is that nobody knows the exact training data and neither users nor companies have any influence on the answers. This means that companies have no control over what OpenAI’s chatbot ChatGPT tells their customers. As a result, the use of the ChatGPT chatbot seems somewhat risky for some companies at the moment. Companies fear a loss of control and have too much respect for the supposed black box. The issue of data protection or liability has also not yet been clearly clarified in all cases.

However, this does NOT mean that companies cannot benefit from the new developments surrounding GPT. Rather, thanks to ChatGPT, companies can see what a well-trained chatbot with the GPT-3 model can answer.

To repeat, the GPT-3 language model can be trained “individually”.

And now?

ChatGPT and GPT-X are not the same thing. Companies should be careful about using the existing chatbot ChatGPT without further consideration of data protection and loss of control. Reason: ChatGPT is trained by OpenAI.

Used with care, however, the following use cases could be suitable for the use of ChatGPT:

• Integrate the answers from ChatGPT into your own chatbot and display them as a default option if your own bot does not know an answer. However, always indicate when the reply was generated by ChatGPT.
• Integrate ChatGPT’s answers into the company’s own chatbot as part of simple small talk or knowledge questions in order to be able to have more fluid conversations with users and find out more about user preferences in the dialog.

However, when it comes to the GPT-3 or GPT-3.5 language model, it is initially “only” a language model, without any knowledge. When using the pure language model GPT-3, companies themselves are required to maintain data protection and also to take certain ethical requirements into account. The control over the content lies with the companies themselves.

Fancy brainstorming about the use of ChatGPT or GPT? Then simply send a short e-mail to sophie@hundertmark.ch.