The search engine based on ChatGPT appears to be very sensitive to manipulation. This is possible with, for example, ‘prompt injection’. As a result, users of the search engine get to see a manipulated answer.
Recently, OpenAI launched its ChatGPT-based search engine ChatGPT search for subscripted end users of the AI assistant. According to the developer, this prompt-based search engine helps get much better search results than existing search engines. Users are, therefore, urged to use ChatGPT search as their default search engine.
ChatGPT search to be manipulated
However, recent research by The Guardian shows that the search results provided by ChatGPT search can be easily manipulated. That can create significant security risks.
Among other things, ChatGPT’s search functionality responds poorly when prompted to summarize web pages that contain hidden content. This hidden content may include, for example, an obligation to change the responses given to ChatGPT. This is also known as prompt injection. In addition, the deliberate instructions in the hidden content can also cause the ChatGPT answers to be nonfactual and, for example, highlight the benefits of products.
This susceptibility to manipulation can also be used for malicious activities. ChatGPT can thereby, for example, surface and spread malicious code from surveyed websites.
High risks
Security experts comment on the research and indicate there have long been risks to combining search functionality and LLMs. Therefore, the answers provided by AI tools should not always be trusted.
Recently, a Microsoft security specialist described an incident in which a crypto-enthusiast used ChatGPT for programming assistance. In the code provided by ChatGPT was a section that, he said, described a legitimate way to access the Solana blockchain. Instead, this turned out to be malicious code that caused the developer’s login credentials to be stolen, thereby stealing $2,500 in cryptocurrency.
Time needed to improve
Despite the preliminary risks of ChatGPT’s new search functionality, security experts do believe they will diminish over time. They indicate that the current version of ChatGPT search is the first development in this area.
OpenAI did not respond to questions from The Guardian.
Also read: OpenAI launches o1 reasoning model exclusively for top developers