OpenAIs artificial intelligence-powered language model ChatGPT has undoubtedly revolutionised the way people search for answers on the internet, but a new study has found that ChatGPT is getting markedly slower and less accurate over since its launch.Researchers at Stanford and UC Berkeley have found that the large language model (LLM) of Open AI – the makers of ChatGPT – has degraded significantly over time in terms of performance and accuracy, and as a result, the performance of ChatGPT has deteriorated.Researchers find ChatGPT degrading in all benchmarksThe researchers involved in the study analysed the different versions of ChatGPT from May 2023 and June 2023 – which employed the GPT-3.5 and GPT-4 models – and used them to perform four primary tasks.The four tasks involved solving mathematical problems, answering sensitive questions, generating code and assessing the models on visual reasoning.The results were clear – the GPT-4 model, and ChatGPT as a result, has degraded in accuracy and speed.In the first benchmark of solving mathematical problems, the researchers tasked the AI bot to identify prime numbers. While the March version of ChatGPT-4 was capable of identifying prime numbers with a 97.6% accuracy rate, the same model in June could correctly answer only 12 questions – a success rate of 2.4%.In terms of code writing, the March version could write code that ran without any changes in 52% of cases, while the June version could only achieve that feat in 10% of the cases.The newer versions of ChatGPT were also more likely to decline to answer sensitive questions.GPT-4 is getting worse over time, not better.Many people have reported noticing a significant degradation in the quality of the model responses, but so far, it was all anecdotal.But now we know.At least one study shows how the June version of GPT-4 is objectively worse than… pic.twitter.com/whhELYY6M4— Santiago (@svpino) July 19, 2023AI expert Santiago Valderrama took to Twitter to asses the probable reasons behind this decline. He wrote, “GPT-4 is getting worse over time, not better.“We assume that OpenAI pushes changes continuously, but we dont know how the process works and how they evaluate whether the models are improving or regressing.“Rumors suggest they are using several smaller and specialized GPT-4 models that act similarly to a large model but are less expensive to run. When a user asks a question, the system decides which model to send the query to.“Cheaper and faster, but could this new approach be the problem behind the degradation in quality”The post triggered an intense debate on Twitter, with many users wondering if the high cost of running ChatGPT is forcing OpenAI to use smaller – and as a result, less accurate – models.However, despite the apparent quality drop in GPT-4, the AI industry is experiencing an all-time boom, with Microsoft going all-in on its Bing AI search engine, while rivals Google launching its own Google Bard chatbot.
