The research from Purdue University, first spotted by news outlet Futurism, was presented earlier this month at the Computer-Human Interaction Conference in Hawaii and looked at 517 programming questions on Stack Overflow that were then fed to ChatGPT.

“Our analysis shows that 52% of ChatGPT answers contain incorrect information and 77% are verbose,” the new study explained. “Nonetheless, our user study participants still preferred ChatGPT answers 35% of the time due to their comprehensiveness and well-articulated language style.”

Disturbingly, programmers in the study didn’t always catch the mistakes being produced by the AI chatbot.

“However, they also overlooked the misinformation in the ChatGPT answers 39% of the time,” according to the study. “This implies the need to counter misinformation in ChatGPT answers to programming questions and raise awareness of the risks associated with seemingly correct answers.”

    • AIhasUse@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      2
      ·
      5 months ago

      Ray Kurzweil has a phenomenal record of making predictions. He’s like 90% or something and has been saying AGI by 2029 for something like 30+ years. Last I heard, he is sticking with it, but he admits he may be a year or two off in either direction. AGI is a pretty broad term, but if you take it as “better than nearly every human in every field of expertise,” then I think 2029 is quite reasonable.

        • explodicle@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          3
          ·
          5 months ago

          Maybe only 51% of the code it writes needs to be good before it can self-improve. In which case, we’re nearly there!

          • AIhasUse@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            1
            ·
            5 months ago

            We are already past that. The 48% is from a version of chatgpt(3.5) that came out a year ago, there has been lots of progress since then.