Stack Exchange is experimenting with AI-generated answers, despite a continuing ban on the use of AI-generated content on Stack Overflow, the section of the site aimed at developers.
The Answer Assistant, referred to as a Labs experiment, uses AI-generated answers. However, it hides those until they are verified, edited and managed by the community, reports DevClass.
Limited number of sites
The introduction of Answer Assistant is being done cautiously. No doubt this is because of the sensitivity of this topic within the Stack Overflow community. The experiment will initially be limited to Stack Exchange sites where moderators have agreed to participate.
Sites can determine which questions AI-generated answers are allowed on. The AI answer will remain private until approved by users of sufficient reputation. Once verified, it becomes publicly visible and attributed to an answer-bot.
The experiment will appear on only a few Stack Exchange sites. Namely Arts and Crafts, Raspberry Pi and User Experience. The Stack Overflow developer site is not yet included. There are hundreds of Stack Exchange sites on topics ranging from coffee to 3D printing, but Stack Overflow is by far the largest, with 1,200 questions per day compared to 155 per day for the second most popular site (Mathematics).
Fewer unanswered questions
Because of Stack Overflow’s dominance within Stack Exchange, it is unlikely that a major new feature such as this one will not eventually land there, should the experiment be put into production. According to the official announcement, the Answer Assistant’s goal is to reduce the number of unanswered questions. And to increase participation within each community.
The context of this is that traffic to Stack Overflow is decreasing significantly. This is probably because developers are using AI assistants directly from their IDE rather than visiting the site. Stack Overflow has already positioned itself as a data source for AI assistants. It also uses AI to help users formulate questions through its experimental Question Assistant.
Users expect human answers
Until now, however, the site has rejected AI-generated content. The ban was imposed in 2022, with a document explaining that AI content is inappropriate for use on Stack Overflow. Users expect human-written answers, and AI cannot reliably cite sources. The paper also stressed that AI answers are readily available elsewhere, and that if a user wanted an AI answer, they probably would have already searched for one.
Another issue is the extent to which AI answers might be influenced by data coming from Stack Exchange itself. Using AI trained on AI-generated data is risky. A study from Rice University in Houston, Texas concluded that without enough new, real data in each generation of a cycle, future generative models are doomed to decline in quality (precision) or diversity (variation).
As an experiment, Answer Assistant may never be widely rolled out. Still, it may seem like a tempting strategy for the site in response to declining traffic. The Stack Exchange team has posted an introduction to the Answer Assistant in the Meta Stack Exchange forum. The response so far has been largely negative.