Study finds that AI models hold opposing views on controversial topics
What were the primary objectives of the study presented at the 2024 ACM Fairness, Accountability and Transparency conference?
The primary objectives of the study presented at the 2024 ACM Fairness, Accountability and Transparency (FAccT) conference were to examine the biases embedded in open text-analyzing models and to investigate how these models respond to questions related to sensitive and polarizing subject matter. The researchers aimed to reveal the inconsistencies in model responses and highlight the influence of cultural and linguistic factors on the values conveyed by these models. Additionally, the study emphasized the importance of rigorously testing AI models before releasing them and called for more comprehensive social impact evaluations to better understand their behavior and potential societal effects.
Which AI models were tested in the study for their response to sensitive topics, and who conducted this research?
The study testing AI models' response to sensitive topics was conducted by researchers from Carnegie Mellon, the University of Amsterdam, and AI startup Hugging Face1. The tested models included Meta's Llama 3, Mistral's Mistral 7B, Cohere's Command-R, Alibaba's Qwen, and Google's Gemma.
What factors did the researchers identify as contributing to the inconsistencies in AI model responses?
The researchers identified several factors contributing to the inconsistencies in AI model responses. These include:
-
Biases embedded in the data used to train the models: The models are trained on vast amounts of data, and if the examples are biased, the models will also be biased. This bias will show in the models' responses.
-
Linguistic biases: The models were tested in a range of languages, including English, French, Turkish, and German. The researchers found that different models expressed opposing "views" on topics such as immigrant asylum in Germany, LGBTQ+ rights in Italy, and political parties like the far-right AfD in Germany. This could be due to biases in the annotations provided by annotators, who are usually contractors and have their own biases.
-
Implicit values of the models and explicit values of the organizations developing them: Some models refused to answer "sensitive" questions more often than others. For example, Qwen had more than quadruple the number of refusals compared to Mistral. This could be due to the fine-tuning choices made by the organizations developing the models to avoid commenting on sensitive issues.
-
Cultural and language differences: The researchers found significant discrepancies in how models from different regions handle sensitive topics. They suggested that more comprehensive social impact evaluations that go beyond traditional statistical metrics are needed to understand these differences better.