In a groundbreaking development, researchers from Pennsylvania State University have developed an AI model designed to detect extremist users and content related to the militant group Islamic State (ISIS) on the social media platform X (formerly Twitter). The researchers used a dataset of tweets spanning from 2009 to 2021 to train the AI. The model employs artificial intelligence techniques such as machine learning and natural language processing to differentiate users sharing ISIS-related content.How the AI model will help restrict extremist activities on social mediaAccording to the author of the paper published in the journal Social Media Analysis and Mining, the primary objective of this innovative model is to assist social media companies in identifying and promptly restricting accounts engaged in extremist activities. By analyzing tweets, the researchers identified potential propaganda messages and their characteristics. Additionally, they developed an image classifier to pinpoint the most frequent categories of images attached to tweets related to ISIS.Younes Karimi, a graduate student at Pennsylvania State University pursuing a doctorate in informatics, emphasized the ongoing manipulation of online communities by the Islamic State group and its affiliates to spread extremist propaganda. Karimi also highlighted the significance of their work in providing social media platforms with tools to identify and restrict such accounts in a more timely manner, thus mitigating their impact on online communities.He said, We believe that users who retweet or quote Islamic State group content are more likely to be affiliates or sympathizers, while those who just mention the content are less likely to be supporters. However, tweets posted by mentioners are still very likely related to ISIS and contain topics similar to ISIS tweets, which make mentioners suitable to be considered as our non-ISIS users and non-trivial counterparts to ISIS users.How the AI was trainedThe researchers used artificial intelligence techniques, including machine learning and natural language processing, to distinguish users sharing ISIS-related content. For the study, ISIS accounts identified before 2015 served as labeled data, while a user classifier was built using an old dataset to identify potential ISIS supporters.Apart from analyzing ISIS-linked tweets, the researchers collected a dataset of tweets from potential ISIS supporters to investigate their recent activities. Karimi pointed out that the Islamic State group increasingly relies on social media to spread propaganda, undermine rivals, and recruit sympathizers, despite countermeasures by platforms like X.Text and image-based detectionThe study provided insights into the pervasive and continuous sharing of identified content, the use of ideology-based words and images designed to elicit an emotional response, and the strategic use of hashtags by supporters and affiliates of the Islamic State group. Researchers said that their approach, focusing on users and user content, could be applied to other social media platforms, enhancing proactive identification and restriction of extremist content online. The longitudinal perspective of the dataset, covering data from before and after 2015, was crucial in understanding the evolving online strategies of extremists following a major crackdown by Twitter.
