Scam Detective: Using Gamification to Improve AI-Powered Scam Awareness

Mar 10, 2025

Scam Detective: Using Gamification to Improve AI-Powered Scam Awareness

Angelica Marie M. Casuela

Read Project

Details

Summary

This project outlines the development of an interactive web application aimed at involving users in understanding the AI skills for both producing believable scams and identifying deceptive content. The game challenges human players to determine if text messages are genuine or fraudulent against an AI. The project tackles the increasing threat of AI-generated fraud while showcasing the capabilities and drawbacks of AI detection systems. The application functions as both a training resource to improve human ability to recognize digital deception and a showcase of present AI capabilities in identifying fraud. By engaging in gameplay, users learn to identify the signs of AI-generated scams and enhance their critical thinking abilities, which are essential for navigating through an increasingly complicated digital world. This project enhances AI safety by equipping users with essential insights regarding AI-generated risks, while underscoring the supportive functions that humans and AI can fulfill in combating fraud.

Download PDF

View Presentation

Cite this work:

@misc {

title={

Scam Detective: Using Gamification to Improve AI-Powered Scam Awareness

author={

Angelica Marie M. Casuela

date={

3/10/25

organization={Apart Research},

note={Research submission to the research sprint hosted by Apart.},

howpublished={https://apartresearch.com}

}

Review

Reviewer Name

Constructive critique

Criteria 1:

Advancing interpretability (Possible questions to answer in this criteria: Does the project contribute to the field of mechanistic interpretability? Does it provide new insights into understanding or steering AI model behavior? How well does it move us towards reprogramming AI models? How original and innovative is the approach?)

2.5

Criteria 2:

Relevance for AI safety (Possible questions to answer in this criteria: How important is the contribution to advancing the field of AI safety? Do we expect the results to generalize beyond the specific case(s) presented in the submission? Does the approach introduce new safety mechanisms or enhance existing ones in innovative ways?))

2.5

Criteria 3:

Methodology & presentation (Possible questions to answer in this criteria: How well is the project executed from a technical standpoint?, Is the code well-structured, documented, and reproducible?, How effectively does it utilize Goodfire's SDK/API and other provided resources?, How clearly and effectively is the research presented in the paper?, Quality of visualizations and demos (if applicable), Clarity of methodology explanation and results interpretation))

2.5

Reviewer's Comments

Tarin

This was a really fun and engaging way to educate audiences on multiple levels. You successfully communicated risks and capabilities of AI while simultaneously empowering users, educating them on how to proactively identify already-common AI scams and protect themselves from harm. The UI itself was also a clean and elegant user experience.

This was a strong MVP, and I'm excited about its continued development opportunities. For example, as you mentioned, the current difficulty level is relatively low. I would love to see more investment in escalating difficulty levels to further demonstrate the strength in capabilities of these systems. This could also further incorporate more explicit learning, like incorporating OpenAI's Persuasion risk-level identifiers which you called out in the paper, or further emphasize the reality of these harms by including and specifically identifying real-life scam examples.

Overall, this was a great multi-purpose game to capture learner attention, educate them on broader AI risk categories, communicate the reality of AI-powered harm, and empower themselves against realistic threats. Well done!

C1:

2.2

C2:

3.6

C3:

2.8

Li-Lian Ang

Great work on this project!

Here are some things I particularly enjoyed:

- I love the clean UI!

- I think this could be a good tool to educate the public on how to detect scam messages by teaching them what to look out for.

Here are some things I thought could be improved:

- I would have loved to see more instructions or context when I first opened the web app. I was a bit confused at first about what the AI assessment was meant to be.

- I think it would be useful to set an end state for the user otherwise it might feel like they would keep playing forever! I'm not sure whether I'm supposed to feel like AI systems are very good at scamming or if I'm simply meant to educate myself on detecting phishing attempts. Perhaps showing different types of phishing attempts?

- An interesting addition to this project might have been to see how AI can create realistic, targeted phishing attempts based on a target's user profile. Perhaps this veers into the realm of dual-use concerns, but an idea nonetheless!

C1:

C2:

C3:

Recent Projects

View All

Feb 20, 2025

Deception Detection Hackathon: Preventing AI deception

Mar 18, 2025

Safe ai

The rapid adoption of AI in critical industries like healthcare and legal services has highlighted the urgent need for robust risk mitigation mechanisms. While domain-specific AI agents offer efficiency, they often lack transparency and accountability, raising concerns about safety, reliability, and compliance. The stakes are high, as AI failures in these sectors can lead to catastrophic outcomes, including loss of life, legal repercussions, and significant financial and reputational damage. Current solutions, such as regulatory frameworks and quality assurance protocols, provide only partial protection against the multifaceted risks associated with AI deployment. This situation underscores the necessity for an innovative approach that combines comprehensive risk assessment with financial safeguards to ensure the responsible and secure implementation of AI technologies across high-stakes industries.

Mar 18, 2025