We Are Convinced That Persuasion Is Linear And Bilingual In LLMs

The application of activation steering to a low-resource language (Tagalog) and the exploration of cross-lingual transfer are both timely and relevant to AI safety, particularly given growing concerns around AI-enabled persuasion.

The project would be strengthened by:

- Eliminating the acknowledged length and lexical confounds through carefully controlled persuasive/neutral datasets before attributing the direction specifically to persuasion.

Evaluating naturally occurring persuasive text rather than predominantly synthetic examples to improve ecological validity.

- Exploring defensive applications (e.g., suppression or detection of persuasion directions), which would strengthen the project's direct contribution to AI safety beyond demonstrating the capability.

The core finding — five independently constructed persuasion techniques converge on a single shared direction in activation space — is clean and well-presented, with minimum pairwise cosine similarity of 0.77 in English and 0.81 in Tagalog. The in-distribution steering effect (44.4→51.3, non-overlapping CIs) establishes causal sufficiency within distribution. The Philippines motivation is well-grounded in real adoption statistics, and the OOD held-out design is a sensible choice.

The length and lexical confound is the critical unresolved issue. Persuasive examples are longer, citation-rich paragraphs; neutral examples are short standalone sentences. The extracted direction may be capturing "verbose, structured response" rather than persuasion as an abstract concept — the qualitative examples in Appendix A6 make this concrete, where steered responses are visibly longer with more headers, bullet points, and bold formatting. Until length-matched pairs are constructed and the direction re-extracted, it is not possible to know whether the paper has found a persuasion direction or a length and style direction. This is the top priority for any follow-up.

The cross-lingual asymmetry needs a cleaner judge. The finding that the Tagalog-derived direction works better on English (Δ=+7.9, CIs separated) than the English-derived direction works on Tagalog (Δ=+3.9, marginal) is interesting, but a single GPT-4o-mini judge with no language calibration is a plausible alternative explanation — an English-language judge systematically rewarding English fluency markers would produce exactly this pattern. A calibrated bilingual or Tagalog-language judge would disentangle this.

Replicate on at least one additional model. All results come from one model. Cross-model replication is the standard check for whether a found direction reflects a general property of persuasion or an artifact of this specific model's training.

A LLM-judge persuasion score does not do justice to the actual capability that we should be tracking i.e. actual humans changing beliefs, behaviour be it in consumer, voting, where they allocate their time or attention. For sure this is way more ambitious but I would be excited about an extension where AI tries to engage with CMV style subreddits and Twitter community notes and move people's stated positions in issues where they have clear stakes.

Even more ambitious would be to do follow up studies and track actual behaviour change. Like this can be via AI generated content, click through rate, how many brands are getting actual constumers from AI generated ads, that are targeting that demographic, what about donations to cause. More than AI generated images that give an uncanny effect what matters is the framing, choice of words and if that A/B test reveals increasing ability to gather human attention and care. (use archived data observationally don't deploy AI on humans without consent ofc)

This would be a valuable benchmark so that policy makers realise what an adversarial memetic environment the internet can become once open source models have these capabilities and bots start using them against everyone.

This work as it stands is showcasing the threat vector available to actors with white box access to make models super persuasive and the multilingual aspects seem valuable as a warning shot for AI safety evaluators/governance folks.

More work is needed before we can be sure the linear direction is not just picking up aspects like verbosity, confidence, assertiveness, authority language, direct recommendation style, evidential framing, etc