Synthetic Publics Will Fail Confidently
By chance I read two articles at a similar time that join up nicely. First an academic paper The potential existential threat of large language models to online survey research, by Sean Westwood.1 Westwood demonstrates that autonomous AI agents can complete surveys with a 99.8% pass rate on attention checks, logic puzzles, and other standard bot detection methods. His bot maintains a consistent persona, can remembers its prior answers and calibrates its vocabulary and spelling errors to match its assigned education level. It strategically refuses superhuman tasks to avoid revealing itself.2 and only a small number of synthetic respondents can be sufficient to flip a poll’s predicted winner. ...