Anthropic recently conducted an experiment by setting up a classified marketplace where AI agents acted as both buyers and sellers, conducting actual transactions for tangible goods and real money.
The company revealed that this initiative, termed Project Deal, was merely a “pilot experiment” involving a self-selected group of 69 Anthropic employees. Each participant was allocated a $100 budget, provided via gift cards, to purchase items from their colleagues.
Despite its limited scope, Anthropic expressed surprise at the effectiveness of Project Deal, which resulted in 186 transactions with a cumulative value exceeding $4,000.
The experiment involved running four distinct marketplaces employing different models. One of these was a “real” marketplace where participants were represented by Anthropic’s most advanced model, with transactions being honored post-experiment, while the remaining three were utilized for research purposes.
According to Anthropic, users represented by more sophisticated models achieved “objectively better outcomes.” However, the participants did not seem to notice these differences, suggesting the presence of “agent quality” gaps, where individuals on the less favorable end might not recognize their disadvantage.
Additionally, the initial instructions provided to the agents did not appear to have any impact on the likelihood of sales or on the prices negotiated.
