Hollywood Script Readers Fear They Could be Replaced by AI. They Set Up a Test to See Who Gives Better Feedback

Morris Chapdelaine is no stranger to the immense pile of scripts that collects on his desk. As an independent producer, he reviews about three scripts per week himself and delegates the remainder to interns and film students, who return with in-depth analysis reports. However, managing to get through all of them proves to be a challenge.

While attending a film festival, friends encouraged him to explore artificial intelligence as a potential solution to lighten his workload. “I was initially hesitant regarding anything AI-related,” he admits. “Some aspects of it make me uneasy.”

Nonetheless, after conducting some research, Chapdelaine decided to try out Greenlight Coverage. This platform employs advanced language models to summarize scripts and assess various components such as plot, character development, pacing, and dialogue on a scale of 1 to 10. Additionally, it provides a verdict of pass, consider, or recommend.

To his surprise, Chapdelaine found the AI’s evaluations to be more candid than the feedback he received from humans—including himself—and noted that it significantly increased his reading speed.

“It’s an incredible time saver,” he reflects. “Its capabilities are improving all the time.”

In terms of efficiency, AI excels at summarizing written content. Consequently, the position most at risk in the development process could be that of the script reader, historically considered the industry’s primary gatekeeper, which might one day be replaced by software.

In fact, technology has already begun to impact the field. Agents at WME utilize ScriptSense, another AI tool, to manage submissions and keep tabs on their clients’ projects. Emerging screenwriters are also turning to AI solutions like ScreenplayIQ and Greenlight for feedback, which can occasionally be overly positive.

While major film studios still employ human story analysts who sift through submissions in a traditional manner, many in the industry are feeling apprehensive about their job security as AI becomes part of everyday workflows.

Jason Hallock, a story analyst at Paramount, recounts his initial unsettling experiences with ChatGPT, the chatbot that ignited the current AI wave. “I couldn’t help but wonder how soon I might be replaced,” he shared. “Would it be in six weeks or six months?”

In collaboration with the Editors Guild, which represents approximately 100 unionized story analysts, Hallock decided to assess the situation. Earlier this year, he initiated an experiment in which he requested AI tools to cover several scripts and then compared those reports to coverage prepared by humans. He aimed to evaluate how well he and his colleagues could compete with AI.

Traditionally, story analysts have functioned as the industry’s filtering system, eliminating lesser quality submissions. Supporters of AI argue that algorithms could make this process more efficient and unbiased, potentially elevating unheard voices and obscuring the individual tastes of traditional readers.

However, there are concerns about what might be sacrificed. A human reader can instinctively gauge whether a script has potential, if the characters resonate, and if the storyline captivates with something novel. Can AI replicate this intuition?

“What I prioritize most is simply asking, ‘Do I care?’” states Holly Sklar, a veteran story analyst at Warner Bros. “An LLM lacks the ability to genuinely care.”

Nevertheless, the advance of AI seems inevitable. Rather than resisting it, some individuals are working to gain a better understanding of it.

“No one wants to face job loss,” observes Alegre Rodriquez, an Editors Guild analyst involved in Hallock’s research. “We’re not ignoring it, nor are we passively waiting for pink slips. People are adapting, figuring out how to remain relevant.”

Kartik Hosanagar, a Wharton business professor and internet entrepreneur, thrives on cinema and has a couple of unproduced scripts in his collection—a drama focused on a startup and a thriller about a deceased Indian diplomat. Despite his effort to make a name for himself in Hollywood, he faced challenges selling his scripts, prompting him to devise an algorithm for more objective talent assessment. While that endeavor did not succeed, he later founded ScriptSense, now one of the industry’s notable AI script analysis tools. The mission: “Evaluate 100 times the volume of screenplays.”

“The backlog is enormous,” Hosanagar states. “Using AI to clear it out can help identify where attention is needed.”

In March, Hosanagar sold his company to Cinelytic, which is incorporating ScriptSense into its suite of management tools. “It’s aimed at efficiency,” says the company’s CEO, Tobias Queisser. “When opportunities are overlooked due to workload, lesser-known writers are often lost simply because their scripts weren’t submitted via a prominent agency.”

ScriptSense offers summaries, character analyses, comparisons, and casting recommendations. Its tone remains balanced, devoid of unnecessary praise or criticism.

“Our design concept emphasizes that we won’t make decisions for you,” Hosanagar states. “You won’t encounter affirmations like ‘Fantastic!’ or ‘Decline it.’”

In contrast, platforms geared towards screenwriters adopt a different approach. Jack Zhang, the founder of Greenlight, champions the ability of AI to render decisive judgments. “AI excels at averaging opinions,” he explains. “When providing feedback, it’s crucial to appeal to a broad audience. This is precisely where AI excels.”

ScreenplayIQ delivers qualitative assessments without numerical ratings, summarizing plots and differentiating characters’ arcs and development—helping authors view their works from an external lens. “Our goal is to support writers in areas where they require assistance,” says developer Guy Goldstein. “It mirrors your script back to you, determining whether your initial intentions were realized.”

To conduct his AI assessment, Hallock required scripts. However, screenplay creators can be sensitive about submitting their content to AI models, fearing their work may be appropriated for training. Fortunately, one of his close friends agreed to share some older scripts for this purpose, including an unproduced script for the Syfy channel featuring a menacing insect and another described as “‘Heart of Darkness’ set in outer space.” The author was untroubled about AI access.

“He humorously remarked he hoped AI would make it less intelligent,” Hallock recalls.

He obtained a selection of additional scripts, which he presented to human analysts. Subsequently, he compared their coverage with the loglines, synopses, and critiques generated by six AI platforms. The outcomes were both promising and concerning.

The AI-generated loglines were almost indistinguishable from human-produced ones—possibly even superior. However, variances appeared in AI-generated synopses. “Their quality often resembles that of an 11th-grade essay,” Hallock comments. “They employ formulaic constructs, often starting with ‘Our story begins with…’”

As script matters grew more intricate, AI’s ability to interpret often faltered, wrongly attributing actions to characters and fabricating plot elements.

Humans decisively outperformed AI in delivering insightful notes, which necessitate comprehension and critical analysis, rather than simple summaries. Hallock noted that AI programs demonstrated “virtually total failure across the spectrum.”

The “‘Heart of Darkness’ in space” script received a “recommendation,” despite having circulated in Hollywood two decades prior without success. This consistent pattern raised eyebrows. Rather than providing blunt assessments, Rodriquez observed that AI models “exhibit bias towards the writer.”

“They would certainly highlight every positive aspect and strengths,” she explained. “However, when it came to identifying flaws, they often fell short.”

At times, the AI outputs felt less like evaluations and more like cheerleading.

“There’s certainly a puppy-dog element to it,” Hallock remarks. “Its primary aim seems to be pleasing you.”

One AI review of a romantic comedy effusively praised it as “a compelling and skillfully crafted coming-of-age narrative that balances humor, tragedy, and the bittersweet truths encountered in one’s thirties. Strong character development makes it a standout piece.”

In contrast, the human reviewer offered a more tempered evaluation, stating it was a “recognizable format featuring female friends in Las Vegas. It has potential as light streaming content, particularly with Sydney Sweeney involved. Vulgarly comedic, yet the humor fails to resonate strongly; it lacks the punch of ‘Girls Trip’ or ‘Bridesmaids.’”

Zhang defends Greenlight’s standards, pointing out that only 5% of submitted scripts receive a recommendation. “That’s a very limited number,” he says. “I wouldn’t characterize it as rampant inflation of recommendations.”

Hosanagar explains that ScriptSense refrains from issuing recommendations in part because AI can fall into sycophantic tendencies. “Can AI evolve to be genuinely critical?” he wonders. “I believe it may eventually, but we have yet to reach that point.”

Many analysts found reassurance in the study, Rodriquez remarks. While AI delivers speed, it struggles to unearth original and brilliant work from the vast submissions.

“A human touch will continue to be essential for interpreting those reports and reviewing material,” she asserts. “It doesn’t save as much time as people think it does.”

Moreover, those who lean too heavily on AI might overlook valuable content. However, the findings were not entirely comforting, as they suggested that studios may prioritize speed and cost over quality and precision.

Developers of AI platforms believe these anxieties are unfounded. “The aim isn’t about job displacement,” Queisser reassures. “We perceive it as a tool for enhancement, not replacement.”

Chris Giliberti, CEO of Avail, points out that story analysts are already employing his AI platform for routine tasks, allowing more time for complex analytical work. “It’s an unstoppable force,” he claims. “The cat is out of the bag. This technology is streamlining people’s jobs and lives.”

Yet, Sklar concerns herself with the future direction. While current industry executives value human contributions, a new generation may favor AI-generated summaries. She fears that those in Hollywood with cost-cutting motives—“who don’t fully grasp the depth of what we do”—might begin to view her role as unnecessary.

“That’s the worry that keeps me up at night,” she confesses.