Why GenAI Adds $5 to User Conversion but Fails to Grow the Cart: Lessons From 7 Production Experiments

When product managers ask engineering teams to integrate generative AI into an e-commerce platform, the business case is almost always framed around revenue growth. The pitch describes conversational shopping assistants that increase order volume, personalized product descriptions that cross-sell higher-tier items, and automated search query expansion that turns casual browsers into buyers.

Before committing infrastructure budget to multi-turn conversational agents and continuous LLM prompt pipelines, technical architects need to know what those systems actually deliver to the bottom line.

A recent empirical study published in Management Science provides concrete data from production systems. In "Generative Artificial Intelligence and Sales Productivity: Field Experiments in Online Retail" (doi:10.1287/mnsc.2026.00623), researchers evaluated seven distinct generative AI deployments across millions of users on a major international e-commerce platform between late 2023 and mid 2024.

The seven workflows spanned three core operational areas:

1. Customer support (pre-sale chatbots and live translation)
2. Catalog matching (search query refinement and automated product descriptions)
3. Promotion (push notification copy and ad title generation)

Across four workflows with positive revenue results, the aggregate annual incremental revenue was roughly five dollars per consumer (between $4.60 and $5.20).

For an e-commerce engineering lead, how that five dollars was generated matters more than the topline number.

First, the entire revenue lift came from conversion rate improvements. Average cart value remained completely flat across every test group. Language models did not persuade consumers to spend more money, add complementary items to their carts, or upgrade to higher-priced products. Instead, models acted as friction reducers. They helped users, particularly first-time shoppers who were unfamiliar with catalog nomenclature or navigating cross-border translation, locate items they intended to purchase without abandoning the session.

Second, the performance of generative models depended on what software ran before them.

In pre-sale customer support, where the platform previously offered no interactive support, the generative chatbot produced a 16.3% lift in consumer expenditure. It replaced an empty workflow. In search query refinement, where the baseline was already a mature machine learning search engine with lexical translation, adding generative query expansion yielded a 2.1% sales increase. In product description generation, the lift was 2.7%.

In creative marketing copy, the models delivered no advantage. Generating automated push notifications and ad titles failed to produce any statistically detectable improvement over standard human copy. In high-entropy consumer feeds, algorithmic variation did not shift engagement.

These numbers dictate the unit economics of generative retail architecture.

If an active user generates five dollars in annual incremental gross merchandise value, the engineering team cannot afford an architecture that burns fractions of a cent on every page view. A user who visits a platform twenty times a year, runs thirty semantic searches, and interacts with a conversational assistant will easily trigger hundreds of model inferences. If those requests are routed through proprietary frontier APIs or commercial enterprise vendor platforms charging high monthly seat fees, the inference bill quickly swallows the five-dollar margin.

To make generative e-commerce workflows cash-flow positive, engineering teams should follow two architectural boundaries:

Focus deployments on zero-to-one operational gaps where no prior automated assistance exists, rather than replacing tuned classical machine learning pipelines for marginal single-digit percentage gains.

Deploy compact, task-specific open-weights models running on dedicated or quantized infrastructure. When downstream economic value is bounded by a five-dollar friction reduction ceiling, inference serving costs must be kept at sub-cent levels per active user session.

Originally published on deanlee.info.