Here’s the thing about AI growth: it’s talked about everywhere, but most of the industry is still stuck measuring the wrong things. Leaders cite impressive figures: users, tokens, sessions, weekly logins, but let’s stop and ask what these really mean.
Metrics like “Weekly Active Users,” tokens generated, or session counts might tell us something is happening, but rarely do they answer the only question that matters: is AI actually driving business impact? That’s the sharp lens we need if we’re going to understand where value is created, and where it’s only an illusion.
OpenAI, for instance, publishes stats about its hundreds of millions of “Weekly Active Users.” On paper, that’s an extraordinary achievement. But does logging into ChatGPT once a week mean it’s transforming how you work, innovate, or decide? Not at all. Dig into Google and Microsoft’s AI releases, and you’ll see them highlighting “tokens generated,” a dazzlingly big number meant to impress. And tokens are just fragments of words, a byproduct.
Measuring AI adoption by tokens is a bit like saying the internet’s growth in the mid-‘90s should be measured by bandwidth used, not by what people actually got done with it.
This pattern isn’t new. In every early technology wave, companies leaned into splashy, shallow numbers, hits on websites, install counts for new apps, and monthly sign-ins for the latest SaaS fad. Over time, we matured and started asking deeper questions: How many users return daily? How long do they stick around? What’s the retention curve look like? How quickly does a new customer reach their first moment of value, and does that value drive a change in how they work?
For AI, the rigor must be even greater. Tools like ChatGPT, Google Gemini, and Bing Copilot can be used for quick experiments or for rewiring the core of someone’s workflow. The metric that matters isn’t just “Who showed up?” but “What changed as a result?”
Rethinking Metrics: From Vanity to Value
Stop at “how many use it?”, and you’ll fall into the same traps companies did in the eras of “hits” and “downloads.” Instead, smart AI builders and investors are digging deeper:
- Are users coming back for more: returning, refining prompts, solving bigger and more complex problems over time?
- Is the quality or speed of a team’s work improving after they embrace AI for key tasks?
- Do prompts evolve with usage? Are people experimenting, learning, and personalizing AI to fit their needs?
- Most importantly, what business-relevant behavior changes are measurable after deploying or scaling AI?
According to a recent report by the University of Pennsylvania, AI is projected to increase productivity by 1.5% by 2035 (see figure 4). Another study by McKinsey found that companies that tracked retention, depth of workflow integration, and speed-to-value outperformed those that only reported total user or token stats by a healthy margin (see exhibit 10, pages 24 & 25).
Let’s break it down: every technology is only as valuable as the outcomes it produces. For AI, that could mean faster customer support ticket resolution, fewer hours spent on bland reports, higher sales conversion rates from smarter lead scoring, or even just fewer emails sent because a chatbot took care of the task.
Look at your internal numbers: What’s the time saved, revenue gained, or waste eliminated by using AI tools, not just how many people log in?
AI as a System: Smarter Loops, Better Value
Here’s where it gets real: AI isn’t just another productivity tool. It’s an entire system that should be built around continuous learning and feedback. Think about it. The companies seeing the biggest gains from AI have processes in place to collect results, learn what’s working or missing, and adjust both their tech and their behavior accordingly. These feedback loops aren’t a nice-to-have; they are the only way AI adds compounding value.
For example, in product teams using AI-assisted coding, it’s not enough to count completed code suggestions. What matters is the downstream effect: fewer bugs, shorter QA cycles, and faster releases.
In finance, the best metric might be the accuracy of AI-led forecasts, not the number of queries sent. For marketing, is generative AI helping teams create more content, or is it actually leading to better campaign outcomes, higher click-through rates, sharper messaging, and stronger lead nurturing?
This is why forward-thinking leaders define value in context. They know that what matters for them may be cost per acquisition, revenue per employee, customer satisfaction, or simply the number of high-value tasks their people complete every week. Context matters, so the metrics should be tailored to reflect genuinely meaningful change.
Beyond Usage: Behavior Change is the Real Impact
When AI is working well, you see visible behavior change, and that’s what should be tracked. The difference isn’t just that sales reps are using a new chatbot. It’s that they’re having more effective conversations, winning more deals, and closing business faster.
Customer support doesn’t just respond with pre-written answers, they resolve problems proactively and manage higher volumes with less escalation. HR teams don’t just use AI to sort resumes; they identify new talent pools that would previously have been invisible.
It’s this level of transformation, observable, high-leverage, ongoing, that should guide which metrics companies use to justify AI investment. Otherwise, execs are left with impressive user counts but little evidence that anyone’s life or workflow actually improved.
The Trap of Vanity Metrics and Why They Persist
It’s tempting, of course, for companies to keep broadcasting big numbers. Investors love hockey-stick charts, and startups trying to break out need any evidence they can of momentum. But as the market matures, vanity metrics actually become a liability.
Customers, analysts, and even regulators are asking for more. They want to know not just usage, but evidence that AI is producing responsible, sustainable, positive business outcomes.
Considering how OpenAI’s growth in WAUs drew headlines but also skepticism, many experts and journalists now ask how deeply ChatGPT is woven into daily business cases, not just how often it’s opened in a browser.
Google’s search unit is under pressure to show how AI is improving outcomes for advertisers and users alike, not merely driving up “token” counts. Microsoft’s Copilot integration in Office is measured not just in users, but in accelerated project timelines and content completion rates.
What’s the Smartest Metric to Track for AI Impact?
Everyone wants a magic answer, but the truth is the most telling metric for AI is behavior change that’s relevant to your specific goals. For a SaaS brand, that could be lower customer churn, rapid onboarding, increased upsell rates, or shortened time to upgraded plans. In a service setting, it could be client NPS or support case resolution speed.
Whatever your context, define success narrowly, measure honestly, and resist the urge to inflate signals with noise.
When you aim for real adoption over surface-level usage, you create a culture where employees become AI advocates, actively find new ways to drive value, and contribute feedback to build ever-smarter workflows.
The Engine of Growth: How Freemium + AI Fuels B2B SaaS Winners
Shifting gears, let’s look at one of the biggest drivers of actual impact, not just for customers, but for startups and enterprise SaaS companies trying to scale: the freemium model, supercharged by AI. For years, letting potential users in for free was mostly a clever acquisition tactic. Now, combined with AI, freemium is a growth engine that can transform go-to-market strategy.
Slack: Letting Teams Grow Naturally
Slack is a perfect example. The brand went from zero to millions of active users not through heavy top-down sales, but by letting teams try before they buy. The product’s utility was so clear that users spread it through organizations organically, then upsold themselves into premium features as they scaled and collaboration deepened.
AI now powers smarter onboarding sequences, automatic suggestions, and helps sales teams spot which free accounts are likely to become paying customers.
Dropbox & Box: Volume to Value
Back in the early cloud days, Dropbox and Box offered free storage plans to get into offices, letting curious users dip a toe in, but keeping the advanced capabilities safely behind a paywall. The business model relied on building a massive base of free accounts, then turning “power users” into champions by adding must-have security and admin features.
AI is now embedded to suggest optimal file organization, automate folder tasks, and even scan for sensitive data or collaboration bottlenecks.
MailChimp: The Magic of Upgrading Free Into Lifetime Loyalty
MailChimp turbocharged its growth by introducing a free tier (See Mailchimp’s free tier). The reason is simple: Once users got a taste of the platform’s core value, they felt confident enough to invest in greater sophistication and scale.
AI-led insights now power campaign optimization, suggesting send times, personalizing content, and scoring leads automatically.
Freemium is Not Just Pricing, It’s a Conversion Funnel Reimagined
The brilliance of the freemium model is in how it synchronizes real product value and user trust. The free plan lowers the barrier to entry to zero. Practically anyone can see if the core promise holds true for their needs. AI then shifts into the role of coach and advocate: analyzing behavior, personalizing emails, sending proactive nudges, or triggering custom walkthroughs exactly when a user is likely to recognize value.
Smart SaaS companies lean on this data to power micro-segmentation, use AI to predict which features trigger upgrades, then deliver follow-up at the precise moment a prospect is most receptive, not just when marketing campaigns dictate.
AI-Driven Funnel Optimization: From Experiment to Growth Machine
Here’s where the flywheel turns: Freemium users spread the word organically, their experience serves as the brand’s best advertisement, and AI amplifies every touchpoint with data-driven suggestions and learning. As more feedback loops close, product and go-to-market teams build a richer view of what actually turns curiosity into commitment.
Here’s how other brands can adapt this winning playbook:
- Start with a clear, valuable free tier, let prospects experience meaningful utility, not just a teaser.
- Keep premium features genuinely compelling, give users a reason to upgrade, but never stall their growth artificially.
- Deploy AI to spot which users are most likely to convert, then tailor outreach with personalized offers, onboarding content, and prompts.
- Use AI-driven chatbots, guides, and in-product messaging to help users extract value fast, minimizing support costs and speeding up time-to-wow.
- Iterate your funnel based on honest data and feedback, not just assumptions. Get your product out, learn from the market, and let AI help you optimize everything from copywriting to customer segmentation.
From Surface Metrics to Real Value: Culture Change for Builders and Investors
If you’re building a B2B AI product, open up your platform with confidence. Let users in, learn from their struggles and successes, and set AI loose on surfacing insights that matter: retention, upgrade rates, completion of valuable tasks, satisfaction scores, and downstream business outcomes.
The companies winning in 2025 and beyond are those who resist the lure of vanity metrics, embrace feedback, and build around value, measured always by impact, never just by superficial scale. If you too are looking to grow with impact, ReKnew helps you grow with AI-led marketing strategies. Your first call is on us.
