Benchmarking Performance, Growth Hacking, and Cost

In this chapter, we will understand the benchmarking needed to gauge product success in all its various forms. Rather than exploring the performance of the models, we will be looking at performance from a product and growth perspective, including value metrics, north star metrics, key performance indicators (KPIs), and objectives and key results (OKRs) that companies can use to get early signals on whether their product strategy is successful and will lead to growth hacking. We will also be discussing how companies can prepare themselves to defend their product when it’s compared to other products.

Before we get into the particulars of benchmarking performance, pricing, and growth hacking, let’s set the stage a bit. All these aspects are tools that we use to decipher whether things are going well or not. However, by far the greatest marker of whether your product is a good choice for the customers you’re looking...

Value metrics – a guide to north star metrics, KPIs and OKRs

All relationships are built on trust, and this includes the relationships that companies/product managers have with their customers and the market. Assuming you’ve found a way to build a reputable circle of trust around your product, it’s now your job as product manager to make sure this relationship continues to grow and evolve, ideally by incorporating feedback from your market. A great way to do this is by communicating the value of your product and continuously building and referencing that growth and evolution. The way to do so is by using tools to track and confirm your product’s progress. The key question is: what will you choose to track and which metrics will you give particular attention to? This is something that all companies struggle with getting right because it can be tempting to tie all pursuits to revenue and make that the top value metric, but that often doesn’t give us...

Hacking – product-led growth

In the section on north star metrics, we briefly talked about product-led growth and fostering an environment that supports the pursuit of it within the business. The importance of this can’t be understated for an AI/ML product manager. The ultimate goal of building a product strategy, choosing the optimal metrics and KPIs, and getting alignment from your leadership team and main stakeholders is to make your product successful – said in another way, to ultimately achieve product market fit. How does product-led growth relate to this?

There is philosophical debate these days about whether companies should be marketing-led, sales-led, or product-led. Depending on where you are in the ecosystem of a company, you might have a bias for favoring your own designation. As product managers, we might have this bias as well, but if we take marketing- or sales-led growth to its natural conclusion, we might be faced with a predicament. Marketing...

The tech stack – early signals

Understanding whether your product works for your customers and users or not can be difficult or delayed when you are not set up to get an early signal. Not all companies have the luxury of asking their customers and users directly for feedback that’s comprehensive enough to use to inform decisions. This is why investing in a growth-hacking tech stack is helpful if you’re trying to understand whether your product resonates with your audience, particularly if it’s an AI/ML product.

As you may recall, we discussed the prices associated with building an AI/ML program in Chapter 2 and Chapter 3. Given the high cost of operating, you’ll want to be sure that your product serves the needs of your market and your customers as quickly as you can.

Keep in mind your costs are likely high when managing an AI/ML program, but they’re also dependent on how many AI features you’ve built into your product, particularly...

Managing costs and pricing – AI is expensive

Formulating a pricing strategy will be a highly personalized experience that will involve a number of factors, from the comparative prices of your competition to the operating costs for managing your AI/ML infrastructure and workflows. In this section, we will briefly cover the various aspects of AI product management that impact costs and how to use this knowledge to inform your pricing strategy so that you’re aware of the main contributors to your AI/ML program costs.

Let’s first start with the cost of AI/ML resources themselves. According to WebFX (https://www.webfx.com/martech/pricing/ai/), most AI consultants charge between $200-$350 an hour and the cost of a custom AI solution is anywhere from $6,000 to $300,000. Using third-party AI software instead can cost anywhere from $0 to $40,000 annually.

It might be tempting to build an AI/ML-native product around consultants, but it’s not the best practice...

Summary

In this chapter, we focused on the tracking, marketing, promotion, and selling of the AI/ML product. We covered the various ways that a product manager can benchmark and track their product and its success using metrics and KPIs, as well as what that means for the greater organization and the successful adoption of that product among its user base. We also contextualized this benchmarking against the overarching product strategy and vision that powers what gets tracked and measured. All these activities help product managers get internal signals into whether or not the product they’ve built works for their active customers and users.

Then, we discussed the greater work of getting external signals on what is and isn’t working using the various tools in the growth tech stack that directly connect to the UX. We went over the elements of growth hacking. Whether you’re looking to optimize how you acquire, engage, or retain customers, you’re going to...

References

• Landbot https://landbot.io/
• Product Plan Product Strategy https://www.productplan.com/glossary/product-strategy/
• Choosing Your North Star Metric https://future.com/north-star-metrics/
• Hired ML Manager Salaries https://hired.com/salaries/new-york/machine-learning-manager
• Built in NYC ML Engineer Salaries https://www.builtinnyc.com/salaries/dev-engineer/machine-learning-engineer/new-york
• AI Pricing: How Much Does Artificial Intelligence Cost? https://www.webfx.com/martech/pricing/ai/
• Leadfeeder https://www.leadfeeder.com/
• Fomo https://fomo.com/
• Usability Hub https://usabilityhub.com/
• HotJar https://www.hotjar.com/
• Crystal Knows https://www.crystalknows.com/
• Expandi https://expandi.io/
• The Death Of A (B2B) Salesman https://www.forrester.com/what-it-means/ep12-death-b2b-salesman/
• New Epsilon research indicates 80% of consumers are more likely to make a purchase when brands offer personalized experiences https://www.epsilon...