You're PM for Facebook ads. How would you define success metrics for the team? You're considering adding GenAI capabilities to the ad creation process for new sellers. How would you decide whether to ship this?

To define success metrics for Facebook Ads, I'd focus on key user groups and their objectives:

• Advertisers: Their success is tied to campaign performance and ROI. Key metrics include:
  • Acquisition: Onboarding completion rate for new sellers.
  • Engagement: Click-through rate (CTR), conversion rate, cost per acquisition (CPA).
  • Retention: Advertiser retention rate, lifetime value (LTV).
  • Satisfaction: Net Promoter Score (NPS) or customer satisfaction (CSAT) surveys.
• Potential Buyers: Their success is measured by relevant ad experiences and fulfilling their needs. Metrics include:
  • Engagement: Ad recall, click-through rate (CTR).
  • Conversion: Purchase completion rate, average order value (AOV).

For deciding whether to ship GenAI capabilities for ad creation for new sellers:

1. Define Objectives: What specific problem are we solving for new sellers (e.g., reducing time to launch, improving ad quality, increasing initial campaign performance)?
2. Identify Key Metrics: How will we measure success for this specific feature? Potential metrics include:
   • Ad Creation Time: Reduction in time for new sellers to create their first ad.
   • Ad Quality Score: Improvement in automated quality scores for AI-generated ads.
   • Initial Campaign Performance: Higher CTR or conversion rates for campaigns using AI-generated ads compared to manual ones.
   • Adoption Rate: Percentage of new sellers utilizing the GenAI feature.
   • Seller Satisfaction: Feedback from new sellers on the ease of use and effectiveness of the GenAI tool.
3. Develop Hypothesis: "We believe that by providing GenAI-powered ad creation tools for new sellers, we will reduce their time to launch and improve their initial campaign performance, leading to higher satisfaction and retention."
4. Experimentation (A/B Test): Roll out the feature to a small segment of new sellers and compare their metrics against a control group using the existing ad creation process.
5. Analyze Results: Evaluate the A/B test data against the defined key metrics and hypothesis.
6. Decision: Ship if the GenAI feature demonstrably improves key metrics and provides a positive ROI, or iterate based on learnings if results are mixed or negative.