You are designing an A/B test on Bing. When would you run a user-tied flight versus an untied flight, and what are the benefits and drawbacks of each?

User-tied flights are ideal when the experiment's impact is dependent on individual user behavior or history, such as testing personalized recommendations or long-term user engagement features. The benefit is a more accurate understanding of how a feature affects specific user segments over time. However, drawbacks include potential for user experience disruption if a user is frequently switched between variants, and increased complexity in tracking and analysis due to user state management.

Untied flights, on the other hand, are suitable for testing features with immediate, stateless impacts, like changes to search result layout or ad serving algorithms. The primary benefit is simplicity in implementation and analysis, as users are randomly assigned to a variant for each interaction. The drawback is that it may not capture the cumulative or personalized effects of a feature, potentially leading to a less nuanced understanding of its true impact on user loyalty or long-term value.