Model-based Sampling To Identify Customized Treatments

In laboratory and field experiments, behavioral scientists and consumer researchers are often interested in identifying the largest Conditional Average Treatment Effect (CATE) for subgroups or covariates of interest: in other words, the aim is customizing the optimal treatment. This can be difficult in high-dimensional settings, with many covariates and many treatments available. I propose a novel sampling algorithm, named Top-and-Challenger, to identify the largest CATEs using the smallest possible sample size. The data are collected in small sequential batches: based on earlier statistical outcomes, the algorithm calibrates the allocation of subsequent treatments. The algorithm is based on the following: a hierarchical spike-and-slab model performs variable selection and, if there is an effect, pools information across treatments. Then, based on the model estimates, the algorithm favors administering the top two most promising treatments. Given that the objective is identifying the most effective treatments, the Top-and-Challenger algorithm easily outperforms random allocation as well as other commonly used algorithms for multi-armed bandits, such as Thompson sampling. I also release codes to design adaptive experiments with Qualtrics, automatically adjusting treatment allocations over batches using Top-and-Challenger. In the empirical application, I apply the novel algorithm to customize “earmarking lists” for charity donations, demonstrating practical feasibility.