Benchmarking for Deep Uplift Modeling in Online Marketing

Uplift modeling seeks to determine the incremental impact of an intervention (or treatment) on an individual’s behavior, where the intervention (or treatment) corresponds to an incentive in online marketing. This is achieved by estimating an individual’s potential outcome under different scenarios, such as with or without intervention. In this paper, without loss of generality, we use the binary treatment indicator $T\in\{0,1\}$, where $T=0$ represents the control group (i.e., without intervention) and $T=1$ represents the treatment group (i.e., with intervention). In line with the potential outcome framework (Rubin, 2005), each individual $i$ has two possible outcomes: $Y_{i}(0)$ in the absence of intervention and $Y_{i}(1)$ in the presence of intervention. The uplift of intervention $T$ on an individual $i$, represented by $\tau_{i}$, is equal to the difference between the two potential outcomes:

However, individuals can only observe one of the two potential outcomes in real-world scenarios. The unobserved outcome is known as the counterfactual outcome, while the observed outcome can be represented as:

Since we can not observe both potential outcomes for each individual $i$, the uplift cannot be directly estimated. However, with some adequate assumptions, it can be obtained in a manner similar to the conditional average treatment effect (CATE), which is the average treatment effect conditioning on a set of covariates. Specially, let $X=\boldsymbol{x}$ be a $k$-dimensional covariate vector, then the CATE $\tau(\boldsymbol{x})$ can be derived as:

The objective of uplift modeling in Eq.(3) involves estimating two conditional expectations of the observed outcomes, which is feasible with available data. Next, we describe the critical components of a DUM model architecture.

Based on the input covariate vector $\mathbf{x}$, DUM usually transforms it into a low-dimensional dense vector $\mathbf{e}$ through an embedding layer. Then, a network architecture with predictive capabilities will be used to obtain the response of the control group $\hat{Y}_{i}(0)$ from this low-dimensional dense vector.

Similarly, there is usually another network architecture with predictive capabilities responsible for predicting the response of the treatment group $\hat{Y}_{i}(1)$. Note that in some DUM models, a shared module shares information at the bottom of the above two prediction networks.

Since accurately predicting whether a user belongs to the control group or the treatment group helps to make the model robust to the distribution imbalance of the group to a certain extent, a crucial technical branch in DUM is whether to additionally predict the indicator variable of the treatment $\hat{T}_{i}$. An independent prediction network will perform this process.

The least-square error (MSE) loss is widely used in DUM tasks, which is defined as follows:

where $\mathcal{D}$ is the training set with $N$ instances. Depending on the group to which each instance $X$ belongs, the prediction of $Y_{i}$ may be generated by $\hat{Y}_{i}(0)=f(\mathbf{e}|T_{i}=0,\theta_{0})$ oder $\hat{Y}_{i}(1)=f(\mathbf{e}|T_{i}=1,\theta_{1})$, where $\theta_{0}$ and $\theta_{1}$ are respective model parameters. Note that for brevity, we omit the optimization loss term for $T$.

In practical applications, DUM pays more attention to the response changes that each incentive may bring to the user than the response itself. Therefore, in the inference stage, we will make predictions for each instance based on the trained model on its response in the control group and the intervention group and then use the difference between the two to perform a ranking of delivery strategies, i.e.,

Since it is necessary to determine the group to which each instance belongs during loss optimization, the corresponding prediction branch is selected to obtain the prediction label. Therefore, the methods of this category only use the treatment indicator as a switch to switch the model branch and do not let it participate in the model’s training. Representative models in this category include T-Learner (Künzel et al., 2019), TARNet (Shalit et al., 2017), CFRNet (Shalit et al., 2017), FlexTENet (Curth and van der Schaar, 2021b), and EUEN (Ke et al., 2021).

Since the intervention itself may also carry helpful information or needs to be exploited to perform some steps that help alleviate the distribution shift, the methods in this category incorporate the treatment indicators as features into the model’s training. Representative models in this category include BNN (Johansson et al., 2016), S-Learner (Künzel et al., 2019), DragonNet (Shi et al., 2019), SNet (Curth and van der Schaar, 2021a), GANITE (Yoon et al., 2018), CEVAE (Louizos et al., 2017), DESCN (Zhong et al., 2022), and EFIN (Liu et al., 2023)

For models in the first category,

• •

  T-Learner (Künzel et al., 2019): it is similar to S-Learner, which uses two estimators for the treatment and control groups.

• •

  TARNet (Shalit et al., 2017): a deep learning-based method that extends the two-model approach, which can be seen as the extension of BNN.

• •

  CFRNet (Shalit et al., 2017): it is built upon the TARNet, which adds the minimization of the discrepancy measured by either the maximum mean discrepancy or the Wasserstein distance between the two distributions.

• •

  FlexTENet (Curth and van der Schaar, 2021b): it builds on ideas from multi-task learning to design a new architecture for CATE estimation, which adaptively learns what to share between the potential outcome functions.

• •

  EUEN (Ke et al., 2021): it is an uplift model for large-scale online advertising, which uses two sub-networks to estimate the control conversion probability and the uplift effect, respectively.

For models in the second category,

• •

  BNN (Johansson et al., 2016): it leverages deep learning for counterfactual inference with theoretical guarantees, combining ideas from domain adaptation and representation learning.

• •

  S-Learner (Künzel et al., 2019): it is a kind of meta-learner method that uses a single estimator to estimate the outcome without giving the treatment a special role. Varying the treatment while fixing other features yields different variants and uses their differences to estimate CATE.

• •

  DragonNet (Shi et al., 2019): it designs a new architecture with the propensity score for estimation adjustment and uses a targeted regularization procedure to induce a bias toward models.

• •

  SNet (Curth and van der Schaar, 2021a): it is a new classification of meta-learners inspired by the ATE estimator taxonomy, which estimates unbiased pseudo-outcomes based on regression adjustment, propensity weighting, or both.

• •

  GANITE (Yoon et al., 2018): it utilizes a generative adversarial network (GAN) to model treatment effect heterogeneity, which consists of two components, a counterfactual block that generates the counterfactual outcome and an ITE block that generates the ITE.

• •

  CEVAE (Louizos et al., 2017): it is a variational autoencoder (VAE)-based treatment effect heterogeneity modeling method, which uses a VAE to learn a latent confounding set from the observed covariates and then uses it to estimate the treatment effect.

• •

  DESCN (Zhong et al., 2022): it jointly learns the treatment and response functions in the entire sample space to avoid treatment bias and employs an intermediate pseudo treatment effect prediction network to relieve sample imbalance.

• •

  EFIN (Liu et al., 2023): it is a feature interaction-aware uplift method, which consists of four customized modules: a feature encoding module, a self-interaction module, a treatment-aware interaction module, and an intervention-constrained module.

Our experiments adopt two representative industrial datasets to better adapt to different marketing scenarios: 1) Lazada (Zhong et al., 2022). It is a dataset collected on the Lazada platform, containing 83 continuous features, a processing indicator, and a label. It provides data sets from two different sources, where the training set is collected from a production environment with a treatment bias, the treatment allocation is selective due to the operation target policy, and the test set is collected from the users who are not affected by the targeting strategy and the treatment assignment follows the randomized controlled trials (RCT). 2) Criteo (Diemert et al., 2021) is a dataset from real advertising scenarios provided by Criteo AI Labs. It contains nearly 14 million instances with similar treatment bias, 12 continuous features, a treatment indicator, and 2 labels (i.e., visits and conversions). The statistics of the two public datasets are shown in Table 2.

Following most existing studies (Liu et al., 2023), we randomly split the Criteo dataset into three sets, with a ratio of 8:1:1 for the training, validation, and testing. For the Lazada dataset, to make it accurately reproducible and easy to compare with existing work, we follow the splitting strategy reported by DESCN (Zhong et al., 2022): we randomly split the training set into two subsets with a ratio of 9:1 for training and validation, and the RCT test data is directly used as the test set. We set the random seed to 0-9, respectively, to ensure reproducibility when performing random splits.

Since all feature values are continuous and do not require any special processing, we use all features from both datasets. Considering that our benchmark investigates the effect of data normalization on the results, we perform batch normalization (Ioffe and Szegedy, 2015) to the input features when choosing to perform normalization on the dataset. The expression can be referenced with Eq.(6).

where $\gamma$ is the scaling factor and $\beta$ is the bias term. During the network’s training process, both $\gamma$ and $\beta$ are learnable parameters that can be updated by the back-propagation algorithm.

To evaluate each model’s effectiveness comprehensively, we adopt four widely used metrics in DUM: AUUC, QINI, WAU, and the uplift score at the first $k$ percentile (LIFT@$k$). We report the results with $k$ set to 30 and compute these metrics using a standard python package scikit-uplift³³3https://www.uplift-modeling.com.

We select all representative methods described in Section 2 as benchmark models. Because PyTorch can avoid non-determinism on GPU devices better than Tensorflow (Zhu et al., 2021), we choose it to implement all models. We utilized the AdamW optimizer and set a maximum of 20 iterations. Our model training is performed on V100 GPU clusters. Due to the different workloads of the GPU cluster, each model’s epoch training time is for reference only. We record all data settings and model hyperparameters in configuration files to make configurations more convenient. To promote more reproducible research, we plan to open-source the implementation and settings of all models to the community after the paper is accepted.

To ensure a fair and objective evaluation of all models’ true performance, we conduct a thorough search for hyperparameters using QINI as the main evaluation metric. In addition, we adopt an early-stopping mechanism with a patience of 5 to avoid overfitting the training set. To improve efficiency, we use the hyperparameter search library Optuna⁴⁴4https://optuna.org/ to speed up the tuning process. The search range of all hyperparameters is shown in Table 3. Note that different models may have different numbers of auxiliary losses, and we use the same parameter search range across all these losses.

We report the corresponding results in Figure 2, where the left and right represent those obtained on Criteo and Lazada, respectively. Based on the results of Figure 2, we can have the following observations: 1) Without performing feature normalization operations, by performing instance deduplication operations on Criteo, i.e., when there is no apparent inconsistency between the training distribution and the test distribution, the performance of the DUM model can be significantly improved in most cases, except for CEVAE and DESCN, and 2) However, by performing instance deduplication on Lazada, i.e., when there is an apparent inconsistency between the training distribution and the test distribution, most DUM models will suffer, although a small number of DUM models can benefit from it.

We report the corresponding results in Figure 3, where the left and right represent those obtained on Criteo and Lazada, respectively. Based on the results of Figure 3, we can have the following observations: 1) Compared with the case without feature normalization (i.e., Figure 2), the trends of most DUM models on Criteo are similar, and 2) However, most DUM models on Ladaza will have an opposite trend compared to the case without feature normalization (i.e., Figure 2).

Based on the above results, we can find that when deploying a specific DUM model in practical applications, practitioners must consider its compatibility with duplicate instances to determine whether to perform instance deduplication. In addition, it also shows that most DUM models are not robust to the presence of duplicate instances, which requires more research attention on data evaluation, denoising, and selection of DUM models.

We report the corresponding results in Figure 4, where the left and right represent those obtained on Criteo and Lazada, respectively. Based on the results of Figure 4, we can have the following observations: 1) On Criteo, performing feature normalization operations has almost no obvious impact without performing instance deduplication operations, except for a slight decrease in the performance of some DUM models. 2) However, we can find that the impact of performing feature normalization operations on Ladaza is bipolar, with significant improvements to S-Learner, BNN, TARNet, and CFRNet but damage to others.

We report the corresponding results in Figure 5, where the left and right represent those obtained on Criteo and Lazada, respectively. Based on the results of Figure 5, we can have the following observations: 1) Compared to the case where instance deduplication is not performed (i.e., Figure 4), the trends are similar on most DUMs on Criteo, except for more significant fluctuations on several models, such as CFRNet and GANITE. 2) The performance trend changes on Ladaza are similar to those observed on Criteo.

The above results suggest that practitioners also need to make more considerations in feature processing to determine the best operation when deploying a specific DUM model. Furthermore, it also shows that most DUM models are not robust to different feature processing, requiring more research on feature selection and feature embedding learning of DUM.