Updating Reporting Windows for Event-level reports in ARA

[chandan-giri]

Problem
The event-level reports in ARA are subject to transmission loss and excessive delays: some conversions registered by the Event API may not be reported or may be delayed. Transmission loss can occur when, for example, the user’s device is not active at the time of the reporting window or if the user clears their cache and deletes the API reports.

Early OT data and associated debug reports suggest that the total magnitude of transmission loss is not ignorable. We find that, on average, the longer a conversion sits in the browser waiting for the next reporting window, the higher the chance of transmission loss through the Event API.

Proposal
We propose to configure the Event API with shorter reporting windows to reduce transmission loss. Currently, Click-Through Conversions are endowed with three windows set to (2 days, 7 days, source expiry). The benefit of shorter windows is that early-arriving conversions will be subject to lower transmission loss rates. However, there are also costs of shorter windows. Conversions that happen after the shorter first window will now be subject to higher loss rates due to a longer gap between the shorter first window and the default 7-day second window, hence we propose changing the second window to a shorter value. We propose a new window configuration of (1 hour, 2 days, source expiry) as a new default.

Analysis framework
Evaluating alternate window configurations requires a target metric. In our analysis, we measure the expected conversion loss as a function of a window configuration W=(w1, w2, w3).

Measuring expected conversion loss requires two inputs. The first is a conversion profile, which tracks the number of conversions observed (ObsConv) as a function of the conversion delay time (conversion time minus click time). Conversion delay time is represented by the t index in the equation above. The conversion profile is directly observable in the logs for any Adtech. The second is a transmission loss profile, which measures the share of reports not sent as a function of the sitting in browser time (reporting window minus conversion time). The transmission loss profile can be learned from OT data.