1. Introduction
As the page is loading and while the user is interacting with the page afterwards, both the application and browser queue various events that are then executed by the browser -- e.g. user agent schedules input events based on user’s activity, the application schedules callbacks for requestAnimationFrame and other callbacks, etc. Once in the queue, the browser dequeues these events one-by-one and executes them.
However, some tasks can take a long time (multiple frames) and if/when that happens, the UI thread may become blocked and block all other tasks as well. To the user, this is commonly visible as a "locked up" page where the browser is unable to respond to user input; this is a major source of bad user experience on the web today:
- Delayed "time to Interactive":
-
while the page is loading, or even completely visually rendered, long tasks often tie up the main thread and prevent the user from interacting with the page. Poorly designed third-party content is frequently the culprit.
- High/variable input latency:
-
critical user-interaction events (e.g. tap, click, scroll, wheel, etc.) are queued behind long tasks which yields janky and unpredictable user experience.
- High/variable event handling latency:
-
like input, processing event callbacks (e.g. onload events, etc.) delay application updates.
- Janky animations and scrolling:
-
some animation and scrolling interactions require coordination between compositor and main threads; if a long task is blocking the main thread it can affect responsiveness of animations and scrolling.
Some applications (and RUM vendors) are already attempting to identify and track cases where "long tasks" happen. For example, one known pattern is to install a ~short periodic timer and inspect the elapsed time between the successive expirations: if the elapsed time is greater than the timer period, then there is high likelihood that one or more long tasks have delayed execution of the event loop. This approach mostly works but has several bad performance implications: by polling to detect long tasks, the application prevents quiescence and long idle blocks (see requestIdleCallback); it’s bad for battery life; there is no way to know what is causing the delay (e.g. first party or third party code).
The RAIL performance model suggests that applications should respond to user input in less than 100ms (for touch move and scrolling, the threshold is 16ms). The goal of this API is to surface notifications about tasks that may prevent the application from hitting these targets. This API surfaces tasks that take 50ms or more. A website without these tasks should respond to user input in under 100ms: it will take less than 50ms to finish the task that is being executed when the user input is received and less than 50ms to execute the task to react to such user input.
1.1. Usage Example
const observer= new PerformanceObserver( function ( list) { for ( const entryof list. getEntries()) { // Process long task notifications: // report back for analytics and monitoring // ... } }); // Register observer for previous and future long task notifications. observer. observe({ type: "long-animation-frame" , buffered: true }); // Long script execution after this will result in queueing // and receiving "long-animation-frame" entries in the observer. // Register observer for previous and future long animation frame notifications. // After this, long periods where the main thread is busy will result in queueing // and receiving "long-animation-frame" entries in the observer. observer. observe({ type: "long-animation-frame" , buffered: true });
1.2. Long Animation Frames vs. Long Tasks
While both long tasks and long animation frames measure congestion and jank, long animation frames provide information that has a better correlation with how users perceive this type of congestion. That’s because long animation frames measure a sequence that begins when the main thread is idle, and end when the frame either renders or the user agents decides there is nothing to render.
The task term is somewhat of an implementation detail, and the long animation frame addition attempts to remedy that by introducing a more user-centric metric of the same phenomenon of main thread congestion/jank.
Because long animation frames are guaranteed to have a maximum of one rendering phase, we can also
use them to expose additional information about the rendering phase itself, such as renderStart
and styleAndLayoutStart
.
2. Long Animation Frame Timing
A long animation frame refers to any of the following occurrences whose duration exceeds 50ms:
-
A task, after which updating the rendering is not necessary.
-
A task after which updating the rendering is necessary, up until rendering is updated.
Long Animation Frame timing involves the following new interfaces:
2.1. PerformanceLongAnimationFrameTiming
interface
[Exposed =Window ]interface :
PerformanceLongAnimationFrameTiming PerformanceEntry { /* Overloading PerformanceEntry */readonly attribute DOMHighResTimeStamp ;
startTime readonly attribute DOMHighResTimeStamp ;
duration readonly attribute DOMString ;
name readonly attribute DOMString ;
entryType readonly attribute DOMHighResTimeStamp ;
renderStart readonly attribute DOMHighResTimeStamp ;
styleAndLayoutStart readonly attribute DOMHighResTimeStamp ;
blockingDuration readonly attribute DOMHighResTimeStamp ; [
firstUIEventTimestamp SameObject ]readonly attribute FrozenArray <PerformanceScriptTiming >; [
scripts Default ]object (); };
toJSON
A PerformanceLongAnimationFrameTiming
has a frame timing info timing info.
The entryType
attribute’s getter step is to return
.
The name
attribute’s getter step is to return
.
The startTime
attribute’s getter step is to return the relative high resolution time given this's timing info's start time and this's relevant global object.
The duration
attribute’s getter step is to return the duration between this's startTime
and the relative high resolution time given this's timing info's end time and this's relevant global object.
The renderStart
attribute’s getter step is to return the relative high resolution time given this's timing info's update the rendering start time and this's relevant global object.
The styleAndLayoutStart
attribute’s getter step is to return the relative high resolution time given this's timing info's style and layout start time and this's relevant global object.
The firstUIEventTimestamp
attribute’s getter step is to return the relative high resolution time given this's timing info's first ui event timestamp and this's relevant global object.
The blockingDuration
attribute’s getter steps are:
-
Let sortedTaskDurations be timing info's task durations, sorted in descending order.
-
If this's timing info's update the rendering start time is not zero, then:
-
Let renderDuration be the duration between this's
renderStart
and the relative high resolution time given this's timing info's end time. -
Increment sortedTaskDurations[0] by renderDuration.
Note: This makes it so that the longest task duration + render duration would be considered blocking if their total duration is >50ms.
-
-
Let totalBlockingDuration be 0.
-
For each duration in sortedTaskDurations, if duration is greater than 50 then increment totalBlockingDuration by duration - 50.
-
Return totalBlockingDuration.
The scripts
attribute’s getter steps are:
-
Let scripts be a list « ».
-
Let entryWindow be this’s relevant global object.
-
For each scriptInfo in this's frame timing info's scripts:
-
Let scriptWindow be scriptInfo’s window.
-
Let scriptEntry be a new
PerformanceScriptTiming
in this's relevant realm, whose timing info is scriptInfo and whose window attribution is the value corresponding to the first matching statement:- scriptWindow is undefined
- scriptWindow is entryWindow
- entryWindow’s associated
Document
's node navigable's ancestor navigables contains scriptWindow’s associatedDocument
's node navigable - scriptWindow’s associated
Document
's node navigable's ancestor navigables contains entryWindow’s associatedDocument
's node navigable - entryWindow’s associated
Document
's node navigable's top-level traversable is scriptWindow’s associatedDocument
's node navigable's top-level traversable - Otherwise
-
Append scriptEntry to scripts.
-
-
Return scripts.
2.2. PerformanceScriptTiming
interface
enum {
ScriptInvokerType ,
"classic-script" ,
"module-script" ,
"event-listener" ,
"user-callback" ,
"resolve-promise" };
"reject-promise" enum {
ScriptWindowAttribution ,
"self" ,
"descendant" ,
"ancestor" ,
"same-page" }; [
"other" Exposed =Window ]interface :
PerformanceScriptTiming PerformanceEntry { /* Overloading PerformanceEntry */readonly attribute DOMHighResTimeStamp ;
startTime readonly attribute DOMHighResTimeStamp ;
duration readonly attribute DOMString ;
name readonly attribute DOMString ;
entryType readonly attribute ScriptInvokerType ;
invokerType readonly attribute DOMString ;
invoker readonly attribute DOMHighResTimeStamp ;
executionStart readonly attribute DOMString ;
sourceURL readonly attribute DOMString ;
sourceFunctionName readonly attribute long long ;
sourceCharPosition readonly attribute DOMHighResTimeStamp ;
pauseDuration readonly attribute DOMHighResTimeStamp ;
forcedStyleAndLayoutDuration readonly attribute Window ?;
window readonly attribute ScriptWindowAttribution ; [
windowAttribution Default ]object (); };
toJSON
A PerformanceScriptTiming
has an associated script timing info timing info.
A PerformanceScriptTiming
has an associated ScriptWindowAttribution
window attribution.
The entryType
attribute’s getter step is to return
.
The name
attribute’s getter step is to return
.
The invokerType
attribute’s getter step is to return this's timing info's invoker type.
The invoker
attribute’s getter steps are:
-
Switch on this’s
invokerType
:- "`classic-script`"
- "`module-script`"
-
Return this’s timing info's source url.
- "`event-listener`"
-
-
Let targetName be this’s timing info's invoker name.
-
If this’s timing info's event target element id is not the empty string, then: Set targetName to the concatenation of « targetName, "#", this’s timing info's event target element id ».
-
Otherwise, If this’s timing info's event target element src attribute is not the empty string, then: Set targetName to the concatenation of « targetName, '[src=', this’s timing info's event target element src attribute, ']' ».
-
Return the concatenation of « targetName, ".on", this's timing info's event type ».
-
- "`user-callback`"
-
Return this’s timing info's invoker name.
- "`resolve-promise`"
- "`reject-promise`"
-
-
If this’s timing info's invoker name is the empty string, then:
-
If this’s
invokerType
is "`resolve-promise`", then return "`Promise.resolve`". -
Otherwise, return "`Promise.reject`".
-
-
Let thenOrCatch be "`then`" if
invokerType
is "`resolve-promise`"; otherwise "`reject-promise`". -
Return the concatenation of « invoker name, ".", thenOrCatch ».
-
The startTime
attribute’s getter step is to return the relative high resolution time given this's timing info's start time and this's relevant global object.
The duration
attribute’s getter step is to return the duration between this's startTime
and the relative high resolution time given this's timing info's end time and this's relevant global object.
The executionStart
attribute’s getter step is to return 0 if this's timing info's execution start time is 0; Otherwise the relative high resolution time given this's timing info's execution start time and this's relevant global object.
The forcedStyleAndLayoutDuration
attribute’s getter step is to return an implementation-defined value that represents time spent performing style and layout synchronously, e.g. by calling getComputedStyle()
oder getBoundingClientRect()
.
Find a way to make this interoperable/normative. Perhaps mark those functions in WebIDL as requiring synchronous style/layout? Also move to timing info once that’s resolved.
The pauseDuration
attribute’s getter step is to return this's timing info's pause duration.
The sourceURL
attribute’s getter step is to return this's timing info's source url.
The sourceFunctionName
attribute’s getter step is to return this's timing info's source function name.
The sourceCharPosition
attribute’s getter step is to return this's timing info's source character position.
The window
attribute’s getter steps are:
-
Let window be the result of calling deref on this's timing info's window.
-
If window is undefined, then return null; Otherwise return window.
The windowAttribution
attribute’s getter step is to return this's window attribution.
3. Processing model
Note: A user agent implementing the Long Animation Frame API would need to include
in supportedEntryTypes
for Window
contexts, respectively.
3.1. Frame Timing Info
frame timing info is a struct used as a bookkeeping detail by the long animation frame algorithms. It has the following items:- start time
- current task start time
- update the rendering start time
- style and layout start time
- first ui event timestamp
- end time
-
A
DOMHighResTimeStamp
, initially 0. Note: all the above are [=monotonic clock/unsafe current time=|unsafe=], and should be coarsened when exposed via an API. - task durations
-
A list of
DOMHighResTimeStamp
, initially empty. - scripts
-
A list of script timing info, initially empty.
- pending script
-
Null or a script timing info, initially null.
script timing info is a struct. It has the following items:
- invoker type
- start time
- end time
- execution start time
-
An unsafe
DOMHighResTimeStamp
, initially 0. - pause duration
-
A
DOMHighResTimeStamp
representing a number of milliseconds, initially 0. - invoker name
- source url
- source function name
- event type
- event target element id
- event target element src attribute
-
A string, initially the empty string.
- source character position
-
A number, initially -1.
- window
A Document
has a null or frame timing info current frame timing info, initially null.
3.2. Report Long Animation Frames
3.2.1. Long Animation Frame Monitoring {#loaf-monitoring}
Document
document:
-
Let ancestors be the ancestor navigables of document.
-
For each ancestorNavigable in ancestors: If ancestorNavigable’s active document's origin is same origin with document’s origin, and ancestorNavigable’s active document's relevant agent is document’s relevant agent, then return ancestorNavigable’s active document.
-
Return document.
The relevant frame timing info for a Document
document is its nearest same-origin root's current frame timing info.
DOMHighResTimeStamp
unsafeTaskStartTime, and a Document
document:
-
Let root be document’s nearest same-origin root.
-
If root’s current frame timing info is null, then set root’s current frame timing info to a new frame timing info whose start time is unsafeTaskStartTime.
-
Set root’s current frame timing info's current task start time to unsafeTaskStartTime.
-
If root’s current frame timing info's 's start time is 0, then set root’s current frame timing info's start time to unsafeTaskStartTime.
DOMHighResTimeStamp
unsafeTaskEndTime, and a Document
document:
-
Let timingInfo be document’s relevant frame timing info.
-
If timingInfo is null, then return.
Note: This can occur if the browser becomes hidden during the sequence.
-
Let safeTaskEndTime be the relative high resolution time given unsafeTaskEndTime and document’s relevant global object.
-
Let safeTaskStartTime be the relative high resolution time given timingInfo’s current task start time and document’s relevant global object.
-
Append the duration between safeTaskStartTime and safeTaskEndTime to timingInfo’s task durations.
-
If the user agent believes that updating the rendering of document’s node navigable would have no visible effect, then report frame timing given document and return.
Note: even though there was no actual visual update, we mark a long animation frame here because it would be blocking in a scenario where it coincided with an unrelated visual update.
Document
document, and a DOMHighResTimeStamp
unsafeStyleAndLayoutStart:
-
Let timingInfo be document’s relevant frame timing info.
-
If timingInfo is null, then return.
Note: This can occur if the browser becomes hidden during the sequence.
-
Set timingInfo’s update the rendering start time to timingInfo’s current task start time.
-
Set timingInfo’s style and layout start time to unsafeStyleAndLayoutStart.
-
report frame timing given document and the unsafe shared current time.
Document
document and a DOMHighResTimeStamp
unsafeEndTime:
-
Let timingInfo be document’s relevant frame timing info.
-
Assert: timingInfo is not null.
-
Let global be document’s relevant global object.
-
Let frameDuration be the duration between the relative high resolution time given timingInfo’s start time and global, and the relative high resolution time given unsafeEndTime and global.
-
If frameDuration is greater than 50 milliseconds, then Queue a new
PerformanceLongAnimationFrameTiming
in document’s relevant realm, whose timing info is timingInfo. -
set document’s nearest same-origin root's current frame timing info to null.
3.2.2. Long Script Monitoring
-
Set scriptTimingInfo’s invoker name to callback’s identifier.
-
Apply source location for scriptTimingInfo given callback.
Function
handler, an environment settings object settings, and a boolean repeat: Create script entry point given settings, "`user-callback`",
and the following steps given a script timing info scriptTimingInfo:
-
Let setTimeoutOrInterval be "setInterval" if repeat is true, "setTimeout" otherwise.
-
Set scriptTimingInfo’s invoker name to concatenation of « TimerHandler:", setTimeoutOrInterval ».
-
If handler is a
Function
, then apply source location for scriptTimingInfo given handler.
Event
event and an EventListener
listener: Create script entry point given listener’s relevant settings object, "`event-listener`",
and the following steps given a script timing info scriptTimingInfo and a frame timing info frameTimingInfo:
-
Set scriptTimingInfo’s event type to event’s
type
. -
Let target be event’s
currentTarget
. -
If target is a
Node
, then:-
Set scriptTimingInfo’s invoker name to target’s
nodeName
. -
If target is an
Element
, then:-
Set scriptTimingInfo’s event target element id to target’s id.
-
Set scriptTimingInfo’s event target element src attribute to the result of getting an attribute value by name "`src`" and target.
-
-
-
Else, set scriptTimingInfo’s invoker name to target’s interface name.
-
Apply source location for scriptTimingInfo given listener’s callback.
-
If event is a
UIEvent
, and frameTimingInfo’s first ui event timestamp is 0, then set frameTimingInfo’s first ui event timestamp to event’stimeStamp
.
Promise
promise and a "`resolve-promise`" or "`reject-promise`" type:
-
Create script entry point given promise’s relevant realm's settings object, type, and the following steps given a script timing info scriptTimingInfo:
-
Set scriptTimingInfo’s invoker name to promise’s invoker name when created.
-
Set scriptTimingInfo’s source url to promise’s script url when created.
-
-
If url is null, then return.
-
If url’s scheme is "`http`" or "`https`", then set scriptTimingInfo’s source url to script’s base URL.
-
Otherwise, if url’s scheme is "`blob`" or "`data`" then set scriptTimingInfo’s source url to the concatenation of « url’s scheme, ":"" ».
-
Create script entry point with script’s settings object, "`classic-script`", and the following step given a script timing info scriptTimingInfo: Set source url for script block given scriptTimingInfo, script, and originalSourceURL.
-
Let settings be script’s settings object.
-
If script’s muted errors is true, then return.
-
If settings is not a
Window
, then return. -
Let document be settings’s
document
. -
Let frameTimingInfo be document’s relevant frame timing info.
-
If frameTimingInfo is null or if frameTimingInfo’s pending script is not null, then return.
-
Assert: frameTimingInfo’s pending script's invoker type is "`classic-script`".
-
Set frameTimingInfo’s pending script's execution start time to the unsafe shared current time.
-
Set scriptTimingInfo’s execution start time to script’s scriptTimingInfo’s start time.
-
Set source url for script block given scriptTimingInfo, script, and script’s base URL.
ScriptInvokerType
invokerType, and steps,
which is an algorithm that takes a script timing info and an optional frame timing info:
-
If settings is not a
Window
, then return. -
Let document be settings’s
document
. -
If document is not fully active oder
, then return.
-
Let frameTimingInfo be document’s relevant frame timing info.
-
If frameTimingInfo is null, then return.
-
If frameTimingInfo’s pending script is not null, then return.
-
Let scriptTimingInfo be a new script timing info whose start time is the unsafe shared current time, and whose invoker type is invokerType.
-
Run steps given scriptTimingInfo and frameTimingInfo.
-
Set scriptTimingInfo’s window to settings.
-
Set frameTimingInfo’s pending script to scriptTimingInfo.
-
Let script be the running script.
-
Let settings be script’s settings object.
-
Let document be settings’s
document
. -
If document is not fully active oder
, then return.
-
Let frameTimingInfo be document’s relevant frame timing info.
-
Let scriptTimingInfo be frameTimingInfo’s pending script.
-
Set frameTimingInfo’s pending script to null.
-
If scriptTimingInfo is null, then return.
-
Set scriptTimingInfo’s end time to the unsafe shared current time.
-
If script is a classic script whose muted errors is true, then:
-
set scriptTimingInfo’s source url to the empty string.
-
set scriptTimingInfo’s source character position to -1.
-
set scriptTimingInfo’s source function name to the empty string.
-
-
If the duration between scriptTimingInfo’s start time and scriptTimingInfo’s end time is greater than 5 milliseconds, then append scriptTimingInfo to frameTimingInfo’s scripts.
Function
callback:
-
The user agent may set scriptTimingInfo’s source url to the source URL of the script where callback was defined.
-
The user agent may set scriptTimingInfo’s source function name to the function name of callback.
-
The user agent may set scriptTimingInfo’s source character position to the character position where callback was defined.
-
Let script be the running script.
-
Let settings be script’s settings object.
-
If settings is not a
Window
, then return. -
Let document be settings’s
document
. -
If document is not fully active oder
, then return.
-
Let frameTimingInfo be document’s relevant frame timing info.
-
If frameTimingInfo is null, then return.
-
If frameTimingInfo’s pending script is null, then return.
-
Increment frameTimingInfo’s pending script's pause duration by the milliseconds value of duration.
4. Additions to existing standards
4.1. Monkey-patches to the WebIDL standard
Promise
interface has an associated string invoker name when created, initially "`Promise`".
The Promise
interface has an associated string script url when created, initially the empty string.
Append the following steps to creating a new promise, before returning the Promise
:
-
Let interfaceName be a string representing the interface responsible for creating this promise.
-
Let attributeName be a string representing the attribute in the interface responsible for creating this promise.
-
Set the created
Promise
's script url when created to the running script's base URL. -
The user-agent may set the created
Promise
's invoker name when created to the last known concatenation of « interfaceName, ".", attributeName »this is quite handwavy, because this is difficult to do in a normative way. Need to see if that can be improved, or if the source location for promise handlers would remain a bit implementation-defined.
Prepend the following step to resolve a promise given Promise
p: Record timing info for promise resolver given p and "`resolve-promise`".
Prepend the following step to reject a promise given Promise
p: Record timing info for promise resolver given p and "`reject-promise`".
5. Security & privacy considerations
The Long Animation Frames API adheres to the same-origin policy by including origin-safe attribution information about the source of the long task. There is a 50ms threshold for long tasks. Durations are only provided in 1 ms granularity. Together this provides adequate protection against cross-origin leaks.
The Long Animation Frames API provides timing information about the duration and type of tasks executed by the user, as well as attribution such as the browsing context causing the function calls. This could enable an attacker to perform side-channel timing attacks to guess the user’s action, or identify the user. For example, a pattern of long script followed by a long render could be put together to guess user’s interaction with a social widget. Detailed function call attribution would be used to determine the user’s action.
5.1. What is Exposed to Observers?
All observers within the top level page (i.e. all iframes in the page and the main frame) will receive notifications about presence of long animation frames. We expose the start time of the task, its duration (with 1 ms granularity), and a pointer to the culprit frame. This information can already be observed today, and with higher resolution, using setTimeout. An attacker can do this by clearing everything else on the page and adding the vulnerable cross-origin resource to ensure that delays from the setTimeout are caused by that resource. Observers in other different pages (tabs or windows) should not receive notifications, regardless of the architecture of the user agent.
5.2. Attack Scenarios Considered
The following are the timing attacks considered:
-
Traditional timing attacks: using external resource load time to reveal the size of private data. For instance the number of hidden pictures in a gallery, whether username is valid, etc. See an example.
-
Side-channel timing attacks: using time for video parsing, script parsing, App Cache reads or Cache API (service workers) usage to uniquely identify a user, or to create a profile of the user’s age, gender, location, and interests etc. For instance, status updates from a social network can be limited to certain demographic (eg. females of age 20-30) the file size of the permalink page can be used to determine whether the user is in the target demographic.
These scenarios are addressed by the 50ms threshold AND respecting cross-origin boundary i.e. not showing task type or additional attribution to untrusted cross origin observers.
5.3. Additional information exposed by the Long Animation Frames API
Since several cross-origin documents can share the same event loop, they can also render as part of the same frame sequence and influence each other’s rendering time. This makes it so that these timings are already somewhat observable cross-origin, e.g. by requesting an animation frame and observing if it is delayed, though long animation frames exposes them at a higher fidelity.To mitigate this, long animation frames are only reported to "participating local roots": only documents that are associated with a work task that contributed to the sequence, or that were rendered as part of the frame, are eligible to observe the long animation frame, and that long animation frame would be available only in their nearest ancestor that is either topmost or has a cross-origin parent.
5.4. PerformanceScriptTiming
and opaque scripts
Since PerformanceScriptTiming
exposes information about script execution, we need to make sure it
doesn’t expose too much information about CORS cross-origin scripts that cannot be easily deduced otherwise.
To do that, we use the existing muted errors boolean, and report an empty sourceURL
in such cases.