Computer Science > Databases
[Submitted on 30 Aug 2024]
Title:Updateable Data-Driven Cardinality Estimator with Bounded Q-error
View PDF HTML (experimental)Abstract:Modern Cardinality Estimators struggle with data updates. This research tackles this challenge within single-table. We introduce ICE, an Index-based Cardinality Estimator, the first data-driven estimator that enables instant, tuple-leveled updates.
ICE has learned two key lessons from the multidimensional index and applied them to solve cardinality estimation in dynamic scenarios: (1) Index possesses the capability for swift training and seamless updating amidst vast multidimensional data. (2) Index offers precise data distribution, staying synchronized with the latest database version. These insights endow the index with the ability to be a highly accurate, data-driven model that rapidly adapts to data updates and is resilient to out-of-distribution challenges during query testing. To make a solitary index support cardinality estimation, we have crafted sophisticated algorithms for training, updating, and estimating, analyzing unbiasedness and variance.
Extensive experiments demonstrate the superiority of ICE. ICE offers precise estimations and fast updates/construction across diverse workloads. Compared to state-of-the-art real-time query-driven models, ICE boasts superior accuracy (2-3 orders of magnitude more precise), faster updates (4.7-6.9 times faster), and significantly reduced training time (up to 1-3 orders of magnitude faster).
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.