XLDB

XLDB refers to eXtremely Large Data Bases. The definition of extremely large refers to data sets that are too big in terms of volume (too much), and/or velocity (too fast), and/of variety (too many places, too many formats) to be handled using conventional solutions.

History

In October 2007 the XLDB experts gathered at SLAC for the First Workshop on Extremely Large Databases. As a result, the XLDB research community was formed. to meet rapidly growing demands, in addition to the original invitational workshop, an open conference, tutorials, and annual satellite events on different continents were added. The main event, held annually at Stanford gathers over 300 technically savvy attendees.

Goals

The main goals of this community include:

• Identify trends, commonalities and major roadblocks related to building extremely large databases
• Bridge the gap between users trying to build extremely large databases and database solution providers worldwide
• Facilitate development and growth of practical technologies for extremely large data stores

XLDB Community

As of 2013, the community consisted of about a thousand members including:

1. Scientists who develop, use, or plan to develop or use XLDB for their research, from laboratories
2. Commercial users of XLDB
3. Providers of database products, including commercial vendors and representatives from open source database communities
4. Academic database researchers.

XLDB Conferences, Workshops and Tutorials

The community meets annually at Stanford where the main event is held each fall, usually in September. These who live too far from California to attend have the opportunity to attend satellite events, organized annually around May/June either in Asia or in Europe.

A detailed report is produced after each workshop.

Year	Place	Link	Bericht	Kommentare
2012	Stanford	[1]	not available yet	conference, workshop and tutorials
2012	Beijing, China	[2]	[3]	satellite conference in Asia
2011	SLAC	[4]	[5]	conference and workshop
2011	Edinburgh, UK	[6]	not available	satellite workshop in Europe
2010	SLAC	[7]	[8]	conference and workshop
2009	Lyon, France	[9]	[10]	workshop
2008	SLAC	[11]	[12]	workshop
2007	SLAC	[13]	[14]	workshop

Tangible results

The XLDB events led to initiating the effort of building a new open source, science database, SciDB.

The XLDB organizers started defining a benchmark for scientific data management systems called SS-DB.

At 2012 the XLDB organizers announced that two major databases that support arrays as first-class objects (MonetDB SciQL and SciDB) have formed a working group in conjunction with XLDB. This working group is proposing a common syntax (provisionally named “ArrayQL”) for manipulating arrays, including array creation and query.

Further reading

• Pavlo A., Paulson E., Rasin A., Abadi D. J., Dewitt D. J., Madden S., and Stonebraker M., A Comparison of Approaches to Large-Scale Data Analysis," Proceedings of the 2009 ACM SIGMOD, http://database.cs.brown.edu/sigmod09/benchmarks-sigmod09.pdf
• Becla, J., et al. 2006, Designing a multi-petabyte database for LSST, http://arxiv.org/abs/cs/0604112.
• Becla, J., & Wang, D. L. 2005, Lessons Learned from Managing a Petabyte, downloaded from http://www.slac.stanford.edu/pubs/slacpubs/10750/slac-pub-10963.pdf on 2007-11-25.
• Bell, G., Gray, J., & Szalay, A. 2005, Petascale computations systems: Balanced cyberinfrastructure in a data-centric world, http://arxiv.org/abs/cs/0701165.
• Duellmann, D. 1999, Petabyte Databases, ACM SIGMOD Record, vol. 28, p. 506, http://www.sigmod.org/sigmod/record/issues/9906/index.html#TutorialSessions.
• Hanushevsky, A., & Nowak, M. 1999, Pursuit of a Scalable High Performance Multi-Petabyte Database, 16th IEEE Symposium on Mass Storage Systems, pp. 169–175, http://citeseer.ist.psu.edu/217883.html.
• Shiers, J., Building Very Large, Distributed Object Databases, downloaded from http://wwwasd.web.cern.ch/wwwasd/cernlib/rd45/papers/dbprog.html on 2007-11-25.