OakDB

A Robust Tuning Paradigm

The increasing demand for hybrid transacational/analytical processing (HTAP) applications has created the need for new database paradigms. This has resulted in databases facing the challenge of solving increasingly complex tuning problems. Yet with increasing complexity, databases do not take into consideration variability in access patterns and resource availability. We propose a design for a robust database, OakDB, that aims to answer the following

Can we design and tune systems that perform close to optimal subject to variability in access patterns and resource availability?

Sources of Variability

Modern state-of-the-art database management systems (DBMS) utilize many tools for the tuning process, and generally these tools will assume some workload knowledge. This may be in the form of a previous workload sample or model estimate, however, these assumptions are not always accurate. We have identified two major sources of unpredictabilty that DBMS tuning tools fail to consider, workload drift and resource availability.

Figure 1: Beta distribution ~ Beta(5, 2) demonstrating shifts across logical blocks of the database.

One part of workload drift occurs when access patterns shift across the logical blocks of the database. Figure 1 shows an example of a beta distribution shifted at varying percentages. Drift such as this can occur due to changes in the usage landscape. For example, DBMS servicing social media applications may experience workload drift when viral events cause drastic changes in the user usage. The other part of workload drift occurs between different operations. Tunings may expect a certain percentage of point queries versus inserts, but HTAP applications can clearly cause these percentages to flucuate.

Additionally, DBMS experience unpredictabilty in resource availability. For many valuable reasons, DBMS have been migrating to the cloud over the years. The massive amounts of resources allow for high performance systems while additionally creating environments that can service multiple applications on a single computing unit. Clearly this provides many benefits as our data collection continues to grow, however, the cloud also introduces a problem with resource availability. With multiple applications living on the same node, applications may not recieve the exact amount of resources they request or expect. This can be extremely deterimental to a DBMS performance.

Research

To address these sources of unpredictabilty we break down this project into five distinct stages; (i) quantifying workload drift, (ii) modeling system performance, (iii) measuring robustness, (iv) formulating a robust optimization problem, and (v) building OakDB. Each stage layouts a strategy for exploring a issue, and tying things together results in a deliverable system to show how to incorporate robustness into a DBMS.

Current Work

We have tested various access pattern distributions of point queries and inserts. Figure 2 shows a mirrored configuration of access patterns for two operations and figure 3 shows a configuration where point queries are constant while inserts are heavily focused on towards earlier blocks. For rotational drift, we shifted both distributions by integer amounts, and for mass drift, we shaved percentages off the point queries distribution and scaled up inserts accordingly. Our cost function is based off of the databases latency per operation.

Figure 2: Point queries distributed Beta(1, 5). Inserts distributed Beta(5, 1). Cost increases is in reference to the zero uncertainity point the DBMS was tuned for.

Figure 3: Point queries distributed Beta(1, 1). Inserts distributed Beta(2, 5). Note the scale is not as wide as figure 2.

We see certain types of access pattern combinations result in a database tuning that can be stable or unstable in respect to workload drift. The symmetric workload, when pertubated either via rotational noise or mass results in extreme cost increases of more than 10x compared to figure 3. This exploration gives us an insight on the space of heuristics to explore inorder to create robust data systems.

Meet the Team