He.ro db: A concept for parallel data processing on heterogeneous hardware

Abstract

Due to the growing demand on processing power and energy efficiency by today's data-intensive applications developers have to deal with heterogeneous hardware platforms composed of specialized computing resources. These are highly efficient for certain workloads but difficult to handle from the software engineering perspective. Even state-of-the-art database management systems do not exploit all heterogeneous hardware components, as their characteristics differ significantly. They are thus hard to integrate within a coherent database architecture. To address this problem, we propose a design concept that is based on a layered system software architecture: He.ro DB transforms a data-flow graph that describes the data-processing application to a task-based execution plan. Task implementations for the different computing resources and a reasonable degree of parallelism are chosen automatically based on available resources. The concept can cover any hardware configuration and application scenario. It is versatile and offers opportunities for independent optimization on each layer. © Springer Nature Switzerland AG 2020.