Abstract The ongoing rise in the availability of data and the increasing sophistication of data-driven analysis methods have motivated companies to gather and examine vast quantities of data for competitive benefits. Various tools for managing analytical data have been created to support these activities. Traditionally, data warehouses have been the go-to option for business analysts for reporting and OLAP. At the... The ongoing rise in the availability of data and the increasing sophistication of data-driven analysis methods have motivated companies to gather and examine vast quantities of data for competitive benefits. Various tools for managing analytical data have been created to support these activities. Traditionally, data warehouses have been the go-to option for business analysts for reporting and OLAP. At the same time, data lakes have introduced a new approach supporting complex analytics. Given the distinct features and intended audiences of these two primary data platforms, companies often find it necessary to use both, leading to complicated, prone to errors, and costly systems. To tackle these challenges, there has been a recent push to merge the capabilities of data warehouses and data lakes into what is known as lake houses, aiming to offer a unified platform for various analytics needs. This work offers a summary of the development of analytical data platforms from data warehouses to data lakes to lake houses and discusses the unique attributes of lake houses. It also explores what functionalities data lakes lack, hindering their transition to lake houses. Read More Read Less

Abstract Recommendation systems have advanced beyond simple user-item matching techniques in studies. However, these matching techniques are still widely used in practical applications, mainly because they are simpler to troubleshoot and adjust. The current systems, while effective, fall short of supporting the optimization of algorithms. They prioritize the consistency of top-performing accuracy over the simplicity of algorithm creation and upkeep, which... Recommendation systems have advanced beyond simple user-item matching techniques in studies. However, these matching techniques are still widely used in practical applications, mainly because they are simpler to troubleshoot and adjust. The current systems, while effective, fall short of supporting the optimization of algorithms. They prioritize the consistency of top-performing accuracy over the simplicity of algorithm creation and upkeep, which can impede quick and repeated testing and fixing. This project introduces an AutoML-driven framework with a modular code base for deep session-based recommendation systems and a built-in component for automated HyperParameter Tuning (HPT4Rec). This framework streamlines the process of finding the optimal session-based model for a specific dataset, enabling the model to be regularly updated as the nature and volume of the data change in real-world settings. Our tests on the RecSys 2015 dataset have shown that HPT4Rec achieves results comparable to the best available. The outcomes of our tests underscore the importance of ongoing and iterative adjustment of parameters, especially in real-world applications, for maintaining high recommendation precision. Read More Read Less

Abstract Over time, changes in the evolution of data pipelines are inevitable, particularly in terms of their structure, meaning, and the roles of the pipeline operators. Addressing these changes or providing ongoing maintenance is expensive. This current study investigates the necessity of having the ability to evolve within pipeline frameworks. This paper describes evolution as a two-step process involving self-awareness and... Over time, changes in the evolution of data pipelines are inevitable, particularly in terms of their structure, meaning, and the roles of the pipeline operators. Addressing these changes or providing ongoing maintenance is expensive. This current study investigates the necessity of having the ability to evolve within pipeline frameworks. This paper describes evolution as a two-step process involving self-awareness and self-adaptation. Additionally, a conceptual requirements model is introduced, which includes criteria for self-awareness and self-adaptation, as well as addressing the aspects of data, operators, pipelines, and the environment. The absence of these capabilities in current frameworks highlights a significant deficiency. Addressing this deficiency would be a major benefit for both practitioners and researchers. This study proposes and establishes the groundwork for a framework capable of managing evolutionary changes. Read More Read Less