Data Engineering Process Fundamentals - Data Analysis and Visualization
Navigating data analysis with established principles and communicating insights through visually engaging dashboards empowers us to extract value from data.
Recent posts
Data Engineering Process Fundamentals - Data Warehouse Model and Transformation Exercise
During this exercise, we delve into the Data Warehouse design and implementation step, crafting robust data models, and designing transformation tasks. We explore how to efficiently load, cleanse, and merge data, ultimately creating dimension and ...
Data Engineering Process Fundamentals - Data Warehouse Model and Transformation
Design and implementation are two pivotal phases for our data warehouse solution. In the design phase, we lay the groundwork by defining the database system, schema model, and technology stack required to support the data warehouse’s implementatio...
Data Engineering Process Fundamentals - Pipeline and Orchestration Exercise
When it comes to writing a data pipeline using Python, there are several options to consider. Apache Spark provides a powerful distributed processing framework, Apache Airflow offers a flexible and scalable solution for workflow orchestration, Pre...
Data Engineering Process Fundamentals - Pipeline and Orchestration
A data pipeline is a workflow of tasks that can be executed in Docker containers. The execution, scheduling, managing and monitoring of the pipeline is refer as orchestration. In order to support the operations of the pipeline and its orchestratio...