Recent posts

Data Engineering Process Fundamentals - Design and Planning Exercise

8 minute read

Ozkary Data Engineering Process Design and Planning Exercise

Infrastructure planning is a critical aspect of every technical project, laying the foundation for successful project delivery. In the case of a Data Engineering project, it becomes even more crucial. To support our project’s objectives, we need t...

Data Engineering Process Fundamentals - Design and Planning

4 minute read

Ozkary Data Engineering Process Design and Planning

A data engineering design is the actual plan to build the technical solution. It includes the system architecture, data integration, flow and pipeline orchestration, the data storage platform, transformation and management, data processing and ana...

Data Engineering Process Fundamentals - Discovery Exercise

9 minute read

Ozkary Data Engineering Process Discovery Exercise with Jupyter

In this discovery exercise lab, we review a problem statement and do the analysis to define the scope of work and requirements. Preliminary data analysis must be done by downloading some sample files with Python and running some queries using Jupy...

Data Engineering Process Fundamentals - Discovery

4 minute read

Ozkary Data Engineering Process Discovery

During the discovery step of a Data Engineering Process, we look to identify and clearly document a problem statement, which helps us have an understanding of what we are trying to solve. We also look at our analytical approach to make observation...

Data Engineering Process Fundamentals

4 minute read

Ozkary Data Engineering Process

Data Engineering is changing constantly. From cloud data platforms and pipeline automation to data streaming and visualizations tools, new innovations are impacting that way we build today’s data and analytical solutions. This is an overview of Da...