Data Engineering

A practical view of how data engineers build and run data systems.

Explore various data sources and understand how data is generated and collected.

Explore various data formats used in data engineering.

Learn the essentials of data ingestion in data engineering.

Explore the methods and technologies used to store data efficiently.

Explore the intricacies of ETL processes in data engineering, focusing on how data is extracted, transformed, and loaded efficiently.

Explore the complexities of data warehouses and how they store analytical data.

Explore the intricacies of data lakes and how they organize raw data efficiently.

Explore the intricacies of data pipelines and how data flows between systems.

Explore the intricacies of ensuring data accuracy and reliability in data engineering.

Advanced mastery of real-time data handling, focusing on ambiguous trade-offs and delayed consequences in stream processing.

Advanced mastery of large-scale data platforms, focusing on ambiguous decisions, trade-offs, and long-term consequences.

Advanced judgment in distributed data processing: recognizing limits, trade-offs, and hidden costs.

Advanced judgment in structuring data for ambiguous, high-stakes systems. Focus on trade-offs, delayed effects, and restraint.

Advanced mastery of optimizing data pipelines under ambiguous and shifting constraints. Focus on judgment, restraint, and calibration when common strategies fail.

Mastering judgment in controlling and managing data under ambiguity and risk.

Advanced judgment in protecting data systems under ambiguous and stressful conditions.

Mastering scalability in data engineering means recognizing when growth strategies become harmful, and calibrating judgment under pressure.

Mastering judgment in designing data platform architectures under uncertainty and pressure.

Advanced judgment in integrating data ingestion, storage, and processing within enterprise systems. For those who already build and maintain pipelines, this course focuses on subtle failures, trade-offs, and restraint.