Kom i kontakt

Kursplan

Databricks Platform and Lakehouse Fundamentals

  • Databricks Lakehouse architecture and components
  • Organizing workspaces and catalogs

Databricks Workspace and Notebooks

  • Workspace navigation and notebook-based development
  • Structuring code into reusable notebooks

Apache Spark Architecture and Execution

  • Spark runtime architecture and execution model
  • Lazy evaluation and the job DAG

PySpark DataFrames and the DataFrame API

  • DataFrame abstractions and schemas
  • Core DataFrame operations and column expressions

Translating SQL to PySpark DataFrames

  • Translating core SQL clauses to DataFrame operations
  • Window functions and aggregations in PySpark

Reading and Writing Data in Databricks

  • Reading from common file and database sources
  • Writing and partitioning data in the Lakehouse

Delta Lake and Table Management

  • Delta tables and ACID transactions
  • Time travel and schema evolution

Data Cleaning and Transformation Patterns

  • Data cleaning and type conversion
  • Building reusable transformation logic

User-Defined Functions and Modular Code

  • Python UDFs and pandas UDFs
  • Modularizing procedural logic into functions

Performance Tuning and Optimization

  • Partitioning and caching strategies
  • Diagnosing bottlenecks with the Spark UI

Structured Streaming Fundamentals

  • Batch versus streaming processing models
  • Streaming DataFrames and basic aggregations

Databricks Jobs and Workflow Orchestration

  • Scheduling notebooks as jobs and tasks
  • Building multi-step workflows with dependencies

Unity Catalog and Data Governance

  • Unity Catalog architecture and namespaces
  • Access control and data lineage

Testing, Debugging, and Production Practices

  • Unit testing PySpark logic
  • Debugging and code quality standards

End-to-End Financial Services Use Cases

  • Building an end-to-end banking ETL pipeline
  • Translating legacy SQL processes to PySpark

Migrating SQL Workloads to PySpark

  • Migration strategy and planning patterns
  • Incremental conversion of SQL workflows to PySpark

Krav

  • Erarenhet av programmering i Python, inklusive funktioner och datatyper.
  • Förståelse för SQL, inklusive joins, aggregationer och underfrågor.
  • Ingen tidigare erfarenhet av Databricks eller PySpark krävs.

Målgrupp

  • Dataingenjörer, dataanalytiker och databrukare.
  • Team som migrerar existerande SQL-baserade arbetsflöden till Databricks och PySpark.
 35 Timmar

Antal deltagare


Pris per deltagare

Vittnesmål (1)

Kommande Kurser

Relaterade Kategorier