AcademyRulesData and reporting

PySpark ETL Best Practices for Cursor

Data engineering teams that want PySpark jobs written in a consistent, testable way.

  • Recommendedour rating
  • 40,892repository stars
  • CC0-1.0licence
  • 4 months agolast update
mkdir -p .cursor/rules && curl -fsSL https://raw.githubusercontent.com/PatrickJS/awesome-cursorrules/main/rules/pyspark-etl-best-practices-cursorrules-prompt-file.mdc -o .cursor/rules/pyspark-etl-best-practices.mdc
pyspark-etl-best-practices.mdc

Loading the file...

What it does

A Cursor rules file for writing and reviewing PySpark ETL code. It covers a base ETL class that manages the Spark session, configuration through a factory function, and code that is testable and safe for cumulative or snapshot tables, with Spark SQL and Apache Iceberg in mind.

Use cases

  1. 01Scaffold a new PySpark ETL job
  2. 02Review a job for snapshot table safety
  3. 03Make a job configurable without touching sys.argv

Questions about
PySpark ETL Best Practices for Cursor.

What is PySpark ETL Best Practices for Cursor used for?

Data engineering teams that want PySpark jobs written in a consistent, testable way. A Cursor rules file for writing and reviewing PySpark ETL code. It covers a base ETL class that manages the Spark session, configuration through a factory function, and code that is testable and safe for cumulative or snapshot tables, with Spark SQL and Apache Iceberg in mind.

How do I install PySpark ETL Best Practices for Cursor?

Run this in your terminal: mkdir -p .cursor/rules && curl -fsSL https://raw.githubusercontent.com/PatrickJS/awesome-cursorrules/main/rules/pyspark-etl-best-practices-cursorrules-prompt-file.mdc -o .cursor/rules/pyspark-etl-best-practices.mdc

Is PySpark ETL Best Practices for Cursor open source?

Yes. The code is on GitHub (PatrickJS/awesome-cursorrules) under the CC0-1.0 licence.

Is PySpark ETL Best Practices for Cursor safe to use?

It passed our automatic scan for credential theft, hidden instructions and risky install commands. Third party open source software. Sabemos AI does not maintain it. Check the code and permissions before you connect it to company data.