PySpark

Jump to bottom

Don Jayamanne edited this page Nov 29, 2016 · 2 revisions

You can now develop and debug PySpark code directly using vscode. More information on Apache Spark can be found here

Features

Develop PySpark code with intellisense support
Running and debugging PySpark code locally

Getting Started

Before developing PySpark code, download the Apache Apark and setup
Create environment variable SPARK_HOME point to installation home and add installation bin directory to global path
Start a PySpark project from a folder

Features in detail

Ctrl+Shift+P to activate command "Python: Update Workspace PySpark Libraries", this enables intellisense
Debug PySpark code locally using PySpark debug option

Code & Debug