python airflow.contrib.hooks.spark_submit_hook.SparkSubmitHook is a hook that provides a way to interact with Apache Spark's spark-submit command. It allows users to easily submit Spark applications from within a Python script. This hook can be used to pass various parameters and configurations to the spark-submit command, such as the application name, master URL, Spark configuration properties, and the path to the Spark application jar or python file. It provides methods to submit the Spark application, track its progress, and obtain the application ID and logs. This hook is a useful utility for integrating Spark applications with Apache Airflow workflows.
Python SparkSubmitHook - 52 examples found. These are the top rated real world Python examples of airflow.contrib.hooks.spark_submit_hook.SparkSubmitHook extracted from open source projects. You can rate examples to help us improve the quality of examples.