pyspark.sql.DataFrame.foreachPartition

DataFrame.foreachPartition(f)

Applies the f function to each partition of this DataFrame.

This a shorthand for df.rdd.foreachPartition().

New in version 1.3.0.

Examples

>>> def f(people):
...     for person in people:
...         print(person.name)
>>> df.foreachPartition(f)

pyspark.sql.DataFrame.foreach pyspark.sql.DataFrame.freqItems