DataFrame.LocalCheckpoint(Boolean) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Returns a locally checkpointed version of this DataFrame.
public Microsoft.Spark.Sql.DataFrame LocalCheckpoint(bool eager = true);member this.LocalCheckpoint : bool -> Microsoft.Spark.Sql.DataFramePublic Function LocalCheckpoint (Optional eager As Boolean = true) As DataFrameParameters
- eager
- Boolean
Whether to checkpoint this DataFrame immediately
Returns
DataFrame object
Remarks
Checkpointing can be used to truncate the logical plan of this `DataFrame`, which is especially useful in iterative algorithms where the plan may grow exponentially. Local checkpoints are written to executor storage and despite potentially faster they are unreliable and may compromise job completion.