Package org.apache.spark.status

Class LiveRDD

Object

org.apache.spark.status.LiveRDD

public class LiveRDD
extends Object

Tracker for data related to a persisted RDD.

The RDD storage level is immutable, following the current behavior of RDD.persist(), even though it is mutable in the RDDInfo structure. Since the listener does not track unpersisted RDDs, this covers the case where an early stage is run on the unpersisted RDD, and a later stage it started after the RDD is marked for caching.

Constructor Summary

Constructors

Constructor	Description
LiveRDD(RDDInfo info, StorageLevel storageLevel)

Method Summary

Modifier and Type	Method	Description
long	diskUsed()
LiveRDDDistribution	distribution(org.apache.spark.status.LiveExecutor exec)
scala.Option<LiveRDDDistribution>	distributionOpt(org.apache.spark.status.LiveExecutor exec)
scala.collection.Map<String,LiveRDDDistribution>	getDistributions()
scala.collection.Map<String,LiveRDDPartition>	getPartitions()
RDDInfo	info()
long	lastWriteTime()
long	memoryUsed()
LiveRDDPartition	partition(String blockName)
boolean	removeDistribution(org.apache.spark.status.LiveExecutor exec)
void	removePartition(String blockName)
void	write(org.apache.spark.status.ElementTrackingStore store, long now, boolean checkTriggers)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LiveRDD

public LiveRDD(RDDInfo info,
 StorageLevel storageLevel)

Method Details

diskUsed

public long diskUsed()

distribution

public LiveRDDDistribution distribution(org.apache.spark.status.LiveExecutor exec)

distributionOpt

public scala.Option<LiveRDDDistribution> distributionOpt(org.apache.spark.status.LiveExecutor exec)

getDistributions

public scala.collection.Map<String,LiveRDDDistribution> getDistributions()

getPartitions

public scala.collection.Map<String,LiveRDDPartition> getPartitions()

info

public RDDInfo info()

memoryUsed

public long memoryUsed()

partition

public LiveRDDPartition partition(String blockName)

removeDistribution

public boolean removeDistribution(org.apache.spark.status.LiveExecutor exec)

removePartition

public void removePartition(String blockName)

lastWriteTime

public long lastWriteTime()

write

public void write(org.apache.spark.status.ElementTrackingStore store,
 long now,
 boolean checkTriggers)