class AllPaths extends Arguments with Serializable with WithDirection with WithLocalCheckpoints with WithCheckpointInterval with WithIntermediateStorageLevel
Computes all simple paths between source and destination vertices.
This algorithm enumerates paths up to maxPathLength hops. It supports directed and undirected
traversal as well as optional edge filtering. It returns all simple paths between source and
destination vertices. Here the term "simple" means no repeated vertices. For example, if there
are paths A-B-C, A-D-C and the edge B-A, user asked to find all the paths between "A" and "C"
only A-B-C and A-D-C will be returned, but not the A-B-A-D-C. The default value of the
maxPathLength is 5. Keep in mind that requesting maxPathLength of the scale of the graph
diameter may tend this algorithm will try to return (almost) all simple paths in the graph that
can create huge performance degradation or even OOM-like errors. Algorithm supports both
directed and undirected graphs.
Returned DataFrame schema:
path: array of vertex ids in traversal orderlen: number of edges in the path (Long)
Note: in the case of undirected graph an algorithm run on the internal graph made by union edges and reversed edges. It is assumed that graph does not have multi-edges. Results may be unstable and unpredictable for the graph with multi-edges.
- Alphabetic
- By Inheritance
- AllPaths
- WithIntermediateStorageLevel
- WithCheckpointInterval
- Logging
- WithLocalCheckpoints
- WithDirection
- Serializable
- Arguments
- AnyRef
- Any
- by any2stringadd
- by StringFormat
- by Ensuring
- by ArrowAssoc
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- def +(other: String): String
- def ->[B](y: B): (AllPaths, B)
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- val checkpointInterval: Int
- Attributes
- protected
- Definition Classes
- WithCheckpointInterval
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def edgeFilter(value: String): AllPaths.this.type
Sets an optional filter expression applied to edges during traversal.
Sets an optional filter expression applied to edges during traversal. Only edges satisfying this condition will be considered.
- value
a SQL expression string evaluated against edge attributes
- returns
this instance for method chaining
- def edgeFilter(value: Column): AllPaths.this.type
Sets an optional filter expression applied to edges during traversal.
Sets an optional filter expression applied to edges during traversal. Only edges satisfying this condition will be considered.
- value
a Column expression evaluated against edge attributes
- returns
this instance for method chaining
- def ensuring(cond: (AllPaths) => Boolean, msg: => Any): AllPaths
- def ensuring(cond: (AllPaths) => Boolean): AllPaths
- def ensuring(cond: Boolean, msg: => Any): AllPaths
- def ensuring(cond: Boolean): AllPaths
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def fromExpr(value: String): AllPaths.this.type
Sets the expression identifying the source (starting) vertices.
Sets the expression identifying the source (starting) vertices.
- value
a SQL expression string evaluated against vertex attributes to select source vertices
- returns
this instance for method chaining
- def fromExpr(value: Column): AllPaths.this.type
Sets the expression identifying the source (starting) vertices.
Sets the expression identifying the source (starting) vertices.
- value
a Column expression evaluated against vertex attributes to select source vertices
- returns
this instance for method chaining
- def getCheckpointInterval: Int
Gets checkpoint interval.
Gets checkpoint interval.
- Definition Classes
- WithCheckpointInterval
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getIntermediateStorageLevel: StorageLevel
Gets storage level for intermediate datasets that require multiple passes.
Gets storage level for intermediate datasets that require multiple passes.
- Definition Classes
- WithIntermediateStorageLevel
- def getIsDirected: Boolean
Gets should graph be considered as directed.
Gets should graph be considered as directed.
- returns
true if directed
- Definition Classes
- WithDirection
- def getUseLocalCheckpoints: Boolean
Gets whether local checkpoints are being used instead of regular checkpoints.
Gets whether local checkpoints are being used instead of regular checkpoints.
- returns
true if local checkpoints are enabled, false otherwise
- Definition Classes
- WithLocalCheckpoints
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- val intermediateStorageLevel: StorageLevel
- Attributes
- protected
- Definition Classes
- WithIntermediateStorageLevel
- val isDirected: Boolean
- Attributes
- protected
- Definition Classes
- WithDirection
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def logDebug(s: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(s: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(s: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarn(s: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def maxPathLength(value: Int): AllPaths.this.type
Sets the maximum path length (number of edges) for the enumerated paths.
Sets the maximum path length (number of edges) for the enumerated paths.
Setting a large value (e.g. on the scale of the graph diameter) may cause the algorithm to attempt to collect a very large number of paths, leading to severe performance degradation or out-of-memory errors. Use with caution on large or densely connected graphs.
- value
the maximum number of edges in a path; must be greater than 0. Default is 5.
- returns
this instance for method chaining
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def resultIsPersistent(): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def run(): DataFrame
Executes the AllPaths algorithm and returns all simple paths between the specified source and destination vertices.
Executes the AllPaths algorithm and returns all simple paths between the specified source and destination vertices.
- returns
a DataFrame with the following columns:
path: an array of vertex ids in traversal orderlen: the number of edges in the path (Long)
- def setCheckpointInterval(value: Int): AllPaths.this.type
Sets checkpoint interval in terms of number of iterations (default: 2).
Sets checkpoint interval in terms of number of iterations (default: 2). Checkpointing regularly helps recover from failures, clean shuffle files, shorten the lineage of the computation graph, and reduce the complexity of plan optimization. As of Spark 2.0, the complexity of plan optimization would grow exponentially without checkpointing. Hence, disabling or setting longer-than-default checkpoint intervals are not recommended. Checkpoint data is saved under
org.apache.spark.SparkContext.getCheckpointDirwith prefix of the algorithm name. If the checkpoint directory is not set, this throws ajava.io.IOException. Set a nonpositive value to disable checkpointing. This parameter is only used when the algorithm is set to "graphframes". Its default value might change in the future.- Definition Classes
- WithCheckpointInterval
- See also
org.apache.spark.SparkContext.setCheckpointDirin Spark API doc
- def setIntermediateStorageLevel(value: StorageLevel): AllPaths.this.type
Sets storage level for intermediate datasets that require multiple passes (default:
MEMORY_AND_DISKSets storage level for intermediate datasets that require multiple passes (default:
MEMORY_AND_DISK- Definition Classes
- WithIntermediateStorageLevel
- def setIsDirected(value: Boolean): AllPaths.this.type
Sets should graph be cosidered as directed.
Sets should graph be cosidered as directed.
- value
true to handle graph as directed
- Definition Classes
- WithDirection
- def setUseLocalCheckpoints(value: Boolean): AllPaths.this.type
Sets whether to use local checkpoints instead of regular checkpoints (default: false).
Sets whether to use local checkpoints instead of regular checkpoints (default: false). Local checkpoints are faster but less reliable as they don't survive node failures.
- value
true to use local checkpoints, false for regular checkpoints
- returns
this instance
- Definition Classes
- WithLocalCheckpoints
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toExpr(value: String): AllPaths.this.type
Sets the expression identifying the destination (target) vertices.
Sets the expression identifying the destination (target) vertices.
- value
a SQL expression string evaluated against vertex attributes to select destination vertices
- returns
this instance for method chaining
- def toExpr(value: Column): AllPaths.this.type
Sets the expression identifying the destination (target) vertices.
Sets the expression identifying the destination (target) vertices.
- value
a Column expression evaluated against vertex attributes to select destination vertices
- returns
this instance for method chaining
- def toString(): String
- Definition Classes
- AnyRef → Any
- val useLocalCheckpoints: Boolean
- Attributes
- protected
- Definition Classes
- WithLocalCheckpoints
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)
- def formatted(fmtstr: String): String
- Implicit
- This member is added by an implicit conversion from AllPaths toStringFormat[AllPaths] performed by method StringFormat in scala.Predef.
- Definition Classes
- StringFormat
- Annotations
- @deprecated @inline()
- Deprecated
(Since version 2.12.16) Use
formatString.format(value)instead ofvalue.formatted(formatString), or use thef""string interpolator. In Java 15 and later,formattedresolves to the new method in String which has reversed parameters.
- def →[B](y: B): (AllPaths, B)
- Implicit
- This member is added by an implicit conversion from AllPaths toArrowAssoc[AllPaths] performed by method ArrowAssoc in scala.Predef.
- Definition Classes
- ArrowAssoc
- Annotations
- @deprecated
- Deprecated
(Since version 2.13.0) Use
->instead. If you still wish to display it as one character, consider using a font with programming ligatures such as Fira Code.