Packages

trait RandomWalkBase extends Serializable with Logging with WithIntermediateStorageLevel

Base trait for implementing random walk algorithms on graph data. Provides common functionality for generating random walks across a graph structure.

Linear Supertypes
WithIntermediateStorageLevel, Logging, Serializable, AnyRef, Any
Known Subclasses
RandomWalkWithRestart
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. RandomWalkBase
  2. WithIntermediateStorageLevel
  3. Logging
  4. Serializable
  5. AnyRef
  6. Any
Implicitly
  1. by any2stringadd
  2. by StringFormat
  3. by Ensuring
  4. by ArrowAssoc
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Abstract Value Members

  1. abstract def runIter(graph: GraphFrame, prevIterationDF: Option[DataFrame], iterSeed: Long): DataFrame

    Runs a single iteration of the random walk.

    Runs a single iteration of the random walk.

    graph

    prepared graph

    prevIterationDF

    DataFrame from previous iteration (if any)

    iterSeed

    seed for this iteration

    returns

    DataFrame result of this iteration

    Attributes
    protected

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. def +(other: String): String
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toany2stringadd[RandomWalkBase] performed by method any2stringadd in scala.Predef.
    Definition Classes
    any2stringadd
  4. def ->[B](y: B): (RandomWalkBase, B)
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toArrowAssoc[RandomWalkBase] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  5. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. val batchSize: Int

    Size of each batch in the random walk process.

    Size of each batch in the random walk process.

    Attributes
    protected
  8. def cleanUp(): Unit

    Deletes all temporary files associated with a given instance.

    Deletes all temporary files associated with a given instance. This method uses Hadoop FileSystem to remove the directory containing batch files for the specified run ID. The temporary prefix must be set and accessible via the current SparkContext's Hadoop configuration.

  9. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  10. def ensuring(cond: (RandomWalkBase) => Boolean, msg: => Any): RandomWalkBase
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toEnsuring[RandomWalkBase] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  11. def ensuring(cond: (RandomWalkBase) => Boolean): RandomWalkBase
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toEnsuring[RandomWalkBase] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  12. def ensuring(cond: Boolean, msg: => Any): RandomWalkBase
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toEnsuring[RandomWalkBase] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  13. def ensuring(cond: Boolean): RandomWalkBase
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toEnsuring[RandomWalkBase] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  14. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  16. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  17. def getIntermediateStorageLevel: StorageLevel

    Gets storage level for intermediate datasets that require multiple passes.

    Gets storage level for intermediate datasets that require multiple passes.

    Definition Classes
    WithIntermediateStorageLevel
  18. def getRunId(): String

    Get the generated (or provided) runID.

    Get the generated (or provided) runID. This method returns current runID!

  19. val globalSeed: Long

    Global random seed for reproducibility.

    Global random seed for reproducibility.

    Attributes
    protected
  20. val graph: GraphFrame

    GraphFrame on which random walks are performed.

    GraphFrame on which random walks are performed.

    Attributes
    protected
  21. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  22. val intermediateStorageLevel: StorageLevel
    Attributes
    protected
    Definition Classes
    WithIntermediateStorageLevel
  23. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  24. def logDebug(s: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  25. def logInfo(s: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  26. def logTrace(s: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  27. def logWarn(s: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  28. val maxNbrs: Int

    Maximum number of neighbors to consider per vertex during random walks.

    Maximum number of neighbors to consider per vertex during random walks.

    Attributes
    protected
  29. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  30. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  31. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  32. val numBatches: Int

    Number of batches to run in the random walk process.

    Number of batches to run in the random walk process.

    Attributes
    protected
  33. val numWalksPerNode: Int

    Number of random walks to generate per node.

    Number of random walks to generate per node.

    Attributes
    protected
  34. def onGraph(graph: GraphFrame): RandomWalkBase.this.type

    Sets the graph to perform random walks on.

    Sets the graph to perform random walks on.

    graph

    the GraphFrame to run random walks on

    returns

    this RandomWalkBase instance for chaining

  35. def prepareGraph(iterationSeed: Long): GraphFrame

    Prepares the graph for random walk by limiting neighbors and handling direction.

    Prepares the graph for random walk by limiting neighbors and handling direction.

    returns

    prepared GraphFrame

    Attributes
    protected
  36. def resultIsPersistent(): Unit
    Attributes
    protected
    Definition Classes
    Logging
  37. def run(): DataFrame

    Executes the random walk algorithm on the set graph.

    Executes the random walk algorithm on the set graph.

    returns

    DataFrame containing the random walks

  38. val runID: String

    Unique identifier for the current random walk run.

    Unique identifier for the current random walk run.

    Attributes
    protected
  39. def setBatchSize(value: Int): RandomWalkBase.this.type

    Sets the batch size.

    Sets the batch size.

    value

    batch size

    returns

    this RandomWalkBase instance for chaining

  40. def setGlobalSeed(value: Long): RandomWalkBase.this.type

    Sets the global random seed.

    Sets the global random seed.

    value

    the seed value

    returns

    this RandomWalkBase instance for chaining

  41. def setIntermediateStorageLevel(value: StorageLevel): RandomWalkBase.this.type

    Sets storage level for intermediate datasets that require multiple passes (default: MEMORY_AND_DISK).

    Sets storage level for intermediate datasets that require multiple passes (default: MEMORY_AND_DISK).

    Definition Classes
    WithIntermediateStorageLevel
  42. def setMaxNbrsPerVertex(value: Int): RandomWalkBase.this.type

    Sets the maximum number of neighbors per vertex.

    Sets the maximum number of neighbors per vertex.

    value

    the max number of neighbors

    returns

    this RandomWalkBase instance for chaining

  43. def setNumBatches(value: Int): RandomWalkBase.this.type

    Sets the number of batches.

    Sets the number of batches.

    value

    number of batches

    returns

    this RandomWalkBase instance for chaining

  44. def setNumWalksPerNode(value: Int): RandomWalkBase.this.type

    Sets the number of walks per node.

    Sets the number of walks per node.

    value

    number of walks

    returns

    this RandomWalkBase instance for chaining

  45. def setRunId(value: String): RandomWalkBase.this.type

    Sets the random walk runID.

    Sets the random walk runID. If provided, cached batches from existing random walk run will be reused. User should be careful, that temporary prefix points to the right direction as well the cached data starting from the set index exists.

  46. def setStartingFromBatch(value: Int): RandomWalkBase.this.type

    Sets the startng batch index for the continous mode.

    Sets the startng batch index for the continous mode. See @setWalkId comment for details.

  47. def setTemporaryPrefix(value: String): RandomWalkBase.this.type

    Sets the temporary prefix for storing intermediate results.

    Sets the temporary prefix for storing intermediate results.

    value

    the prefix string

    returns

    this RandomWalkBase instance for chaining

  48. def setUseEdgeDirection(value: Boolean): RandomWalkBase.this.type

    Sets whether to use edge direction.

    Sets whether to use edge direction.

    value

    true if the graph is directed

    returns

    this RandomWalkBase instance for chaining

  49. val startingIteration: Int

    Starting batch index for continous mode

    Starting batch index for continous mode

    Attributes
    protected
  50. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  51. val temporaryPrefix: Option[String]

    Optional prefix for temporary storage during random walks.

    Optional prefix for temporary storage during random walks.

    Attributes
    protected
  52. def toString(): String
    Definition Classes
    AnyRef → Any
  53. val useEdgeDirection: Boolean

    Whether to respect edge direction in the graph (true for directed graphs).

    Whether to respect edge direction in the graph (true for directed graphs).

    Attributes
    protected
  54. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  55. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  56. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

  2. def formatted(fmtstr: String): String
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toStringFormat[RandomWalkBase] performed by method StringFormat in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @deprecated @inline()
    Deprecated

    (Since version 2.12.16) Use formatString.format(value) instead of value.formatted(formatString), or use the f"" string interpolator. In Java 15 and later, formatted resolves to the new method in String which has reversed parameters.

  3. def →[B](y: B): (RandomWalkBase, B)
    Implicit
    This member is added by an implicit conversion from RandomWalkBase toArrowAssoc[RandomWalkBase] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @deprecated
    Deprecated

    (Since version 2.13.0) Use -> instead. If you still wish to display it as one character, consider using a font with programming ligatures such as Fira Code.

Inherited from WithIntermediateStorageLevel

Inherited from Logging

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd fromRandomWalkBase to any2stringadd[RandomWalkBase]

Inherited by implicit conversion StringFormat fromRandomWalkBase to StringFormat[RandomWalkBase]

Inherited by implicit conversion Ensuring fromRandomWalkBase to Ensuring[RandomWalkBase]

Inherited by implicit conversion ArrowAssoc fromRandomWalkBase to ArrowAssoc[RandomWalkBase]

Ungrouped