Packages

object GraphFrame extends Serializable with Logging

Linear Supertypes
Logging, Serializable, Serializable, AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. GraphFrame
  2. Logging
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val DST: String

    Column name for destination vertices of edges.

    Column name for destination vertices of edges.

  5. val EDGE: String

    Column name for edge in GraphFrame.triplets.

    Column name for edge in GraphFrame.triplets. In GraphFrame.triplets, this is a column of edges with schema matching GraphFrame.edges.

  6. val ID: String

    Column name for vertex IDs in GraphFrame.vertices Note that GraphFrame assigns a unique long ID to each vertex, If the vertex ID type is one of byte / int / long / short type, GraphFrame casts the original IDs to long as the unique long ID, otherwise GraphFrame generates the unique long ID by Spark function monotonically_increasing_id which is less performant.

  7. val SRC: String

    Column name for source vertices of edges.

    Column name for source vertices of edges.

  8. def apply(vertices: DataFrame, edges: DataFrame): GraphFrame

    Create a new GraphFrame from vertex and edge DataFrames.

    Create a new GraphFrame from vertex and edge DataFrames.

    vertices

    Vertex DataFrame. This must include a column "id" containing unique vertex IDs. All other columns are treated as vertex attributes.

    edges

    Edge DataFrame. This must include columns "src" and "dst" containing source and destination vertex IDs. All other columns are treated as edge attributes.

    returns

    New GraphFrame instance

  9. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  10. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def fromEdges(e: DataFrame): GraphFrame

    Create a new GraphFrame from an edge DataFrame.

    Create a new GraphFrame from an edge DataFrame. The resulting GraphFrame will have GraphFrame.vertices with a single "id" column.

    Note: The GraphFrame.vertices DataFrame will be persisted at level StorageLevel.MEMORY_AND_DISK.

    e

    Edge DataFrame. This must include columns "src" and "dst" containing source and destination vertex IDs. All other columns are treated as edge attributes.

    returns

    New GraphFrame instance

  14. def fromGraphX[V, E](originalGraph: GraphFrame, graph: Graph[V, E], vertexNames: Seq[String] = Nil, edgeNames: Seq[String] = Nil)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[V], arg1: scala.reflect.api.JavaUniverse.TypeTag[E]): GraphFrame

    Given:

    Given:

    • a GraphFrame originalGraph
    • a GraphX graph derived from the GraphFrame using GraphFrame.toGraphX this method merges attributes from the GraphX graph into the original GraphFrame.

    This method is useful for doing computations using the GraphX API and then merging the results with a GraphFrame. For example, given:

    • GraphFrame originalGraph
    • GraphX Graph[String, Int] graph with a String vertex attribute we want to call "category" and an Int edge attribute we want to call "count" We can call fromGraphX(originalGraph, graph, Seq("category"), Seq("count")) to produce a new GraphFrame. The new GraphFrame will be an augmented version of originalGraph, with new GraphFrame.vertices column "category" and new GraphFrame.edges column "count" added.

    See org.graphframes.examples.BeliefPropagation for example usage.

    V

    the type of the vertex data

    E

    the type of the edge data

    originalGraph

    Original GraphFrame used to compute the GraphX graph.

    graph

    GraphX graph. Vertex and edge attributes, if any, will be merged into the original graph as new columns. If the attributes are Product types such as tuples, then each element of the Product will be put in a separate column. If the attributes are other types, then the entire GraphX attribute will become a single new column.

    vertexNames

    Column name(s) for vertex attributes in the GraphX graph. If there is no vertex attribute, this should be empty. If there is a singleton attribute, this should have a single column name. If the attribute is a Product type, this should be a list of names matching the order of the attribute elements.

    edgeNames

    Column name(s) for edge attributes in the GraphX graph. If there is no edge attribute, this should be empty. If there is a singleton attribute, this should have a single column name. If the attribute is a Product type, this should be a list of names matching the order of the attribute elements.

    returns

    original graph augmented with vertex and column attributes from the GraphX graph

  15. def fromGraphX[VD, ED](graph: Graph[VD, ED])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[VD], arg1: scala.reflect.api.JavaUniverse.TypeTag[ED]): GraphFrame

    Converts a GraphX Graph instance into a GraphFrame.

    Converts a GraphX Graph instance into a GraphFrame.

    This converts each org.apache.spark.rdd.RDD in the Graph to a DataFrame using schema inference.

    Vertex ID column names will be converted to "id" for the vertex DataFrame, and to "src" and "dst" for the edge DataFrame.

  16. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  17. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  18. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  19. def logDebug(s: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  20. def logInfo(s: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  21. def logTrace(s: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  22. def logWarn(s: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  23. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  24. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  25. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  26. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  27. def toString(): String
    Definition Classes
    AnyRef → Any
  28. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  30. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from Logging

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

conversions

Ungrouped