This is a primitive for implementing graph algorithms. This method aggregates values from the neighboring edges and vertices of each vertex. See AggregateMessages for detailed documentation.

Breadth-first search (BFS)

Refer to the documentation of org.graphframes.lib.BFS for the description of the output.

Connected component algorithm.

See org.graphframes.lib.ConnectedComponents for more details.

The degree of each vertex in the graph, returned as a DataFrame with two columns:

• GraphFrame.ID the ID of the vertex
• 'degree' (integer) the degree of the vertex Note that vertices with 0 edges are not returned in the result.

The vertex names in the vertices DataFrame, in order.

Helper method for toGraphX which specifies the schema of edge attributes. The edge attributes of the returned edges are given as a Row, and this method defines the column ordering in that Row.

The dataframe representation of the edges of the graph.

It contains two columns called GraphFrame.SRC and GraphFrame.DST that contain the ids of the source vertex and the destination vertex of each edge, respectively. It may also contain various other columns with user-defined attributes for each edge.

For symmetric graphs, both pairs src -> dst and dst -> src are present with the same attributes for each pair.

The order of the columns is available in edgeColumns.

Motif finding: Searching the graph for structural patterns

Motif finding uses a simple Domain-Specific Language (DSL) for expressing structural queries. For example, graph.find("(a)-[e]->(b); (b)-[e2]->(a)") will search for pairs of vertices a,b connected by edges in both directions. It will return a DataFrame of all such structures in the graph, with columns for each of the named elements (vertices or edges) in the motif. In this case, the returned columns will be in order of the pattern: "a, e, b, e2."

DSL for expressing structural patterns:

• The basic unit of a pattern is an edge. For example, "(a)-[e]->(b)" expresses an edge e from vertex a to vertex b. Note that vertices are denoted by parentheses (a), while edges are denoted by square brackets [e].
• A pattern is expressed as a union of edges. Edge patterns can be joined with semicolons. Motif "(a)-[e]->(b); (b)-[e2]->(c)" specifies two edges from a to b to c.
• Within a pattern, names can be assigned to vertices and edges. For example, "(a)-[e]->(b)" has three named elements: vertices a,b and edge e. These names serve two purposes:
  • The names can identify common elements among edges. For example, "(a)-[e]->(b); (b)-[e2]->(c)" specifies that the same vertex b is the destination of edge e and source of edge e2.
  • The names are used as column names in the result DataFrame. If a motif contains named vertex a, then the result DataFrame will contain a column "a" which is a StructType with sub-fields equivalent to the schema (columns) of GraphFrame.vertices. Similarly, an edge e in a motif will produce a column "e" in the result DataFrame with sub-fields equivalent to the schema (columns) of GraphFrame.edges.
  • Be aware that names do *not* identify *distinct* elements: two elements with different names may refer to the same graph element. For example, in the motif "(a)-[e]->(b); (b)-[e2]->(c)", the names a and c could refer to the same vertex. To restrict named elements to be distinct vertices or edges, use post-hoc filters such as resultDataframe.filter("a.id != c.id").
• It is acceptable to omit names for vertices or edges in motifs when not needed. E.g., "(a)-[]->(b)" expresses an edge between vertices a,b but does not assign a name to the edge. There will be no column for the anonymous edge in the result DataFrame. Similarly, "(a)-[e]->()" indicates an out-edge of vertex a but does not name the destination vertex. These are called *anonymous* vertices and edges.
• An edge can be negated to indicate that the edge should *not* be present in the graph. E.g., "(a)-[]->(b); !(b)-[]->(a)" finds edges from a to b for which there is *no* edge from b to a.

Restrictions:

• Motifs are not allowed to contain edges without any named elements: "()-[]->()" and "!()-[]->()" are prohibited terms.
• Motifs are not allowed to contain named edges within negated terms (since these named edges would never appear within results). E.g., "!(a)-[ab]->(b)" is invalid, but "!(a)-[]->(b)" is valid.

More complex queries, such as queries which operate on vertex or edge attributes, can be expressed by applying filters to the result DataFrame.

This can return duplicate rows. E.g., a query "(u)-[]->()" will return a result for each matching edge, even if those edges share the same vertex u.

The in-degree of each vertex in the graph, returned as a DataFame with two columns:

• GraphFrame.ID the ID of the vertex "- "inDegree" (int) storing the in-degree of the vertex Note that vertices with 0 in-edges are not returned in the result.

Label propagation algorithm.

See org.graphframes.lib.LabelPropagation for more details.

The out-degree of each vertex in the graph, returned as a DataFrame with two columns:

• GraphFrame.ID the ID of the vertex
• "outDegree" (integer) storing the out-degree of the vertex Note that vertices with 0 out-edges are not returned in the result.

PageRank algorithm.

See org.graphframes.lib.PageRank for more details.

Parallel personalized PageRank algorithm.

See org.graphframes.lib.ParallelPersonalizedPageRank for more details.

Persist the dataframe representation of vertices and edges of the graph with the given storage level.

Power Iteration Clustering (PIC), a scalable graph clustering algorithm developed by Lin and Cohen. From the abstract: PIC finds a very low-dimensional embedding of a dataset using truncated power iteration on a normalized pair-wise similarity matrix of the data.

PowerIterationClustering algorithm.

Pregel algorithm.

Shortest paths algorithm.

See org.graphframes.lib.ShortestPaths for more details.

Strongly connected components algorithm.

See org.graphframes.lib.StronglyConnectedComponents for more details.

SVD++ algorithm.

See org.graphframes.lib.SVDPlusPlus for more details.

Converts this GraphFrame instance to a GraphX Graph. Vertex and edge attributes are the original rows in vertices and edges, respectively.

Note that vertex (and edge) attributes include vertex IDs (and source, destination IDs) in order to support non-Long vertex IDs. If the vertex IDs are not convertible to Long values, then the values are indexed in order to generate corresponding Long vertex IDs (which is an expensive operation).

The column ordering of the returned Graph vertex and edge attributes are specified by vertexColumns and edgeColumns, respectively.

Triangle count algorithm.

See org.graphframes.lib.TriangleCount for more details.

Returns triplets: (source vertex)-[edge]->(destination vertex) for all edges in the graph. The DataFrame returned has 3 columns, with names: GraphFrame.SRC, GraphFrame.EDGE, and GraphFrame.DST. The 2 vertex columns have schema matching GraphFrame.vertices, and the edge column has a schema matching GraphFrame.edges.

Mark the dataframe representation of vertices and edges of the graph as non-persistent, and remove all blocks for it from memory and disk.

The column names in the vertices DataFrame, in order.

Helper method for toGraphX which specifies the schema of vertex attributes. The vertex attributes of the returned Graph are given as a Row, and this method defines the column ordering in that Row.

The dataframe representation of the vertices of the graph.

It contains a column called GraphFrame.ID with the id of the vertex, and various other user-defined attributes with other attributes.

The order of the columns is available in vertexColumns.