GraphFrames Tutorials

This section contains step-by-step tutorials to help you learn and master specific features of GraphFrames.

Currently available tutorials:

• Data Setup Tutorial: Set up the Stack Exchange dataset used by the Motif Finding and Pregel tutorials. Download the data dump, install dependencies, and convert XML to Parquet.

• Motif Finding Tutorial: Learn how to perform pattern recognition in graphs using network motif finding. This tutorial demonstrates the technique using the stackexchange.com knowledge graph, showing you how to identify and analyze recurring patterns and substructures within the graph data.

• Pregel Tutorial: Learn to write scalable graph algorithms using GraphFrames' Pregel API. This tutorial teaches the vertex-centric programming model through progressive examples — from simple degree centrality to custom reputation propagation — using real Stack Exchange data.

• Neo4j Integration Tutorial: Offload expensive algorithms like connected components from Neo4j to Spark/GraphFrames and back.

More tutorials will be added in future releases to cover additional GraphFrames features and use cases.

Other Resources

These tutorials are not part of the official GraphFrames documentation.

• GraphFrames on Databricks
• Advanced Deduplication Using Apache Spark: A Guide for Machine Learning Pipelines
• Parallelized Community Detection in Social Networks
• Determining Communities In Graphs Using Label Propagation Algorithm
• Top 10 Graph Processing Algorithms in Spark - A Comprehensive Guide for Data Scientists
• Exploring GraphFrames in PySpark