Explain the differences between wide and narrow dependencies in Apache Spark.

In Apache Spark, dependencies between tasks can be categorized as either wide or narrow.

Narrow dependencies occur when each partition of the parent RDD is used by at most one partition of the child RDD. Operations like map, filter, and flatMap create narrow dependencies. These are efficient because they can be pipelined and executed in a single stage.

Wide dependencies occur when each partition of the parent RDD can be used by multiple partitions of the child RDD. Operations like groupByKey, reduceByKey, and join create wide dependencies. These require a shuffle operation, where data is redistributed across the network, making them more computationally expensive and typically requiring multiple stages.