Demo entry 6357066

scala

   

Submitted by anonymous on Apr 21, 2017 at 03:58
Language: Scala. Code size: 528 Bytes.

val conf = new SparkConf().setAppName("Simple Application")
                          .setMaster("spark://192.16.7.65:7077")
		  	  .setJars(Array("/home/lgl/project/FreshQoE/test.jar"))
val sc = new SparkContext(conf)
val arr = 1 to 30
// create RDD by collection, four partitions
val rdd = sc.parallelize(arr, 4)
// filter by key 'mod(3)'
val rdd1 = rdd.map(x => (x%3, x))
              .filter(_._1 == 0)
  	      .map(_._2)
// aggregation the result
val result = rdd1.reduce(_ + _)
print(result)
// output: 165

This snippet took 0.00 seconds to highlight.

Back to the Entry List or Home.

Delete this entry (admin only).