distributive data processing framework -Apache Spark Report writing
- Country :
Australia
Tasks:
- Using Spark, write a program to count the number of words in book.txt.
Example:
Input:A distributed system is a collection of autonomous computing elements that appears to its users as a single coherent system.
Output:system: 2, distributed: 1, ..
- Using Spark, write a program to count how many times each letter appeared in the book.txt.
- Using Spark, write a program to replace the words to lowercase letters and write it the file words_lower.txt.
- Using Spark, write a program to replace spaces with - in the book.txt and write it to words-.txt.
- Using Spark, compute the sum of the numbers given in numbers.txt in the numbers.zip file.
- Additionally, you are given files Numbers2.txt, Numbers4.txt, Numbers8.txt, Numbers16.txt, and Numbers32.txt.
- Compute the sum of the numbers in the individual files and plot a bar-graph. On the x-axis plot the size of the file and on y-axis plot the time taken by the Spark to compute the result.
Report
Write a 1-page report on Spark and mention its main features & use cases. For instance, what kind of data can be processed in it. What are RDDs?
