Skip to content

Latest commit

 

History

History
8 lines (4 loc) · 649 Bytes

File metadata and controls

8 lines (4 loc) · 649 Bytes

Introduction

In this guide, I'm going to introduce you some techniques for tuning your Apache Spark jobs for optimal efficiency. Using Spark to deal with massive datasets can become nontrivial, especially when you are dealing with a terabyte or higher volume of data. The first thing that comes up could be to use a large cluster of hundreds of machines with hundreds of cores and petabytes of RAM, but using a super-sized cluster has a cost that can exponentially grow. That's why I wrote this guide to help you to achieve better performance and saving costs.