Spark: Big Data Cluster Computing in Production

Spark: Big Data Cluster Computing in Production goes beyond general Spark overviews to provide targeted guidance toward using lightning-fast big-data clustering in production. Written by an expert team well-known in the big data community, this book walks you through the challenges in moving from proof-of-concept or demo Spark applications to live Spark in production. Real use cases provide deep insight into common problems, limitations, challenges, and opportunities, while expert tips and tricks help you get the most out of Spark performance. Coverage includes Spark SQL, Tachyon, Kerberos, ML Lib, YARN, and Mesos, with clear, actionable guidance on resource scheduling, db connectors, streaming, security, and much more.

Spark has become the tool of choice for many Big Data problems, with more active contributors than any other Apache Software project. General introductory books abound, but this book is the first to provide deep insight and real-world advice on using Spark in production. Specific guidance, expert tips, and invaluable foresight make this guide an incredibly useful resource for real production settings.

• Review Spark hardware requirements and estimate cluster size
• Gain insight from real-world production use cases
• Tighten security, schedule resources, and fine-tune performance
• Overcome common problems encountered using Spark in production

Spark works with other big data tools including MapReduce and Hadoop, and uses languages you already know like Java, Scala, Python, and R. Lightning speed makes Spark too good to pass up, but understanding limitations and challenges in advance goes a long way toward easing actual production implementation. Spark: Big Data Cluster Computing in Production tells you everything you need to know, with real-world production insight and expert guidance, tips, and tricks.

Keywords: Spark: Big Data Cluster Computing in Production; Ema Orhian; Kai Sasaki; Ilya Ganelin; Brennon York; Atiego; Cloudera; Yahoo; Concur; Capital One; Spark in production; Spark guide; production cluster computing; big data in production; big data tools; open-source big data framework; Apache Software; Spark applications; Spark security; Spark SQL; Spark on Yarn; Spark on Mesos; Spark ML Lib; Spark with Tachyon; Spark data warehousing; Spark database; live Spark in production; Spark production use cases