Big Data: Hadoop, Hive, Spark, ZooKeeper, Solr, Kafka, Nutch, MongoDB, …

Big Data

Big Data ecosystem
Apache Hadoop
- Install and configure Apache Hadoop (single node cluster) (3.2.0)
- HDFS Commands
  - HDFS - DFS Commands
  - HDFS - DFSADMIN Commands
- ORC/Parquet/Avro Tools
  - ORC Tools (1.5.4)
  - Parquet Tools (1.9.0)
  - Avro Tools (1.9.0)
Apache Hive
- Install and configure Apache Hive (HiveServer, Hive MetaStore) (3.1.1)
- Manage Hive Databases
Apache Spark
- Install and configure Apache Spark (standalone) (2.4.3)
- Access Hive Tables using Spark SQL
- Spark Tools
  - Spark Interactive Shell (Scala): spark-shell
  - Spark Interactive Shell (Python): pyspark
  - Spark Interactive Shell (R): sparkR
  - Submitting Applications: spark-submit
  - Spark SQL CLI: spark-sql
- Spark API: RDD, DataFrame, Dataset
Apache ZooKeeper (3.6.1)
Apache Solr (8.5.2)
Install and configure Apache Kafka (2.3.0)
Install and configure MongoDB (4.2)
Install and configure Apache Nutch (2.3.1)