Skip to content

erekjiang/BigData_SafeEntry_Analytics

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

BigData_SafeEntry_Analytics

Installing python3

https://docs.python-guide.org/starting/install3/osx/

Installing spark

For MacOS - https://medium.com/beeranddiapers/installing-apache-spark-on-mac-os-ce416007d79f

Spark Master UI - http://localhost:8080
Spark Worker UI - http://localhost:8081

Installing hadoop

For MacOS - https://towardsdatascience.com/installing-hadoop-on-a-mac-ec01c67b003c

hdfs UI - http://localhost:9870/dfshealth.html#tab-overview
hadoop UI - http://localhost:8088/cluster

Installing Kafka

https://kafka.apache.org/quickstart

Tools

big data file viewer - https://github.com/Eugene-Mark/bigdata-file-viewer
Jupiter - https://jupyter.org/install

Spark - 3.0.1
Hadoop - 3.3.0
Kafka - kafka_2.13-2.7.0
Python - 3.9

Distributed or replicated OLTP Multi-datacenter deployments Multi-region deployments Cloud migrations Infrastructure initiatives built for the cloud

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •