rorounifix/pyspark repository overview

⁠Hadoop-Python-Java Docker Image

⁠Overview

Image Name: rorounifix/pyspark:latest
Image Name with Python version: rorounifix/pyspark:3.10

Description: This Docker image provides an Ubuntu-based environment with Python 3.10, OpenJDK 11, PostgreSQL client tools, and Apache Hadoop 3.3.5 pre-installed. It is designed to be a versatile base for developing and running applications that require Hadoop, Java, and Python.

⁠Description

Base Image: ubuntu:22.04

⁠Components Included:

• Python 3.10: A popular high-level programming language for general-purpose programming. Includes python3.10 and pip for package management.
• OpenJDK 11: A widely used Java Development Kit (JDK) version for running Java applications and Hadoop.
• PostgreSQL Client: Command-line tools for interacting with PostgreSQL databases.
• Apache Hadoop 3.3.5: A framework that allows for distributed storage and processing of large data sets across clusters of computers.
• Google Chrome: A web browser for accessing web applications.

⁠Features:

• Hadoop Setup: Installs Apache Hadoop 3.3.5 and configures environment variables (HADOOP_HOME, PATH, and LD_LIBRARY_PATH) for easy access.
• Python and Java: Provides the latest stable Python 3.10 and OpenJDK 11 to support development and execution of Python scripts and Java applications.
• PostgreSQL Client: Enables database operations directly from the container.
• Web Browsing: Includes Google Chrome for web access.

⁠Installation and Usage:

1. Pull the Image:

   docker pull rorounifix/pyspark:latest
   
   

   Copy
2. Run the Container:

   docker run -it --rm rorounifix/pyspark
   

   Copy
3. Verify Installations: Inside the container, you can verify the installations with:

   python3.10 --version
   java -version
   hadoop version
   google-chrome --version
   

   Copy

This setup provides a comprehensive environment for working with Hadoop, Java, Python, PostgreSQL, and browsing capabilities, all in one Docker image.