Dockerfile linksYou can use Python extension modules and libraries with your AWS Glue ETL scripts as long as they are written in pure Python. C libraries such as pandas are not supported at the present time, nor are extensions written in other languages.
-- AWS
AWS Glue Development enviroment based on svajiraya/aws-glue-libs fix.
/opt/samples/glue/opt/samples/cdk/opt/samples/cloudformation# install docker and configure aliases
curl -sSL https://raw.githubusercontent.com/webysther/aws-glue-docker/master/start.sh | sh
# to use pandas
glue
# or pyspark
glue-spark
# here you are inside docker
# Glue PySpark (REPL)
pyspark
# Glue PySpark
# /app is you current folder
glue-spark sparksubmit /app/spark_script.py
# Test
glue pytest
# aliases inside docker (backwards compatibility)
gluesparksubmit == sparksubmit
gluepyspark == pyspark
gluepytest == pytest
MIT License. Please see License File for more information.
Content type
Image
Digest
Size
2.7 GB
Last updated
over 6 years ago
docker pull webysther/aws-glue