AWS Data Processing

AWS Data Processing MCP Server

Data processing and transformation services.

What is an MCP Server?⁠

MCP Info

Image Building Info

Attribute	Details
Dockerfile	https://github.com/awslabs/mcp/blob/76ec6993aec4c6281e5250cbeba8db6a5863f2ff/src/aws-dataprocessing-mcp-server/Dockerfile⁠
Docker Image built by	Docker Inc.
Docker Scout Health Score
Verify Signature	COSIGN_REPOSITORY=mcp/signatures cosign verify mcp/aws-dataprocessing-mcp-server --key https://raw.githubusercontent.com/docker/keyring/refs/heads/main/public/mcp/latest.pub
Licence	Apache License 2.0

Available Tools (32)

Tools Details

Tool: add_inline_policy

Add a new inline policy to an IAM role.

This tool creates a new inline policy with the specified permissions and adds it to an IAM role. Inline policies are embedded within the role and cannot be attached to multiple roles. Commonly used for granting data processing services access to AWS resources, enabling Glue jobs to access data sources, and configuring permissions for CloudWatch logging and S3 access.

Requirements

• The server must be run with the --allow-write flag
• The role must exist in your AWS account
• The policy name must be unique within the role
• You cannot modify existing policies with this tool

Permission Format

The permissions parameter can be either a single policy statement or a list of statements.

Single Statement Example

{
    "Effect": "Allow",
    "Action": ["s3:GetObject", "s3:PutObject"],
    "Resource": "arn:aws:s3:::example-bucket/*"
}

Common Data Processing Permission Examples

Glue Job Permissions

{
    "Effect": "Allow",
    "Action": [
        "glue:*",
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket",
        "iam:PassRole"
    ],
    "Resource": "*"
}

EMR Cluster Permissions

{
    "Effect": "Allow",
    "Action": [
        "elasticmapreduce:*",
        "ec2:DescribeInstances",
        "ec2:DescribeSecurityGroups",
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject"
    ],
    "Resource": "*"
}

Athena Query Permissions

{
    "Effect": "Allow",
    "Action": [
        "athena:*",
        "glue:GetDatabase",
        "glue:GetTable",
        "glue:GetPartition",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject"
    ],
    "Resource": "*"
}

Usage Tips

• Follow the principle of least privilege by granting only necessary permissions
• Use specific resources rather than "*" whenever possible
• Consider using conditions to further restrict permissions
• Group related permissions into logical policies with descriptive names Parameters|Type|Description -|-|- permissions|string|Permissions to include in the policy as IAM policy statements in JSON format. Can be either a single statement object or an array of statement objects. policy_name|string|Name of the inline policy to create. Must be unique within the role. role_name|string|Name of the IAM role to add the policy to. The role must exist.

Tool: analyze_s3_usage_for_data_processing

Analyze S3 bucket usage patterns for data processing services (Glue, EMR, Athena).

This tool helps identify which buckets are actively used by data processing services and which ones might be idle or underutilized.

Tool: create_data_processing_role

Create a new IAM role for data processing services.

This tool creates a new IAM role with the appropriate trust relationship for the specified data processing service (Glue, EMR, or Athena). It can also attach managed policies and add an inline policy to the role.

Requirements

• The server must be run with the --allow-write flag
• The role name must be unique within your AWS account
• Valid AWS credentials with permissions to create IAM roles

Service Types

• glue: Creates a role that can be assumed by the Glue service
• emr: Creates a role that can be assumed by the EMR service
• athena: Creates a role that can be assumed by the Athena service

Common Managed Policies. add these policies

• Glue: 'arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole'
• EMR: 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole'
• Athena: 'arn:aws:iam::aws:policy/service-role/AmazonAthenaFullAccess'

Usage Tips

• Always provide a descriptive name and description for the role
• Attach only the necessary managed policies to follow least privilege
• Use inline policies for custom permissions specific to your use case
• Consider adding S3 access permissions for data sources and targets Parameters|Type|Description -|-|- role_name|string|Name of the IAM role to create. Must be unique within your AWS account. service_type|string|Type of data processing service: 'glue', 'emr', or 'athena'. description|stringoptional|Optional description for the IAM role. inline_policy|stringoptional|Optional inline policy to add to the role. managed_policy_arns|stringoptional|Optional list of managed policy ARNs to attach to the role.

Tool: get_policies_for_role

Get all policies attached to an IAM role.

This tool retrieves all policies associated with an IAM role, providing a comprehensive view of the role's permissions and trust relationships. It helps you understand the current permissions, identify missing or excessive permissions, troubleshoot data processing issues, and verify trust relationships for service roles.

Requirements

• The role must exist in your AWS account
• Valid AWS credentials with permissions to read IAM role information

Response Information

The response includes role ARN, assume role policy document (trust relationships), role description, managed policies with their documents, and inline policies with their documents.

Usage Tips

• Use this tool before adding new permissions to understand existing access
• Check the assume role policy to verify which services or roles can assume this role
• Look for overly permissive policies that might pose security risks
• Use with add_inline_policy to implement least-privilege permissions
• For Glue jobs, ensure the role has access to required data sources and targets
• For EMR clusters, verify EC2 instance profile permissions
• For Athena queries, check S3 bucket access permissions Parameters|Type|Description -|-|- role_name|string|Name of the IAM role to get policies for. The role must exist in your AWS account.

Tool: get_roles_for_service

Get all IAM roles that can be assumed by a specific AWS service.

This tool retrieves all IAM roles in your AWS account that have a trust relationship with the specified service. It helps you identify which roles can be used for services like Glue jobs, EMR clusters, or Athena queries, making it easier to select the appropriate role when creating these resources.

Service Types

Common service types include:

• glue: AWS Glue service (glue.amazonaws.com)
• emr: Amazon EMR service (elasticmapreduce.amazonaws.com)
• athena: Amazon Athena service (athena.amazonaws.com)
• You can also specify other AWS service principals

Response Information

The response includes a list of roles that can be assumed by the specified service, with details such as role name, ARN, description, creation date, and the full assume role policy document.

Usage Tips

• Use this tool to find existing roles before creating new ones
• Verify that roles have the necessary permissions for your use case
• For Glue jobs, look for roles with AWSGlueServiceRole or similar policies
• For EMR clusters, look for roles with AmazonElasticMapReduceRole or similar policies
• For Athena queries, look for roles with AmazonAthenaFullAccess or similar policies Parameters|Type|Description -|-|- service_type|string|Type of data processing service: 'glue', 'emr', 'athena', or other AWS service name.

Tool: list_s3_buckets

List S3 buckets that have 'glue' in their name and are in the specified region.

This tool helps identify S3 buckets commonly used for data processing workflows, particularly those related to AWS Glue operations. It provides usage statistics and idle time information to help with resource management.

Requirements

• Valid AWS credentials with permissions to list S3 buckets
• S3:ListAllMyBuckets permission

Response Information

The response includes bucket name, creation date, region, object count, last modified date, and idle time analysis.

Usage Tips

• Use this tool to find existing data processing buckets before creating new ones
• Monitor idle buckets that haven't been accessed for 90+ days
• Verify bucket regions match your data processing service regions
• Check object counts to understand bucket usage patterns Parameters|Type|Description -|-|- region|stringoptional|AWS region to filter buckets by (defaults to AWS_REGION environment variable)

Tool: manage_aws_athena_data_catalogs

Manage AWS Athena data catalogs with both read and write operations.

This tool provides operations for managing Athena data catalogs, including creating, retrieving, listing, updating, and deleting data catalogs. Data catalogs are used to organize and access data sources in Athena, enabling you to query data across various sources like AWS Glue Data Catalog, Apache Hive metastores, or federated sources.

Requirements

• The server must be run with the --allow-write flag for create-data-catalog, delete-data-catalog, and update-data-catalog operations
• Appropriate AWS permissions for Athena data catalog operations

Operations

• create-data-catalog: Create a new data catalog
• delete-data-catalog: Delete an existing data catalog
• get-data-catalog: Get information about a single data catalog
• list-data-catalogs: List all data catalogs
• update-data-catalog: Update an existing data catalog

Usage Tips

• Use list-data-catalogs to find available data catalogs
• Data catalogs can be of type LAMBDA, GLUE, HIVE, or FEDERATED
• Parameters are specific to the type of data catalog

Example

# List all data catalogs
{'operation': 'list-data-catalogs', 'max_results': 10}

# Create a Glue data catalog
{
    'operation': 'create-data-catalog',
    'name': 'my-glue-catalog',
    'type': 'GLUE',
    'description': 'My Glue Data Catalog',
    'parameters': {'catalog-id': '123456789012'},
}

Tool: manage_aws_athena_databases_and_tables

Manage AWS Athena databases and tables with read operations.

This tool provides operations for retrieving information about databases and tables in Athena data catalogs. These are read-only operations that do not modify any resources.

Requirements

• Appropriate AWS permissions for Athena database and table operations

Operations

• get-database: Get information about a single database
• get-table-metadata: Get metadata for a specific table
• list-databases: List all databases in a data catalog
• list-table-metadata: List metadata for all tables in a database

Usage Tips

• Use list-databases to find available databases in a data catalog
• Use list-table-metadata to find available tables in a database
• The expression parameter for list-table-metadata supports filtering tables by name pattern

Example

# List all databases in a catalog
{'operation': 'list-databases', 'catalog_name': 'AwsDataCatalog', 'max_results': 10}

# Get metadata for a specific table
{
    'operation': 'get-table-metadata',
    'catalog_name': 'AwsDataCatalog',
    'database_name': 'my_database',
    'table_name': 'my_table',
}

Tool: manage_aws_athena_named_queries

Manage saved SQL queries in AWS Athena.

This tool provides operations for creating, retrieving, updating, and deleting named queries in AWS Athena. Named queries are saved SQL statements that can be easily reused, shared with team members, and executed without having to rewrite complex queries.

Requirements

• The server must be run with the --allow-write flag for create-named-query, delete-named-query, and update-named-query operations
• Appropriate AWS permissions for Athena named query operations

Operations

• batch-get-named-query: Get details for up to 50 named queries by their IDs
• create-named-query: Save a new SQL query with a name and description
• delete-named-query: Remove a saved query
• get-named-query: Retrieve a single named query by ID
• list-named-queries: List available named query IDs
• update-named-query: Modify an existing named query

Example

# Create a named query
create_response = await manage_aws_athena_named_queries(
    operation='create-named-query',
    name='Daily Active Users',
    description='Query to calculate daily active users',
    database='analytics',
    query_string='SELECT date, COUNT(DISTINCT user_id) AS active_users FROM user_events GROUP BY date ORDER BY date DESC',
    work_group='primary',
)

# Later, retrieve the named query
query = await manage_aws_athena_named_queries(
    operation='get-named-query', named_query_id=create_response.named_query_id
)

Tool: manage_aws_athena_query_executions

Execute and manage AWS Athena SQL queries.

This tool provides comprehensive operations for AWS Athena query management, including starting new queries, monitoring execution status, retrieving results, and analyzing performance statistics.

Requirements

• The server must be run with the --allow-write flag if start-query-execution contains any write operation for example DDL commands, Insert, Update, Delete Commands or any flag updates
• Appropriate AWS permissions for Athena query operations

Operations

• batch-get-query-execution: Get details for up to 50 query executions by their IDs
• get-query-execution: Get complete information about a single query execution
• get-query-results: Retrieve the results of a completed query
• get-query-runtime-statistics: Get performance statistics for a query execution
• list-query-executions: List available query execution IDs (up to 50)
• start-query-execution: Execute a new SQL query
• stop-query-execution: Cancel a running query

Example

# Start a new query
response = await manage_aws_athena_queries(
    operation='start-query-execution',
    query_string='SELECT * FROM my_database.my_table LIMIT 10',
    query_execution_context={'Database': 'my_database', 'Catalog': 'my_catalog'},
    work_group='primary',
)

# Get the query results
results = await manage_aws_athena_queries(
    operation='get-query-results', query_execution_id=response.query_execution_id
)

Tool: manage_aws_athena_workgroups

Manage AWS Athena workgroups with both read and write operations.

This tool provides operations for managing Athena workgroups, including creating, retrieving, listing, updating, and deleting workgroups. Workgroups allow you to isolate queries for different user groups and control query execution settings.

Requirements

• The server must be run with the --allow-write flag for create-work-group, delete-work-group, and update-work-group operations
• Appropriate AWS permissions for Athena workgroup operations

Operations

• create-work-group: Create a new workgroup
• delete-work-group: Delete an existing workgroup
• get-work-group: Get information about a single workgroup
• list-work-groups: List all workgroups
• update-work-group: Update an existing workgroup

Usage Tips

• Use workgroups to isolate different user groups and control costs
• Configure workgroup settings to enforce query limits and output locations
• Use tags to organize and track workgroups Parameters|Type|Description -|-|- operation|string|Operation to perform: create-work-group, delete-work-group, get-work-group, list-work-groups, update-work-group. Choose read-only operations when write access is disabled. configuration|stringoptional|Configuration for the workgroup, including result configuration, enforcement options, etc. (optional for create-work-group and update-work-group). description|stringoptional|Description of the workgroup (optional for create-work-group and update-work-group). max_results|stringoptional|Maximum number of results to return for list-work-groups operation. name|stringoptional|Name of the workgroup (required for create-work-group, delete-work-group, get-work-group, update-work-group). next_token|stringoptional|Pagination token for list-work-groups operation. recursive_delete_option|stringoptional|Whether to recursively delete the workgroup and its contents (optional for delete-work-group). state|stringoptional|State of the workgroup: ENABLED or DISABLED (optional for create-work-group and update-work-group). tags|stringoptional|Tags for the workgroup (optional for create-work-group). Example {'ResourceType': 'Workgroup'}

Tool: manage_aws_emr_clusters

Manage AWS EMR EC2 clusters with comprehensive control over cluster lifecycle.

This tool provides operations for managing Amazon EMR clusters running on EC2 instances, including creating, configuring, monitoring, modifying, and terminating clusters. It also supports security configuration management for EMR clusters.

Requirements

• The server must be run with the --allow-write flag for create-cluster, modify-cluster, modify-cluster-attributes, terminate-clusters, create-security-configuration, and delete-security-configuration operations
• Appropriate AWS permissions for EMR cluster operations

Operations

• create-cluster: Create a new EMR cluster with specified configurations
• describe-cluster: Get detailed information about a specific EMR cluster
• modify-cluster: Modify the step concurrency level of a running cluster
• modify-cluster-attributes: Modify auto-termination and termination protection settings
• terminate-clusters: Terminate one or more EMR clusters
• list-clusters: List all EMR clusters with optional filtering
• create-security-configuration: Create a new EMR security configuration
• delete-security-configuration: Delete an existing EMR security configuration
• describe-security-configuration: Get details about a specific security configuration
• list-security-configurations: List all available security configurations

Example

# Create a basic EMR cluster with Spark
{
    'operation': 'create-cluster',
    'name': 'SparkCluster',
    'release_label': 'emr-7.9.0',
    'applications': [{'Name': 'Spark'}],
    'instances': {
        'InstanceGroups': [
            {
                'Name': 'Master',
                'InstanceRole': 'MASTER',
                'InstanceType': 'm5.xlarge',
                'InstanceCount': 1,
            },
            {
                'Name': 'Core',
                'InstanceRole': 'CORE',
                'InstanceType': 'm5.xlarge',
                'InstanceCount': 2,
            },
        ],
        'Ec2KeyName': 'my-key-pair',
        'KeepJobFlowAliveWhenNoSteps': true,
    },
}

Usage Tips

• Use list-clusters to find cluster IDs before performing operations on specific clusters
• Check cluster state before performing operations that require specific states
• For large result sets, use pagination with marker parameter
• When creating clusters, consider using security configurations for encryption and authentication Parameters|Type|Description -|-|- operation|string|Operation to perform: create-cluster, describe-cluster, modify-cluster, modify-cluster-attributes, terminate-clusters, list-clusters, create-security-configuration, delete-security-configuration, describe-security-configuration, list-security-configurations. Choose read-only operations when write access is disabled. applications|stringoptional|The applications to be installed on the cluster (optional for create-cluster). Example: [{"Name": "Hadoop"}, {"Name": "Spark"}] auto_scaling_role|stringoptional|An IAM role for automatic scaling policies (optional for create-cluster). Default role is EMR_AutoScaling_DefaultRole. auto_terminate|stringoptional|Whether the cluster should auto-terminate after completing steps (optional for modify-cluster-attributes). bootstrap_actions|stringoptional|A list of bootstrap actions to run on the cluster (optional for create-cluster). Each action contains Name, ScriptBootstrapAction properties. cluster_id|stringoptional|ID of the EMR cluster (required for describe-cluster, modify-cluster, modify-cluster-attributes). cluster_ids|stringoptional|List of EMR cluster IDs (required for terminate-clusters). cluster_states|stringoptional|The cluster state filters to apply when listing clusters (optional for list-clusters). configurations|stringoptional|A list of configurations to apply to the cluster (optional for create-cluster). Applies only to EMR releases 4.x and later. created_after|stringoptional|The creation date and time beginning value filter for listing clusters (optional for list-clusters). created_before|stringoptional|The creation date and time end value filter for listing clusters (optional for list-clusters). custom_ami_id|stringoptional|A custom Amazon Linux AMI for the cluster (optional for create-cluster). Available only in EMR releases 5.7.0 and later. ebs_root_volume_iops|stringoptional|The IOPS of the EBS root device volume of the Linux AMI (optional for create-cluster). Available in EMR releases 6.15.0 and later. ebs_root_volume_size|stringoptional|The size, in GiB, of the EBS root device volume of the Linux AMI (optional for create-cluster). Available in EMR releases 4.x and later. ebs_root_volume_throughput|stringoptional|The throughput, in MiB/s, of the EBS root device volume of the Linux AMI (optional for create-cluster). Available in EMR releases 6.15.0 and later. instances|stringoptional|A specification of the number and type of Amazon EC2 instances (required for create-cluster). Must include instance groups or instance fleets configuration. job_flow_role|stringoptional|The IAM role for EC2 instances running the job flow (required for create-cluster when using temporary credentials). kerberos_attributes|stringoptional|Attributes for Kerberos configuration when Kerberos authentication is enabled (optional for create-cluster). log_encryption_kms_key_id|stringoptional|The KMS key used for encrypting log files. Available with EMR 5.30.0 and later, excluding EMR 6.0.0 (optional for create-cluster). log_uri|stringoptional|The path to the Amazon S3 location where logs for the cluster are stored (optional for create-cluster). marker|stringoptional|The pagination token for list-clusters operation. name|stringoptional|Name of the EMR cluster (required for create-cluster). Cannot contain <, >, $, |, or (backtick).os_release_label|string*optional*|The Amazon Linux release for the cluster (optional for create-cluster).placement_groups|string*optional*|Placement group configuration for the cluster (optional for create-cluster).release_label|string*optional*|The Amazon EMR release label, which determines the version of open-source application packages installed on the cluster (required for create-cluster). Format: emr-x.x.xrepo_upgrade_on_boot|string*optional*|Applies only when CustomAmiID is used. Specifies the type of updates that are applied from the Amazon Linux AMI package repositories when an instance boots (optional for create-cluster).scale_down_behavior|string*optional*|The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs (optional for create-cluster). Values: TERMINATE_AT_INSTANCE_HOUR, TERMINATE_AT_TASK_COMPLETION.security_configuration|string*optional*|The name of a security configuration to apply to the cluster (optional for create-cluster).security_configuration_json|string*optional*|JSON format security configuration (required for create-security-configuration).security_configuration_name|string*optional*|Name of the security configuration (required for create-security-configuration, delete-security-configuration, describe-security-configuration).service_role|string*optional*|The IAM role that Amazon EMR assumes to access AWS resources on your behalf (optional for create-cluster).step_concurrency_level|string*optional*|The number of steps that can be executed concurrently (required for modify-cluster). Range: 1-256.steps|string*optional*|A list of steps to run on the cluster (optional for create-cluster). Each step contains Name, ActionOnFailure, and HadoopJarStep properties.termination_protected|string*optional*|Whether the cluster is protected from termination (optional for modify-cluster-attributes).unhealthy_node_replacement|string*optional*|Whether Amazon EMR should gracefully replace Amazon EC2 core instances that have degraded within the cluster (optional for create-cluster).visible_to_all_users|string` optional|Whether the cluster is visible to all IAM users of the AWS account (optional for create-cluster, default: true).

Tool: manage_aws_emr_ec2_instances

Manage AWS EMR EC2 instances with both read and write operations.

This tool provides comprehensive operations for managing Amazon EMR EC2 instances, including adding and modifying instance fleets and groups, as well as listing instance details. It enables scaling cluster capacity, configuring instance specifications, and monitoring instance status.

Requirements

• The server must be run with the --allow-write flag for add-instance-fleet, add-instance-groups, modify-instance-fleet, and modify-instance-groups operations
• Appropriate AWS permissions for EMR instance operations

Operations

• add-instance-fleet: Add an instance fleet to an existing EMR cluster

  • Required: cluster_id, instance_fleet (with InstanceFleetType)
  • Returns: cluster_id, instance_fleet_id, cluster_arn

• add-instance-groups: Add instance groups to an existing EMR cluster

  • Required: cluster_id, instance_groups (each with InstanceRole, InstanceType, InstanceCount)
  • Returns: cluster_id (as job_flow_id), instance_group_ids, cluster_arn

• modify-instance-fleet: Modify an instance fleet in an EMR cluster

  • Required: cluster_id, instance_fleet_id, instance_fleet_config
  • Returns: confirmation of modification

• modify-instance-groups: Modify instance groups in an EMR cluster

  • Required: instance_group_configs (each with InstanceGroupId)
  • Optional: cluster_id
  • Returns: confirmation of modification

• list-instance-fleets: List all instance fleets in an EMR cluster

  • Required: cluster_id
  • Optional: marker
  • Returns: instance_fleets, marker for pagination

• list-instances: List all instances in an EMR cluster

  • Required: cluster_id
  • Optional: instance_group_id, instance_group_types, instance_fleet_id, instance_fleet_type, instance_states, marker
  • Returns: instances, marker for pagination

• list-supported-instance-types: List all supported instance types for EMR

  • Required: release_label
  • Optional: marker
  • Returns: instance_types, marker for pagination

Example

# Add a task instance fleet with mixed instance types
response = await manage_aws_emr_ec2_instances(
    operation='add-instance-fleet',
    cluster_id='j-123ABC456DEF',
    instance_fleet={
        'InstanceFleetType': 'TASK',
        'Name': 'TaskFleet',
        'TargetOnDemandCapacity': 2,
        'TargetSpotCapacity': 3,
        'InstanceTypeConfigs': [
            {
                'InstanceType': 'm5.xlarge',
                'WeightedCapacity': 1,
                'BidPriceAsPercentageOfOnDemandPrice': 80,
            },
            {
                'InstanceType': 'm5.2xlarge',
                'WeightedCapacity': 2,
                'BidPriceAsPercentageOfOnDemandPrice': 75,
            },
        ],
    },
)

Tool: manage_aws_emr_ec2_steps

Manage AWS EMR EC2 steps for processing data on EMR clusters.

This tool provides comprehensive operations for managing EMR steps, which are units of work submitted to an EMR cluster for execution. Steps typically consist of Hadoop or Spark jobs that process and analyze data.

Requirements

• The server must be run with the --allow-write flag for add-steps and cancel-steps operations
• Appropriate AWS permissions for EMR step operations

Operations

• add-steps: Add new steps to a running EMR cluster (max 256 steps per job flow)
• cancel-steps: Cancel pending or running steps on an EMR cluster (EMR 4.8.0+ except 5.0.0)
• describe-step: Get detailed information about a specific step's configuration and status
• list-steps: List and filter steps for an EMR cluster with pagination support

Usage Tips

• Each step consists of a JAR file, its main class, and arguments
• Steps are executed in the order listed and must exit with zero code to be considered complete
• For cancel-steps, you can specify SEND_INTERRUPT (default) or TERMINATE_PROCESS as cancellation option
• When listing steps, filter by step states: PENDING, CANCEL_PENDING, RUNNING, COMPLETED, CANCELLED, FAILED, INTERRUPTED
• For large result sets, use pagination with marker parameter

Example

# Add a Spark step to process data
{
    'operation': 'add-steps',
    'cluster_id': 'j-2AXXXXXXGAPLF',
    'steps': [
        {
            'Name': 'Spark Data Processing',
            'ActionOnFailure': 'CONTINUE',
            'HadoopJarStep': {
                'Jar': 'command-runner.jar',
                'Args': [
                    'spark-submit',
                    '--class',
                    'com.example.SparkProcessor',
                    's3://mybucket/myapp.jar',
                    'arg1',
                    'arg2',
                ],
            },
        }
    ],
}

Tool: manage_aws_glue_catalog

Manage AWS Glue Data Catalog with both read and write operations.

This tool provides operations for managing the Glue Data Catalog itself, including creating custom catalogs, importing from external sources, and managing catalog-level configurations.

Requirements

• The server must be run with the --allow-write flag for create-catalog, delete-catalog, and import operations
• Appropriate AWS permissions for Glue Data Catalog operations
• For import operations, access to the external data source is required

Operations

• create-catalog: Create a new data catalog
• delete-catalog: Delete an existing data catalog
• get-catalog: Retrieve detailed information about a specific catalog
• list-catalogs: List all available catalogs
• import-catalog-to-glue: Import metadata from external sources into Glue Data Catalog

Usage Tips

• The default catalog ID is your AWS account ID
• Custom catalogs allow for better organization and access control
• Import operations can take significant time depending on source size Parameters|Type|Description -|-|- operation|string|Operation to perform: create-catalog, delete-catalog, get-catalog, list-catalogs, or import-catalog-to-glue. Choose "get-catalog" or "list-catalogs" for read-only operations. catalog_id|stringoptional|ID of the catalog (required for create-catalog, delete-catalog, get-catalog, and import-catalog-to-glue operations). catalog_input|stringoptional|Catalog definition for create-catalog operations. max_results|stringoptional|The maximum number of catalogs to return in one response. next_token|stringoptional|A continuation token, if this is a continuation call. parent_catalog_id|stringoptional|The ID of the parent catalog in which the catalog resides. If none is provided, the AWS Account Number is used by default.

Tool: manage_aws_glue_classifiers

Manage AWS Glue classifiers to determine data formats and schemas.

This tool provides operations for AWS Glue classifiers, which help determine the schema of your data. Classifiers analyze data samples to infer formats and structures, enabling accurate schema creation when crawlers process your data sources.

Requirements

• The server must be run with the --allow-write flag for create, delete, and update operations
• Appropriate AWS permissions for Glue classifier operations

Operations

• create-classifier: Create a new custom classifier (CSV, JSON, XML, or GROK)
• delete-classifier: Remove an existing classifier
• get-classifier: Retrieve detailed information about a specific classifier
• get-classifiers: List all available classifiers
• update-classifier: Modify an existing classifier's configuration

Example

# Create a CSV classifier
{
    'operation': 'create-classifier',
    'classifier_definition': {
        'CsvClassifier': {
            'Name': 'my-csv-classifier',
            'Delimiter': ',',
            'QuoteSymbol': '"',
            'ContainsHeader': 'PRESENT',
            'Header': ['id', 'name', 'date', 'value'],
            'AllowSingleColumn': false,
        }
    },
}

Tool: manage_aws_glue_connections

Manage AWS Glue Data Catalog connections with both read and write operations.

Connections in AWS Glue store connection information for data stores, such as databases, data warehouses, and other data sources. They contain connection properties like JDBC URLs, usernames, and other metadata needed to connect to external data sources.

Requirements

• The server must be run with the --allow-write flag for create, update, and delete operations
• Appropriate AWS permissions for Glue Data Catalog operations
• Connection properties must be valid for the connection type

Operations

• create-connection: Create a new connection
• delete-connection: Delete an existing connection
• get-connection: Retrieve detailed information about a specific connection
• list-connections: List all connections
• update-connection: Update an existing connection's properties

Usage Tips

• Connection names must be unique within your catalog
• Connection input should include ConnectionType and ConnectionProperties
• Use get or list operations to check existing connections before creating Parameters|Type|Description -|-|- operation|string|Operation to perform: create-connection, delete-connection, get-connection, list-connections, or update-connection. Choose "get-connection" or "list-connections" for read-only operations. catalog_id|stringoptional|Catalog ID for the connection (optional, defaults to account ID). connection_input|stringoptional|Connection definition for create and update operations. connection_name|stringoptional|Name of the connection (required for create-connection, delete-connection, get-connection, and update-connection operations). hide_password|stringoptional|Flag to retrieve the connection metadata without returning the password(for get-connection and list-connections operation). max_results|stringoptional|The maximum number of connections to return in one response. next_token|stringoptional|A continuation token, if this is a continuation call.

Tool: manage_aws_glue_crawler_management

Manage AWS Glue crawler schedules and monitor performance metrics.

This tool provides operations for controlling crawler schedules and retrieving performance metrics. Use it to automate crawler runs on a schedule and monitor crawler efficiency and status.

Requirements

• The server must be run with the --allow-write flag for schedule management operations
• Appropriate AWS permissions for Glue crawler operations

Operations

• get-crawler-metrics: Retrieve performance statistics about crawlers
• start-crawler-schedule: Activate a crawler's schedule
• stop-crawler-schedule: Deactivate a crawler's schedule
• update-crawler-schedule: Modify a crawler's schedule with a new cron expression

Example

# Update a crawler's schedule to run daily at 2:30 AM UTC
{
    'operation': 'update-crawler-schedule',
    'crawler_name': 'my-s3-data-crawler',
    'schedule': 'cron(30 2 * * ? *)',
}

# Get metrics for specific crawlers
{
    'operation': 'get-crawler-metrics',
    'crawler_name_list': ['my-s3-data-crawler', 'my-jdbc-crawler'],
}

Tool: manage_aws_glue_crawlers

Manage AWS Glue crawlers to discover and catalog data sources.

This tool provides comprehensive operations for AWS Glue crawlers, which automatically discover and catalog data from various sources like S3, JDBC databases, DynamoDB, and more. Crawlers examine your data sources, determine schemas, and register metadata in the AWS Glue Data Catalog.

Requirements

• The server must be run with the --allow-write flag for create, delete, start, stop, and update operations
• Appropriate AWS permissions for Glue crawler operations

Operations

• create-crawler: Create a new crawler with specified targets, role, and configuration
• delete-crawler: Remove an existing crawler from AWS Glue
• get-crawler: Retrieve detailed information about a specific crawler
• get-crawlers: List all crawlers with pagination
• batch-get-crawlers: Retrieve multiple specific crawlers in a single call
• list-crawlers: List all crawlers with tag-based filtering
• start-crawler: Initiate a crawler run immediately
• stop-crawler: Halt a currently running crawler
• update-crawler: Modify an existing crawler's configuration

Example

# Create a new S3 crawler
{
    'operation': 'create-crawler',
    'crawler_name': 'my-s3-data-crawler',
    'crawler_definition': {
        'Role': 'arn:aws:iam::123456789012:role/GlueServiceRole',
        'Targets': {'S3Targets': [{'Path': 's3://my-bucket/data/'}]},
        'DatabaseName': 'my_catalog_db',
        'Description': 'Crawler for S3 data files',
        'Schedule': 'cron(0 0 * * ? *)',
        'TablePrefix': 'raw_',
    },
}

Tool: manage_aws_glue_databases

Manage AWS Glue Data Catalog databases with both read and write operations.

This tool provides operations for managing Glue Data Catalog databases, including creating, updating, retrieving, listing, and deleting databases. It serves as the primary mechanism for database management within the AWS Glue Data Catalog.

Requirements

• The server must be run with the --allow-write flag for create-database, update-database, and delete-database operations
• Appropriate AWS permissions for Glue Data Catalog operations

Operations

• create-database: Create a new database in the Glue Data Catalog
• delete-database: Delete an existing database from the Glue Data Catalog
• get-database: Retrieve detailed information about a specific database
• list-databases: List all databases in the Glue Data Catalog
• update-database: Update an existing database's properties

Usage Tips

• Use the get-database or list-databases operations first to check existing databases
• Database names must be unique within your AWS account and region
• Deleting a database will also delete all tables within it Parameters|Type|Description -|-|- operation|string|Operation to perform: create-database, delete-database, get-database, list-databases, or update-database. Choose "get-database" or "list-databases" for read-only operations when write access is disabled. catalog_id|stringoptional|ID of the catalog (optional, defaults to account ID). database_name|stringoptional|Name of the database (required for create-database, delete-database, get-database, and update-database operations). description|stringoptional|Description of the database (for create-database and update-database operations). location_uri|stringoptional|Location URI of the database (for create-database and update-database operations). max_results|stringoptional|The maximum number of databases to return in one response. next_token|stringoptional|A continuation token, if this is a continuation call. parameters|stringoptional|Key-value pairs that define parameters and properties of the database.

Tool: manage_aws_glue_encryption

Manage AWS Glue Data Catalog Encryption Settings for data protection.

This tool allows you to retrieve and update AWS Glue Data Catalog Encryption Settings, which control how metadata and connection passwords are encrypted in the Data Catalog.

Requirements

• The server must be run with the --allow-write flag for put-catalog-encryption-settings operation
• Appropriate AWS permissions for Glue Data Catalog Encryption operations

Operations

• get-catalog-encryption-settings: Retrieve the current encryption settings for the Data Catalog
• put-catalog-encryption-settings: Update the encryption settings for the Data Catalog

Example

{
  "operation": "put-catalog-encryption-settings",
  "encryption_at_rest": {
    "CatalogEncryptionMode": "SSE-KMS",
    "SseAwsKmsKeyId": "arn:aws:kms:region:account-id:key/key-id"
  },
  "connection_password_encryption": {
    "ReturnConnectionPasswordEncrypted": true,
    "AwsKmsKeyId": "arn:aws:kms:region:account-id:key/key-id"
  }
}

Tool: manage_aws_glue_jobs

Manage AWS Glue ETL jobs and job runs with both read and write operations.

This tool provides comprehensive operations for managing AWS Glue ETL jobs and job runs, including creating, updating, retrieving, listing, starting, stopping, and monitoring jobs.

Requirements

• The server must be run with the --allow-write flag for create-job, delete-job, update-job, start-job-run, stop-job-run, and batch-stop-job-run operations
• Appropriate AWS permissions for Glue ETL job operations

Job Operations

• create-job: Create a new ETL job in AWS Glue
• delete-job: Delete an existing ETL job from AWS Glue
• get-job: Retrieve detailed information about a specific job
• get-jobs: List all jobs in your AWS Glue account
• update-job: Update an existing job's properties
• start-job-run: Start a job run using a job name

Job Run Operations

• stop-job-run: Stop a job run using a job name and run ID
• get-job-run: Retrieve detailed information about a specific job run
• get-job-runs: List all job runs for a specific job
• batch-stop-job-run: Stop one or more running jobs

Usage Tips

• Job names must be unique within your AWS account and region
• Create a script required by the customer and push the script to a customer S3 Location. Ask for S3 Location if not provided.
• Verify if the IAM role used has glue trusted entities in the role if not update the role or create a new one
• Job definitions should include command, role, and other required parameters
• As rule of thumb use Glue Version 5.0 or latest to create jobs

Examples

# Create a new Spark ETL job
{
    'operation': 'create-job',
    'job_name': 'my-etl-job',
    'job_definition': {
        'Role': 'arn:aws:iam::123456789012:role/GlueETLRole',
        'Command': {
            'Name': 'glueetl',
            'ScriptLocation': 's3://my-bucket/scripts/etl-script.py',
        },
        'GlueVersion': '5.0',
        'MaxRetries': 2,
        'Timeout': 120,
        'WorkerType': 'G.1X',
        'NumberOfWorkers': 5,
    },
}

# Start a job run
{
    'operation': 'start-job-run',
    'job_name': 'my-etl-job',
    'worker_type': 'G.1X',
    'number_of_workers': 5,
}

# Get details of a specific job run
{
    'operation': 'get-job-run',
    'job_name': 'my-etl-job',
    'job_run_id': 'jr_1234567890abcdef0',
}

Tool: manage_aws_glue_partitions

Manage AWS Glue Data Catalog partitions with both read and write operations.

Partitions in AWS Glue represent a way to organize table data based on the values of one or more columns. They enable efficient querying and processing of large datasets by allowing queries to target specific subsets of data.

Requirements

• The server must be run with the --allow-write flag for create-partition, update-partition, and delete-partition operations
• Database and table must exist before creating partitions
• Partition values must match the partition schema defined in the table

Operations

• create-partition: Create a new partition in the specified table
• delete-partition: Delete an existing partition from the table
• get-partition: Retrieve detailed information about a specific partition
• list-partitions: List all partitions in the specified table
• update-partition: Update an existing partition's properties

Usage Tips

• Partition values must be provided in the same order as partition columns in the table
• Use get-partition or list-partitions operations to check existing partitions before creating
• Partition input should include storage descriptor and location information Parameters|Type|Description -|-|- database_name|string|Name of the database containing the table. operation|string|Operation to perform: create-partition, delete-partition, get-partition, list-partitions, or update-partition. Choose "get-partition" or "list-partitions" for read-only operations. table_name|string|Name of the table containing the partition. catalog_id|stringoptional|ID of the catalog (optional, defaults to account ID). expression|stringoptional|Filter expression for list-partitions operation. max_results|stringoptional|Maximum number of results to return for list-partitions operation. next_token|stringoptional|A continuation token, if this is not the first call to retrieve these partitions. partition_input|stringoptional|Partition definition for create-partition and update-partition operations. partition_values|stringoptional|Values that define the partition (required for create-partition, delete-partition, get-partition, and update-partition operations).

Tool: manage_aws_glue_resource_policies

Manage AWS Glue Resource Policies for access control.

This tool allows you to retrieve, create, update, and delete AWS Glue Resource Policies, which control access to Glue resources through IAM policy documents.

Requirements

• The server must be run with the --allow-write flag for put-resource-policy and delete-resource-policy operations
• Appropriate AWS permissions for Glue Resource Policy operations

Operations

• get-resource-policy: Retrieve the current resource policy
• put-resource-policy: Create or update the resource policy
• delete-resource-policy: Delete the resource policy

Example

{
  "operation": "put-resource-policy",
  "policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::123456789…
  "policy_exists_condition": "NOT_EXIST",
  "enable_hybrid": true
}

Tool: manage_aws_glue_security_configurations

Manage AWS Glue Security Configurations for data encryption.

This tool allows you to create, retrieve, and delete AWS Glue Security Configurations, which define encryption settings for Glue jobs, crawlers, and development endpoints.

Requirements

• The server must be run with the --allow-write flag for create-security-configuration and delete-security-configuration operations
• Appropriate AWS permissions for Glue Security Configuration operations

Operations

• create-security-configuration: Create a new security configuration with encryption settings
• delete-security-configuration: Delete an existing security configuration
• get-security-configuration: Retrieve detailed information about a specific security configuration

Example

{
  "operation": "create-security-configuration",
  "config_name": "my-encryption-config",
  "encryption_configuration": {
    "S3Encryption": [
      {
        "S3EncryptionMode": "SSE-KMS",
        "KmsKeyArn": "arn:aws:kms:region:account-id:key/key-id"
      }
    ],
    "CloudWatchEncryption": {
      "CloudWatchEncryptionMode": "DISABLED"
    },
    "JobBookmarksEncryption": {
      "JobBookmarksEncryptionMode": "CSE-KMS",
      "KmsKeyArn": "arn:aws:kms:region:account-id:key/key-id"
    }
  }
}

Tool: manage_aws_glue_sessions

Manage AWS Glue Interactive Sessions for running Spark and Ray workloads.

This tool provides operations for creating and managing Glue Interactive Sessions, which enable interactive development and execution of Spark ETL scripts and Ray applications. Interactive sessions provide a responsive environment for data exploration, debugging, and iterative development.

Requirements

• The server must be run with the --allow-write flag for create-session, delete-session, and stop-session operations
• Appropriate AWS permissions for Glue Interactive Session operations

Operations

• create-session: Create a new interactive session with specified configuration
• delete-session: Delete an existing interactive session
• get-session: Retrieve detailed information about a specific session
• list-sessions: List all interactive sessions with optional filtering
• stop-session: Stop a running interactive session

Example

# Create a new Spark ETL session
{
    'operation': 'create-session',
    'session_id': 'my-spark-session',
    'role': 'arn:aws:iam::123456789012:role/GlueInteractiveSessionRole',
    'command': {'Name': 'glueetl', 'PythonVersion': '3'},
    'glue_version': '3.0',
}

Tool: manage_aws_glue_statements

Manage AWS Glue Interactive Session Statements for executing code and retrieving results.

This tool provides operations for executing code, canceling running statements, and retrieving results within Glue Interactive Sessions. It enables interactive data processing, exploration, and analysis using Spark or Ray in AWS Glue.

Requirements

• The server must be run with the --allow-write flag for run-statement and cancel-statement operations
• Appropriate AWS permissions for Glue Interactive Session Statement operations
• A valid session ID is required for all operations

Operations

• run-statement: Execute code in an interactive session and get a statement ID
• cancel-statement: Cancel a running statement by ID
• get-statement: Retrieve detailed information and results of a specific statement
• list-statements: List all statements in a session with their status

Example

# Run a PySpark statement in a session
{
    'operation': 'run-statement',
    'session_id': 'my-spark-session',
    'code': "df = spark.read.csv('s3://my-bucket/data.csv', header=True)\ndf.show(5)",
}

Tool: manage_aws_glue_tables

Manage AWS Glue Data Catalog tables with both read and write operations.

This tool provides comprehensive operations for managing Glue Data Catalog tables, including creating, updating, retrieving, listing, searching, and deleting tables. Tables define the schema and metadata for data stored in various formats and locations.

Requirements

• The server must be run with the --allow-write flag for create-table, update-table, and delete-table operations
• Database must exist before creating tables within it
• Appropriate AWS permissions for Glue Data Catalog operations

Operations

• create-table: Create a new table in the specified database
• delete-table: Delete an existing table from the database
• get-table: Retrieve detailed information about a specific table
• list-tables: List all tables in the specified database
• update-table: Update an existing table's properties
• search-tables: Search for tables using text matching

Usage Tips

• Table names must be unique within a database
• Use get-table or list-tables operations to check existing tables before creating
• Table input should include storage descriptor, columns, and partitioning information Parameters|Type|Description -|-|- database_name|string|Name of the database containing the table. operation|string|Operation to perform: create-table, delete-table, get-table, list-tables, update-table, or search-tables. Choose "get-table", "list-tables", or "search-tables" for read-only operations. catalog_id|stringoptional|ID of the catalog (optional, defaults to account ID). max_results|stringoptional|Maximum number of results to return for list and search-tables operations. next_token|stringoptional|A continuation token, included if this is a continuation call. search_text|stringoptional|Search text for search-tables operation. table_input|stringoptional|Table definition for create-table and update-table operations. table_name|stringoptional|Name of the table (required for create-table, delete-table, get-table, and update-table operations).

Tool: manage_aws_glue_triggers

Manage AWS Glue triggers to automate workflow and job execution.

This tool allows you to create, delete, retrieve, list, start, and stop AWS Glue triggers. Triggers define the conditions that automatically start jobs or workflows, enabling scheduled or event-based execution of your ETL processes.

Requirements

• The server must be run with the --allow-write flag for create-trigger, delete-trigger, start-trigger, and stop-trigger operations
• Appropriate AWS permissions for Glue trigger operations

Operations

• create-trigger: Create a new trigger with specified type (SCHEDULED, CONDITIONAL, ON_DEMAND, EVENT) and actions
• delete-trigger: Delete an existing trigger by name
• get-trigger: Retrieve detailed information about a specific trigger
• get-triggers: List all triggers with pagination support
• start-trigger: Activate a trigger to begin monitoring for its firing conditions
• stop-trigger: Deactivate a trigger to pause its monitoring

Trigger Types

• SCHEDULED: Time-based triggers that run on a cron schedule
• CONDITIONAL: Event-based triggers that run when specified conditions are met
• ON_DEMAND: Manually activated triggers
• EVENT: EventBridge event-based triggers

Example

# Create a scheduled trigger
manage_aws_glue_triggers(
    operation='create-trigger',
    trigger_name='daily-etl-trigger',
    trigger_definition={
        'Type': 'SCHEDULED',
        'Schedule': 'cron(0 12 * * ? *)',  # Run daily at 12:00 UTC
        'Actions': [{'JobName': 'process-daily-data'}],
        'Description': 'Trigger for daily ETL job',
        'StartOnCreation': True,
    },
)

# Create a conditional trigger
manage_aws_glue_triggers(
    operation='create-trigger',
    trigger_name='data-arrival-trigger',
    trigger_definition={
        'Type': 'CONDITIONAL',
        'Actions': [{'JobName': 'process-new-data'}],
        'Predicate': {
            'Conditions': [
                {
                    'LogicalOperator': 'EQUALS',
                    'JobName': 'crawl-new-data',
                    'State': 'SUCCEEDED',
                }
            ]
        },
        'Description': 'Trigger that runs when data crawling completes',
    },
)

Tool: manage_aws_glue_usage_profiles

Manage AWS Glue Usage Profiles for resource allocation and cost management.

This tool allows you to create, retrieve, update, and delete AWS Glue Usage Profiles, which define resource allocation and cost management settings for Glue jobs and interactive sessions.

Requirements

• The server must be run with the --allow-write flag for create-profile, delete-profile, and update-profile operations
• Appropriate AWS permissions for Glue Usage Profile operations

Operations

• create-profile: Create a new usage profile with specified resource allocations
• delete-profile: Delete an existing usage profile
• get-profile: Retrieve detailed information about a specific usage profile
• update-profile: Update an existing usage profile's configuration

Example

{
  "operation": "create-profile",
  "profile_name": "my-standard-profile",
  "description": "Standard resource allocation for ETL jobs",
  "configuration": {
      "JobConfiguration": {
        "numberOfWorkers": {
          "DefaultValue": "10",
          "MinValue": "1",
          "MaxValue": "10"
        },
        "workerType": {
          "DefaultValue": "G.2X",
          "AllowedValues": [
            "G.2X",
            "G.4X",
            "G.8X"
          ]
        },
    }
}

Tool: manage_aws_glue_workflows

Manage AWS Glue workflows to orchestrate complex ETL activities.

This tool allows you to create, delete, retrieve, list, and start AWS Glue workflows. Workflows help you design and visualize complex ETL activities as a series of dependent jobs and crawlers, making it easier to manage and monitor your data processing pipelines.

Requirements

• The server must be run with the --allow-write flag for create-workflow, delete-workflow, and start-workflow-run operations
• Appropriate AWS permissions for Glue workflow operations

Operations

• create-workflow: Create a new workflow with optional description, default run properties, tags, and max concurrent runs
• delete-workflow: Delete an existing workflow by name
• get-workflow: Retrieve detailed information about a specific workflow with optional graph inclusion
• list-workflows: List all workflows with pagination support
• start-workflow-run: Start a workflow run with optional run properties

Example

# Create a new workflow
manage_aws_glue_workflows(
    operation='create-workflow',
    workflow_name='my-etl-workflow',
    workflow_definition={
        'Description': 'ETL workflow for daily data processing',
        'DefaultRunProperties': {'ENV': 'production'},
        'MaxConcurrentRuns': 1,
    },
)

# Start a workflow run
manage_aws_glue_workflows(
    operation='start-workflow-run',
    workflow_name='my-etl-workflow',
    workflow_definition={'run_properties': {'EXECUTION_DATE': '2023-06-19'}},
)

Tool: upload_to_s3

Upload Python code content directly to an S3 bucket using putObject.

This tool uploads Python code content directly to an S3 bucket, commonly used for storing Glue job scripts, EMR step scripts, or other data processing code. The uploaded file can be referenced by data processing services.

Requirements

• The server must be run with the --allow-write flag
• Valid AWS credentials with permissions to write to the specified S3 bucket
• The bucket must exist and be accessible

Usage Tips

• Use descriptive S3 keys that include version information or timestamps
• Store scripts in organized folder structures (e.g., glue-jobs/, emr-steps/)
• Consider using versioning on the S3 bucket for script history
• The returned S3 URI can be used directly in Glue job configurations Parameters|Type|Description -|-|- bucket_name|string|Name of the S3 bucket code_content|string|String containing Python code to upload s3_key|string|S3 object key (path within the bucket) make_public|booleanoptional|Whether to make the file publicly accessible (default: False)

Use this MCP Server

{
  "mcpServers": {
    "aws-dataprocessing": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "mcp/aws-dataprocessing-mcp-server"
      ]
    }
  }
}

Why is it safer to run MCP Servers with Docker?⁠