Sign inSign up

affinefoundation/agentgym

By affinefoundation

•Updated 10 months ago

The AgentGym environment provides interactive agent evaluation across multiple benchmark tasks.

Image
Machine learning & AI
0

2.5K

affinefoundation/agentgym repository overview

⁠AgentGym Environment

⁠Task Description

The AgentGym environment provides interactive agent evaluation across multiple benchmark tasks. Each task evaluates an agent's ability to interact with different environments and complete specific objectives through multi-turn conversations.

⁠Supported Tasks
  • webshop: E-commerce shopping tasks
  • alfworld: Household task completion in text-based environments
  • babyai: Grid-world navigation and instruction following
  • sciworld: Scientific reasoning and experimentation
  • textcraft: Text-based crafting game

⁠How to Use with Affinetes

⁠Option 1: Pull Pre-built Docker Image
import affinetes as af
import asyncio
import os

async def main():
    # Load environment from Docker Hub (example: sciworld)
    env = af.load_env(
        image="bignickeye/agentgym:sciworld-v2",
        env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
    )
    
    # Evaluate on a specific task
    result = await env.evaluate(
        model="deepseek-ai/DeepSeek-V3",
        base_url="https://llm.chutes.ai/v1",
        task_id=10,
        max_round=30
    )
    
    print(f"Score: {result['score']}")
    print(f"Success: {result['success']}")
    
    await env.cleanup()

asyncio.run(main())

Tag summary

Content type

Image

Digest

sha256:c531e612e…

Size

466.8 MB

Last updated

10 months ago

docker pull affinefoundation/agentgym:textcraft