henrybit/dify-api

By henrybit

•Updated 6 months ago

Dify is an open-source LLM app development platform.

Image

Machine learning & AI

Developer tools

Web servers

10K+

Overview Tags

henrybit/dify-api repository overview

⁠Dify-API (K8S Deployment)

[Import!] If you want deploy dify on k8s cluster, you can use this document.

Dify is an LLM application development platform that has already facilitated the creation of over 100,000 applications using Dify.AI. It combines the principles of Backend as a Service and LLMOps, encompassing the essential technology stack needed for developing AI-native applications, incorporating a built-in RAG engine. With Dify, you can independently deploy functionalities akin to Assistants API and GPTs based on any LLMs.

⁠1. Dify-API is the api component of Dify.

Alt

⁠2. How to use Dify-API?

If your dify-api domain is "https://dify-api.xxx.yyy⁠" and your dify-web domain is "https://dify-web.xxx.yyy⁠"
Vector db is Milvus
DB is PostgreSQL
Cache is redis
file storage is Aliyun oss
Your need add domain "dify-plugin-daemon" for plugin daemon when you use dify version >= 1.0.0, The domain name can be an internal Kubernetes domain. In this case, communication can be done through ports 5002 and 5003.

api & worker use this same image, just use the MODE environment to distinguish

⁠2.1 [Version] >= 1.0.0

The new version is a major upgrade, so please be cautious if you plan to update. It removes the old model configuration mode and replaces it with a plugin-based strategy. This change will require significant adjustments to the knowledge base and all orchestration nodes, especially those using vector models, reasoning models, and ranking models.

A new plugin service has been introduced to manage local plugin installations. Plugins are now responsible for extending capabilities like models, agents, and tools. All default model settings from previous versions have been removed, so you’ll need to install the required plugins yourself before use.

Additionally, the new version supports the popular MCP protocol, a key technology in the large model ecosystem.

⁠2.1.1 api

The environment in a Kubernetes YAML file looks like this.:

env:
  - name: SECRET_KEY
    value: "MFnQ128032819024170983219038YYp5A"
  - name: CONSOLE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: CONSOLE_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: SERVICE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: APP_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: CELERY_BROKER_URL
    value: "redis://:[password]@[redis_host]:6379/1"
  - name: REDIS_HOST
    value: "redis"
  - name: REDIS_PORT
    value: 6379
  - name: REDIS_PASSWORD
    value: ""
  - name: DB_USERNAME
    value: ""
  - name: DB_PASSWORD
    value: ""
  - name: DB_HOST
    value: ""
  - name: DB_PORT
    value: 5432
  - name: DB_DATABASE
    value: "dify"
  - name: STORAGE_TYPE
    value: "aliyun-oss"
  - name: ALIYUN_OSS_ENDPOINT
    value: ""
  - name: ALIYUN_OSS_PATH
    value: ""
  - name: ALIYUN_OSS_BUCKET_NAME
    value: ""
  - name: ALIYUN_OSS_ACCESS_KEY
    value: ""
  - name: ALIYUN_OSS_SECRET_KEY
    value: ""
  - name: VECTOR_STORE
    value: "milvus"
  - name: MILVUS_HOST
    value: "my-release-milvus"
  - name: MILVUS_PORT
    value: 19530
  - name: MILVUS_USER
    value: "root"
  - name: MILVUS_PASSWORD
    value: ""
  - name: MILVUS_SECURE
    value: false
  - name: 
  - name: WEB_API_CORS_ALLOW_ORIGINS
    value: "*"
  - name: CONSOLE_CORS_ALLOW_ORIGINS
    value: "*"
  - name: MAIL_TYPE
    value: "resend"
  - name: MAIL_DEFAULT_SEND_FROM
    value: ""
  - name: RESEND_API_KEY
    value: ""
  - name: CODE_EXECUTION_ENDPOINT
    value: http://sandbox:8194
  - name: CODE_EXECUTION_API_KEY
    value: "xYaiVMrBUtVppEv6N7cW"
  - name: CODE_MAX_NUMBER
    value: 9223372036854775807
  - name: CODE_MIN_NUMBER
    value: -9223372036854775808
  - name: CODE_MAX_DEPTH
    value: 5
  - name: CODE_MAX_PRECISION
    value: 20
  - name: CODE_MAX_STRING_LENGTH
    value: 80000
  - name: TEMPLATE_TRANSFORM_MAX_LENGTH
    value: 80000
  - name: CODE_MAX_STRING_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_OBJECT_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_NUMBER_ARRAY_LENGTH
    value: 1000
  - name: TEMPLATE_TRANSFORM_MAX_LENGTH
    value: 80000
  - name: PLUGIN_REMOTE_INSTALL_HOST
    value: "dify-plugin-daemon"
  - name: PLUGIN_REMOTE_INSTALL_PORT
    value: 5003
  - name: PLUGIN_MAX_PACKAGE_SIZE
    value: 52428800
  - name: INNER_API_KEY_FOR_PLUGIN
    value: "MFnQ128032819024170983219038YYp5A"
  - name: PLUGIN_DAEMON_KEY
    value: "6ovaezWAjpCjf+Sjc3ZtU+qUEi"
  - name: PLUGIN_DAEMON_URL
    value: "http://dify-plugin-daemon:5002"

SECRET_KEY - is A secret key that is used for securely signing the session cookie and encrypting sensitive information on the database. You can generate a strong key using openssl rand -base64 42.
CELERY_BROKER_URL - Use redis as the broker, and redis db 1 for celery broker. example - redis://:difyai123456@redis:6379/1
[DB example] -
- DB_USERNAME - postgres
- DB_PASSWORD: dify_test123
- DB_HOST: pgsql_local
- DB_PORT: 5432
- DB_DATABASE: dify
[Milvus example] -
- VECTOR_STORE - milvus
- MILVUS_HOST: milvus_local
- MILVUS_PORT: 19530
- MILVUS_USER: root
- MILVUS_PASSWORD: milvus_test12
MILVUS_SECURE - must be set false if your milvus don't need password (Important!)
MAIL_TYPE - support resend or smtp, resend link is https://www.resend.com/⁠, An online email server.
STORAGE_TYPE - aliyun-oss, fill these fields ALIYUN_OSS_ENDPOINT/ALIYUN_OSS_BUCKET_NAME/ALIYUN_OSS_ACCESS_KEY/ALIYUN_OSS_SECRET_KEY
ALIYUN_OSS_PATH - version >= 0.8.2, must be set, this is the root path.
CODE_EXECUTION_ENDPOINT - code execute sandbox, if it's empty, custom code execute node will fail.
CODE_EXECUTION_API_KEY - If CODE_EXECUTION_ENDPOINT is not empty, CODE_EXECUTION_API_KEY must be set the same as the dify-sandbox image's running configuration.
PLUGIN_REMOTE_INSTALL_HOST - The host of the plugin installation service. This is the host where the plugin installation service is running.
PLUGIN_REMOTE_INSTALL_PORT - The port of the plugin installation service. This is the port where the plugin installation service is running.
PLUGIN_MAX_PACKAGE_SIZE - The maximum size of the plugin package that can be installed. This is the maximum size of the plugin package that can be installed in bytes.
INNER_API_KEY_FOR_PLUGIN - The API key used for authentication when installing plugins. The key from plugin daemon's envrionment variable SERVER_KEY in k8s yaml.
PLUGIN_DAEMON_KEY - The key used for authentication when communicating with the plugin daemon. This is the key used for authentication when communicating with the plugin daemon.
PLUGIN_DAEMON_URL - The URL of the plugin daemon. This is the URL of the plugin daemon that is used for communication with the plugin installation service, you can use the internal Kubernetes domain of plugin daemon server.

⁠2.1.2 worker

The environment in a Kubernetes YAML file looks like this.:

env:
  - name: SECRET_KEY
    value: "MFnQ128032819024170983219038YYp5A"
  - name: CONSOLE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: CONSOLE_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: SERVICE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: APP_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: CELERY_BROKER_URL
    value: "redis://:[password]@[redis_host]:6379/1"
  - name: REDIS_HOST
    value: "redis"
  - name: REDIS_PORT
    value: 6379
  - name: REDIS_PASSWORD
    value: ""
  - name: DB_USERNAME
    value: ""
  - name: DB_PASSWORD
    value: ""
  - name: DB_HOST
    value: ""
  - name: DB_PORT
    value: 5432
  - name: DB_DATABASE
    value: "dify"
  - name: STORAGE_TYPE
    value: "aliyun-oss"
  - name: ALIYUN_OSS_ENDPOINT
    value: ""
  - name: ALIYUN_OSS_PATH
    value: ""
  - name: ALIYUN_OSS_BUCKET_NAME
    value: ""
  - name: ALIYUN_OSS_ACCESS_KEY
    value: ""
  - name: ALIYUN_OSS_SECRET_KEY
    value: ""
  - name: VECTOR_STORE
    value: "milvus"
  - name: MILVUS_HOST
    value: "my-release-milvus"
  - name: MILVUS_PORT
    value: 19530
  - name: MILVUS_USER
    value: "root"
  - name: MILVUS_PASSWORD
    value: ""
  - name: MILVUS_SECURE
    value: false
  - name: 
  - name: WEB_API_CORS_ALLOW_ORIGINS
    value: "*"
  - name: CONSOLE_CORS_ALLOW_ORIGINS
    value: "*"
  - name: MAIL_TYPE
    value: "resend"
  - name: MAIL_DEFAULT_SEND_FROM
    value: ""
  - name: RESEND_API_KEY
    value: ""
  - name: MODE
    value: "worker"
  - name: LOG_LEVEL
    value: "INFO"
  - name: CODE_EXECUTION_ENDPOINT
    value: http://sandbox:8194
  - name: CODE_EXECUTION_API_KEY
    value: "xYaiVMrBUtVppEv6N7cW"
  - name: CODE_MAX_NUMBER
    value: 9223372036854775807
  - name: CODE_MIN_NUMBER
    value: -9223372036854775808
  - name: CODE_MAX_DEPTH
    value: 5
  - name: CODE_MAX_PRECISION
    value: 20
  - name: CODE_MAX_STRING_LENGTH
    value: 80000
  - name: TEMPLATE_TRANSFORM_MAX_LENGTH
    value: 80000
  - name: CODE_MAX_STRING_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_OBJECT_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_NUMBER_ARRAY_LENGTH
    value: 1000
  - name: TEMPLATE_TRANSFORM_MAX_LENGTH
    value: 80000000
  - name: INNER_API_KEY_FOR_PLUGIN
    value: "MFnQ128032819024170983219038YYp5A"

MODE - must be set "worker"

⁠2.1.3 plugin-daemon

use this image to run plugin-daemon offical download link⁠.

Recommend to use the latest version of plugin-daemon image.

The version <= 0.0.6-local works fine during testing.

For local deployment, use the version with the "local" suffix.

The environment in a Kubernetes YAML file looks like this.:

Volume mounts for the plugin-daemon container:(You must mount the volume to the plugin-daemon container, otherwise the plugin will lose when your Kubernetes pod restarts.)

volumeMounts:
    - name: dify-plugin-data
    mountPath: /app/storage

Environment variables for the plugin-daemon container:

env:
    - name: SERVER_PORT
      value: 5002
    - name: SERVER_KEY
      value: "6ovaezWAjpCjf+Sjc3ZtU+qUEi"
    - name: GIN_MODE
      value: "release"
    - name: PLATFORM
      value: "local"
    - name: DIFY_INNER_API_KEY
      value: "MFnQ128032819024170983219038YYp5A"
    - name: DIFY_INNER_API_URL
      value: "http://dify-api:5001"
    - name: PLUGIN_REMOTE_INSTALLING_ENABLED
      value: true
    - name: PLUGIN_REMOTE_INSTALLING_HOST
      value: "0.0.0.0"
    - name: PLUGIN_REMOTE_INSTALLING_PORT
      value: "5003"
    - name: PLUGIN_STORAGE_TYPE
      value: "local"
    - name: PLUGIN_STORAGE_OSS_BUCKET
      value: ""
    - name: PLUGIN_STORAGE_LOCAL_ROOT
      value: "/app/storage"
    - name: PLUGIN_INSTALLED_PATH
      value: "/app/storage/plugin"
    - name: PLUGIN_WORKING_PATH
      value: "/app/storage/cwd"
    - name: PERSISTENCE_STORAGE_PATH
      value: "/app/storage/persistence"
    - name: PERSISTENCE_STORAGE_MAX_SIZE
      value: 104857600
    - name: PLUGIN_WEBHOOK_ENABLED
      value: true
    - name: ROUTINE_POOL_SIZE
      value: 1024
    - name: REDIS_HOST
      value: "redis"
    - name: REDIS_PORT
      value: 6379
    - name: REDIS_PASSWORD
      value: "xxxx"
    - name: DB_USERNAME
      value: "dify"
    - name: DB_PASSWORD
      value: "xxx"
    - name: DB_HOST
      value: "xxxxxxx"
    - name: DB_PORT
      value: 5432
    - name: DB_DATABASE
      value: "dify"   
    - name: DIFY_INVOCATION_CONNECTION_IDLE_TIMEOUT
      value: 120
    - name: MAX_PLUGIN_PACKAGE_SIZE
      value: 52428800
    - name: PYTHON_ENV_INIT_TIMEOUT
      value: 120
    - name: PPROF_ENABLED
      value: false
    - name: FORCE_VERIFYING_SIGNATURE
      value: true

SERVER_PORT - The port on which the plugin daemon server will listen for incoming requests, The default port is 5002.
SERVER_KEY - The key used for authentication when communicating with the plugin daemon. This is the key used for authentication when communicating with the plugin daemon.
GIN_MODE - The mode in which the Gin web framework will run. It can be set to "release" for production or "debug" for development.
PLATFORM - The platform on which the plugin daemon is running. It can be set to "local" for local deployment or "k8s" for Kubernetes deployment.
DIFY_INNER_API_KEY - The API key used for authentication when communicating with the Dify API. This is the key used for authentication when communicating with the Dify API.
DIFY_INNER_API_URL - The URL of the Dify API. This is the URL of the Dify API that is used for communication with the plugin daemon.
PLUGIN_REMOTE_INSTALLING_ENABLED - Whether remote plugin installation is enabled. It can be set to true or false.
PLUGIN_REMOTE_INSTALLING_HOST - The host on which the remote plugin installation service will run. This is the host where the remote plugin installation service is running.
PLUGIN_REMOTE_INSTALLING_PORT - The port on which the remote plugin installation service will run. Default port is 5003.
PLUGIN_STORAGE_TYPE - The type of storage used for storing plugins. It can be set to "local" for local storage or "oss" for object storage, Unfortunately, it does not support domestic S3 services like Alibaba Cloud or Tencent Cloud, So, I recommend using a local PVC (Persistent Volume Claim).

⁠2.2 [Version] <= 0.15.x

⁠2.2.1 api

The environment in a Kubernetes YAML file looks like this.:

env:
  - name: SECRET_KEY
    value: "MFnQ128032819024170983219038YYp5A"
  - name: CONSOLE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: CONSOLE_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: SERVICE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: APP_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: CELERY_BROKER_URL
    value: "redis://:[password]@[redis_host]:6379/1"
  - name: REDIS_HOST
    value: "redis"
  - name: REDIS_PORT
    value: 6379
  - name: REDIS_PASSWORD
    value: ""
  - name: DB_USERNAME
    value: ""
  - name: DB_PASSWORD
    value: ""
  - name: DB_HOST
    value: ""
  - name: DB_PORT
    value: 5432
  - name: DB_DATABASE
    value: "dify"
  - name: STORAGE_TYPE
    value: "aliyun-oss"
  - name: ALIYUN_OSS_ENDPOINT
    value: ""
  - name: ALIYUN_OSS_PATH
    value: ""
  - name: ALIYUN_OSS_BUCKET_NAME
    value: ""
  - name: ALIYUN_OSS_ACCESS_KEY
    value: ""
  - name: ALIYUN_OSS_SECRET_KEY
    value: ""
  - name: VECTOR_STORE
    value: "milvus"
  - name: MILVUS_HOST
    value: "my-release-milvus"
  - name: MILVUS_PORT
    value: 19530
  - name: MILVUS_USER
    value: "root"
  - name: MILVUS_PASSWORD
    value: ""
  - name: MILVUS_SECURE
    value: false
  - name: 
  - name: WEB_API_CORS_ALLOW_ORIGINS
    value: "*"
  - name: CONSOLE_CORS_ALLOW_ORIGINS
    value: "*"
  - name: MAIL_TYPE
    value: "resend"
  - name: MAIL_DEFAULT_SEND_FROM
    value: ""
  - name: RESEND_API_KEY
    value: ""
  - name: CODE_EXECUTION_ENDPOINT
    value: http://sandbox:8194
  - name: CODE_EXECUTION_API_KEY
    value: "xYaiVMrBUtVppEv6N7cW"
  - name: CODE_MAX_NUMBER
    value: 9223372036854775807
  - name: CODE_MIN_NUMBER
    value: -9223372036854775808
  - name: CODE_MAX_DEPTH
    value: 5
  - name: CODE_MAX_PRECISION
    value: 20
  - name: CODE_MAX_STRING_LENGTH
    value: 80000
  - name: TEMPLATE_TRANSFORM_MAX_LENGTH
    value: 80000
  - name: CODE_MAX_STRING_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_OBJECT_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_NUMBER_ARRAY_LENGTH
    value: 1000

SECRET_KEY - is A secret key that is used for securely signing the session cookie and encrypting sensitive information on the database. You can generate a strong key using openssl rand -base64 42.
CELERY_BROKER_URL - Use redis as the broker, and redis db 1 for celery broker. example - redis://:difyai123456@redis:6379/1
[DB example] -
- DB_USERNAME: postgres
- DB_PASSWORD: dify_test123
- DB_HOST: pgsql_local
- DB_PORT: 5432
- DB_DATABASE: dify
[Milvus example] -
- VECTOR_STORE: milvus
- MILVUS_HOST: milvus_local
- MILVUS_PORT: 19530
- MILVUS_USER: root
- MILVUS_PASSWORD: milvus_test12
MILVUS_SECURE - must be set false if your milvus don't need password (Important!)
MAIL_TYPE - support resend or smtp, resend link is https://www.resend.com/⁠, An online email server.
STORAGE_TYPE - aliyun-oss, fill these fields ALIYUN_OSS_ENDPOINT/ALIYUN_OSS_BUCKET_NAME/ALIYUN_OSS_ACCESS_KEY/ALIYUN_OSS_SECRET_KEY
ALIYUN_OSS_PATH - version >= 0.8.2, must be set, this is the root path.
CODE_EXECUTION_ENDPOINT - code execute sandbox, if it's empty, custom code execute node will fail.
CODE_EXECUTION_API_KEY - If CODE_EXECUTION_ENDPOINT is not empty, CODE_EXECUTION_API_KEY must be set the same as the dify-sandbox image's running configuration.

⁠2.2.2 worker

The environment in a Kubernetes YAML file looks like this.:

env:
  - name: SECRET_KEY
    value: "MFnQ128032819024170983219038YYp5A"
  - name: CONSOLE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: CONSOLE_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: SERVICE_API_URL
    value: "https://dify-api.xxx.yyy"
  - name: APP_WEB_URL
    value: "https://dify-web.xxx.yyy"
  - name: CELERY_BROKER_URL
    value: "redis://:[password]@[redis_host]:6379/1"
  - name: REDIS_HOST
    value: "redis"
  - name: REDIS_PORT
    value: 6379
  - name: REDIS_PASSWORD
    value: ""
  - name: DB_USERNAME
    value: ""
  - name: DB_PASSWORD
    value: ""
  - name: DB_HOST
    value: ""
  - name: DB_PORT
    value: 5432
  - name: DB_DATABASE
    value: "dify"
  - name: STORAGE_TYPE
    value: "aliyun-oss"
  - name: ALIYUN_OSS_ENDPOINT
    value: ""
  - name: ALIYUN_OSS_PATH
    value: ""
  - name: ALIYUN_OSS_BUCKET_NAME
    value: ""
  - name: ALIYUN_OSS_ACCESS_KEY
    value: ""
  - name: ALIYUN_OSS_SECRET_KEY
    value: ""
  - name: VECTOR_STORE
    value: "milvus"
  - name: MILVUS_HOST
    value: "my-release-milvus"
  - name: MILVUS_PORT
    value: 19530
  - name: MILVUS_USER
    value: "root"
  - name: MILVUS_PASSWORD
    value: ""
  - name: MILVUS_SECURE
    value: false
  - name: 
  - name: WEB_API_CORS_ALLOW_ORIGINS
    value: "*"
  - name: CONSOLE_CORS_ALLOW_ORIGINS
    value: "*"
  - name: MAIL_TYPE
    value: "resend"
  - name: MAIL_DEFAULT_SEND_FROM
    value: ""
  - name: RESEND_API_KEY
    value: ""
  - name: MODE
    value: "worker"
  - name: LOG_LEVEL
    value: "INFO"
  - name: CODE_EXECUTION_ENDPOINT
    value: http://sandbox:8194
  - name: CODE_EXECUTION_API_KEY
    value: "xYaiVMrBUtVppEv6N7cW"
  - name: CODE_MAX_NUMBER
    value: 9223372036854775807
  - name: CODE_MIN_NUMBER
    value: -9223372036854775808
  - name: CODE_MAX_DEPTH
    value: 5
  - name: CODE_MAX_PRECISION
    value: 20
  - name: CODE_MAX_STRING_LENGTH
    value: 80000
  - name: TEMPLATE_TRANSFORM_MAX_LENGTH
    value: 80000
  - name: CODE_MAX_STRING_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_OBJECT_ARRAY_LENGTH
    value: 30
  - name: CODE_MAX_NUMBER_ARRAY_LENGTH
    value: 1000

MODE - must be set "worker"

⁠3. Which port does dify-api listen on?

API port: 5001

worker port: 8194

⁠4.【Important】- Upgrade

when you upgrade dify, you need to run this command in dify-api pod

flask db upgrade

⁠5. issues

when you upgrade the image version in [0.7.2, 0.8.0], the worker will fail, because of an issue in entrypoint.sh.

upgrade 0.8.0-a version to fix this bug!

https://github.com/langgenius/dify/issues/7698⁠

⁠5.1 error info

Usage: celery worker [OPTIONS]
Try 'celery worker --help' for help.
Error: Invalid value for '-l' / '--loglevel': '-Q' is not one of 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL', 'FATAL'.

⁠5.2. fix

change this content in entrypoint.sh

Old content

exec celery -A app.celery worker -P ${CELERY_WORKER_CLASS:-gevent} $CONCURRENCY_OPTION --loglevel ${LOG_LEVEL} \
    -Q ${CELERY_QUEUES:-dataset,generation,mail,ops_trace,app_deletion}

New content

exec celery -A app.celery worker -P ${CELERY_WORKER_CLASS:-gevent} $CONCURRENCY_OPTION -Q ${CELERY_QUEUES:-dataset,generation,mail,ops_trace,app_deletion} --loglevel ${LOG_LEVEL}

⁠6. Other links

Tag summary

Recent tags

Content type

Image

Digest

sha256:9d52f0f1e…

Size

840.8 MB

Last updated

6 months ago

docker pull henrybit/dify-api