Outline (BUSL) fork with per-character CJK tokenization for full-text search across CJK content
489
A fork of outline/outline that adds per-character CJK tokenization to PostgreSQL full-text search, fixing the long-standing issue where Chinese / Japanese / Korean queries miss matches because the default english tokenizer treats CJK runs as a single opaque token.
This is the runtime image — pull this one to deploy.
searchVectorCJK tsvector column on documents, with a GIN index.outline_split_cjk(text) SQL helper that pads each CJK character (Han / Hiragana / Katakana / Hangul) with spaces so PostgreSQL tokenizes per-character.documents_search_trigger() to maintain both the original searchVector and the new searchVectorCJK, so the rest of the codebase keeps working.PostgresSearchProvider reads from searchVectorCJK and applies the same per-character splitting to user queries, keeping query and index in sync.项目 or 计划 now match documents where those characters appear, with no ICU/zhparser/jieba extension required.Latin / mixed-script content is unaffected — it still goes through the standard english tokenizer.
vX.Y.Z-cjk.N — pinned to upstream Outline vX.Y.Z, fork iteration N. Example: 1.7.0-cjk.1.X.Y — latest patch on a minor release.latest — most recent published tag.Same as upstream — replace the image in your docker-compose.yml:
services:
outline:
image: reveever/outline:latest
# ...env, volumes, depends_on as in upstream docs
The first run after upgrading from upstream Outline will execute a migration that adds the new column, index, helper function, and trigger, then backfills searchVectorCJK for all existing documents (UPDATE documents SET "title" = "title"). On large datasets this can take several minutes.
cjk-search)linux/amd64 only.
-- searchVectorCJK column + index exist
\d documents
-- trigger maintains both vectors
SELECT pg_get_functiondef('documents_search_trigger'::regproc);
-- body must assign to BOTH "searchVector" AND "searchVectorCJK"
Content type
Image
Digest
sha256:36e7e7395…
Size
259.5 MB
Last updated
6 months ago
docker pull reveever/outline