Simple embeddings api endpoint. Fetches model from hf, uses sbert sentence-transformers.
620
This is a quick and dirty way to get an embeddings api endpoint that will allow you to encode sentences into vectors using huggingface models and sbert sentence transformers.
It uses a rudimentary flask server.
This image should not be run in production under any circumstances.
Store the hf_cache folder as a volume or mounted share - it contains the downloaded hf models.
See also:
BATCH_SIZE=32 # Any integer you'd like.
DEVICE=cpu # By default this uses max 4 cpus, to make it use more - use cpu0,cpu1,cpu2,cpu3,cpu4,cpu5...
REPO_ID=sentence-transformers/all-mpnet-base-v2 # This is a reasonable default for me
docker run -it --rm -p 5000:5000 \
-v /path/on/host:/app/hf_cache \
-e REPO_ID=<repo_id> \
-e BATCH_SIZE=<batch_size> \
-e DEVICE=<device> \
meltir/simple-embeddings-api:tagname
1. /embed_demo: a GET endpoint that embeds hardcoded sentences.
2. /embed_sentences: a POST endpoint that takes a list of sentences as a JSON payload and returns their embeddings.
3. /embed_sentences/normalize: a POST endpoint similar to `/embed_sentences` but it normalizes the embeddings.
These endpoints will accept post requests containing json arrays of sentences, which will encoded and returned as json.
POST localhost:5000/embed_sentences
Content-Type: application/json
[
"Test number one.",
"Test 2."
]
The output will be in the below format (number of vectors has been truncated):
[
{
"embedding": [
-0.005890531465411186,
0.024965772405266762,
-0.0500827319920063,
-0.018901731818914413
],
"parameters": {
"batch_size": 32,
"device": "cpu",
"normalize_embeddings": false
},
"repo": "sentence-transformers/all-mpnet-base-v2",
"sentence": "Test number one."
},
{
"embedding": [
0.008925802074372768,
-0.0020000915974378586,
-0.06097005680203438,
-0.03114527463912964,
0.009883358143270016,
0.03417069837450981,
-0.019230583682656288,
-0.06188041716814041,
-0.019297847524285316
],
"parameters": {
"batch_size": 32,
"device": "cpu",
"normalize_embeddings": false
},
"repo": "sentence-transformers/all-mpnet-base-v2",
"sentence": "Test 2."
}
]
Content type
Image
Digest
sha256:0847a7b1e…
Size
724 MB
Last updated
over 2 years ago
docker pull meltir/simple-embeddings-api:beta0.3-aarch64