Synesis — API Usage Guide

Multimodal embedding and reranking service powered by Qwen3-VL-2B. Supports text, image, and mixed-modal inputs over a simple REST API.

Overview

Embeddings

Generate dense vector representations for text, images, or both. Vectors are suitable for semantic search, retrieval, clustering, and classification.

POST /v1/embeddings

Reranking

Given a query and a list of candidate documents, score and sort them by relevance. Use after an initial retrieval step to improve precision.

POST /v1/rerank

Similarity

Convenience endpoint to compute cosine similarity between two inputs without managing vectors yourself.

POST /v1/similarity

Interactive API Explorer

Full request/response schemas, try-it-out functionality, and auto-generated curl examples are available at http://<host>:8400/docs (Swagger UI). Use it to experiment with every endpoint interactively.

Base URL

All endpoints are served from a single base URL. Configure this in your consuming application:

http://<synesis-host>:8400

Default port is 8400. No authentication is required (secure via network policy / firewall).

Architecture

Service Architecture

Synesis loads two Qwen3-VL-2B models into GPU memory at startup: one for embeddings and one for reranking. Both share the same NVIDIA 3090 (24 GB VRAM).

Request Flow

graph LR Client["Client Application"] -->|HTTP POST| FastAPI["FastAPI
:8400"] FastAPI -->|/v1/embeddings| Embedder["Qwen3-VL
Embedder 2B"] FastAPI -->|/v1/rerank| Reranker["Qwen3-VL
Reranker 2B"] FastAPI -->|/v1/similarity| Embedder Embedder --> GPU["NVIDIA 3090
24 GB VRAM"] Reranker --> GPU FastAPI -->|/metrics| Prometheus["Prometheus"]

Typical RAG Integration

sequenceDiagram participant App as Your Application participant Synesis as Synesis API participant VDB as Vector Database Note over App: Indexing Phase App->>Synesis: POST /v1/embeddings (documents) Synesis-->>App: embedding vectors App->>VDB: Store vectors + metadata Note over App: Query Phase App->>Synesis: POST /v1/embeddings (query) Synesis-->>App: query vector App->>VDB: ANN search (top 50) VDB-->>App: candidate documents App->>Synesis: POST /v1/rerank (query + candidates) Synesis-->>App: ranked results with scores App->>App: Use top 5-10 results

Embeddings API

POST /v1/embeddings

Generate dense vector embeddings for one or more inputs. Each input can be text, an image, or both (multimodal).

Request Body

Field Type Required Description
inputs array Yes List of items to embed (1 to max_batch_size).
inputs[].text string * Text content. At least one of text or image is required.
inputs[].image string * Image file path or URL. At least one of text or image is required.
inputs[].instruction string No Optional task instruction to guide embedding (e.g. "Represent this document for retrieval").
dimension int No Output vector dimension (64–2048). Default: 2048. See Dimensions.
normalize bool No L2-normalize output vectors. Default: true.

Response Body

Field Type Description
embeddings[] array One embedding per input, in order.
embeddings[].index int Position in the input array.
embeddings[].embedding float[] The dense vector (length = dimension).
usage.input_count int Number of inputs processed.
usage.dimension int Dimension of returned vectors.
usage.elapsed_ms float Server-side processing time in milliseconds.

Input Modalities

Text Only

{
  "inputs": [
    {"text": "quantum computing basics"},
    {"text": "machine learning tutorial"}
  ]
}

Image Only

{
  "inputs": [
    {"image": "/data/photos/cat.jpg"},
    {"image": "https://example.com/dog.png"}
  ]
}

Multimodal

{
  "inputs": [
    {
      "text": "product photo",
      "image": "/data/products/shoe.jpg"
    }
  ]
}

Reranking API

POST /v1/rerank

Score and rank a list of candidate documents against a query. Returns documents sorted by relevance (highest score first).

Request Body

Field Type Required Description
query object Yes The query to rank against. Must contain text, image, or both.
query.text string * Query text. At least one of text or image required.
query.image string * Query image path or URL.
documents array Yes Candidate documents to rerank (1 to max_batch_size).
documents[].text string * Document text. At least one of text or image required per document.
documents[].image string * Document image path or URL.
instruction string No Task instruction (e.g. "Retrieve images relevant to the query.").
top_n int No Return only the top N results. Default: return all.

Response Body

Field Type Description
results[] array Documents sorted by relevance score (descending).
results[].index int Original position of this document in the input array.
results[].score float Relevance score (higher = more relevant).
results[].document object The document that was ranked (echoed back).
usage.query_count int Always 1.
usage.document_count int Total documents scored.
usage.returned_count int Number of results returned (respects top_n).
usage.elapsed_ms float Server-side processing time in milliseconds.

Example: Text Query → Text Documents

{
  "query": {"text": "How do neural networks learn?"},
  "documents": [
    {"text": "Neural networks adjust weights through backpropagation..."},
    {"text": "The stock market experienced a downturn in Q3..."},
    {"text": "Deep learning uses gradient descent to minimize loss..."},
    {"text": "Photosynthesis converts sunlight into chemical energy..."}
  ],
  "top_n": 2
}

Example: Text Query → Image Documents

{
  "query": {"text": "melancholy album artwork"},
  "documents": [
    {"image": "/data/covers/cover1.jpg"},
    {"image": "/data/covers/cover2.jpg"},
    {"text": "dark moody painting", "image": "/data/covers/cover3.jpg"}
  ],
  "instruction": "Retrieve images relevant to the query.",
  "top_n": 2
}

Dimensions, Batches & Performance

Matryoshka Dimension Truncation

Synesis uses Matryoshka Representation Learning (MRL). The model always computes full 2048-dimensional vectors internally, then truncates to your requested dimension. This means you can choose a dimension that balances quality vs. storage/speed.

Dimension Vector Size Quality Use Case
2048 (default) 8 KB / vector (float32) Maximum Highest accuracy retrieval, small collections
1024 4 KB / vector Very high Good balance for most production systems
512 2 KB / vector High Large-scale search with reasonable quality
256 1 KB / vector Good Very large collections, cost-sensitive
128 512 B / vector Moderate Rough filtering, pre-screening
64 256 B / vector Basic Coarse clustering, topic grouping

Important: Consistency

All vectors in the same index/collection must use the same dimension. Choose a dimension at index creation time and use it consistently for both indexing and querying. You cannot mix 512-d and 1024-d vectors in the same vector database index.

Batch Size & Microbatching

The max_batch_size setting (default: 32) controls the maximum number of inputs per API call. This is tuned for the 3090's 24 GB VRAM.

  • Text-only inputs: Batch sizes up to 32 are safe.
  • Image inputs: Images consume significantly more VRAM. Reduce batch sizes to 8–16 when embedding images, depending on resolution.
  • Mixed-modal inputs: Treat as image batches for sizing purposes.

Microbatching Strategy

When processing large datasets (thousands of documents), do not send all items in a single request. Instead, implement client-side microbatching:

  1. Split your dataset into chunks of 16–32 items.
  2. Send each chunk as a separate /v1/embeddings request.
  3. Collect and concatenate the resulting vectors.
  4. For images, use smaller chunk sizes (8–16) to avoid OOM errors.
  5. Add a small delay between requests if processing thousands of items to avoid GPU thermal throttling.

Reranking Batch Limits

The reranker also respects max_batch_size for the number of candidate documents. If you have more than 32 candidates, either pre-filter with embeddings first (recommended) or split into multiple rerank calls and merge results.

Integration Guide

Configuring a Consuming Application

To integrate Synesis into another system, configure these settings:

Setting Value Notes
Embedding API URL http://<host>:8400/v1/embeddings POST, JSON body
Rerank API URL http://<host>:8400/v1/rerank POST, JSON body
Health check URL http://<host>:8400/ready/ GET, 200 = ready
Embedding dimension 2048 (or your chosen value) Must match vector DB index config
Authentication None Secure via network policy
Content-Type application/json All endpoints
Timeout 30–60 seconds Image inputs take longer; adjust for batch size

Python Integration Example

import requests

SYNESIS_URL = "http://synesis-host:8400"

# --- Generate embeddings ---
resp = requests.post(f"{SYNESIS_URL}/v1/embeddings", json={
    "inputs": [
        {"text": "How to train a neural network"},
        {"text": "Best practices for deep learning"},
    ],
    "dimension": 1024,
})
data = resp.json()
vectors = [e["embedding"] for e in data["embeddings"]]
# vectors[0] is a list of 1024 floats

# --- Rerank candidates ---
resp = requests.post(f"{SYNESIS_URL}/v1/rerank", json={
    "query": {"text": "neural network training"},
    "documents": [
        {"text": "Backpropagation adjusts weights using gradients..."},
        {"text": "The weather forecast for tomorrow is sunny..."},
        {"text": "Stochastic gradient descent is an optimization method..."},
    ],
    "top_n": 2,
})
ranked = resp.json()
for result in ranked["results"]:
    print(f"  #{result['index']} score={result['score']:.4f}")
    print(f"    {result['document']['text'][:80]}")

Typical Two-Stage Retrieval Pipeline

  1. Index time: Embed all documents via /v1/embeddings and store vectors in your vector database (e.g. pgvector, Qdrant, Milvus, Weaviate).
  2. Query time — Stage 1 (Recall): Embed the query via /v1/embeddings, perform approximate nearest neighbour (ANN) search in the vector DB to retrieve top 20–50 candidates.
  3. Query time — Stage 2 (Precision): Pass the query and candidates to /v1/rerank to get precise relevance scores. Return the top 5–10 to the user or LLM context.

Similarity API

POST /v1/similarity

Compute cosine similarity between exactly two inputs. A convenience wrapper — embeds both, normalizes, and returns the dot product.

Request Body

Field Type Required Description
a object Yes First input (text, image, or both).
b object Yes Second input (text, image, or both).
dimension int No Embedding dimension for comparison (64–2048). Default: 2048.

Response Body

Field Type Description
score float Cosine similarity (−1.0 to 1.0). Higher = more similar.
dimension int Dimension used for the comparison.

Operations & Monitoring

Health & Readiness Endpoints

Endpoint Method Purpose
/ready/ GET Readiness probe. Returns 200 when both models are loaded and GPU is available. 503 otherwise. Use for load balancer health checks.
/live/ GET Liveness probe. Returns 200 if the process is alive. Use for container restart decisions.
/health GET Detailed status: model paths, loaded state, GPU device name, VRAM usage.
/models
/v1/models		 GET List available models (OpenAI-compatible). Returns model IDs, capabilities, and metadata. Used by OpenAI SDK clients for model discovery.
/metrics GET Prometheus metrics (request counts, latency histograms, GPU memory, model status).

Prometheus Metrics

Key custom metrics exposed:

  • embedding_model_loaded — Gauge (1 = loaded)
  • reranker_model_loaded — Gauge (1 = loaded)
  • embedding_gpu_memory_bytes — Gauge (current GPU allocation)
  • embedding_inference_requests_total{endpoint} — Counter per endpoint (embeddings, similarity, rerank)
  • embedding_inference_duration_seconds{endpoint} — Histogram of inference latency
  • Plus standard HTTP metrics from prometheus-fastapi-instrumentator

Environment Configuration

All settings use the EMBEDDING_ prefix and can be overridden via environment variables or /etc/default/synesis:

Variable Default Description
EMBEDDING_MODEL_PATH ./models/Qwen3-VL-Embedding-2B Path to embedding model weights
EMBEDDING_RERANKER_MODEL_PATH ./models/Qwen3-VL-Reranker-2B Path to reranker model weights
EMBEDDING_TORCH_DTYPE float16 Model precision (float16 or bfloat16)
EMBEDDING_USE_FLASH_ATTENTION true Enable Flash Attention 2
EMBEDDING_DEFAULT_DIMENSION 2048 Default embedding dimension when not specified per request
EMBEDDING_MAX_BATCH_SIZE 32 Maximum inputs per request (both embeddings and rerank)
EMBEDDING_HOST 0.0.0.0 Bind address
EMBEDDING_PORT 8400 Listen port

Error Handling

HTTP Status Codes

Code Meaning Action
200 Success Process the response.
422 Validation error Check your request body. Batch size may exceed max_batch_size, or required fields are missing.
500 Inference error Model failed during processing. Check server logs. May indicate OOM with large image batches.
503 Model not loaded Service is starting up or a model failed to load. Retry after checking /ready/.

Synesis v0.2.0 — Qwen3-VL Embedding & Reranking Service. For interactive API exploration, visit /docs on the running service.