Adds the REST API surface that Daedalus calls to manage workspace
lifecycle and dispatch file ingestion. All endpoints under /library/api/:
POST /workspaces/ create workspace (idempotent on
workspace_id; library_type frozen)
GET /workspaces/{workspace_id}/ workspace status with item/chunk
counts
DELETE /workspaces/{workspace_id}/ delete workspace + reachable
content; concept-safe (orphan-only
Concept GC; concepts referenced
elsewhere are preserved)
POST /ingest/ queue a file for ingest. Idempotent
on (library, source_ref, hash):
same triple → return existing job;
new hash → supersede.
GET /jobs/{job_id}/ poll job status
POST /jobs/{job_id}/retry/ re-dispatch a failed job
GET /jobs/?status=&library_uid= list recent jobs
Workspace-Library lookup uses the unique workspace_id index added in the
schema commit. Concept GC runs as a separate transaction after item/chunk
delete so partial failures don't leave the global graph corrupted.
Tests cover serializer validation, IngestJob ORM behavior, the
(library, source_ref, hash) idempotency query pattern, and auth
boundaries on every new endpoint. Cypher correctness is validated by
manual end-to-end testing — no live Neo4j in unit tests.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds DAEDALUS_S3_* settings (read-only credentials for the Daedalus bucket)
and a small `daedalus_s3.py` helper that fetches a file from Daedalus's
bucket and writes it into Mnemosyne's bucket via default_storage.
Adds the Celery task `library.tasks.ingest_from_daedalus`. Given an
IngestJob row, it:
1. Resolves the target Library (by library_uid).
2. Supersedes a prior Item with the same source_ref but different
content_hash by deleting the old Item + chunks first.
3. Fetches from Daedalus S3, copies into items/{item_uid}/original.{ext}.
4. Creates the Item node, links it to a default Collection.
5. Runs the existing EmbeddingPipeline.process_item.
6. Marks the job completed with chunks/concepts counts.
Failures retry up to 3× with exponential backoff; final failure marks
the job failed with the exception text. Routed to the embedding queue
so single-worker setups must consume it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds two new content-type-aware library types — `business` for
proposals/marketing/strategy (used by the work-team agents) and `finance`
for statements/tax/market commentary (used by Garth). Each ships with
chunking config, embedding/reranker instructions, an LLM-context prompt
that forbids fabricating financial figures, and a vision prompt.
Adds a unique-indexed `workspace_id` property to `Library` so a node
can be scoped to a Daedalus workspace. Null means a global library;
non-null means workspace-scoped. Search Cypher (added in a later
commit) enforces the boundary.
Adds an `IngestJob` Django ORM model — separate from neomodel — that
tracks asynchronous ingestion lifecycle (Daedalus → S3 → Celery →
embedding pipeline) with idempotency on (library, source_ref, hash).
Migration 0001_initial creates the table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace plaintext token storage with SHA-256 hashes so leaked database
contents cannot be used to authenticate. Plaintext is generated, shown
once at creation time, and never persisted.
- Add `hash_token()` helper and `MCPTokenManager.create_token()` that
returns `(instance, plaintext)`.
- Replace `token` field with indexed `token_hash`; look up bearers by
hashing the incoming value.
- Update dashboard, management command, and admin to surface plaintext
only at creation. Disable admin "add" since it cannot reveal plaintext.
- Migration drops the old `token` column and adds `token_hash`;
pre-existing tokens are invalidated and must be reissued.
- Remove Phase 4 RAG pipeline in favor of retrieval-only architecture
- Add FastMCP server exposing search, get_chunk, list_libraries tools
- Mount MCP endpoints (streamable HTTP + SSE) via Starlette in ASGI config
- Update README to clarify Mnemosyne is a retrieval engine, not RAG
- Let calling LLMs drive synthesis and iterative retrieval themselves
Replace OPTIONAL MATCH with MATCH for Library-Collection-Item paths to
ensure results are properly scoped to libraries, and remove per-query
score normalization since RRF fuses results by rank rather than score
magnitude.
Replace the single `DATABASE_URL` connection string with individual
environment variables (`APP_DB_NAME`, `APP_DB_USER`, `APP_DB_PASSWORD`,
`DB_HOST`, `DB_PORT`) for more granular database configuration control.
Implement hybrid search pipeline combining vector, fulltext, and graph
search across Neo4j, with cross-attention reranking via Synesis
(Qwen3-VL-Reranker-2B) `/v1/rerank` endpoint.
- Add SearchService with vector, fulltext, and graph search strategies
- Add SynesisRerankerClient for multimodal reranking via HTTP API
- Add search API endpoint (POST /search/) with filtering by library,
collection, and library_type
- Add SearchRequest/Response serializers and image search results
- Add "nonfiction" to library_type choices
- Consolidate reranker stack from two models to single Synesis service
- Handle image analysis_status as "skipped" when analysis is unavailable
- Add comprehensive tests for search pipeline and reranker client
- Introduced a new vision analysis service to classify, describe, and extract text from images.
- Enhanced the Image model with fields for OCR text, vision model name, and analysis status.
- Added a new "nonfiction" library type with specific chunking and embedding configurations.
- Updated content types to include vision prompts for various library types.
- Integrated vision analysis into the embedding pipeline, allowing for image analysis during document processing.
- Implemented metrics to track vision analysis performance and usage.
- Updated UI components to display vision analysis results and statuses in item details and the embedding dashboard.
- Added migration for new vision model fields and usage tracking.
- Implemented custom form widgets for date, time, and datetime fields with DaisyUI styling.
- Created utility functions for formatting dates, times, and numbers according to user preferences.
- Developed views for profile settings, API key management, and notifications, including health check endpoints.
- Added URL configurations for Themis tests and main application routes.
- Established test cases for custom widgets to ensure proper functionality and integration.
- Defined project metadata and dependencies in pyproject.toml for package management.
