Skip to main content

v1.5.0

HAWKI RAG’s ingestion and retrieval stack has been rebuilt around durable Temporal workflows, dataset-scoped search, and a clearer operations experience. This release also brings richer graph exploration, safer incremental indexing, optional model routing, and more predictable Docker deployments.

Upgrading an existing installation? This release changes core persistence and orchestration. Review the accompanying upgrade guide before replacing a running stack.

What's New​

  • Durable ingestion with Temporal. Website crawls, file conversion, vector indexing, and graph extraction now run as observable workflows with retries, timeouts, persisted history, and independently scalable Python workers.
  • A complete document lifecycle. Documents and processing stages are persisted in PostgreSQL, making progress, failures, retries, and cleanup visible across restarts.
  • Dataset-scoped retrieval. Browser sessions and bearer-token clients now resolve an explicit authorized dataset scope before querying Qdrant or Neo4j.
  • Interactive graph exploration. The playground can visualize Neo4j entities and relationships, highlight newly added triplets, show provenance and relationship counts, save snapshots, expand nodes, and clear graph views.
  • Graph-only ingestion. Operators can extract and persist Neo4j triplets without rebuilding the vector index.
  • Incremental indexing and deletion. Reprocessing avoids duplicate Qdrant points and Neo4j facts, while managed document deletion cleans up dataset records and associated storage.
  • Flexible model routing. Local Ollama remains the default, while the optional LiteLLM profile exposes configured Ollama, OpenAI, and Anthropic aliases for chat, vision, and embedding workloads.
  • Built-in API reference. The Laravel image now ships the Swagger UI and OpenAPI contract, including support for installations served below a reverse proxy path such as /hawki-rag/.

Quality of Life​

  • Three clear startup modes. Use make up-core for the production-shaped local UI, make up-core-local for source-mounted development, or make up-core-server behind an external reverse proxy.
  • Less manual setup. Supported Make targets create required Docker networks, initialize PostgreSQL, wait for health, and run Laravel migrations before writable application services start.
  • Optional-service discovery. Make checks for the configured crawler and converter containers and attempts to attach available services to the shared network without blocking core startup when they are absent.
  • A more useful operations UI. Svelte-powered dataset, pipeline, graph, and settings dashboards provide clearer navigation, responsive layouts, auto-refresh, stage progress, logs, and recovery controls.
  • A friendlier environment template. .env.example is grouped by purpose and documents PostgreSQL, Temporal, shared storage, external tools, model providers, retrieval, Neo4j, and optional integrations.
  • A simpler command surface. make help is the canonical command catalogue, with focused targets for startup, health, workers, tests, cleanup, migrations, model preparation, and graph administration.

Bugfix​

  • More accurate mixed and deep retrieval. Retrieval stages now normalize scores before merging, avoid mixing scaled and unscaled values, deduplicate by chunk identity, preserve ordinal terms such as dritte, and remove query instructions such as “in my dataset” from mandatory lexical matching.
  • Reliable reranking. Mixed-mode ranking no longer allows incompatible score ranges to hide a stronger lexical or secondary retrieval result.
  • Correct dataset boundaries. Query, graph, deletion, and browser flows now retain the complete authorized dataset scope and fail closed when a single active user cannot be resolved safely.
  • Stable long-running crawls. Scraper progress resumes without duplicate retries, profile settings reach the crawler, failed crawls block downstream stages, and status is persisted consistently.
  • Safer conversion handoff. Converted output is validated, metadata is normalized, unsupported files fail clearly, nested conversion trees are not ingested twice, and writes use the shared-storage contract consistently.
  • Cleaner knowledge graphs. Reverse duplicate relationships are folded into canonical facts, provenance is retained through doc_ids, UTF-8 snippets are preserved, and temporary RAG-Anything/LightRAG extraction data no longer pollutes Neo4j statistics.
  • Consistent duplicate handling. Repeated LightRAG documents are recorded as skipped rather than failed, reducing misleading error counts.
  • Predictable proxy and asset URLs. Laravel, Vite, Swagger, and the crawler UI now respect configurable protocols, asset bases, and reverse-proxy prefixes.
  • More dependable Docker health checks. Ollama and RAG health probes handle partial backend availability correctly, and supported startup and migration commands recreate missing external networks.
  • Better cross-platform image selection. Ollama and database administration image selection no longer assumes an x86_64 host.

Internals​

  • Reorganize Python RAG code around explicit API, application, provider, graph, ingestion, and CLI boundaries while retaining compatibility adapters where external callers still rely on them.
  • Tighten Python contracts by replacing broad Any annotations with concrete values, object, callables, and generics where runtime behavior proves the narrower type.
  • Align Laravel controllers, requests, repositories, workflow services, and graph helpers with framework conventions and clearer domain boundaries.
  • Add structured validation, logging, startup health checks, cache cleanup, and error boundaries across ingestion and retrieval services.
  • Pin runtime images and Python dependencies, harden the MinerU path, update vulnerable frontend packages, and correct Composer package and license metadata.
  • Expand Laravel, Python, system, migration, OpenAPI, retrieval, graph, and Temporal regression coverage, including live integration suites that remain opt-in.
  • Build frontend assets in a dedicated Docker stage, exclude the unused nested crawler frontend from the Laravel build context, and retain Swagger assets in the final image.
  • Restore versioned contributor guidance for Laravel, production Python, and RAWKI authorization work.