docs: update SQL-to-FalkorDB migration guide for latest feature set#402
Conversation
Co-Authored-By: Oz <oz-agent@warp.dev>
📝 WalkthroughWalkthroughUpdated SQL-to-FalkorDB migration guide to add BigQuery and Spark SQL support, introduce schema introspection and template scaffolding capabilities, expand control-plane features, and refine loader configuration documentation with execution modes and state backend details. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
operations/migration/sql-to-falkordb.md (2)
316-327: Consider explicitly calling out default-port collisions.Since defaults are reused by multiple loaders, add a brief sentence that concurrent mixed-loader runs will often require explicit per-loader port overrides.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@operations/migration/sql-to-falkordb.md` around lines 316 - 327, The docs list default metrics ports for each loader but don't warn that several defaults duplicate across loaders, causing port collisions when running mixed loaders concurrently; update the paragraph after the port list to explicitly call out that defaults are reused and that concurrent mixed-loader runs will usually require per-loader overrides via the CLI flag `--metrics-port` or each loader's `*_TO_FALKORDB_METRICS_PORT` environment variable so users know to set unique ports for each running loader.
217-217: Normalize source naming for consistency.This line says
Spark, while other sections consistently saySpark SQL (via Livy sessions)/Spark SQL (Livy). Consider using one canonical label throughout to avoid confusion.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@operations/migration/sql-to-falkordb.md` at line 217, Replace the inconsistent "Spark" label in the supported list string "Supported by: BigQuery, ClickHouse, Databricks, MariaDB, MySQL, PostgreSQL, Snowflake, Spark, SQL Server." with the canonical form used elsewhere (e.g., "Spark SQL (via Livy sessions)" or "Spark SQL (Livy)") so the document matches other sections; update that exact phrase in operations/migration/sql-to-falkordb.md to use the chosen canonical label.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@operations/migration/sql-to-falkordb.md`:
- Around line 316-327: The docs list default metrics ports for each loader but
don't warn that several defaults duplicate across loaders, causing port
collisions when running mixed loaders concurrently; update the paragraph after
the port list to explicitly call out that defaults are reused and that
concurrent mixed-loader runs will usually require per-loader overrides via the
CLI flag `--metrics-port` or each loader's `*_TO_FALKORDB_METRICS_PORT`
environment variable so users know to set unique ports for each running loader.
- Line 217: Replace the inconsistent "Spark" label in the supported list string
"Supported by: BigQuery, ClickHouse, Databricks, MariaDB, MySQL, PostgreSQL,
Snowflake, Spark, SQL Server." with the canonical form used elsewhere (e.g.,
"Spark SQL (via Livy sessions)" or "Spark SQL (Livy)") so the document matches
other sections; update that exact phrase in
operations/migration/sql-to-falkordb.md to use the chosen canonical label.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fe7bd30f-4387-4ae3-87c3-fa387ff08612
📒 Files selected for processing (1)
operations/migration/sql-to-falkordb.md
There was a problem hiding this comment.
Pull request overview
Updates the SQL-to-FalkorDB online migration documentation to reflect the current DM-SQL-to-FalkorDB loader/control-plane feature set, including new sources and CLI/control-plane capabilities.
Changes:
- Added BigQuery and Spark SQL (Livy) as documented migration sources, including CLI run examples and capability notes.
- Documented schema introspection and template scaffolding flags (
--introspect-schema,--generate-template) and clarified purge-mode support. - Refreshed control-plane feature list, API endpoints, and metrics/operational guidance.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - Snowflake: `9992` | ||
| - Spark: `9997` | ||
| - SQL Server: `9996` | ||
|
|
There was a problem hiding this comment.
The default metrics port list includes duplicates (BigQuery/MySQL both 9995; MariaDB/Spark both 9997). If two loaders are started with defaults on the same host, the metrics HTTP server will fail to bind or one loader’s metrics will be unavailable. Consider assigning unique defaults (if the loaders support it) or explicitly calling out these collisions in the doc and recommending setting --metrics-port when running those pairs together.
| Note: BigQuery and MySQL share the same default metrics port (`9995`), and MariaDB and Spark share `9997`. When running any of these pairs concurrently on the same host, configure distinct metrics ports for each loader. |
| - The Metrics UI uses persisted snapshots and does not expose internal scrape endpoint/port settings. | ||
| - Runs execute locally on the host machine running the control plane server. | ||
| - Runtime artifacts are persisted under `CONTROL_PLANE_DATA_DIR`, including a SQLite DB and per-run files. | ||
| - SSE auth with API key may use query-string token fallback because browser `EventSource` does not support custom auth headers. |
There was a problem hiding this comment.
This note recommends using an API key via query-string for SSE auth. Query-string tokens are easy to leak via browser history, reverse-proxy/access logs, and the Referer header on subsequent requests. Please add a short warning and mitigation guidance (e.g., prefer HTTPS, short-lived tokens, disable/refine access logging, or use a cookie-based/session approach / SSE polyfill when possible).
| - SSE auth with API key may use query-string token fallback because browser `EventSource` does not support custom auth headers. | |
| - SSE auth with API key may use query-string token fallback because browser `EventSource` does not support custom auth headers. Be aware that query-string tokens can leak via browser history, logs, and `Referer` headers; mitigate by using HTTPS, short-lived tokens, minimized/filtered access logging, or a cookie/session-based approach or SSE polyfill where possible. |
Summary
operations/migration/sql-to-falkordb.mdto match current DM-SQL-to-FalkorDB capabilitiesValidation
Artifacts
Co-Authored-By: Oz oz-agent@warp.dev
Summary by CodeRabbit
--introspect-schema,--generate-template).