Skip to content

feat: web ui#155

Open
MistEO wants to merge 28 commits intomainfrom
feat/web
Open

feat: web ui#155
MistEO wants to merge 28 commits intomainfrom
feat/web

Conversation

@MistEO
Copy link
Copy Markdown
Owner

@MistEO MistEO commented Apr 6, 2026

Summary by Sourcery

添加一个可通过浏览器访问的 Web UI 和 HTTP/WebSocket 后端,与现有的 Tauri IPC 并行工作,实现对 Maa 实例的远程控制与配置。

New Features:

  • 通过内嵌的 axum HTTP 服务器向外暴露 Maa 核心操作,为浏览器客户端提供 JSON API 和 WebSocket 事件。
  • 支持 MXU 的完整浏览器模式功能(设备/窗口发现、实例管理、任务、截图、回调、agent 日志),使用 REST 和 WebSocket 替代 Tauri invoke。
  • 在设置中新增局域网访问开关,用于控制 Web UI 绑定到 localhost 还是 0.0.0.0,并在调试面板中展示实际生效的 Web 服务器地址。
  • 通过后端 API 持久化并同步配置(支持多客户端变更通知),并确保在页面卸载时刷新配置快照。
  • 从内嵌资源或外部 dist 目录提供构建后的 React 前端,并在 Vite 开发服务器中代理 /api 请求,以支持无缝的 Web UI 开发体验。

Enhancements:

  • 重构 Maa 核心 Rust 命令,使 Tauri 命令与 HTTP 处理器在控制器连接、任务运行/停止、设备/窗口发现、截图等内部实现上得以复用。
  • 统一回调和 agent 输出的投递方式,使 Tauri WebView 与浏览器客户端都能通过通用的广播层接收 Maa 事件。
  • 扩展前端服务(maaServiceinterfaceLoadercontentResolverconfigService),集成 backendApiwsService,以便在 Tauri IPC 与 HTTP/WS 之间动态选择。
  • 改进窗口/图标/背景的处理逻辑,确保在桌面和浏览器环境中都能正确加载资源与元数据,包括 favicon 更新和更安全的窗口定位。
  • 调整工具链与运行时配置(tokio 特性、开发服务器 host/proxy、JSON 库目标、构建脚本),以支持内嵌 Web 服务器和基于浏览器的 UI。
Original summary in English

Summary by Sourcery

Add a browser-accessible Web UI and HTTP/WebSocket backend alongside existing Tauri IPC, enabling remote control and configuration of Maa instances.

New Features:

  • Expose Maa core operations over an embedded axum HTTP server with JSON APIs and WebSocket events for browser clients.
  • Support full-featured browser mode for MXU (device/window discovery, instance management, tasks, screenshots, callbacks, agent logs) via REST and WebSocket instead of Tauri invoke.
  • Introduce a LAN access toggle in settings that controls whether the Web UI binds to localhost or 0.0.0.0, and surface the effective web server address in the debug panel.
  • Persist and synchronize configuration through the backend API (with multi-client change notifications) and ensure config snapshots are flushed on page unload.
  • Serve the built React frontend from embedded assets or an external dist directory, and proxy /api requests in Vite dev server for seamless Web UI development.

Enhancements:

  • Refactor Maa core Rust commands to share internal implementations between Tauri commands and HTTP handlers (controller connection, task run/stop, devices/windows discovery, screenshots).
  • Unify callback and agent output delivery so both Tauri WebView and browser clients receive Maa events via a common broadcast layer.
  • Extend frontend services (maaService, interfaceLoader, contentResolver, configService) with backendApi and wsService integration to dynamically choose between Tauri IPC and HTTP/WS.
  • Improve window/icon/background handling so assets and metadata are loaded correctly in both desktop and browser environments, including favicon updates and safer window positioning.
  • Adjust tooling and runtime configuration (tokio features, dev server host/proxy, JSON lib target, build script) to support the embedded web server and browser-based UI.

Copilot AI review requested due to automatic review settings April 6, 2026 12:52
sourcery-ai[bot]

This comment was marked as outdated.

This comment was marked as outdated.

@MistEO MistEO marked this pull request as draft April 6, 2026 13:08
@MistEO MistEO linked an issue Apr 8, 2026 that may be closed by this pull request
@MistEO MistEO marked this pull request as ready for review April 8, 2026 15:21
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @MistEO, your pull request is larger than the review limit of 150000 diff characters

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 52 changed files in this pull request and generated 8 comments.

Comment on lines +58 to +70
export async function apiPost<T>(path: string, body?: unknown): Promise<T> {
const url = `${getApiBase()}${path}`;
const resp = await fetch(url, {
method: 'POST',
headers: body ? { 'Content-Type': 'application/json' } : {},
body: body ? JSON.stringify(body) : undefined,
});
if (!resp.ok) {
const text = await resp.text().catch(() => resp.statusText);
throw new Error(`API POST ${path} failed (${resp.status}): ${text}`);
}
return resp.json() as Promise<T>;
}
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apiPost(以及 apiPut / apiGet)无条件执行 resp.json():当后端返回 204 No Content(例如 /api/logs/:id 的 POST/DELETE)或返回空 body 时会抛出解析异常,导致调用方只能通过 catch 静默吞掉错误。建议在解析前根据 resp.status === 204Content-LengthContent-Type 判断是否有 JSON body,并在无 body 时返回 undefined(或提供 apiPostVoid/apiPutVoid 这类不解析响应体的辅助函数)。

Copilot uses AI. Check for mistakes.
Comment on lines +179 to +186
// 通知 Rust 后端更新内存缓存并广播 config-changed 给所有其他客户端
try {
markSelfSave();
const { invoke } = await import('@tauri-apps/api/core');
await invoke('notify_config_changed', { config });
} catch (err) {
log.debug('notify_config_changed 调用失败(不影响保存):', err);
}
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里在调用 notify_config_changed 前先 markSelfSave();如果 invoke 失败,计数不会回滚,可能让本客户端错误跳过下一次 config-changed(来自其他客户端)。建议只在 notify_config_changed 成功后计数,或在失败时回滚计数。

Copilot uses AI. Check for mistakes.
Comment on lines 251 to +256
async destroyInstance(instanceId: string): Promise<void> {
if (!isTauri()) return;
log.info('销毁实例:', instanceId);
if (!isTauri()) {
await fetch(`${getApiBase()}/maa/instances/${instanceId}`, { method: 'DELETE' });
log.info('销毁实例成功 (HTTP):', instanceId);
return;
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

浏览器环境下销毁实例使用了裸 fetch(..., { method: 'DELETE' }) 且未检查 resp.ok,即使后端返回 4xx/5xx 也会记录“销毁实例成功”。建议改用 apiDelete 或显式检查响应状态并在失败时抛错/记录错误。

Copilot uses AI. Check for mistakes.
Comment on lines 367 to +386
async runTask(
instanceId: string,
entry: string,
pipelineOverride: string = '{}',
): Promise<number> {
log.info(
'运行任务, 实例:',
instanceId,
', 入口:',
entry,
', pipelineOverride:',
pipelineOverride,
);
if (!isTauri()) {
return Math.floor(Math.random() * 10000);
const result = await apiPost<{ taskIds: number[] }>(
`/maa/instances/${instanceId}/tasks/run`,
[{ entry, pipeline_override: pipelineOverride }],
);
const taskId = result.taskIds[0] ?? 0;
log.info('任务已提交 (HTTP), taskId:', taskId);
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runTask(HTTP 分支)没有传递 selected_task_id,且后端当前的 run_task_impl 路径不会初始化/更新 task_run_state 映射与逐任务状态;这会导致“运行中追加任务”场景下前端无法通过 instanceTaskRunStatus 正确展示 pending/running 状态,也会影响通过 selectedTaskId 反查 taskId 的能力。建议为 runTask 增加 selected_task_id 入参并在后端同步更新 TaskRunState(pending_task_ids/mappings/statuses)。

Copilot uses AI. Check for mistakes.
.map_err(|e| e.to_string())?;
let task_id = job.id;

instance.task_ids.push(task_id);
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run_task_impl 只把 task_id push 到 instance.task_ids,但没有同步更新 task_run_state(pending_task_ids/mappings/statuses/overall_status)。在当前“后端为任务状态单一真相来源”的设计下,这会导致通过 maa_run_task / /tasks/run 提交的任务无法被 maa_get_all_states 正确恢复到前端任务状态。建议在这里补齐对 TaskRunState 的初始化/追加逻辑,并在 sink 中复用 handle_task_callback 来更新逐任务状态。

Suggested change
instance.task_ids.push(task_id);
if !instance.task_ids.contains(&task_id) {
instance.task_ids.push(task_id);
}
if !instance.task_run_state.pending_task_ids.contains(&task_id) {
instance.task_run_state.pending_task_ids.push(task_id);
}
instance
.task_run_state
.mappings
.insert(task_id, entry.to_string());
instance
.task_run_state
.statuses
.insert(task_id, TaskStatus::Pending);
instance.task_run_state.overall_status = TaskStatus::Running;

Copilot uses AI. Check for mistakes.
Comment on lines +92 to +112
// 从已加载的配置中读取 allowLanAccess 设置
let allow_lan_access = app_config
.config
.lock()
.unwrap()
.get("settings")
.and_then(|s| s.get("allowLanAccess"))
.and_then(|v| v.as_bool())
.unwrap_or(false);

tauri::async_runtime::spawn(async move {
web_server::start_web_server(
cfg_clone,
maa_clone,
app_handle,
ws_clone,
web_server::DEFAULT_PORT,
allow_lan_access,
)
.await;
});
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

当前应用启动时会自动启动内嵌 HTTP/WebSocket 服务。结合后端实现里对 CORS 的放开策略(permissive)与缺少鉴权/CSRF 防护,这会使任意网页都有机会向本机(甚至局域网)后端发起跨域请求,进而远程控制 Maa、读取本地资源代理接口等,风险较高。建议至少:默认仅允许同源/显式白名单 Origin;为写操作与敏感接口增加鉴权(token)或同源校验,并在启用 allowLanAccess 时强制启用认证机制。

Suggested change
// 从已加载的配置中读取 allowLanAccess 设置
let allow_lan_access = app_config
.config
.lock()
.unwrap()
.get("settings")
.and_then(|s| s.get("allowLanAccess"))
.and_then(|v| v.as_bool())
.unwrap_or(false);
tauri::async_runtime::spawn(async move {
web_server::start_web_server(
cfg_clone,
maa_clone,
app_handle,
ws_clone,
web_server::DEFAULT_PORT,
allow_lan_access,
)
.await;
});
let config_guard = app_config.config.lock().unwrap();
let settings = config_guard.get("settings");
// 内嵌 HTTP/WebSocket 服务默认不自动启动,必须显式开启
let enable_embedded_web_server = settings
.and_then(|s| s.get("enableEmbeddedWebServer"))
.and_then(|v| v.as_bool())
.unwrap_or(false);
// 从已加载的配置中读取 allowLanAccess 设置
let allow_lan_access = settings
.and_then(|s| s.get("allowLanAccess"))
.and_then(|v| v.as_bool())
.unwrap_or(false);
drop(config_guard);
if !enable_embedded_web_server {
eprintln!(
"Embedded HTTP/WebSocket server is disabled by default. Set settings.enableEmbeddedWebServer=true to enable it."
);
} else if allow_lan_access {
eprintln!(
"Refusing to start embedded HTTP/WebSocket server with LAN access enabled because authentication is not enforced yet."
);
} else {
tauri::async_runtime::spawn(async move {
web_server::start_web_server(
cfg_clone,
maa_clone,
app_handle,
ws_clone,
web_server::DEFAULT_PORT,
allow_lan_access,
)
.await;
});
}

Copilot uses AI. Check for mistakes.
Comment on lines +2 to +11
// rust-embed 在编译期需要 folder 路径存在,
// dev 模式下可能未运行 pnpm build 导致 ../dist 不存在,创建占位目录避免编译失败
let dist_dir = std::path::Path::new("../dist");
if !dist_dir.exists() {
let _ = std::fs::create_dir_all(dist_dir);
let _ = std::fs::write(
dist_dir.join("index.html"),
"<!DOCTYPE html><html><body>Frontend not built. Run <code>pnpm build</code> first.</body></html>",
);
}
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build 脚本在编译时创建/写入 ../dist/index.html,这会在工作区产生副作用文件(可能污染 git 状态、影响 CI 缓存、并让“未构建前端”变成静默成功)。建议改为:在 rust-embed 路径缺失时使用 build.rs 通过环境变量/feature 切换到空资源集,或仅在 OUT_DIR 下生成占位并让 embed/serve 逻辑从 OUT_DIR 读取,而不是写回仓库目录。

Suggested change
// rust-embed 在编译期需要 folder 路径存在,
// dev 模式下可能未运行 pnpm build 导致 ../dist 不存在,创建占位目录避免编译失败
let dist_dir = std::path::Path::new("../dist");
if !dist_dir.exists() {
let _ = std::fs::create_dir_all(dist_dir);
let _ = std::fs::write(
dist_dir.join("index.html"),
"<!DOCTYPE html><html><body>Frontend not built. Run <code>pnpm build</code> first.</body></html>",
);
}
// rust-embed 在编译期需要 folder 路径存在。
// 不要在 build.rs 中写回工作区生成占位文件;若前端未构建,应显式失败并提示先构建。
let dist_dir = std::path::Path::new("../dist");
let index_file = dist_dir.join("index.html");
println!("cargo:rerun-if-changed=../dist");
if !dist_dir.exists() || !index_file.exists() {
panic!(
"missing frontend build output at {}. Run `pnpm build` first.",
index_file.display()
);
}

Copilot uses AI. Check for mistakes.
Comment on lines 291 to 299

log.info(`特殊任务已追加, maaTaskId:`, maaTaskId);

// 注册映射关系
registerMaaTaskMapping(instance.id, maaTaskId, addedTask.id);
// 注册 task_id 与任务名的映射(用 t() 翻译特殊任务 label)
// 注册 task_id 与任务名的映射(用于日志显示)
registerTaskIdName(
maaTaskId,
addedTask.customName || t(specialTask.taskDef.label || specialTask.taskName),
);

// 设置任务状态为 pending
setTaskRunStatus(instance.id, addedTask.id, 'pending');

// 追加到任务队列
appendPendingTaskId(instance.id, maaTaskId);
} catch (err) {
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里虽然注册了 task_id→名称映射,但“运行中追加任务”仍依赖 maaService.runTask 提交任务;该提交路径目前不会向后端提供 selectedTaskId,也未见后端对 run_task 更新 TaskRunState 的逻辑,因此 UI 的 instanceTaskRunStatus 很可能始终是 idle、无法反映追加任务的运行状态。建议让追加任务路径携带 selected_task_id,并由后端把该任务追加进 pending 队列并跟踪状态。

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request 支持暴露http端口

2 participants