Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b939101565 | ||
|
|
ae4e0909a6 | ||
|
|
530e768a5b | ||
|
|
3367bb6454 | ||
|
|
2a90201bfa | ||
|
|
b776ce58eb | ||
|
|
ffc0548e1f | ||
|
|
264fee5127 | ||
|
|
3b8a93e12c | ||
|
|
cf879c9265 | ||
|
|
01d1b54caa | ||
|
|
f3d7423d02 | ||
|
|
62f143fe9f | ||
|
|
9837ca581b | ||
|
|
c787ef325b | ||
|
|
3c4e6e7e37 | ||
|
|
ba5b13667b | ||
|
|
1baf908ba4 | ||
|
|
6d0b92c515 | ||
|
|
7d5ea814ec | ||
|
|
b21960cda1 | ||
|
|
ce42acc654 | ||
|
|
1ee519e5d2 | ||
|
|
bc91b98385 | ||
|
|
9c4ccc5bf5 | ||
|
|
46bcafd7a3 | ||
|
|
e46982af6c | ||
|
|
2a9398ab50 | ||
|
|
faad50df80 | ||
|
|
74c1cb752c | ||
|
|
a8358d90d7 | ||
|
|
486ea05ad5 | ||
|
|
4b78ca0f27 | ||
|
|
0151552f55 | ||
|
|
2c88be99ce | ||
|
|
a011e8eab6 | ||
|
|
9a3dff2f14 | ||
|
|
bdb096fb88 | ||
|
|
89c7c603f6 | ||
|
|
681403a4d2 | ||
|
|
a94276494a | ||
|
|
00861a59d6 | ||
|
|
832ab5d563 | ||
|
|
e15ee259f8 | ||
|
|
e4ef6a533e | ||
|
|
3b3b648580 |
No files matched your search
@@ -4,72 +4,164 @@
|
||||
|
||||
面向 **Windows + DxDesigner / Expedition Enterprise 7.9.5** 的器件批量替换工具。一个工程对应一个任务,可以只替换同料号下的部分位号;支持直接编辑和 Excel BOM 导入,同批工程按顺序执行。
|
||||
|
||||
**最新版本:V1.1.3** · [下载便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.3/DxPartReplace-1.1.3.zip) · [版本说明与附件](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/tag/v1.1.3) · [完整使用说明](docs/user/v1.1.0-project-tasks.md)
|
||||
**最新版本:V1.1.4 Beta 3(预发布)** · [下载便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.4-beta.3/DxPartReplace-1.1.4-beta.3.zip) · [版本说明与附件](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/tag/v1.1.4-beta.3) · [完整使用说明](docs/user/v1.1.0-project-tasks.md)
|
||||
|
||||
所有发布均标记为 **Beta 预发布**。V1.1.4 Beta 1 因 BOM 导入缺少 `win32timezone` 已撤回,请使用 Beta 3;旧 Git 标签保留供回溯。升级前备份 `data`,完整更新 EXE 与 `_internal`,不要只替换 EXE。
|
||||
|
||||
Beta 3 面向工单 #1/#3 复测:修复内部长路径访问,将后台资源移交独立批次监督进程,补齐安全重建、释放诊断与可选关闭归档。[本版变更和复测说明](docs/releases/v1.1.4-beta.3.md)。
|
||||
|
||||
本版提供[目标封装选择](docs/user/target-cell-selection.md):目标封装可留空,此时工具不主动覆写封装,可后续与中心库同步;需要指定时,在料号规则中读取并选择合法封装。BOM 保持原 11 列。
|
||||
|
||||
## 窗口操作教程
|
||||
## 图文教程
|
||||
|
||||
下面的 GIF 录制于 V1.1.2,操作步骤仍适用于 V1.1.3,图标与版本号以新版为准。包含创建任务、BOM 导入、错误行提示、保存、顺序执行、查看结果和开始新一轮。共 20 个步骤、47.5 秒;仅记录工具及其弹窗,CAD 等待已压缩,Excel 填表示例在录制之外完成。
|
||||
按下面 **13 步**,完成“创建任务 → 填 BOM → 导入校验 → 保存计划 → 替换 → 查看结果”。本例有两个工程 `Audio_D2128`、`Audio_T605A`,各自只替换 `RV05`,保留同料号下的 `RV11`。
|
||||
|
||||

|
||||
截图来自真实窗口:起始页为 V1.1.3,后续操作图取自已验收的 V1.1.2 业务流程,操作位置相同。点击图片可以放大查看;BOM 填写图为示意。
|
||||
|
||||
如果图片没有自动播放,可以[单独打开 GIF](docs/user/media/v1.1.2-window-bom-tutorial.gif)。
|
||||
### 第 1 步:解压并打开应用
|
||||
|
||||
## 开始使用
|
||||
[下载 V1.1.4 Beta 3 便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.4-beta.3/DxPartReplace-1.1.4-beta.3.zip),**完整解压**,双击 `DxPartReplace.exe`。保留它与 `_internal` 的相对位置,无需安装 Python;本机需要可用的 DxDesigner、许可证和中央库。
|
||||
|
||||
1. 下载并完整解压便携包,保留 `DxPartReplace.exe` 和 `_internal` 的相对位置。**便携包无需安装 Python**,本机仍需安装可用的 DxDesigner、许可证和中央库。
|
||||
2. 双击 `DxPartReplace.exe`。首次运行会自动创建同目录的 `data`,保存任务、日志和执行证据。
|
||||
3. 点击左侧 **新建**,在弹窗填写工程根目录,点击 **读取工程**,勾选需要处理的 `.prj`,再点击 **创建任务**。每个工程自动创建一个任务。
|
||||
打开后看到下图主窗口。首次使用请准备完整的工程副本,执行前保存并关闭待处理工程。
|
||||
|
||||
首次使用请在完整工程副本上验证映射与库配置。执行前保存并关闭待处理工程;工具会启动自己的后台 CAD,主桌面仍可查看其他文件。
|
||||

|
||||
|
||||
## 方式一:通过 BOM 批量配置
|
||||
### 第 2 步:新建任务,读取工程目录
|
||||
|
||||
### 1. 导出本批模板
|
||||
点击左下方 **新建**。在弹窗中填写工程根目录,或点击 **选择目录**,再点击 **读取工程**。
|
||||
|
||||
选择任务,打开顶部 **本批位号计划**,点击 **BOM 模板**。只要已导入工程,就能导出同批所有工程的一张空白表,**不需要先采集位号或填写料号规则**。
|
||||
**完成标志:** 弹窗列表显示该目录下找到的 `.prj` 工程。
|
||||
|
||||
工程名称、工程路径自动填写。工程路径用于唯一分配任务,请保持原样;同一工程有多条替换时,可以复制该工程行。
|
||||

|
||||
|
||||
### 2. 在 Excel 填写替换项
|
||||
### 第 3 步:勾选工程,创建任务
|
||||
|
||||
保留原模板的 11 列表头,按需要填写:
|
||||
勾选这次要处理的工程,点击右下角 **创建任务**。本例勾选两个工程。
|
||||
|
||||
| 列名 | 填写内容 |
|
||||
**完成标志:** 左侧出现两个任务,**一个工程对应一个任务**。
|
||||
|
||||

|
||||
|
||||
### 第 4 步:导出本批 BOM 模板
|
||||
|
||||
选中左侧任意一个任务,切换到顶部 **本批位号计划**,点击 **BOM 模板**,选择文件保存位置。
|
||||
|
||||
**完成标志:** 导出的 Excel 包含本批所有工程,工程名称和完整路径已经填写好。此时不需要先采集位号,也不需要先添加料号规则。
|
||||
|
||||

|
||||
|
||||
### 第 5 步:在 Excel 中填写替换项
|
||||
|
||||
打开刚导出的模板,**保留全部 11 列表头以及原工程路径**。同一工程需要填写多条替换时,复制该工程行后再填写。
|
||||
|
||||
本例在两个工程行中分别填写相同的旧、新料号和 `RV05`:
|
||||
|
||||

|
||||
|
||||
| 要填写的列 | 填法 |
|
||||
| --- | --- |
|
||||
| `old_part_number` | 工程当前的旧料号 |
|
||||
| `new_part_number` | 中央库中的目标新料号 |
|
||||
| `refdes` | 要处理的位号,例如 `RV05`;一行可以写多个位号,以逗号、分号、空格或顿号分隔 |
|
||||
| `enabled` | `TRUE` / `是` 表示替换,`FALSE` / `否` 表示保留 |
|
||||
| `sync_nc` | 是否同步 NC,默认开启,可逐行关闭 |
|
||||
| `custom_value` | 可选的自定义 Value;填写后优先写入,留空沿用任务的 Value 策略 |
|
||||
| `old_part_number` | 工程当前的旧料号,本例为 `4100-CA0000-T7` |
|
||||
| `new_part_number` | 中央库中的目标新料号,本例为 `4100-CA0000-T0` |
|
||||
| `refdes` | 要替换的位号,本例为 `RV05`;多个位号可用逗号、分号、空格或顿号分隔 |
|
||||
| `enabled` | `TRUE` / `是` 表示替换,`FALSE` / `否` 表示保留;留空默认替换,建议显式填写 |
|
||||
| `sync_nc` | 是否同步 NC,默认开启;按需要填 `TRUE` 或 `FALSE` |
|
||||
| `custom_value` | 可选的自定义 Value;留空时使用任务的 Value 策略 |
|
||||
|
||||
例如,首次配置的旧料号同时对应 `RV05`、`RV11`,只想替换 `RV05`,可以只填写 `RV05` 的启用项;首次采集的其余位号默认不替换。
|
||||
料号按**文本**填写,不要使用公式。相同位号出现在多个图页时,补填 `schematic_name`、`sheet_number` 或 `uid` 消除歧义。旧新相同、链式或循环替换会被拒绝。
|
||||
|
||||
**再次导入时,BOM 中未列出的既有计划行会保留原勾选和编辑状态。** 如果这轮只想执行 BOM 列出的目标,请在预览中检查其他启用行,并明确取消勾选,或在 BOM 中填写对应的 `enabled=FALSE` 行。
|
||||
**本例首次导入只列出 RV05,采集到的 RV11 默认保留。再次导入时,未列出的既有计划行会保留原勾选;需要停用时,请取消勾选或明确填写 `enabled=FALSE`。**
|
||||
|
||||
同名位号分布在不同图页时,补填 `schematic_name`、`sheet_number` 或 `uid` 消除歧义。`enabled` 留空默认替换,`sync_nc` 留空默认开启;建议显式填写。料号按文本填写;导入不接受公式,也不接受旧新相同、链式或循环替换。
|
||||
填完后保存 Excel 文件。
|
||||
|
||||
### 3. 导入并检查报告
|
||||
### 第 6 步:导入填写好的 BOM
|
||||
|
||||
点击 **导入 BOM**,选择填好的文件。工具先校验表格格式、工程归属,以及能从表中确定的重复声明和料号冲突;通过后,只核验本表涉及的工程。已有位号库存覆盖旧料号时直接匹配;缺少库存时优先通过原生 CLI 采集,不可用时才回退到后台 CAD 只读采集。
|
||||
返回应用,在 **本批位号计划** 点击 **导入 BOM**,选择刚保存的 Excel,等待校验结果。
|
||||
|
||||
新一轮导入开始后,旧报告会清除,界面显示读表、获取库存、匹配位号等进度;取消选择文件则保留原报告。导入失败或取消会保留已有预览和手动编辑。保存和执行前仍会核验工程。
|
||||
导入会检查表格和工程归属,并匹配相关工程的位号。缺少库存时优先调用原生 CLI 采集,不可用时回退到后台 CAD 只读采集。**导入不会执行替换,也不会直接保存计划。**
|
||||
|
||||
- 校验报告显示通过、失败、未检查的行数,并标明 Excel 行号及原因。
|
||||
- 有错误时,整批不会生成新的可保存预览,也不会部分写入计划。修正后重新导入。
|
||||
- 全部通过后,先得到预览;可以切换任务,检查或修改各工程的位号计划。
|
||||

|
||||
|
||||
### 4. 保存并执行
|
||||
### 第 7 步:检查导入报告,有错误先修正
|
||||
|
||||
确认预览后点击 **保存本批位号计划**;普通单工程编辑时按钮显示 **保存位号计划**。
|
||||
点击校验状态条中的 **详情**。报告会显示通过、失败、未检查的行数,以及 Excel 行号和错误原因。
|
||||
|
||||
点击 **执行当前任务** 处理当前工程,或点击左侧 **顺序执行本批任务** 处理同批工程。确认范围后,工具逐项执行原生 Replace Part、保存、关闭并重新打开核验。
|
||||
下图是故意填入不存在位号的示例:**通过 2 行、失败 1 行**,错误定位到 Excel 第 4 行。按报告修正 Excel、保存文件,再次点击 **导入 BOM**。
|
||||
|
||||
**左侧搜索和显示排序只影响浏览,不改变本批执行范围。** 调整执行顺序请使用上移、下移任务按钮。
|
||||
**有错误时不会部分保存。全部通过后再继续下一步。** 新一次导入开始后旧报告会清除;导入失败仍保留已有预览和手动编辑,取消文件选择则保留原报告。
|
||||
|
||||
## 方式二:直接在界面编辑
|
||||

|
||||
|
||||
### 第 8 步:核对每个工程的预览,保存计划
|
||||
|
||||
报告全部通过后,关闭详情,依次选中左侧各任务,核对:
|
||||
|
||||
- 需要替换的 `RV05` 已勾选,新料号正确。
|
||||
- 需要保留的 `RV11` 未勾选。
|
||||
- 自定义 Value、同步 NC 等逐行设置符合预期。
|
||||
|
||||
确认后点击 **保存本批位号计划**;单工程编辑或保存完成后,按钮显示 **保存位号计划**。
|
||||
|
||||
**完成标志:** 底部提示计划保存完成。下图为保存后的状态,只有 RV05 启用。
|
||||
|
||||

|
||||
|
||||
### 第 9 步:按任务确认 Value 与封装设置
|
||||
|
||||
选中一个任务,按需设置顶部 **Value 策略**:勾选后,将替换后的 **Part Label 写入 Value**;逐行自定义 Value 优先。处理多个工程时,分别确认各任务的开关。
|
||||
|
||||
本例开启该选项。**复制的是 Part Label,不是把新料号直接写进 Value。**
|
||||
|
||||

|
||||
|
||||
目标封装可留空,工具不主动覆写 Cell Name,可后续与中心库同步。需要本轮指定时,在 **本批料号规则** 中编辑对应旧、新料号规则,读取并选择合法封装,核对计划中的选择后保存。详见[封装操作说明](docs/user/target-cell-selection.md)。
|
||||
|
||||
### 第 10 步:选择执行范围,确认后开始
|
||||
|
||||
只处理选中工程,点击底部 **执行当前任务**;处理同批工程,点击左侧 **顺序执行本批任务**。本例使用后者。
|
||||
|
||||
在确认弹窗中核对工程数量和执行范围,确认无误再点击 **Yes / 是**。
|
||||
|
||||

|
||||
|
||||
**搜索和显示排序只影响浏览,不改变本批执行范围。** 调整实际执行顺序,请使用左下方 **上移任务 / 下移任务**。
|
||||
|
||||
### 第 11 步:等待后台执行和保存重开核验
|
||||
|
||||
工具按工程顺序执行替换,保存、关闭并重新打开核验。左侧显示各任务状态,底部显示当前阶段和进度。
|
||||
|
||||

|
||||
|
||||
运行期间可以在主桌面查看其他文件,请勿打开或修改本次正在处理的同一工程。需要暂停时,点击 **完成当前项后暂停**,等待当前步骤安全收尾。
|
||||
|
||||
**完成标志:** 任务显示已完成,底部提示本批执行结束。还要打开结果核对,不能只看进度条。
|
||||
|
||||
### 第 12 步:查看实际修改结果与日志
|
||||
|
||||
选中要检查的任务,点击右下角 **替换结果 / 日志**,查看 **位号替换结果**。
|
||||
|
||||
本例应看到:`RV05` 的旧料号为 `4100-CA0000-T7`,回读料号为 `4100-CA0000-T0`,核验状态为 **已保存并核验**。
|
||||
|
||||

|
||||
|
||||
需要排查原因时,切到 **完整日志**,查看具体阶段和耗时;可用 **上一页日志 / 最新日志** 翻页。失败、未命中或未核验不会被当作已保存成功。
|
||||
|
||||

|
||||
|
||||
### 第 13 步:需要继续替换时,开始新一轮
|
||||
|
||||
已完成任务可以点击底部 **开始新一轮**:
|
||||
|
||||

|
||||
|
||||
核对弹窗中的任务范围,再确认。
|
||||
|
||||

|
||||
|
||||
**完成标志:** 已完成任务恢复为待配置,旧计划、命令、快照和日志归档保留。接下来按工程**当前料号**重新填 BOM 或修改规则,重新采集、保存后执行;旧轮次通过 **查看轮次证据** 打开。
|
||||
|
||||
出现状态不确定或恢复提示时,先使用 **核验并恢复**,按[恢复说明](docs/user/interrupted-recovery.md)处理。
|
||||
|
||||
## 不使用 Excel:直接在界面编辑
|
||||
|
||||
1. 在 **本批料号规则** 添加旧、新料号,按需配置 Value 与同步 NC;规则提供采集条件和默认值。
|
||||
2. 切到 **本批位号计划**,点击 **采集所有匹配位号**。
|
||||
@@ -78,7 +170,7 @@
|
||||
|
||||
两种方式可以配合使用,不必为了编辑而先导出 Excel。切换任务会保留编辑草稿;关闭时有未保存内容会提示保存全部、放弃或取消。
|
||||
|
||||
## Value、暂停与执行结果
|
||||
## Value、NC 与日志选项
|
||||
|
||||
- 顶部 **Value 策略** 开关控制当前任务是否将替换后的 Part Label 写入 Value;逐行自定义 Value 优先。处理多个任务时,分别确认各任务的开关。
|
||||
- **同步 NC** 是逐行选项,默认开启,可以关闭。
|
||||
@@ -86,12 +178,6 @@
|
||||
- 点击 **替换结果 / 日志** 查看图页、位号、旧料号、实际回读料号和核验状态。失败、未命中、未核验不会显示为已保存成功;完整日志支持翻页。
|
||||
- 日志和 BOM 详情按需打开,关闭后释放窗口;数据和证据仍保留。
|
||||
|
||||
## 完成后再执行一轮
|
||||
|
||||
已完成任务可以点击 **开始新一轮**。工具归档旧计划、命令、快照与执行证据,清空过期位号基线,将同批已完成任务恢复为待配置。
|
||||
|
||||
接着按工程**当前料号**重新填 BOM 或修改规则,重新采集、保存并执行。旧轮次可通过 **查看轮次证据** 打开。出现状态不确定或恢复提示时,先使用 **核验并恢复**,不要直接重复执行。
|
||||
|
||||
## 更新程序与保留数据
|
||||
|
||||
先关闭旧程序并备份 `data`。完整解压新包,用新 EXE 和 `_internal` 配合原 `data` 使用;不要混用不同版本的依赖目录,也不要删除原数据。
|
||||
@@ -110,6 +196,10 @@ DxPartReplace/
|
||||
|
||||
## 验证范围与已知限制
|
||||
|
||||
V1.1.4 Beta 3 新增批次后台资源管理:一次 BOM 导入或顺序执行共用一个私有桌面,CAD 按需启动并在同批工程间复用,批次结束统一退出。若提示 **资源释放待处理**,已保存核验的结果保留,请点击 **重新清理资源**;该操作只清理后台资源,不会再次替换。旧的工程保护锁仍按“核验并恢复”处理。设计、异常边界与实机结果见[批次运行时验收记录](docs/testing/2026-10-21-batch-private-runtime.md)。
|
||||
|
||||
资源释放会追加日志:在 **替换结果 / 日志 → 完整日志** 查看本批开始释放、释放成功/失败及重新清理的结果。独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`,记录批次、桌面、CAD PID、退出码、释放耗时、句柄关闭和桌面消失的分别确认,以及失败原因;同时保留 `session.json` 的原始释放证明。重启后重新检查也会追加记录,不覆盖原失败日志。
|
||||
|
||||
V1.1.3 的业务代码沿用本机已验收候选:两工程实际替换和独立回读、362 字符工程路径、BOM 失败/成功导入、Part Label 写入 Value、可选封装、旧版任务数据副本、新轮次和自有后台资源释放。新增图标及版本元数据另做冻结包检查。详见[验收报告](docs/testing/2026-10-19-release-preparation.md)。
|
||||
|
||||
当前按 **evaluation-only(评估试用)** 交付。不同机器、中央库和 CAD 版本仍需用工程副本确认兼容性;替换回读通过不代表已通过 Packager、ERC 或网表检查。
|
||||
@@ -128,6 +218,7 @@ py -3.12 -m venv .venv
|
||||
|
||||
源码默认数据目录为 `%LOCALAPPDATA%\DxProductivity\trial-task-center`,可用 `DX_TASK_CENTER_HOME` 指定独立目录。运行回归:`python -m pytest -q`;构建使用 `packaging/Build-DxPartReplace.ps1`。
|
||||
|
||||
- [历史窗口 GIF(补充演示,录制于 V1.1.2)](docs/user/media/v1.1.2-window-bom-tutorial.gif)
|
||||
- [完整桌面使用说明](docs/user/v1.1.0-project-tasks.md)
|
||||
- [中断核验与恢复](docs/user/interrupted-recovery.md)
|
||||
- [Value 与属性继承](docs/user/value-copy-and-layout.md)
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"version": "1.1.4-beta.1",
|
||||
"release_tag": "v1.1.4-beta.1",
|
||||
"classification": "beta",
|
||||
"prerelease": true,
|
||||
"code_commit": "e46982af6c3ea145cfa04486a54227ca02ecdd89",
|
||||
"application_entry_matches_source": true,
|
||||
"vbs_match_source": {
|
||||
"QuickPartReplace.vbs": true,
|
||||
"NativeSnapshotReader.vbs": true
|
||||
},
|
||||
"data_directory_excluded": true,
|
||||
"gif_recorded_version": "1.1.2",
|
||||
"acceptance_report": "docs/testing/2026-10-21-batch-private-runtime.md",
|
||||
"build_log": "build/release-1.1.4-beta.1-build.log",
|
||||
"zip_sha256": "acf9f13617b5c25ddfe4c9e4df90481f6ca7838329bca1c201e7c1fdfe27d4a1",
|
||||
"zip_bytes": 60266953,
|
||||
"file_count": 396,
|
||||
"extracted_zip_smoke": true,
|
||||
"exe_sha256": "c0a43c302568206c9bbfc02203daded1df8e19df1d2083c2f0961eddd213a900",
|
||||
"build_verification": {
|
||||
"passed": 822,
|
||||
"warnings": 1,
|
||||
"seconds": 211.19
|
||||
},
|
||||
"frozen_helper_verification": {
|
||||
"passed": true,
|
||||
"version": "1.1.4-beta.1",
|
||||
"windows_numeric_version": "1.1.4.1",
|
||||
"windows_prerelease_flag": true,
|
||||
"frozen_marker_written": true,
|
||||
"stale_marker_rejected": true,
|
||||
"commands": [
|
||||
"probe",
|
||||
"checkpoint",
|
||||
"quit"
|
||||
],
|
||||
"opened_projects": [],
|
||||
"resources_released": true,
|
||||
"observer_errors": []
|
||||
},
|
||||
"business_module_count": 84,
|
||||
"business_modules_all_match_source": true
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
# V1.1.4 Beta 1 构建与发布记录
|
||||
|
||||
源提交:`e46982af6c3ea145cfa04486a54227ca02ecdd89`。本次后续提交只补充构建记录,程序代码不变。版本显示为 `1.1.4-beta.1`,Python 包元数据为等价的 `1.1.4b1`;Windows 数字版本为 `1.1.4.1`,设置 `VS_FF_PRERELEASE`。
|
||||
|
||||
## 本次检查
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 标准打包检查 | 822 passed,211.19 秒;1 条已知 pywinauto COM 模式提示 |
|
||||
| 冻结代码对应源码 | 84 个业务模块及应用入口编译码全部一致,两份 VBS SHA256 一致 |
|
||||
| ZIP 解压 | 完整性检查、逐文件校验和解压后的 EXE 启动/干净退出通过 |
|
||||
| 版本信息 | 数字版本 1.1.4.1 与 Windows 预发布标志通过 |
|
||||
| 冻结保护标记 CLI | 当前尝试写入成功;旧代次被拒绝且原记录不变 |
|
||||
| 冻结 CAD 助手 | 实际调用 ZIP 解压 EXE 的 probe → checkpoint → quit,通过 |
|
||||
| CAD 与桌面释放 | CAD/退出助手码均为 0,批次 released,pending 为空,释放日志保留 |
|
||||
| 主桌面 | Default 保持不变,无自有 CAD 前台/主桌面污染,无观察错误 |
|
||||
| 用户数据 | 包内不含运行数据目录、任务数据库、现场日志或工单附件 |
|
||||
|
||||
本轮冻结助手验证没有打开任何工程,也没有重复替换;真实两工程替换范围和限制沿用[已记录的源码实机验收](../testing/2026-10-21-batch-private-runtime-live.md)。
|
||||
|
||||
初次便携包暂存被隐私目录检查阻止:仓库公开的 `docs/testing/data` 内有三份已提交的验收 JSON,被宽泛的 `data` 名称检查误判。保留失败暂存,调整为明确禁止包根运行数据和所有数据库/日志文件,在新暂存目录重新生成并通过;未放行用户数据。
|
||||
|
||||
## 附件
|
||||
|
||||
- `DxPartReplace-1.1.4-beta.1.zip`:60,266,953 字节;包含程序、完整离线文档、GIF 和空 BOM 模板。
|
||||
- `BOM-template.xlsx`、`window-tutorial-1.1.4-beta.1.gif`、`release-manifest-1.1.4-beta.1.json`、`SHA256SUMS-1.1.4-beta.1.txt`。
|
||||
- ZIP SHA256:`acf9f13617b5c25ddfe4c9e4df90481f6ca7838329bca1c201e7c1fdfe27d4a1`。
|
||||
- [紧凑构建记录](v1.1.4-beta.1-build.json);完整逐文件清单在发布附件的 manifest。
|
||||
|
||||
新发布与四个旧发布都设为 Beta 预发布。旧发布只增加 Beta 标题并调整 prerelease 状态,逐一确认原标签、正文和附件 ID/名称/大小不变。新附件先上传到草稿,逐项回下载核对 SHA256,再转为公开 Beta。
|
||||
|
||||
本机原始证据位于 `.worktrees/batch-runtime/build/release-1.1.4-beta.1`,构建日志为同工作树 `build/release-1.1.4-beta.1-build.log`。
|
||||
@@ -0,0 +1,31 @@
|
||||
# DxPartReplace V1.1.4 Beta 1
|
||||
|
||||
发布日期:2026-10-21。**Beta 预发布版**。此前发布也统一标记为 Beta,原标签、附件及下载地址保留。
|
||||
|
||||
## 下载与升级
|
||||
|
||||
完整解压 `DxPartReplace-1.1.4-beta.1.zip`,运行 `DxPartReplace.exe`。无需安装 Python;本机需要可用的 DxDesigner / Expedition Enterprise 7.9.5、许可证和中央库。
|
||||
|
||||
升级前关闭旧程序并备份 `data`,保留原数据,配套更新 EXE 与 `_internal`。不要让不同版本同时打开同一数据目录;历史保护锁不会自动解除。
|
||||
|
||||
[中文图文教程](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/README.md)。包内包含离线图文教程、GIF、默认 11 列 BOM 模板、构建清单和 SHA256 校验。教程图片及 GIF 沿用已验收的旧版录制,新增的“重新清理资源”与释放日志以本版界面为准。
|
||||
|
||||
## 本版更新
|
||||
|
||||
- 每次 BOM 导入和顺序执行分别作为一个批次,共用一个私有桌面;CAD 按需启动,在批内跨工程复用,批末统一退出。
|
||||
- 工程之间确认上一工程已关闭,重新核验路径、中心库、图页及目标身份;保留保存、关闭、重开回读流程。
|
||||
- 首次可能写入前持久化保护标记。只有明确证明尚未修改且旧资源已释放时,允许重建重试一次;身份冲突、已修改或结果不确定时不自动重放。
|
||||
- 释放失败保留已保存并核验的任务结果,阻止新批次,提供“重新清理资源”;清理操作不重复替换,重启后仍检查未收尾批次。
|
||||
- 释放日志显示开始、成功、失败和重新检查,包含耗时、CAD退出码、句柄关闭与桌面消失的分别确认、失败原因。同批各任务“完整日志”可见,独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`。
|
||||
- 修复 BOM 错误回报与清理错误同时发生时丢失逐行报告的问题;清理失败只保留诊断,不提供可应用的部分预览。
|
||||
- 保留原生 CLI 优先采集,减少不必要的 CAD 回退;原生会话重试耗尽时停止整批。
|
||||
|
||||
## 验证与范围
|
||||
|
||||
两个全新工程副本完成 BOM 导入、实际替换、保存重开、独立完整属性/连接回读和资源释放。两版本保存结果一致;各工程 57 个组件、103 个引脚连接,56 个非目标组件属性和连接不变,源工程及中心库 `.lmc` 哈希不变。
|
||||
|
||||
相同样例单次对照:BOM 加执行总耗时 170.860 → 151.859 秒,私有桌面创建 6 → 2 次,CAD 启动 2 → 1 次。该结果不代表固定加速比例;BOM 单次测量 2.250 → 3.187 秒。
|
||||
|
||||
工单完整库存覆盖的 31 行回放为 20 通过、11 失败,未强行放行不匹配项;其余 406 条替换行、96 个工程未获对应完整库存,不能宣称全部工程已实机通过。用户机器的外部桌面引用残留根因尚未在本机复现。
|
||||
|
||||
详见[批次运行时验收](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-batch-private-runtime.md)、[实机对照](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-batch-private-runtime-live.md)和[释放日志验证](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-resource-release-logging.md)。
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"version": "1.1.4-beta.2",
|
||||
"release_tag": "v1.1.4-beta.2",
|
||||
"classification": "beta",
|
||||
"prerelease": true,
|
||||
"code_commit": "1ee519e5d25e4052f20b7b32ad18dcd837eb948d",
|
||||
"application_entry_matches_source": true,
|
||||
"vbs_match_source": {
|
||||
"QuickPartReplace.vbs": true,
|
||||
"NativeSnapshotReader.vbs": true
|
||||
},
|
||||
"data_directory_excluded": true,
|
||||
"gif_recorded_version": "1.1.2",
|
||||
"build_log": "build/release-1.1.4-beta.2-build.log",
|
||||
"zip_sha256": "21d242d1c4bd015f3b140d01f08e6779be0b6408f4d0deef09b457ec22a00325",
|
||||
"zip_bytes": 60682629,
|
||||
"file_count": 402,
|
||||
"extracted_zip_smoke": true,
|
||||
"exe_sha256": "0247347e33389025e9dab61065040a1589ed14876eaa9db5cf8fabd24604c360",
|
||||
"build_verification": {
|
||||
"passed": 824,
|
||||
"warnings": 1,
|
||||
"seconds": 177.39
|
||||
},
|
||||
"packaging_checks_passed": 2,
|
||||
"acceptance_guard_checks_passed": 15,
|
||||
"business_module_count": 84,
|
||||
"business_modules_all_match_source": true,
|
||||
"frozen_main": {
|
||||
"passed": true,
|
||||
"win32timezone_archived": true,
|
||||
"process_creation_time_read": true,
|
||||
"resources_released": true,
|
||||
"windows_numeric_version": "1.1.4.2",
|
||||
"windows_prerelease_flag": true
|
||||
},
|
||||
"ui_acceptance": {
|
||||
"full_exe_all_program_files_match_zip": true,
|
||||
"template_columns": 11,
|
||||
"imports": [
|
||||
{
|
||||
"file": "BOM-含错误位号.xlsx",
|
||||
"seconds": 9.889999999999418,
|
||||
"title": "本批 BOM 校验未通过 · 通过 2 / 失败 1 / 未检查 0 行 · 详情"
|
||||
},
|
||||
{
|
||||
"file": "BOM-替换RV05.xlsx",
|
||||
"seconds": 4.734000000011292,
|
||||
"title": "本批 BOM 校验通过 · 通过 2 / 失败 0 / 未检查 0 行 · 详情"
|
||||
}
|
||||
],
|
||||
"search_preserves_tasks": true,
|
||||
"execution_seconds": 209.40600000001723,
|
||||
"completed_tasks": 2,
|
||||
"cad_starts": 1,
|
||||
"queue_desktop_generations": 1,
|
||||
"all_batches_released": true,
|
||||
"results_window_reopened": true,
|
||||
"observer_initial_path_mismatch_retained": true,
|
||||
"read_only_post_verification_passed": true,
|
||||
"replacement_replayed": false
|
||||
},
|
||||
"independent_graphs": {
|
||||
"passed": true,
|
||||
"components_each": 57,
|
||||
"pin_connections_each": 103,
|
||||
"unchanged_non_targets_each": 56,
|
||||
"source_hashes_unchanged": true,
|
||||
"library_lmc_hash_unchanged": true
|
||||
},
|
||||
"acceptance_report": "docs/testing/2026-10-23-issue-2-frozen-runtime.md",
|
||||
"old_release": {
|
||||
"tag": "v1.1.4-beta.1",
|
||||
"withdrawn_to_draft": true,
|
||||
"public_release_api_status": 404,
|
||||
"git_tag_retained": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# V1.1.4 Beta 2 构建与发布记录
|
||||
|
||||
日期:2026-10-23。源快照 `1ee519e5d25e4052f20b7b32ad18dcd837eb948d`;后续提交仅补充本次构建记录。
|
||||
|
||||
修复工单 #2 的 pywin32 动态依赖漏打包。版本 `1.1.4-beta.2`,Python 元数据 `1.1.4b2`,Windows 数字版本 `1.1.4.2`,带预发布标志。
|
||||
|
||||
| 验证 | 结果 |
|
||||
| --- | --- |
|
||||
| 发布门禁 | 824 passed,177.39 秒;1 条既有 pywinauto STA COM 警告 |
|
||||
| 额外检查 | 打包配置 2 项,验收脚本 15 项通过 |
|
||||
| 冻结主进程 | 最终 ZIP 解压 EXE 实际读取工作进程创建时间,确认进程退出和桌面释放 |
|
||||
| 源码一致性 | 84 个业务模块、应用入口编译码、两份 VBS 哈希一致 |
|
||||
| 完整 EXE 双工程 UI | 创建任务、默认模板导出、错误/正确 BOM 导入、保存计划、实际替换、保存重开、结果日志重开通过 |
|
||||
| 执行批次 | 209.406 秒;一次 CAD 启动,一个桌面代次;资源最终释放 |
|
||||
| 独立前后对照 | 每工程 57 组件、103 引脚连接,56 个非目标组件属性/连接不变,目标料号和 Value 正确 |
|
||||
| 原始数据 | 源工程目录完整哈希、中央库 .lmc 哈希不变 |
|
||||
| 最终 ZIP | 完整性、解压 smoke、逐文件与已验收 UI 包比对通过;不含运行 data、任务数据库或现场日志 |
|
||||
|
||||
旧外部观察器未扫描批次资源目录造成的验收断言保留在原始记录中。已补齐观察器并完成测试,采用只读续验确认已完成的执行和资源释放,没有再次替换。细节见[工单修复与实机记录](../testing/2026-10-23-issue-2-frozen-runtime.md)。
|
||||
|
||||
ZIP:`DxPartReplace-1.1.4-beta.2.zip`,60,682,629 字节。SHA256:`21d242d1c4bd015f3b140d01f08e6779be0b6408f4d0deef09b457ec22a00325`。
|
||||
|
||||
附件包括便携 ZIP、默认 BOM 模板、GIF、SHA256SUMS 和逐文件 manifest。GIF 沿用 1.1.2 教程,包内说明已标明。
|
||||
|
||||
Beta 1 已撤回为草稿,匿名发布 API 返回 404;其 Git 标签和原附件保留用于回溯。Beta 2 先以草稿上传,回下载核对附件哈希后再公开为 Beta。紧凑证据见[构建 JSON](v1.1.4-beta.2-build.json)。本机原始证据为 `build/release-1.1.4-beta.2`。
|
||||
@@ -0,0 +1,43 @@
|
||||
# DxPartReplace V1.1.4 Beta 2
|
||||
|
||||
发布日期:2026-10-23。**Beta 预发布版**。替代已撤回的 V1.1.4 Beta 1;旧 Git 标签保留供回溯,旧发布不再提供公开下载。
|
||||
|
||||
## 下载与升级
|
||||
|
||||
完整解压 `DxPartReplace-1.1.4-beta.2.zip`,运行 `DxPartReplace.exe`。无需安装 Python;本机需要可用的 DxDesigner / Expedition Enterprise 7.9.5、许可证和中央库。
|
||||
|
||||
升级前关闭旧程序并备份 `data`,保留原数据,配套更新 EXE 与 `_internal`。不要让不同版本同时打开同一数据目录;历史保护锁不会自动解除。
|
||||
|
||||
[中文图文教程](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/README.md)。包内包含离线图文教程、GIF、默认 11 列 BOM 模板、构建清单和 SHA256 校验。教程图片及 GIF 沿用已验收的旧版录制,新增的“重新清理资源”与释放日志以本版界面为准。
|
||||
|
||||
## Beta 2 修复
|
||||
|
||||
- 修复工单 #2(DXT-20261023-A1F7FADF):发布包缺失 `win32timezone`,导致 BOM 导入记录采集进程创建时间时失败。显式打包 pywin32 的动态依赖,无需用户安装 Python。
|
||||
- 发布门禁改为在完整 EXE 主进程中验证私有桌面、子进程启动、创建时间记录及最终释放,防止源码 Python 的完整环境掩盖打包缺失。
|
||||
- 不改变替换、保存回读、保护锁及资源恢复规则。
|
||||
|
||||
## 继承的批次功能
|
||||
|
||||
- 每次 BOM 导入和顺序执行分别作为一个批次,共用一个私有桌面;CAD 按需启动,在批内跨工程复用,批末统一退出。
|
||||
- 工程之间确认上一工程已关闭,重新核验路径、中心库、图页及目标身份;保留保存、关闭、重开回读流程。
|
||||
- 首次可能写入前持久化保护标记。只有明确证明尚未修改且旧资源已释放时,允许重建重试一次;身份冲突、已修改或结果不确定时不自动重放。
|
||||
- 释放失败保留已保存并核验的任务结果,阻止新批次,提供“重新清理资源”;清理操作不重复替换,重启后仍检查未收尾批次。
|
||||
- 释放日志显示开始、成功、失败和重新检查,包含耗时、CAD退出码、句柄关闭与桌面消失的分别确认、失败原因。同批各任务“完整日志”可见,独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`。
|
||||
- 修复 BOM 错误回报与清理错误同时发生时丢失逐行报告的问题;清理失败只保留诊断,不提供可应用的部分预览。
|
||||
- 保留原生 CLI 优先采集,减少不必要的 CAD 回退;原生会话重试耗尽时停止整批。
|
||||
|
||||
## Beta 2 发布验证
|
||||
|
||||
发布门禁 824 项通过;打包配置检查 2 项、外部验收脚本检查 15 项通过。完整 EXE 在两个全新工程副本上完成创建任务、导出默认模板、错误 BOM 报告(2 通过 / 1 失败)、修正后导入(2 通过 / 0 失败)、保存计划、实际替换、保存重开回读及结果日志查看。两工程各 57 个组件、103 个引脚连接,56 个非目标组件不变;源文件与中心库 `.lmc` 哈希不变。执行批次共一次 CAD 启动、一个桌面代次,最终释放已独立确认。
|
||||
|
||||
本轮[工单修复与验收记录](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-23-issue-2-frozen-runtime.md)包含原验证漏检范围及外部观察器修正后的只读续验说明;已完成替换没有重放。
|
||||
|
||||
## 继承的历史验证与范围
|
||||
|
||||
两个全新工程副本完成 BOM 导入、实际替换、保存重开、独立完整属性/连接回读和资源释放。两版本保存结果一致;各工程 57 个组件、103 个引脚连接,56 个非目标组件属性和连接不变,源工程及中心库 `.lmc` 哈希不变。
|
||||
|
||||
相同样例单次对照:BOM 加执行总耗时 170.860 → 151.859 秒,私有桌面创建 6 → 2 次,CAD 启动 2 → 1 次。该结果不代表固定加速比例;BOM 单次测量 2.250 → 3.187 秒。
|
||||
|
||||
工单完整库存覆盖的 31 行回放为 20 通过、11 失败,未强行放行不匹配项;其余 406 条替换行、96 个工程未获对应完整库存,不能宣称全部工程已实机通过。用户机器的外部桌面引用残留根因尚未在本机复现。
|
||||
|
||||
详见[批次运行时验收](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-21-batch-private-runtime.md)、[实机对照](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-21-batch-private-runtime-live.md)和[释放日志验证](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-21-resource-release-logging.md)。
|
||||
@@ -0,0 +1,37 @@
|
||||
# DxPartReplace V1.1.4 Beta 3
|
||||
|
||||
2026-10-23,**Beta 预发布/工单现场复测版**,对应工单 [#1](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/1)、[#3](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/3)。
|
||||
|
||||
## 下载与升级
|
||||
|
||||
完整解压 `DxPartReplace-1.1.4-beta.3.zip`,运行 `DxPartReplace.exe`,无需安装 Python。保留 EXE 与 `_internal` 的相对位置;本机需要可用的 DxDesigner、许可证和中央库。
|
||||
|
||||
建议在新目录解压,用工程副本复测。需要沿用任务时,先关闭旧程序、备份旧 `data`,再完整复制数据目录;不同版本不能同时打开同一数据目录。旧保护锁保留,不能通过删除保护记录来重试。
|
||||
|
||||
包内包含中文图文教程、窗口 GIF、空白 11 列 BOM 模板、发布清单。教程沿用旧版已录制的基本操作,新增加的恢复与归档选项以本版界面为准。
|
||||
|
||||
## 本版修改
|
||||
|
||||
- 修复内部长路径访问:缩短批次 attempt 目录,补齐扩展路径、兼容工作目录、深目录工程扫描、日志和保护记录访问;不要求修改系统注册表。
|
||||
- 后台资源由独立批次监督进程持有,批内复用桌面与按需启动的 CAD,批末退出后由主界面独立复核。主界面不持有私有桌面或 COM 对象。
|
||||
- 只有持久化证据证明尚未修改、工程身份一致、旧资源已释放,才允许每工程自动重试一次;修改后、保存核验失败或结果不确定时不重放。
|
||||
- 自有 CAD 启动期间退出时及时检测,避免探针继续空等 120 秒。真实连续启动故障的检测与收尾阶段约 0.25–0.33 秒,完整任务仍受 CAD 启动、替换及保存回读耗时影响。
|
||||
- 分别保留首发错误、清理错误与 BOM 逐行报告。清理资源不重新执行已完成替换;恢复请求和消息检查批次、代次及尝试身份。
|
||||
- 补充自有进程/句柄归属诊断,提供“导出诊断包”,包含独立的批次释放日志并脱敏。
|
||||
- 增加“正常关闭时归档并清空当前任务列表”,默认关闭。启用后安全任务移至回收站;存在保护、中断或未释放资源的任务保留,不物理删除工程、日志及执行证据。
|
||||
- 持久化只读采集缓存,绑定工程指纹及完整载荷摘要;损坏缓存隔离后重新读取。
|
||||
|
||||
## 工单复测步骤
|
||||
|
||||
1. 使用原问题目录长度及工程副本,创建任务并导入原 BOM;确认通过、失败、未检查行数和逐行原因都能显示,失败后再次导入会刷新状态。
|
||||
2. 先选择两个工程的少量位号,保存计划并顺序执行;核对目标料号、未选位号、保存重开结果及释放日志,再扩大批次。
|
||||
3. 遇到资源待处理时,使用“重新清理资源”,确认不会重复替换。仍失败请点击“导出诊断包”,将 ZIP、版本及触发步骤附到原工单。
|
||||
4. 需要关闭后清空列表时,启用正常关闭归档选项;重开后可从回收站恢复。保护或待核验任务应继续保留。
|
||||
|
||||
## 验证范围与已知边界
|
||||
|
||||
实现阶段完成两份新工程副本的完整 EXE BOM 流程、真实替换、保存重开与独立属性/连接对照:每工程 57 器件、103 引脚,56 个非目标器件不变;顺序执行共一个桌面、一次 CAD 启动。另完成深中文路径的冻结 EXE 检查、真实外部桌面持有者、标记写入失败、连续启动失败及写后监督进程崩溃的测试。
|
||||
|
||||
故障实机入口部分为源码非 GUI parent+真实冻结 supervisor/helper/VBS,不等同于完整 GUI 硬退出或机器掉电测试。不同候选的具体哈希、命令和范围见[实施与验收记录](../testing/2026-10-23-issue-3-safe-recovery.md)。本发布包重新构建并检查冻结源码一致性、解压启动及资源释放,结果写入发布清单。
|
||||
|
||||
**仍需现场验证:** 关闭系统长路径支持且没有 8.3 的组合环境尚未完整验收;原工单 102 个工程没有完成原工程实机替换,其中 96 个连报告回放也未覆盖,见[逐工程清单](../testing/2026-10-23-issue-3-uncovered-projects.md)。工单 #1 现场残留桌面的具体持有者仍未确定,不能宣称根因已根治。两个工单保持打开,等待现场反馈。
|
||||
@@ -0,0 +1,22 @@
|
||||
# 批次私有桌面实施计划
|
||||
|
||||
用户已在会话中明确批准完整设计及实施;自动重试策略为仅修改前重试一次。基线 e15ee25,不发布安装包。
|
||||
|
||||
## 实施任务
|
||||
|
||||
- [x] T1:批次运行时。显式传递资源,惰性共用桌面/CAD;项目关闭检查点独立于批末桌面释放;持久化生命周期,资源失败阻止新批次,支持重新清理。正常退出优先,无法确认旧资源释放不重建。
|
||||
- [x] T2:写前持久化。每次尝试有新的身份与代次,首个可能修改调用前 fsync 写入标记;失败禁止写入。仅当前尝试明确尚未写入且身份与只读指纹安全时允许一次重建重试。
|
||||
- [x] T3:入口及界面。BOM/任务队列分别拥有一次运行时;单独操作兼容;跨任务复用 CAD,同工程多步骤继续复用。清理状态独立于已核验任务结果,提供清理入口;原模板和队列可读。
|
||||
- [x] T4:验证与审查。故障注入与取消竞争、旧结果隔离、共享数量和缓存全命中;工单回放及两份真实工程副本替换/保存回读/图连接校验;独立审查并修复,分阶段 Git 提交后推送。
|
||||
|
||||
## 接口约定
|
||||
|
||||
`BatchCadRuntime(root, kind, stop_event=None)` 为上下文管理器,提供 `native_scope(evidence)`、`run_worker(command, timeout, abort_event=None, cooperative_abort=False)`;`root` 是应用数据根。原生 scope 提供桌面式 start_process/close 接口,关闭只回收该次 CLI 子进程。CAD run_worker 仅在需要时启动并保存项目关闭检查点。
|
||||
|
||||
监督端不保存 COM 对象,运行时通过可选 `runtime` 参数显式传递;工作进程中继续核验桌面、CAD PID/GUID 和执行标识。可调用 `guard_pending_resources(root)` 与 `cleanup_pending_resources(root)`,后者仅清理资源,不执行替换。
|
||||
|
||||
写前协议模块独立于 Qt/COM。每个进程尝试写入 `DXPART_ATTEMPT_FILE`,由监督端先落盘准备记录,原生写入前验证并置为 mutation_possible。标记和身份缺失不能证明安全。
|
||||
|
||||
## 验收约束
|
||||
|
||||
所有真实 CAD 写入仅对原仓库 build 下的隔离副本进行,不触碰源工程与中心库。不删除原 feedback 或历史证据,不放宽保护锁。性能按同一副本和相同输入比较,不承诺实机残留根因已治愈。
|
||||
@@ -0,0 +1,73 @@
|
||||
# 批次私有桌面实机验收,2026-10-21
|
||||
|
||||
本机单组对照通过:两个工程的 BOM 导入及一次顺序替换合计从 170.860 秒降至 151.859 秒,减少 19.001 秒,约 11.12%。业务流程的私有桌面数从 6 个降至 2 个,CAD 启动数从 2 次降至 1 次。该时间不包含独立核验器额外启动 CAD 的开销,属于本次对照的实测值。
|
||||
|
||||
## 输入与隔离范围
|
||||
|
||||
- 基线:原仓库 `e15ee25`;候选:`.worktrees/batch-runtime` 的批次运行时实现。
|
||||
- 原始 fixture:原仓库 `build/cbb-bom-import-live-20261015/projects/Audio_D2128` 与 `Audio_T605A`。
|
||||
- 两版本分别复制到全新目录,使用相同源文件及输入;源工程全文件 SHA-256 在结束后保持一致。
|
||||
- 中心库:`F:/Sync Tool Skyworth/Work Maps/Skyworth_CBB_Lib/Mentor_lib_EE7.9/mentor_lib.lmc`,只读使用,`.lmc` SHA-256 不变。
|
||||
- 两工程均只替换 `RV05`,由 `4100-CA0000-T7` 替换为 `4100-CA0000-T0`。`sync_nc=True`,`copy_part_number_to_value=False`,队列策略为 `preserve`。
|
||||
- 证据根:工作树 `build/batch-runtime-live-20261021-144210`,实际基线结果在 `baseline2`,候选在 `candidate`。`comparison.json` 汇总跨版本对照。
|
||||
|
||||
`preserve` 保持原生目标库器件的 Value,而非恢复旧器件文字。此目标的原生 Value 属性缺失,最终期望为 Part Number=`4100-CA0000-T0`、Part Label=`0R`、Value 属性不存在。该语义来自 `docs/superpowers/specs/2026-10-14-v1.1.0-project-tasks-design.md`,与 `QPRestoreComponent`、`QPVerifyValuePolicy` 的存在性及内容核验一致。两版本的最终实际状态均符合该期望。
|
||||
|
||||
## 测量结果
|
||||
|
||||
| 指标 | 基线 | 候选 |
|
||||
|---|---:|---:|
|
||||
| BOM 导入两工程 | 2.250 s | 3.187 s |
|
||||
| 两工程顺序替换、保存、回读和流程收尾 | 168.610 s | 148.672 s |
|
||||
| 上述业务流程合计 | 170.860 s | 151.859 s |
|
||||
| BOM 阶段私有桌面 | 2 | 1 |
|
||||
| 执行阶段私有桌面 | 4 | 1 |
|
||||
| 执行阶段 CAD 启动 | 2 | 1 |
|
||||
|
||||
基线执行阶段含两个只读 native CLI 桌面和两个 CAD 桌面;候选在该阶段共用一个私有桌面和同一个 CAD PID `9192`,generation 为 `1`。独立核验器使用的额外 CAD 会话单独记录在 `desktop_starts`、`cad_starts` 的 `independent_before`、`independent_after` 阶段,不计入表中的业务资源数量。
|
||||
|
||||
## 保存结果与释放核验
|
||||
|
||||
每份工程均通过独立 Python COM 读取器,从空 CAD 开始重新打开磁盘工程,直接读取图页组件、全部属性和引脚网络。读取器不调用生产恢复/替换代码。
|
||||
|
||||
- 两版本替换前的完整独立读取 JSON 完全相等;替换后的完整 JSON 也完全相等,包含目标器件的所有属性。
|
||||
- 每份工程保持 57 个组件身份、103 个引脚连接;56 个非目标器件的全部属性、位置、变换和引脚状态完全不变。
|
||||
- `RV05` 的 UID、Refdes、位置、方向、比例和网络连接保持一致,目标引脚编号非空;Part Number/Part Label/Value 符合上述策略。
|
||||
- 候选两份写前记录具有相同 batch session/generation、不同 attempt ID,并均为 `mutation_possible`。两个逻辑工程分别记录标记。
|
||||
- 候选 BOM 和任务队列资源记录均为 `released`,无自动重试、恢复或已成功步骤的重放。
|
||||
- 观察器确认所有拥有的 CAD 退出、私有桌面消失,输入桌面未切换,拥有的 CAD 未出现在用户桌面或成为前台窗口。
|
||||
- 候选源代码/VBS 清单的 SHA-256 在实机执行期间保持一致。两个工程源副本及中心库 `.lmc` 的 SHA-256 均不变。
|
||||
|
||||
关键证据:
|
||||
|
||||
- `comparison.json`
|
||||
- `candidate/report.json`
|
||||
- `candidate/desktop-observation.json`
|
||||
- `candidate/source-under-test-manifest.json`
|
||||
- `candidate/data/batch-resources/*/session.json`
|
||||
- `candidate/data/tasks/*/quick-run/batch-attempts/*/*.json`
|
||||
- `baseline2/read-only-verification/saved-verification.json`
|
||||
- `baseline2/{before,after}-Audio_D2128.json`、`baseline2/{before,after}-Audio_T605A.json` 及 candidate 对应文件
|
||||
|
||||
初次探针在 `baseline` 目录因属性 COM 对象未强制转为 `IVdAttr` 而停止,尚未执行替换。修复读取器后使用全新 `baseline2` 副本执行。`baseline2` 两次替换均已完成,随后旧核验断言误把默认关闭复制选项视为 Part Label 复制;保留原错误报告,依据既有 Value 策略修正独立预期,再仅补充缺少的磁盘读取。该补充过程没有调用队列、恢复或替换。原始错误记录和历史输出均保留。
|
||||
|
||||
## 复现
|
||||
|
||||
`tools/probe_batch_runtime_live.py` 使用服务层 BOM 导入和任务队列入口,自动创建全新工程副本,无恢复/重放选项。`--repo` 指向被测版本,工具本身保持相同。
|
||||
|
||||
```powershell
|
||||
$batchProbeRepo = 'F:/Sync Tool Skyworth/Work Maps/dxdesigner-part-replacement'
|
||||
$batchProbeWorktree = "$batchProbeRepo/.worktrees/batch-runtime"
|
||||
$batchProbeSources = "$batchProbeRepo/build/cbb-bom-import-live-20261015/projects"
|
||||
$batchProbeLibrary = 'F:/Sync Tool Skyworth/Work Maps/Skyworth_CBB_Lib/Mentor_lib_EE7.9/mentor_lib.lmc'
|
||||
$batchProbeOutput = "$batchProbeWorktree/build/batch-runtime-live-NEW-TIMESTAMP"
|
||||
$batchProbeBaseline = "$batchProbeWorktree/build/baseline-e15ee25"
|
||||
|
||||
git -C $batchProbeRepo worktree add --detach $batchProbeBaseline e15ee25
|
||||
python "$batchProbeWorktree/tools/probe_batch_runtime_live.py" --repo $batchProbeBaseline --source-root $batchProbeSources --library $batchProbeLibrary --output "$batchProbeOutput/baseline" --run-native
|
||||
python "$batchProbeWorktree/tools/probe_batch_runtime_live.py" --repo $batchProbeWorktree --source-root $batchProbeSources --library $batchProbeLibrary --output "$batchProbeOutput/candidate" --run-native
|
||||
```
|
||||
|
||||
基线须固定到 `e15ee25` 的独立目录;合入后原仓库已经是新实现,不能继续把原仓库当前 HEAD 当作旧基线。上述输出及基线目录均应使用尚不存在的新路径。
|
||||
|
||||
仅在两个任务均已完成而独立核验需补读时,可使用 `--verify-existing <已有输出目录>`。该分支只读已保存工程,不执行任务队列;核验结果写入独立的 `read-only-verification` 目录,且拒绝覆盖已有核验目录。
|
||||
@@ -0,0 +1,84 @@
|
||||
# 批次私有桌面复用与安全重建验收
|
||||
|
||||
本次仅更新源码和 Git,不修改 V1.1.3 发布包。实施基线为 `e15ee25`。
|
||||
|
||||
## 已实施的边界
|
||||
|
||||
一次 BOM 导入、一次顺序执行分别拥有一个 `BatchCadRuntime`。库存缓存全部命中时不创建 Windows 资源;原生 CLI 共用批次桌面,各自进程及 Job 及时回收。需要 CAD 时才启动,同批工程串行复用该实例。工程之间由独立子进程核对桌面、CAD GUID/PID 和活动工程路径为空;每个工程使用新的 VBS 工作进程,组件、引脚及库查询缓存不跨工程传递。原有保存、关闭、重开回读继续执行。
|
||||
|
||||
批次标识、代次、工程/任务/作业和尝试标识保存在独立 JSON 旁文件中,旧 BOM、TSV 和已保存任务格式保持兼容。启动下一次工作前先作废该证据目录的旧关闭证明。监控消息有自己的随机标识,旧代次的资源租约不能启动新进程或把迟到关闭结果登记为新代次。
|
||||
|
||||
原生写入、恢复属性及保存之前,工作进程先验证完整尝试身份并持久化 `mutation_possible`。落盘失败禁止调用写入。修改前自动重试仅一次,还要求整个工程此前所有作业都处于 `prepared`、身份匹配、状态文件未改变、工程完整指纹一致且旧资源已确认释放。当前未把“数据库文件单独变化”视为自动重试许可;缺少独立身份等价证据时保守停止。明确确认的恢复操作不自动重试。
|
||||
|
||||
原生 CLI 仅操作临时数据库副本。启动失败、超时或特定崩溃的重试还要核对当前租约已经关闭、当前批次/代次/尝试一致,以及源 PRJ/数据库 SHA256 未变化。普通表格、匹配、解析和封装错误不触发桌面重建。
|
||||
|
||||
原生会话重试耗尽或预算拒绝时停止整批,不能再回退 CAD 绕过异常。该分支在实机正常路径验收后新增四项异常注入并修复,普通非零退出与格式不支持的兼容回退保留。
|
||||
|
||||
身份冲突单独分类:bootstrap、空工程检查点的绑定损坏/不一致,以及旧 VBS 明确报告的私有 CAD GUID 冲突,都不会被归入普通启动未就绪而自动重建。子进程用专用退出码保留该类别,异常包裹也保留不可重试的原因。
|
||||
|
||||
每个成功释放的代次另外保存在 `generation_releases`,含桌面、CAD PID/GUID、清理证明与释放时间。重建不会覆盖旧代次的释放证据;发布失败后重新检查也不会重复追加成功记录。
|
||||
|
||||
## 资源清理与 Windows 契约
|
||||
|
||||
- [CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop):关闭自有句柄与确认桌面对象消失分开检查。仍有外部引用时不创建更多桌面绕过问题。
|
||||
- [Job Objects](https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects):工作进程和后代位于自有 Job;正常退出优先,必要终止限定于自有进程及 Job。
|
||||
- [TerminateProcess](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminateprocess):终止请求后等待实际退出,随后回收句柄。
|
||||
- 监督进程不连接 CAD COM,不调用 `SwitchDesktop`。工作进程通过 `STARTUPINFO.lpDesktop` 启动;不改变用户输入桌面。
|
||||
|
||||
`batch-resources/<批次>/session.json` 保存生命周期、进程创建时间、工程关闭证明、重建原因和耗时。批末清理失败独立报告“资源释放待处理”,保留已核验任务状态,阻止后续 CAD 入口。界面提供“重新清理资源”,仅清理/重新检查资源,不重放替换。BOM 错误仍保留逐行诊断,但移除可应用的预览。
|
||||
|
||||
应用重启后只能重新确认已记录进程和桌面消失;不会根据旧 PID 终止新进程、接管 COM 或自动解除历史工程保护锁。损坏或未知格式的所有权记录保持阻塞,不能把缺失进程列表解释成没有资源。清理过程中即使日志写入失败也继续回收自有资源;释放记录发布失败可再次检查并落盘。取消请求不能覆盖清理失败。
|
||||
|
||||
残留时保留窗口 PID/TID/类和查询错误,进一步线程诊断见[Windows 桌面研究](2026-10-21-windows-desktop-lifecycle.md)。没有查询到窗口或线程,不等于不存在外部句柄持有者。
|
||||
|
||||
新增真实 Windows 对象寿命回归:测试自己的独立进程连接批次桌面,批次清理确实进入 blocked 并保留 completed_result;该外部进程仍存活、用户输入桌面不变,也没有生成工程 uncertain.flag。测试关闭自己拥有的独立进程后,资源入口确认 released,generation 仍为 1,没有创建新桌面或重放任务。
|
||||
|
||||
## 回归范围
|
||||
|
||||
针对性注入包含:惰性零启动、多工程一个 CAD、关闭证明作废、一次重建和第二次失败、修改后禁止重放、写前记录失败、任务边界暂停竞争、资源清理失败优先、释放日志发布失败后重新检查、损坏记录、跨重启检查、旧代次租约及检查点拒绝。独立审查发现的关闭证明过期、重复退出首次启动失败实例、独立服务入口绕过门禁、BOM 清理失败丢报告、损坏资源记录误解除等问题均已修复。
|
||||
|
||||
全量检查发现一个基线已有的取消测试:假执行器仅声明清理完成,却没有写出必要的释放证据。已在原基线隔离复现,并修正测试的释放证据,生产校验条件没有放宽。
|
||||
|
||||
扩展旧版本回归还发现两处既有边界:旧队列冻结 Value 策略发生冲突时,原先会先访问工程进行备份,导致真正的冲突被路径错误覆盖。现在以只读检查先拒绝冲突,断言拒绝前不备份、不启动 CAD、证据不变;备份仍先于 CAD 采集。千步骤模拟测试的固定 30 秒限制在基线也失败(41.83 秒,峰值 2.35 MiB),改为直接约束命令/状态全量解析各不超过两次,保留正确性、128 MiB 内存上限及实际耗时记录,没有调高秒数来掩盖回归。
|
||||
|
||||
最终统一执行 `python -m pytest tests/v1_1 tests/trial_task_center -q --basetemp=build/pytest-batch-final-20261021`:**1104 passed、1 failed**,327.33 秒。唯一失败是旧测试要求工作进程路径与长路径字符串逐字相等;本次把证据目录放到较长的 build 路径后,合法的 Windows 短路径触发该断言。生产传参指向正确文件,未改生产代码;将测试改为核对 canonical 文件/目录身份并继续严格核对其余参数,整个 `test_quick_run.py` 在同样较长路径下复测 **34 passed**。两次执行合计覆盖全部 1105 个不同用例,没有未修复的失败;未为测试断言修改重复执行已成功的真实替换。
|
||||
|
||||
最终千步骤模拟指标:20 个工程 × 5 张图页 × 10 条规则,91 保存、909 未命中,全部作业唯一;32.147 秒、峰值 2.05 MiB,命令及状态全量解析各 **1 次**。这是注入工作进程的控制层压力测试,不能作为实际 CAD 替换吞吐量。
|
||||
|
||||
## 实机与工单范围
|
||||
|
||||
实机验收仅对两个全新工程副本进行;源工程、中心库和原始工单附件不修改。分别核对 BOM 导入、替换、保存重开、独立回读、全部非目标元件属性与引脚连接、最终资源释放及原输入桌面。性能按相同样例对照单次运行,不代表大规模实机稳定性或固定加速比例。
|
||||
|
||||
工单只有六个工程有完整已捕获原生库存,回放与真实 CAD 验收分别报告,不能据此宣称全部 102 个工程均已实机通过。用户机器的桌面残留根因尚未在本机复现,本次减少重复创建/释放并补齐恢复边界,没有宣称根因已根治。
|
||||
|
||||
### 两工程实机对照
|
||||
|
||||
完整过程与复现命令见[实机对照报告](2026-10-21-batch-private-runtime-live.md)。计时只包含 BOM 导入和顺序执行,不包含独立检查器额外打开 CAD 的时间。
|
||||
|
||||
| 指标 | 基线 | 候选 |
|
||||
| --- | ---: | ---: |
|
||||
| BOM 导入 | 2.250 秒 | 3.187 秒 |
|
||||
| 替换、保存回读、执行收尾 | 168.610 秒 | 148.672 秒 |
|
||||
| 上述总耗时 | 170.860 秒 | 151.859 秒 |
|
||||
| 私有桌面创建 | 6 | 2 |
|
||||
| CAD 启动 | 2 | 1 |
|
||||
|
||||
本次样例节省 19.001 秒(11.12%),BOM 阶段单次测量略慢。两个工程各保持 57 个组件、103 个引脚连接,56 个非目标组件全部属性及连接不变;两版本独立回读的完整结果一致。只有 RV05 更换到目标料号,RV11 保持旧料号。批次均记录 released,未自动重试,输入桌面未切换,源工程和中心库 `.lmc` 哈希不变。
|
||||
|
||||
正常替换验收之后,最终异常分类及释放历史补丁 `a8358d9` 又做了一次空 CAD 实机冒烟:probe、空工程 checkpoint、quit 全部通过,退出码均为 0,批次 released、pending 为空,CAD/桌面消失,输入桌面仍为 Default。该次实测 59.281 秒,没有打开任何工程,也没有重复替换。证据在工作树 `build/batch-runtime-idle-smoke-20261021-final`。
|
||||
|
||||
### 工单只读回放
|
||||
|
||||
使用[回放工具](../../tools/replay_issue_bom_report.py)复制工单任务库到新的 build 目录,再通过最新 `inspect_batch` 回放完整库存覆盖的原 Excel 行,保留原行号与工程归属。
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 工单原始范围 | 428 个任务;763 行,其中 437 条替换行、102 个工程 |
|
||||
| 已捕获完整库存 | 8 份报告、6 个工程,覆盖 31 条替换行 |
|
||||
| 最新逐行结果 | 20 通过、11 失败,工程/Excel 行号/状态与原报告一致 |
|
||||
| 失败行 | Excel 3、4、6、7、11、12、13、16–19 行 |
|
||||
| 回放副作用 | 0 CAD/桌面/进程创建,0 目标计划写入,复制数据库未变化 |
|
||||
| 原证据 | ZIP、BOM、原生库存哈希不变 |
|
||||
| 未回放 | 406 条替换行、96 个工程,没有相应完整已捕获库存 |
|
||||
|
||||
完整本机回放证据位于原仓库 `build/issue-1-batch-runtime-replay-final-20261021`。这些是报告回放,不能代替对应用户工程的实际替换验收。
|
||||
@@ -0,0 +1,47 @@
|
||||
# 工单 #1:大量工程 BOM 导入中断
|
||||
|
||||
2026-10-21,基于 V1.1.3 修复,未更新发布包。
|
||||
|
||||
工单:https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/1
|
||||
|
||||
## 附件事实
|
||||
|
||||
工单包含应用数据和原模板格式的 BOM。任务库有 428 个草稿任务;BOM 共 763 条数据行,其中 437 条填写了替换内容,涉及 102 个工程,其余 326 条为空白占位。未发现因工程数量触发的固定上限。
|
||||
|
||||
日志显示四次导入均因 CAD 采集后的私有桌面释放确认失败而整体中断;四次已完成的采集均有 `uncertain.flag`。另一次采集在打开工程阶段收到取消,附件没有最终收尾记录,不能据此判断是否成功释放。
|
||||
|
||||
原生清单回退 CAD 的一个原因是:`1PEF7071_PHY` 的一个非本次替换器件,展开报告中的显示位号和 `Ref Designator` 属性不同。类似情况也出现在 `2G5PHY_GPY211` 的非目标测试点。旧代码因此拒绝整个工程的原生库存。另一次回退源于数据库拒绝读取,该权限问题没有被认定为已修复。
|
||||
|
||||
## 修复行为
|
||||
|
||||
- 原生报告仍检查全部 UID、配置、层次路径、器件范围和属性唯一性;只有本次请求料号的位号差异才要求 CAD 核验。没有猜测或改写存在差异的目标位号。
|
||||
- CAD 读取失败,且有明确的会话释放证据时,记录该工程失败并继续检查后续工程。许可由专用异常承载,不能仅凭错误码或保护标记不存在来继续。
|
||||
- 未确认释放、未知操作异常等仍中止后续工程,但附带完整逐行报告,区分通过、失败和未检查;不生成部分预览、不写入部分计划。
|
||||
- 状态机在失败后保留报告,重试时清空旧报告。取消不能覆盖未经确认的清理失败。图页证据解析和缓存错误仍保守中止。
|
||||
- 桌面释放失败时额外记录残留窗口的 PID、线程和窗口类,查询失败则记录诊断错误。不会切换输入桌面或终止外部进程;释放判定和保护锁条件不变。
|
||||
|
||||
## 验证
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 工单 8 份完整原生报告回放 | 全部可解析,包括原先回退的 4 份报告 |
|
||||
| `1PEF7071_PHY` 目标核对 | 35 个目标与附件三份 CAD 清单的原理图、图页、UID、位号、料号和 Value 完全一致 |
|
||||
| 原生目标位号存在差异 | 仍拒绝原生库存,要求 CAD 核验 |
|
||||
| 本地既有工程副本的新鲜原生读取 | 两个工程分别 1.418 秒、1.596 秒,原生资源释放通过 |
|
||||
| 本机空 CAD 会话和完整 CAD 采集尝试 | 桌面均正常释放;完整采集因打开图页失败而拒绝,没有执行替换 |
|
||||
| 实际异步界面 | 失败详情保留 → 重试旧结果清空 → 成功预览;计划未提前写入 |
|
||||
| Windows 外部桌面引用 | 无窗口和有窗口均保持阻断;窗口场景识别到实际外部 PID,未终止其进程 |
|
||||
| BOM、原生库存、工程采集、状态机、桌面回归 | 114 passed,34.30 秒 |
|
||||
| 取消、隔离 CAD、采集缓存回归 | 63 passed,12.32 秒 |
|
||||
|
||||
独立代码审查发现并修复了“清理失败且保护标记写入失败时错误继续”和“同时取消掩盖未确认释放”两处边界,最终复核通过。审查者另行运行 38 项针对性检查通过。
|
||||
|
||||
工单原生报告中只有 6 个不同工程的完整库存。按原 BOM 回放其 31 条替换行,20 条通过,11 条无法匹配对应旧料号和位号:Excel 第 3、4、6、7、11、12、13、16、17、18、19 行。这些行没有被强行导入;其中第 16–19 行对应库存没有请求的旧料号。
|
||||
|
||||
## 限制与后续实机复核
|
||||
|
||||
本地相应源工程只有 10 个,版本与用户机器不完全相同;尚未对用户全部 102 个工程完成真实库存核验。本地新鲜读取耗时不能作为用户全部工程的加速比例。
|
||||
|
||||
用户机器的 CAD 桌面残留尚未在本机复现,不能宣称泄漏根因已解决。此修复减少已证实不必要的 CAD 回退,并补齐失败报告和诊断;后续需在实机重新导入,核对是否还产生 `inventory_cleanup` 及新的桌面诊断。保留原有锁、日志和附件,不自动解除历史保护锁。
|
||||
|
||||
原始工单附件仅在 `build/issue-1-20261021` 保留用于本地诊断,不纳入 Git;`feedback/` 未修改。工程打开尝试仅使用 `build/` 副本,没有执行 Replace Part 或发布新安装包。
|
||||
@@ -0,0 +1,21 @@
|
||||
# 资源释放日志
|
||||
|
||||
在批次资源复用实现上补齐可读历史,不改变资源释放条件、修改前重试规则或任务结果。
|
||||
|
||||
## 查看方式
|
||||
|
||||
- 界面:同批各任务的“替换结果 / 日志 → 完整日志”。
|
||||
- 独立文件:`data/batch-resources/<批次ID>/resource-release.log`。
|
||||
- 汇总诊断:`data/logs/application.log`;原始所有权证明仍为旁边的 `session.json`。
|
||||
|
||||
每次追加开始和最终结果,涵盖批次结束、安全重建、手动重新清理和重启后重新检查。记录批次ID、代次、桌面、CAD PID/GUID、退出码、耗时、句柄关闭、桌面消失、原证明路径及失败原因。不存在 CAD 绑定的历史记录不会被猜成“已正常退出”。缓存全部命中且未获取资源时,不虚构释放成功。
|
||||
|
||||
所有同批任务在启动前绑定日志目标;任务ID也写入批次记录,因此应用重启后的重新检查仍能关联原任务。重复点击已经成功的清理不会重复执行或追加伪造的新释放。读日志和重新清理均不执行工程替换。
|
||||
|
||||
日志写入错误由诊断层隔离:独立文件、各任务事件和应用日志分别尝试,单处失败仍继续其他输出;能落盘时记录 `release_log_errors`。即使日志文件被锁定或整个诊断器抛异常,也不能跳过进程等待/桌面回收、把未释放资源误判成功,或把已核验任务降为失败。所有输出位置均不可写时无法保证留下文本日志,原有资源释放证明及保护规则仍优先执行。
|
||||
|
||||
## 验证
|
||||
|
||||
新增七项验证先复现缺少日志功能,再实现通过,覆盖成功、外部持有失败后重试、多任务显示、日志写入失败、重启关联、CAD退出码及缓存零资源。既有真实 Windows 对象寿命测试增加日志断言:独立进程仍连接桌面时记录句柄关闭但桌面未消失的失败;该进程退出后重新清理记录成功。未切换输入桌面、未终止外部持有者、没有重放任务。
|
||||
|
||||
日志、Windows资源寿命、BOM、顺序任务、服务保护和界面入口针对性检查:63 passed,13.74秒。原有日志、服务、恢复及批次生命周期回归:72 passed,12.76秒。本次没有执行真实器件替换,没有打包。
|
||||
@@ -0,0 +1,67 @@
|
||||
# Windows 桌面生命周期:工单 #1 的继续排查
|
||||
|
||||
2026-10-21。此次没有修改生产替换流程、保护锁或发布包。
|
||||
|
||||
## 工单能证明什么
|
||||
|
||||
工单四次整批 BOM 导入的直接中断原因是 `inventory_cleanup`:受控 CAD、已知子进程退出,自有句柄关闭,但五秒后仍枚举到该私有桌面。它不能证明 Windows 因工程数量多而拒绝导入,也没有足够信息证明残留持有者是谁。BOM 行匹配失败是另外一类结果,详见 [首轮诊断](2026-10-21-issue-1-bom-import.md)。
|
||||
|
||||
## 对照微软官方文档
|
||||
|
||||
1. [CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop) 关闭一个打开的桌面句柄。成功不等于所有引用消失。调用进程的线程正在使用这个句柄,或它是初始桌面句柄时,关闭会失败。
|
||||
2. [GetThreadDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getthreaddesktop) 返回指定线程关联的桌面句柄;文档明确不要对该返回值调用 `CloseDesktop`。它与应用通过 `OpenDesktop` 自己打开的句柄归属不同。
|
||||
3. [Thread Connection to a Desktop](https://learn.microsoft.com/en-us/windows/win32/winstation/thread-connection-to-a-desktop) 说明线程的桌面连接、`STARTUPINFO.lpDesktop` 和继承句柄的关系。线程连接不要求存在可见窗口;不能仅凭窗口列表为空判断没有持有者。
|
||||
4. [Terminating a Process](https://learn.microsoft.com/en-us/windows/win32/procthread/terminating-a-process) 明确进程终止不会自动终止其创建的全部子进程。[TerminateProcess](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminateprocess) 是异步请求,必须等待退出。现有实现已有 Job、等待与句柄关闭逻辑;本轮未直接把它认定为错误或移除检查。
|
||||
5. [Traversing the Thread List](https://learn.microsoft.com/en-us/windows/win32/toolhelp/traversing-the-thread-list) 给出 Toolhelp 线程快照遍历方法。[CreateToolhelp32Snapshot](https://learn.microsoft.com/en-us/windows/win32/api/tlhelp32/nf-tlhelp32-createtoolhelp32snapshot) 要求关闭快照句柄。
|
||||
6. [CreateDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createdesktopw) 说明桌面堆限制。工单已有日志没有桌面创建失败或内存不足的证据,因此本轮没有改注册表的 `SharedSection`、全局资源配额或超时常量。
|
||||
|
||||
这些接口没有提供“按桌面名称强制删除全部引用”的安全操作。不能把关闭自己的句柄解释成强行删除其他进程持有的桌面。
|
||||
|
||||
## 原生受控实验
|
||||
|
||||
实验仅使用临时 Python 进程和 `dx-doc-*` 桌面,不打开真实工程。
|
||||
|
||||
| 实验 | 结果 |
|
||||
| --- | --- |
|
||||
| 无额外引用,GUI 进程正常退出 | 进程清理后桌面立即消失 |
|
||||
| 额外持有已退出进程的内核句柄 | 桌面正常释放,不能将进程句柄残留直接当作本案根因 |
|
||||
| 额外持有已退出线程的内核句柄 | 桌面正常释放 |
|
||||
| 子进程线程仍连接私有桌面,但不创建窗口 | 复现相同保护现象;窗口清单为空,线程扫描能找到实际 PID/TID/程序路径 |
|
||||
| 该外部线程的进程退出后 | 再次核验桌面正常释放 |
|
||||
| 对当前线程正在使用的桌面调用 CloseDesktop | 真实返回 WinError 170;恢复原线程桌面后重试成功,pywin32 没有吞掉失败或丢失重试句柄 |
|
||||
|
||||
实际受控残留实验还发现该会话的 `conhost.exe` 线程。此结果只证明诊断能看到无窗口持有者,不能证明用户现场也是 `conhost.exe` 导致。
|
||||
|
||||
实验记录在 `build/desktop-doc-research-20261021`:`retained-kernel-handles.json`、`close-api-contract.txt`、`retained-thread-session.json`、`retained-thread-diagnostic.json`、`released-after-owner-exit.json`。原始机器日志未修改。
|
||||
|
||||
## 新增只读诊断工具
|
||||
|
||||
源码工具:[diagnose_private_desktop.py](../../tools/diagnose_private_desktop.py)。需要在**发生问题的机器、桌面仍残留时**运行,事后复制到另一台电脑的旧日志不能还原已消失的 Windows 资源。
|
||||
|
||||
在仓库根目录使用已安装项目依赖的 Python:
|
||||
|
||||
```powershell
|
||||
python tools/diagnose_private_desktop.py --record '实际失败会话/private-desktop.json' --output 'build/desktop-diagnostic.json'
|
||||
```
|
||||
|
||||
也可以根据日志中的真实名称传入 `--desktop dx-实际标识`。同路径或硬链接形式的输出覆盖输入记录均被拒绝。
|
||||
|
||||
工具输出桌面是否仍被枚举、残留窗口、连接该桌面的线程 PID/TID/可执行程序路径,以及无法查询的线程错误统计。目标桌面查询句柄只在短生命周期子进程中打开;扫描子进程退出后再检查桌面存在性。借用的 `GetThreadDesktop` 句柄不关闭,自有进程查询句柄和快照关闭。不调用 `SetThreadDesktop`、`SwitchDesktop`、`TerminateProcess`,不关闭外部句柄。
|
||||
|
||||
实测 Windows 对另一进程的 `GetThreadDesktop` 查询,在仅打开 `DESKTOP_READOBJECTS`、`READOBJECTS|ENUMERATE`、`READOBJECTS|WRITEOBJECTS` 时仍返回访问拒绝,而打开拥有者 `GENERIC_ALL` 句柄后可以查询。因此仅在扫描子进程中请求该权限,仍只执行查询;权限不足会报错,不能当作“没有持有者”。
|
||||
|
||||
**工具的边界:**进程若只保留一个 `OpenDesktop` 句柄,却没有线程连接或窗口,线程/窗口扫描可能找不到它。空扫描不是释放证明;`desktop_still_listed` 仍独立报告。
|
||||
|
||||
这种情况下可使用微软官方 [Sysinternals Handle](https://learn.microsoft.com/en-us/sysinternals/downloads/handle),按其文档在管理员终端查询所有类型的句柄:
|
||||
|
||||
```powershell
|
||||
handle64.exe -a 'dx-实际标识'
|
||||
```
|
||||
|
||||
这里只查询,不使用 `-c` 关闭外部句柄。该工具由现场人员按微软文档运行,本轮未下载执行、接受许可或更改系统配置。拿到真实持有 PID 后,才能区分自有程序的生命周期问题、外部进程引用和资源释放延迟,并针对已确认的所有者修复。
|
||||
|
||||
## 验证与当前结论
|
||||
|
||||
原生测试验证了无窗口线程识别、外部进程不受影响、输入桌面不切换、输入记录不修改、诊断不遗留句柄,以及无匹配线程时仍报告桌面存在。四项工具测试通过;已有 Windows 桌面测试另行通过。
|
||||
|
||||
用户现场的持有者尚未查实,不能宣称私有桌面残留已经修好。此次交付是微软文档核对、排除实验和能在现场定位到进程/线程的独立诊断工具;没有删除保护锁、强杀不明进程或放宽准确性校验。
|
||||
@@ -0,0 +1,25 @@
|
||||
# V1.1.4 Beta 3 现场复测包构建记录
|
||||
|
||||
用户在实现验收后授权“压缩发布一版本发给工单回去测试”。本轮交付为工单 #1/#3 的 Beta 复测包,不将此前未满足的严格现场条件写为已解决,不自动关闭工单。
|
||||
|
||||
## 发布构建
|
||||
|
||||
版本字符串 `1.1.4-beta.3`,Python 项目版本 `1.1.4b3`,Windows 文件版本 `1.1.4.3`,Windows 预发布标志启用。
|
||||
|
||||
- `packaging/Build-DxPartReplace.ps1` 退出码 0,源码发布门禁 **889 passed**,195.14 秒;一条既有 pywinauto STA 警告。原始构建输出:`build/release-1.1.4-beta.3-build.log`。
|
||||
- 打包配置与验收脚本检查 **17 passed**,6.11 秒。
|
||||
- 构建门禁在完整冻结 EXE 中启动独立批次监督进程,读取真实进程创建时间,确认 Job/进程/桌面释放,再完成 GUI 启动和正常退出检查。
|
||||
- EXE SHA256:`df08cd328f6f4c9019f232c845e09627551e7b2d24790a466d53efcd291622d1`。
|
||||
- 与上一实现验收候选 `f38411a2…` 比较 91 个冻结 `dx_productivity` 模块,只有 `version` 模块变化;应用入口一致。比较递归标准化 code filename 后的 Python code object,不以时间戳或文件名推断源码一致。
|
||||
|
||||
完整实现、两工程替换和故障注入的候选哈希及范围见[实施验收](2026-10-23-issue-3-safe-recovery.md)。本轮没有对已完成工程重放替换。
|
||||
|
||||
## 压缩与分发校验
|
||||
|
||||
便携包只从发布 EXE、完整 `_internal` 和仓库教程组装,附空白 11 列 BOM 模板、窗口 GIF、清单;不从运行目录打包 `data`、工程、数据库或现场日志。
|
||||
|
||||
发布准备脚本校验冻结模块与当前源码、应用入口、两份 VBS 的一致性,生成程序文件 SHA256 清单。ZIP 校验 CRC、解压路径和所有程序文件哈希,再在清除源码 Python 环境变量后运行解压出来的完整 EXE。具体 ZIP 大小、SHA256、文件数与解压启动结果记录在发布附件 `release-manifest-1.1.4-beta.3.json`,公开校验文件为 `SHA256SUMS-1.1.4-beta.3.txt`。
|
||||
|
||||
先上传为草稿,下载回验全部附件的大小和 SHA256,再标记为 Beta 预发布。工单回复包含版本、下载地址、复测步骤和诊断包反馈入口;工单保持打开。运行时 API 返回与下载校验留存于 `build/release-1.1.4-beta.3`,不将认证凭据写入文件或输出。
|
||||
|
||||
发布说明:[V1.1.4 Beta 3](../releases/v1.1.4-beta.3.md)。尚未覆盖的系统组合、原工程和现场持有者问题继续以该说明为准。
|
||||
@@ -0,0 +1,40 @@
|
||||
# 工单 #2:冻结主进程依赖与实机回归
|
||||
|
||||
日期:2026-10-23。对应修正版:V1.1.4 Beta 2。
|
||||
|
||||
## 根因与复现
|
||||
|
||||
工单:https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/2 。附件 `application.log` 的 SHA256 为 `f81f4ee602089e85cf98f6f8c64dcb4f5325885573b1b210d97f43e4ba3d98ef`。
|
||||
|
||||
Beta 1 在 `inspect_bom_batch → NativeInventoryReader → _DesktopLease.start_process → BatchCadRuntime._track → process_identity → win32process.GetProcessTimes` 抛出 `ModuleNotFoundError: No module named 'win32timezone'`。pywin32 的 C 扩展在转换进程创建时间时动态导入该模块,PyInstaller 未自动收集。已核对与公开 Beta 1 哈希一致的 EXE(`c0a43c302568206c9bbfc02203daded1df8e19df1d2083c2f0961eddd213a900`),其 PYZ 中确实没有该模块;独立进程阻断该导入后调用 GetProcessTimes 可复现。
|
||||
|
||||
附件显示尚未启动 CAD,未进入替换;清理成功,桌面句柄关闭与桌面消失分别确认,耗时 1.281 秒。现有证据不支持归因为 BOM 内容或多工程私有桌面泄漏。
|
||||
|
||||
原发布验证用源码 Python 监督打包后的 COM 子进程,GetProcessTimes 实际运行在依赖完整的源码环境,未覆盖冻结主进程,这是本次漏检的边界。
|
||||
|
||||
## 修复与门禁
|
||||
|
||||
- 在 spec 中显式加入 `win32timezone`。
|
||||
- 冻结 EXE 的 `--smoke-test` 实际创建批次私有桌面,启动自有短命进程,经生产 `_track` 读取创建时间,再释放工作进程和桌面。不打开 CAD 或工程。
|
||||
- `tools/verify_packaged_runtime.py` 清除源码 Python 环境变量,以完整 EXE 验证上述链路及持久化释放证明;构建脚本必须通过该门禁。
|
||||
- 旧版 EXE 在新增门禁中失败:`Missing win32timezone in frozen archive`。新版 EXE 通过,Windows 版本为 1.1.4.2,带预发布标志。
|
||||
- 回归覆盖模拟缺依赖时仍完成资源释放。验收脚本检查每个原生采集操作的批次归属及最终释放,支持批次保留桌面的正常语义。
|
||||
- 不改变替换规则、写入前保护标记、人工恢复边界或已有保护锁。
|
||||
|
||||
## 自动验证
|
||||
|
||||
发布构建门禁:824 passed,177.39 秒;1 条既有 pywinauto STA COM 警告。打包配置检查另有 2 项通过;外部验收脚本防误判检查 15 项通过。新版本在无 PYTHONPATH/PYTHONHOME 的冻结主进程中完成进程创建时间读取和资源释放。
|
||||
|
||||
## 实机验证范围
|
||||
|
||||
使用 Audio_D2128、Audio_T605A 的全新副本,通过 Windows 界面操作完整 EXE。原始工程和中央库不用于写入。证据目录为 `build/release-1.1.4-beta.2/ui-acceptance`。
|
||||
|
||||
工程任务创建、导出默认 11 列模板、填写并导入错误/正确 BOM、保存计划已通过。错误表为通过 2 / 失败 1 / 未检查 0;正确表为通过 2 / 失败 0 / 未检查 0。保存前数据库没有部分计划,保存后每工程仅 RV05 启用,RV11 保留。搜索过滤不改变任务范围。BOM 使用原生 CLI,无 CAD 启动;两个工程共用一个桌面,释放已确认。第二次导入缓存命中,没有新增桌面。
|
||||
|
||||
两个工程均已通过完整 EXE 的实际替换、保存关闭及重新打开核验。队列耗时 209.406 秒,同一代桌面、一次 CAD 启动;未重建、未重放。结果窗口可关闭并重新打开,显示 RV05、新料号及已保存核验状态。
|
||||
|
||||
旧外部观察器遗漏 `data/batch-resources/*/session.json`,导致任务完成后的验收断言失败。已补齐扫描路径并以回归验证;原失败记录保留。未重新执行替换,而是读取完成事件、队列/批次记录,并用 Windows API 独立确认 CAD 已退出、桌面消失。结合原观察器 943 次采样的前台 PID 和输入桌面窗口记录确认,执行 CAD 未成为前台或进入输入桌面。只读续验结果为 `completed-execution-verification.json`。
|
||||
|
||||
独立源属性/连接读取器分别在替换前后打开工程副本:每工程 57 个组件、103 个引脚连接;56 个非目标组件的身份、位置、方向、属性及连接完全相同。目标 RV05 的料号由 4100-CA0000-T7 变为 4100-CA0000-T0;启用 Part Label → Value,回读均为 0R;目标位置、方向和网络连接保持一致。源工程副本来源目录的完整文件哈希及中心库 `.lmc` 哈希不变。完整 EXE 无 ERROR 日志,窗口正常退出。
|
||||
|
||||
本次不是全部工单工程实机覆盖,也不据此承诺所有机器上的桌面残留根因均已消除。原始缺依赖故障已在完整 EXE 主进程、双工程原生采集和替换路径上覆盖。
|
||||
@@ -0,0 +1,27 @@
|
||||
# Issue 3:外部 OpenDesktop 句柄实机验收
|
||||
|
||||
2026-10-23,在 Windows 本机运行 `python tools/probe_external_desktop_holder.py`,退出码为 0。探针使用真实 `BatchCadRuntime` / `PrivateDesktop`,只创建独立 Python 持有者,不启动 CAD,不打开工程,不执行替换。
|
||||
|
||||
真实证据保存在 `build/issue-3-external-holder/run-d0a0af0e6a154c65910fb5f29a645e61/evidence.json`,同时保留该目录下真实 `data/batch-resources/*/session.json`、`business-result.json` 和资源释放日志。每次运行创建全新的 `run-*` 目录,避免覆盖既有证据。
|
||||
|
||||
本次探针 PID 为 37744;桌面为 `dx-d346d3ab310f470c86ffd9d32ccbef4a`;独立持有者 PID 为 18136,使用 `OpenDesktop(..., DESKTOP_READOBJECTS)` 获得句柄 412。桌面所有者记录句柄 552 的创建与成功关闭,包含 PID 37744、原生线程 ID 22468。持有者仍在默认桌面,无测试窗口。
|
||||
|
||||
| 检查 | 实机结果 |
|
||||
| --- | --- |
|
||||
| 首次批次清理 | 自有桌面句柄关闭,但枚举仍发现桌面;等待 5 秒后 `blocked` |
|
||||
| 残留窗口 | `remaining_windows=[]`,仍然阻断,没有据此推断资源已释放 |
|
||||
| 持有者存活 | 首次失败与持有期间资源重试后,持有者均仍在运行 |
|
||||
| 新批次门禁 | `BatchCadRuntime.__enter__` 抛出 `batch_resources_pending`;候选批次 generation=0,未创建桌面 |
|
||||
| 持有中仅重试资源 | `cleanup_pending_resources` 再次确认阻断,没有启动 CAD 或业务操作 |
|
||||
| 持有者自行释放 | 通过探针持有的 stdin 发送 `release`;持有者调用 `CloseDesktop`,正常退出码 0 |
|
||||
| 释放后仅重试资源 | `cleanup_pending_resources` 返回空元组,session 状态为 `released`,桌面枚举确认消失 |
|
||||
| 业务结果 | 合成的已核验 `business-result.json` 逐字节保持一致;内存业务结果亦保持一致 |
|
||||
| CAD 与重放 | CAD 工厂调用次数 0;`cad_start_requested=false`、`cad_pid=null`、`processes=[]`;没有业务重放入口调用 |
|
||||
| 输入桌面 | 前后一致,未切换输入桌面 |
|
||||
| 释放后的准入 | 后续批次可通过准入检查,但探针不启动其桌面/CAD或业务 |
|
||||
|
||||
业务结果前后 SHA256 均为 `80507674ce6b5c1d11419bd6e7b2fdfea3c5a6697cb01d628abf232615e9dc7e`。本轮总耗时 10.609 秒,主要为两次有界的 5 秒释放确认。
|
||||
|
||||
诊断中的外部持有者仍为 `unknown`,未识别具体外部句柄持有 PID。上述 PID 18136 仅由探针自身创建进程返回值与辅助进程握手确认,不能视为产品诊断识别能力。空窗口列表也不能证明没有外部句柄持有者。
|
||||
|
||||
这轮覆盖的是资源门禁与资源清理 API 的真实 Windows 句柄路径。业务结果是独立合成验收夹具,不等同于真实 CAD 工程结果,也不替代打包 EXE 的交互验收。未修改 `src`。
|
||||
@@ -0,0 +1,33 @@
|
||||
# Issue 3:新版冻结 EXE 两次启动故障,最多恢复一次
|
||||
|
||||
2026-10-23,在 CAD 独占窗口运行 `tools/probe_frozen_startup_recovery.py --failures 2`,实机退出码 0。使用另一份全新 `Audio_D2128` 副本,不复用已完成替换或 postwrite 崩溃工程。未修改生产源码/VBS、未降低生产超时。
|
||||
|
||||
冻结 EXE SHA256 为 `3b6a8608bac9d88f93ac71487d5c32647af0894c639d624f380f60345d27d34a`,源码基线为 264fee5。本例入口仍为**源码非 GUI supervised parent + 冻结 worker/helper/VBS**;独立完整图读取为源码只读 inspector,在独立私有 CAD 中运行。
|
||||
|
||||
真实证据目录为 `build/issue-3-frozen-startup-recovery/run-275ef3111c8d423caa4242b9045e9c59`。
|
||||
|
||||
两次终止均使用同一安全约束:仅本轮 request/session/generation 的 CAD PID;核对 Windows 创建时间与拥有者记录相同、私有桌面相同、starting 状态、无 GUID/bootstrap/cad-ready、attempt=prepared;然后只终止该自有 CAD。未终止任何未知或外部进程。
|
||||
|
||||
| 检查 | 实机结果 |
|
||||
| --- | --- |
|
||||
| 第一代故障 | 自有 CAD PID 16192,创建时间 `2026-10-23 08:53:02.663000+00:00`,退出码 71 |
|
||||
| 第二代故障 | 自有 CAD PID 31544,创建时间 `2026-10-23 08:53:04.657000+00:00`,退出码 71 |
|
||||
| 提前退出检测 | reason=`owned private CAD exited during startup: exit code 71`,未等待 bootstrap 总超时 |
|
||||
| 故障阶段耗时 | 生产 `project_worker` 记录分别为 0.328 秒、0.250 秒;包含该阶段的启动/检测/本地清理,不是孤立检测函数的精确计时 |
|
||||
| 唯一预算 | execution generation=2;retries 只有一条且 `count=1`;第二次失败后任务 blocked/paused,无第三代 |
|
||||
| 两代 proof | 都保持 prepared;前后字节与 SHA256 不变;没有 mutation_possible |
|
||||
| 换代安全顺序 | 第一桌面确认消失、released_at 先于第二桌面创建 time_ns |
|
||||
| 业务执行 | 没有 `saved_ok`,未进入正式替换 |
|
||||
| 完整图对照 | before 与 after 结构完全一致:1 图页、57 器件、103 引脚,包括全部已读取属性、位置、方向、缩放、引脚编号/位置及 net UID/name |
|
||||
| RV05 | 仍为 `4100-CA0000-T7`、Value `0R`,没有修改 |
|
||||
| 清理 | 两代资源均确认释放,最终 session `released`,无 pending resources/supervisors |
|
||||
| 用户桌面 | 输入桌面未变化,自有 CAD 未成为前台或出现在输入桌面 |
|
||||
| 原始输入 | 原工程目录与中央库 LMC 哈希不变 |
|
||||
|
||||
第一代 prepared SHA256 为 `89ae4b40d87359a429f38afc9c653c39dab6722a8a2d2526d881aea8cec54f0e`;第二代为 `8d9fe3a23c1dfaf629db066dd6858f91c45103c4d08691f265f27fffa7e282da`。两者分别与自身执行前字节比较一致。
|
||||
|
||||
整个探针耗时 233.891 秒,包含两次独立私有 CAD 的完整图读取。此前旧 EXE SHA `4e251627…` 的同类第一代故障阶段耗时 120.187 秒;新版此处为 0.328 秒。不能把两轮探针总耗时直接视为性能对照:旧轮还完成了一次真实替换,本轮连续失败后没有替换。
|
||||
|
||||
主要证据为 `report.json`(`verified=true`)、`fault-observations.json`(两次准确身份与 prepared 快照)、`before.json`、`after.json`、`desktop-observation.json`、`data/batch-resources/*/session.json`(generation/retry/timings/release)及两代 attempt JSON。
|
||||
|
||||
本轮验证新版提前检测与一次恢复预算用尽后停止。旧 EXE 的首次故障后恢复成功并保存替换已在独立文档记录;没有再次对已完成工程运行替换,也没有将旧 EXE 成功结果改称为新版同一用例通过。完整 GUI 故障注入与机器掉电仍未覆盖。
|
||||
@@ -0,0 +1,38 @@
|
||||
# Issue 3:真实 native 写入后冻结监督进程硬退出
|
||||
|
||||
2026-10-23,`tools/probe_frozen_postwrite_crash.py` 在两份全新工程副本上运行,最终实机退出码 0。冻结 EXE SHA256 为 `3b6a8608bac9d88f93ac71487d5c32647af0894c639d624f380f60345d27d34a`,源码基线为 264fee5。本例是**源码非 GUI parent + 冻结 supervisor/helper/VBS**,不是完整 GUI 故障注入。
|
||||
|
||||
真实通过证据目录为 `build/issue-3-frozen-postwrite-crash/run-c26f51c28caf47e1bf3f612801af9f33`。工程 `Audio_D2128`、`Audio_T605A` 从原 checkout 样例分别复制;不触碰正常 UI、marker 写保护或启动恢复验收副本。生产意图均为 RV05 的 `4100-CA0000-T7 → 4100-CA0000-T0`。
|
||||
|
||||
测试 parent 使用 `TaskCenterService(supervised=True)`,仅在测试进程中覆盖 `_command`,使生产请求调用真实冻结 EXE `--batch-worker`。没有修改生产源码、VBS 或添加生产故障开关。
|
||||
|
||||
监视器仅接受第一任务 helper 原子发布的 `native_replace_applied` / `native_replace_verified` 报告,并同时要求非空 replaced UID、对应 attempt `mutation_possible`、execution session generation=1 / retries 为空、下一任务仍 pending、没有可信最终 business-result。随后打开 supervisor journal 中的进程,核对其 PID 与创建时间,在仍存活时仅硬退出该明确自有冻结监督进程。
|
||||
|
||||
本轮触发瞬间为:
|
||||
|
||||
- `stage=native_replace_verified`,`replaced_uids=["$2I2014"]`;没有使用 `native_replace_started` 冒充写后证据。
|
||||
- mutation marker 为 `mutation_possible`,SHA256 `7f7cfa6dd6cc991289e3e883a74d17e507a88424a44aee1ab818528047137502`。
|
||||
- supervisor PID 38916,创建时间 `2026-10-23 08:48:49.752000+00:00`,与 journal 身份一致。
|
||||
- batch ID `07a50287d6b840b3b160b35c035eb3f1`。
|
||||
|
||||
| 检查 | 实机结果 |
|
||||
| --- | --- |
|
||||
| 正写后硬退出 | 已触发;保存完整 native report、marker、队列及进程身份后 `TerminateProcess` 该自有 supervisor |
|
||||
| parent 业务反馈 | `supervisor_interrupted`;没有可信 business-result,不伪报成功 |
|
||||
| 自动重复 | session generation=1、retries 为空、唯一 attempt;没有重新执行业务 |
|
||||
| 下一任务 | 第一任务 running、第二任务 pending;第二工程没有执行 status 文件,执行前后目录完整文件哈希一致 |
|
||||
| 保护发布 | 两参与任务均保留 `reason=supervisor_interrupted` 的 uncertain.flag |
|
||||
| 重启边界 | 关闭原 parent 的 store 后,启动全新 Python parent 进程打开原数据目录 |
|
||||
| 新 parent 操作 | 只启动冻结 `cleanup_batch_resources` 请求;没有 BOM 或队列执行请求 |
|
||||
| 资源复核 | execution session `released`;supervisor journal `finished / outcome=interrupted` |
|
||||
| 业务保护 | 新 parent 启动与资源清理后,uncertain.flag 内容、mutation marker 字节及其 SHA256 均未变化 |
|
||||
| 外部输入 | 原始两工程目录与库 LMC 哈希不变 |
|
||||
| 桌面 | 所有自有 CAD 确认退出、私有桌面确认消失;输入桌面前后一致,自有 CAD 不成为前台 |
|
||||
|
||||
“第二工程未启动”指执行队列未进入第二任务;BOM 导入仍按生产流程读取两工程。通过轮首工程导入时 native CLI 曾遇到路径 `WinError 5`,生产流程转入只读私有 CAD 读取并完成库存核验。这发生在故障监视启用之前,未额外注入或修改生产处理。总耗时 187.343 秒包含这一导入路径。
|
||||
|
||||
首轮 `run-39fa3d839de54a748992f3c06e86f61c` 的测试监视器断言失败,仅发合作取消;第一任务正常保存后停止,第二任务 pending,没有硬退出。该轮保留为失败证据,不计为 postwrite 故障通过,也未重放或复用其已完成工程。旧监视器未记录具体断言栈,不能据此证明确切失败行。工具随后改进可正常变化阶段的等待处理,并保存详细异常栈;上述通过轮来自另一份全新副本。
|
||||
|
||||
主要证据:`postwrite-crash.json`(真实触发快照)、`report.json`(`verified=true`)、`restart-resource-recheck.json`(全新 parent 的 resource-only 验证)、`inputs.json`(第二工程执行前哈希)、`desktop-observation.json`、生产队列与资源 journal、原 uncertain.flag。
|
||||
|
||||
本例没有声称崩溃后的第一工程是完整成功结果,也未自动打开第一工程做修复或再次替换。真实 native 写入已经发生,最终保存/恢复结果缺少可信业务收尾证据,因此保留保护并等待人工核验符合预期。完整 GUI 硬退出、系统/机器掉电,以及任意时点崩溃仍不在本例覆盖范围。
|
||||
@@ -0,0 +1,32 @@
|
||||
# Issue 3:冻结监督进程中的首次 CAD 启动失败恢复
|
||||
|
||||
2026-10-23,运行 `tools/probe_frozen_startup_recovery.py --failures 1`,实机退出码 0。使用独立全新 `Audio_D2128` 副本,与正常 UI 验收及 marker 写保护验收目录隔离。
|
||||
|
||||
本轮冻结 EXE SHA256 为 `4e251627d1ed6f22786de6febec971f889b6b1f2f69c86f19ccdeebd0d41ed95`。该版本首次启动故障等待约 120 秒的行为如实保留;后续检测 CAD 提前退出的源码/EXE 优化不得套用本轮耗时。
|
||||
|
||||
本轮是**源码非 GUI parent + 真实冻结 worker/helper/VBS**。parent 创建 `TaskCenterService(supervised=True)`,测试中仅替换 `batch_supervisor._command`,使 BOM 与队列请求均启动该冻结 EXE 的 `--batch-worker` 模式。不修改生产源码、生产 VBS 或增加注入开关。完整图读取为独立源码只读 inspector,在独立私有 CAD 中运行。该混合入口不等同于完整 GUI 故障注入。
|
||||
|
||||
真实证据目录为 `build/issue-3-frozen-startup-recovery/run-b266b219d8374db3b926d8a24a7dcc2f`。
|
||||
|
||||
故障线程仅监视本轮 `execute_task_queue` 的 session/私有 CAD 记录。终止前交叉核对 batch/session/generation、桌面名、记录中的 CAD PID、进程创建时间、`state=starting`、无 GUID、无 bootstrap 绑定、无 cad-ready checkpoint,以及对应 `prepared` attempt。所有条件满足后,只终止该明确自有 CAD;不附着外部 COM,不终止未知或外部进程。
|
||||
|
||||
| 检查 | 真实结果 |
|
||||
| --- | --- |
|
||||
| 第一次自有 CAD | PID 40836,创建时间 `2026-10-23 08:32:26.507000+00:00`;确认尚未授权工程 worker 后终止 |
|
||||
| 第一次 attempt | `prepared` 保持不变;SHA256 `79c88c6b81d3488952aa154e50ef878cca1d68462cadb91e8ceb0bad5ea08737` 未变化 |
|
||||
| 第一次桌面 | `dx-3e9ef3b9fd75476d9e7b8825bc6172be` 释放确认成功 |
|
||||
| 安全换代顺序 | 第一代 `released_at` 早于第二代桌面 handle-create `time_ns`;观察第二代时桌面枚举中旧桌面已经消失 |
|
||||
| 第二代 CAD | PID 33116,创建时间 `2026-10-23 08:34:27.099000+00:00`,新桌面 `dx-89dbf9636ccd461db04a95ef1df03dfa` |
|
||||
| 重试预算 | execution session generation=2,项目 retries 只有一条且 `count=1` |
|
||||
| 正式业务结果 | 一次 `saved_ok`,任务 completed;没有重复完成或再次替换 |
|
||||
| 独立图对照 | 57 器件、103 引脚;56 个非目标器件完全一致;RV05 原 UID `$2I2014`、位置/方向/缩放与连接保持不变 |
|
||||
| 目标结果 | 料号 `4100-CA0000-T0`,Part Label `0R`,保留 native target 的 Value 属性不存在状态 |
|
||||
| 清理 | session `released`;所有自有 CAD 退出、桌面消失;无 pending resource/supervisor |
|
||||
| 用户桌面 | 输入桌面未变化;自有 CAD 未成为前台或出现在输入桌面 |
|
||||
| 原始输入 | 原始工程目录与中央库 LMC 哈希保持不变 |
|
||||
|
||||
本轮总耗时 349.859 秒,包含独立前后图读取。第一代 `project_worker` 耗时 120.187 秒,记录的重试原因为 `private CAD bootstrap timed out`;第二代 CAD 启动 18.188 秒,第二代 `project_worker` 89.750 秒。未降低生产超时,也未把等待失败降为成功。
|
||||
|
||||
主要证据:`report.json`(`verified=true`)、`fault-observations.json`、`before.json`、`after.json`、`desktop-observation.json`、两代私有 CAD 记录,以及 `data/batch-resources/*/{request.json,supervisor.json,session.json,result.json,business-result.json}`。
|
||||
|
||||
本轮只验证首次 prewrite 启动失败后安全恢复一次。连续两次启动失败、优化后提前退出检测的完整冻结重测,以及完整 GUI 故障注入不在本轮覆盖范围;postwrite 监督进程崩溃使用另外的新副本单独验收。
|
||||
@@ -0,0 +1,44 @@
|
||||
# Issue 3:真实写保护使 mutation marker 发布失败
|
||||
|
||||
2026-10-23,在 Windows 本机独占 CAD 验收窗口,使用全新复制的 `Audio_D2128` 工程运行源码生产 `TaskCenterService → execute_task_queue → QuickPartReplace.vbs → native_helper`。替换意图为 RV05 的 `4100-CA0000-T7 → 4100-CA0000-T0`,不勾选复制料号到 Value。中央库为 `mentor_lib.lmc`。未修改 `src`、生产 VBS 或根代理的 `build/issue-3-ui-acceptance`。
|
||||
|
||||
实际工作目录为 `build/issue-3-marker-failure/run-e361ef90cd1d4145b1c68cdfb067f4a6`。工程从原 checkout 的 `build/cbb-bom-import-live-20261015/projects/Audio_D2128` 单独复制。执行前检查没有其他 CAD 实机测试进程,并与冻结 UI 验收协调串行进行。
|
||||
|
||||
测试进程仅包装 `BatchCadRuntime._prepare_attempts`:生产方法返回并发布 `prepared` 记录后,对该 JSON 调用真实 `win32file.CreateFile`,以 `GENERIC_READ`、`FILE_SHARE_READ | FILE_SHARE_WRITE` 打开,明确不允许 `FILE_SHARE_DELETE`。该句柄不妨碍读取 prepared 记录,但阻止 helper 的 `os.replace` 原子发布。句柄保持到该次生产队列退出,并在 `finally` 关闭;没有添加生产故障开关。
|
||||
|
||||
实际证据:
|
||||
|
||||
| 检查 | 结果 |
|
||||
| --- | --- |
|
||||
| prepared 发布与读取 | 成功;真实测试句柄 1524,可读取原 JSON |
|
||||
| helper 首次发布 mutation marker | `PermissionError: [WinError 5]`,拒绝原子替换 |
|
||||
| 真实调用栈 | `run_native → mark_snapshot_mutation → mark_mutation_possible → _publish → os.replace` |
|
||||
| 原生提交边界 | 失败发生于 `mark_snapshot_mutation`,在原生 Replace 按钮 `click(...)` 之前 |
|
||||
| VBS 状态 | `awaiting_native_replace → batch_failed`,失败工程关闭且未保存 |
|
||||
| 队列结果 | `blocked`、`paused=true`,没有 `saved_ok` |
|
||||
| 重复执行 | `_prepare_attempts` 只调用 1 次;本批 `retries={}`;没有再次执行替换 |
|
||||
| 标记内容 | 保持 `state=prepared`,执行前后 SHA256 完全一致 |
|
||||
| 资源 | 批次 session 为 `released`;CAD 退出与桌面消失均已确认 |
|
||||
| 独立回读 | 私有 CAD 枚举工程全部图页,并读取所有器件属性、位置、方向、缩放及引脚位置、编号、net UID/name;前后结构化数据完全相同 |
|
||||
| 对照范围 | 1 图页 `Audio-D2128:S905L CVBS-LR`,57 器件,103 引脚 |
|
||||
| RV05 | UID `$2I2014`、料号 `4100-CA0000-T7`、Value `0R`,属性及两条引脚连接保持原样 |
|
||||
| 原始输入 | 源工程目录完整文件哈希与中央库 LMC 哈希保持不变 |
|
||||
| 可见性 | 输入桌面前后均为 Default;自有 CAD 未出现在输入桌面或成为前台 |
|
||||
|
||||
prepared 文件前后 SHA256 为 `a2ee4d65d953613c50fa14fb49a22159a7d4cc3b9c3685281ffb672962c41d17`。
|
||||
|
||||
native report 的 `stage=native_replace_started` 在源码中位于 marker 写入之前,不能仅凭该 stage 判断已提交原生修改。本轮以真实异常调用栈、prepared 标记未变化、未保存状态,以及完整原理图前后独立对照共同判定。
|
||||
|
||||
首轮工具错误地要求出现后置 VBS guard 文案 `durable mutation proof failed`。实际上 native helper 更早的 marker guard 已经失败,因此该工具文案断言退出码为 1;原始 `report.json` 保留此失败,未删除或改写。修正测试识别真实 helper `_publish/os.replace` 异常后,只对同一次已执行工程运行 `--verify-existing` 只读回读,退出码为 0;该分支没有导入或队列执行调用,没有重放业务。
|
||||
|
||||
主要证据文件:
|
||||
|
||||
- `report.json`:生产队列结果、唯一 attempt、真实拒绝删除共享句柄、prepared 前后哈希、原输入不变及首轮工具断言失败记录。
|
||||
- `data/tasks/*/quick-run/status.tsv` 与 `snapshot-*.tsv.native.json`:真实生产状态、WinError 5 与调用栈。
|
||||
- `data/batch-resources/*/session.json`:释放确认与空重试记录。
|
||||
- `before.json`、`readback-only/after.json`:完整结构化读取结果;JSON 对象内容比较一致,属性枚举顺序可能不同,不以文本字节哈希代替结构比较。
|
||||
- `readback-only/verification.json`:`verified=true`、`graph_identical=true`、`marker_unchanged=true`、`resources_released=true`、`business_replayed=false`。
|
||||
- 两阶段的 `desktop-observation.json`:真实 CAD 退出、桌面消失及前台/输入桌面观察。
|
||||
- `source-under-test.json`:本轮源代码与 VBS 哈希清单。
|
||||
|
||||
本例覆盖源码服务/队列/真实 VBS/helper/真实私有 CAD 的实际 Windows 写保护失败。通过测试进程包装方法实施故障注入,没有覆盖冻结 EXE 的同类注入;冻结 UI 正常流程由另一轮验收单独覆盖,不混记为本例通过。
|
||||
@@ -0,0 +1,102 @@
|
||||
# 工单 1/3:长路径与安全自恢复实施、验收记录
|
||||
|
||||
后续用户已授权提供 Beta 3 工单复测包,见[发布构建记录](2026-10-23-beta3-release.md)。下文保留实现阶段各候选的验证边界,不将现场未覆盖项改记为完成。
|
||||
|
||||
本轮在 `codex/issue-3-safe-recovery` 实施,不发布版本、不清除旧保护锁。工程与中心库原件只用于复制和哈希核对。
|
||||
|
||||
## 已实施
|
||||
|
||||
- 写入前记录改存 `data/batch-resources/<批次>/attempts/g<代次>/<尝试>/`。目录、原子写入和兼容入口预检在桌面创建前完成。Python 使用扩展路径,CAD/VBS 保留短路径或受控目录联接入口;身份仍用规范路径。
|
||||
- GUI 通过同一 EXE 的隐藏工作模式启动独立监督进程。桌面、CAD、CLI 和工作 Job 属于监督进程;GUI 只持有监督进程自己的 Job。SQLite 连接不跨进程传递。
|
||||
- 请求有版本、批次与尝试 ID、请求摘要、父子 PID/创建时间和一次性 claim。消息检查序号、代次;结果持久化后通知。缺失或不可信的写结果保留实际任务证据目录的保护标记,不从头执行。
|
||||
- GUI 修改与监督进程准入使用同一跨进程互斥。启动恢复持锁到 runner 初始化结束;已占用则只查询。监督进程构造 runner 时不再次执行启动恢复。
|
||||
- 业务结果在收尾前另存;资源失败保留结果。BOM 的两个预览引用均撤销,防止资源未安全释放时保存部分计划。界面分别显示首发错误与释放错误。
|
||||
- 持久化缓存绑定工程完整指纹与载荷摘要;损坏缓存隔离后重新读取。跨监督进程可复用命中的只读采集结果。
|
||||
- 日志分为 GUI、监督进程和逐批释放日志;资源记录包含自有进程创建时间、句柄归属与关闭结果。诊断包有大小边界并脱敏。
|
||||
- 正常关闭归档默认关闭。启用后使用现有回收站事务;未释放、保护或待核验任务保留。崩溃不归档。
|
||||
|
||||
## 安全裁定与审查
|
||||
|
||||
Ruling: 不为桌面残留创建更多桌面 — 释放未证实时保持阻断 — 外部持有者必须释放后才准入。
|
||||
|
||||
Ruling: 心跳丢失只记通信异常 — 不提供“未修改”证明 — 不能授权重放。
|
||||
|
||||
Ruling: 保持现有 Beta 2 版本标识,仅生成未发布验收产物 — 本轮没有发布授权 — 候选哈希和分支用于区分,不能与公开下载混用。
|
||||
|
||||
独立审查发现并修正:冻结 windowed EXE 没有 `sys.stdout`;保留请求可被重复调用;BOM 清理失败仍带预览;旧式任务保护根目录不一致;启动恢复只瞬时检查锁导致多窗口竞争。最后对 `f3d7423` 和 `c787ef3` 的独立复审未发现新增实质安全问题。
|
||||
|
||||
## 验收边界
|
||||
|
||||
本机 `LongPathsEnabled=1`,尚无关闭系统长路径支持且禁用 8.3 的独立 Windows 环境。扩展路径、真实深目录及强制兼容入口测试只能证明其覆盖的分支,不能替代该环境的完整冻结 EXE 验收。
|
||||
|
||||
工单原始多工程文件不齐备。报告回放只验证已有原生报告对应的行,不算原批量工程替换实机验收。故障注入中的替身测试与真实进程/CAD 测试分别记录,不相互替代。
|
||||
|
||||
外部持有者真实测试见 [独立记录](2026-10-23-issue-3-external-holder.md):保持阻断、持有者退出后仅资源清理恢复、业务结果未变化;产品仍不能识别任意外部持有者。不得据此宣称工单 1 根因已根治。
|
||||
|
||||
## 最终验证
|
||||
|
||||
最终源码整合回归:**896 passed,1 条既有 pywinauto STA 警告,205.29 秒**。命令为 `python -m pytest -q tests/v1_1 tests/trial_task_center/test_application.py tests/trial_task_center/test_service.py tests/trial_task_center/test_execution_safety.py tests/part_replace/test_verification.py`;原始输出在 `build/issue-3-final-source-regression.log`。工作树差异检查通过。
|
||||
|
||||
源码构建门禁 885 项通过。实际冻结深目录检查额外发现 `CreateProcess` 不能使用过长工作目录:首次 274 字符数据目录启动失败,保留在 `build/issue-3-frozen-deep`。修正为经过身份核验的兼容工作目录,并保留首发 transport error;22 项深目录/监督/结果相关回归通过。重建候选后,280 字符数据目录的完整 EXE smoke 成功启动监督进程,创建并释放桌面、Job 和自有进程;证据为 `build/issue-3-frozen-deep-report.json`。
|
||||
|
||||
未发布候选 EXE SHA256:`4e251627d1ed6f22786de6febec971f889b6b1f2f69c86f19ccdeebd0d41ed95`。完整交互证据保留在 `build/issue-3-ui-acceptance`,不是源码服务替代 UI。两份全新副本为 Audio_D2128 和 Audio_T605A,其中前者工程路径含中文、长度 287 字符。
|
||||
|
||||
| 完整 EXE 交互检查 | 结果 |
|
||||
| --- | --- |
|
||||
| 创建两任务、导出默认 11 列模板并填入替换项 | 通过 |
|
||||
| 含错误位号 BOM | 通过 2/失败 1/未检查 0;8.172 秒;未部分保存 |
|
||||
| 修正后 BOM | 通过 2/失败 0/未检查 0;4.891 秒 |
|
||||
| 资源与缓存 | 首次导入原生 CLI,两工程一个桌面、CAD 未启动;第二次命中持久化缓存,不创建桌面 |
|
||||
| 计划与搜索 | 四条目标记录,每工程仅 RV05 启用;RV11 未启用;搜索不改变批次范围 |
|
||||
| 顺序替换、保存、关闭重开回读 | 两任务 completed,每工程一条 saved_ok 事件;175.984 秒 |
|
||||
| 执行资源 | 一个桌面、一代会话、一次 CAD 启动,无重建;最终进程退出、句柄关闭和桌面消失均确认 |
|
||||
| 内部写前证据路径 | 两个 attempt JSON,规范路径最长 254 字符;迁移后不再随工程历史目录嵌套增长 |
|
||||
| 主桌面 | 独立观察未发现自有 CAD 成为前台或进入输入桌面,持续文件读取成功 |
|
||||
| 独立工程读回 | 每工程 57 器件、103 引脚;56 个非目标器件全部字段一致,目标的身份、位置和连接保持,RV05 新料号与 Value/Part Label 为预期值 |
|
||||
| 原件 | 两源工程整目录哈希不变,中心库 `.lmc` 哈希不变;未声称完成整个库目录前后哈希对照 |
|
||||
| 正常关闭与重开 | 归档开关默认关闭;两次正常关闭后两 completed 任务保留;重开仍能查看实际结果 |
|
||||
| 界面诊断导出 | 成功导出 30 个文件,包含释放日志,不包含 DB、工程、BOM、EXE 或中心库文件 |
|
||||
|
||||
与已记录的 Beta 2 同组 Audio_D2128/Audio_T605A、每工程 RV05、开启 Part Label→Value 的一次交互观测对比:
|
||||
|
||||
| 候选 | 执行总耗时 | CAD 启动 | 执行桌面创建 |
|
||||
| --- | --- | --- | --- |
|
||||
| 公开 Beta 2 的既有验收 | 209.406 秒 | 1 | 1 |
|
||||
| 本轮隔离监督候选完整 GUI 验收 | 175.984 秒 | 1 | 1 |
|
||||
|
||||
本轮另包含一个 287 字符工程路径。两次观测的缓存、机器负载及目录不同,没有统计重复,因此不能据此承诺稳定加速比例;主要收益是 GUI 与资源所有权分离,以及释放后独立复核。基线出处为 `2026-10-23-issue-2-frozen-runtime.md`。
|
||||
|
||||
独立读回证据为 `graph-before-report.json`、`graph-after-report.json` 和逐工程 `graph-read`;主桌面观察为 `desktop-observation-execute.json`。默认关闭证据为 `normal-close-report.json`。诊断探针曾在异步 ZIP 生成完毕前读取文件而失败,随后只读验证已完成的同一 ZIP 通过,未重新导出。
|
||||
|
||||
工单 BOM 报告回放为 `build/issue-3-bom-report-replay/report.json`:428 个原任务的副本中,仅现有 6 工程原生采集报告覆盖 31 行,20 通过、11 失败,与原行/工程/状态一致;0 资源创建、0 计划写入、原证据哈希不变。另有 406 行/96 工程缺少采集证据,未覆盖。
|
||||
|
||||
真实标记持久化失败检查见 [标记失败实机记录](2026-10-23-issue-3-marker-write-failure.md):生产 helper 因 deny-delete 持有句柄在原子发布时返回 WinError 5,唯一尝试终止,无 saved_ok;只读回读确认 57 元件、103 引脚完全一致。该故障注入以源码 parent/helper 运行,不替代冻结 GUI 的全部故障案例。
|
||||
|
||||
跨路径共享重试预算、不同工程独立预算以及真实子进程硬退出前后结果保护的证据边界见 [监督安全回归](2026-10-23-supervisor-safety-regressions.md)。
|
||||
|
||||
首次真实冻结启动故障恢复见 [冻结启动记录](2026-10-23-issue-3-frozen-startup-recovery.md):一代故障只重建一次,旧桌面消失后才创建二代,最终 saved_ok 恰一次且完整图对照通过。该实验发现 CAD 已退出后仍等待探针约 120 秒;后续 `264fee5` 在等待期间检查明确自有 CAD 的退出码,及时进入同一清理路径,未增加或绕过重试预算。新增用例先失败再通过,相关 65 项回归通过,独立审查确认没有跳过清理或复用死进程。
|
||||
|
||||
最后三行启动检测优化后的候选 SHA256 为 `3b6a8608bac9d88f93ac71487d5c32647af0894c639d624f380f60345d27d34a`,280 字符数据目录的冻结监督、Job/进程身份及桌面释放再次通过,证据在 `build/issue-3-final-frozen-runtime.json`。上表完整 GUI 交互属于前一候选 `4e251627…`,不能混淆两个产物的实测范围;之后的源码更改仅限 CAD 启动等待检测。
|
||||
|
||||
该候选的真实写后硬退出检查已通过,详见 [写后崩溃与仅资源恢复](2026-10-23-issue-3-frozen-postwrite-crash.md):在 `native_replace_verified`、明确 mutation marker 和自有监督 PID/创建时间均核对后终止监督进程;未知业务结果未伪报成功,无重试,第二工程仍 pending 且文件哈希不变。全新 parent 只请求资源清理,保护与 marker 字节保留。该例仍是源码非 GUI parent+真实冻结 supervisor/helper/VBS,不将其描述为 GUI 硬退出或掉电验收。
|
||||
|
||||
同候选 [连续两次启动失败实机记录](2026-10-23-issue-3-frozen-double-startup-failure.md) 通过:两次故障分别在 0.328/0.250 秒被检测,generation=2、重试 count=1,第二次失败后停止,没有第三代或 saved_ok。两个 prepared 记录字节不变,57 元件/103 引脚完整图前后一致。总 233.891 秒包含前后独立 CAD 图读取,不能把故障检测耗时当成全流程耗时。
|
||||
|
||||
## 最终路径补漏与冻结产物
|
||||
|
||||
`3367bb6` 只补扫描和归档的文件访问边界。受限长路径分支的 failing-first 测试证实:原扫描不能访问深工程,原孤立保护检测可能把没有 handoff 的受保护任务归档。修正后深工程可扫描,孤立保护任务 retained,保护文件未变化。30 项相关回归通过;独立审查确认公开路径身份、祖先/reparse/containment 检查未放宽。
|
||||
|
||||
该补丁后的最终未发布 EXE SHA256 为 `f38411a2d91f30461e1441249120b461a7f56ad3f2a0c4a0a95a817037fd9798`。最终冻结验证:
|
||||
|
||||
- 285 字符数据目录 smoke 通过真实监督、进程身份、Job 与私有桌面释放;`build/issue-3-final-path-frozen-runtime.json`。
|
||||
- 完整 GUI 在 296 字符数据目录扫描 302/152 字符工程路径,创建两任务并导出含两工程的 11 列默认模板;公开路径无扩展前缀泄漏,归档默认关闭,正常退出后任务保留,全程没有 CAD/私有桌面资源;`build/issue-3-final-scan-ui/report.json`。
|
||||
- 完整替换与故障注入对应上文注明的前两候选。最后三个模块只改扫描/孤立保护的 I/O,未再次重放已完成替换;不同候选的证据分别保留,不混写为同一个产物的整套验收。
|
||||
|
||||
## 未满足的严格验收条件
|
||||
|
||||
- 未取得 LongPathsEnabled=0 且 8.3 不可用的独立系统,不能宣称该组合环境完整冻结 EXE 验收完成。
|
||||
- 工单原工程的真实批量替换未覆盖,逐工程/原 Excel 行号见 [102 个原工程清单](2026-10-23-issue-3-uncovered-projects.md);其中 96 个连报告回放也未覆盖。
|
||||
- 工单 1 现场桌面残留的具体持有者仍未确定。新隔离设计与自有/人工外部持有者的释放测试通过,但不能宣称现场根因已根治。
|
||||
- 故障实机入口为源码非 GUI parent+冻结 worker/helper/VBS;没有完整 GUI 硬退出或掉电验收。原库只核对 `.lmc` 哈希,未完成整个库目录内容的前后哈希基线。
|
||||
|
||||
因此本轮代码与证据可审查,仍不标为“所有严格验收条件完成”,不自动发布版本或清除旧保护。
|
||||
@@ -0,0 +1,110 @@
|
||||
# 工单 BOM 原工程实机未覆盖清单
|
||||
|
||||
本清单按原 BOM 工程路径身份分组,只展示文件名与不可逆标识,避免发布客户目录。102 个有替换行的原工程均未取得可用于本轮完整实机替换的工程文件;其中 6 个仅有原生报告覆盖的 31 行回放。两个 Audio 全新样例的实机验收不能替代下列原工程验收。
|
||||
|
||||
其余 96 个工程的 406 行连报告回放也未覆盖。Excel 行号对应原 `reported-bom.xlsx`,不是删减后的新编号。
|
||||
|
||||
| 工程文件/路径身份摘要 | 原 BOM Excel 行 | 报告回放行数 | 原工程实机 |
|
||||
| --- | --- | --- | --- |
|
||||
| 1gphy_rtl8211f.prj · 9a3147d0e1 | 3, 4, 5, 6 | 4 | 未覆盖 |
|
||||
| 1gphy_rtl8211f_a.prj · 03b0761631 | 7, 8, 9, 10 | 4 | 未覆盖 |
|
||||
| 1gphy_yt8511c.prj · 2a63ba7f0c | 11, 12, 13, 14, 15 | 5 | 未覆盖 |
|
||||
| 2g5phy_gpy211.prj · 9f2961f1db | 30, 31, 32, 33, 34, 35 | 6 | 未覆盖 |
|
||||
| 2gphy_zx5201.prj · b704fb0870 | 16, 17, 18, 19 | 4 | 未覆盖 |
|
||||
| audio_d2128.prj · f5c5728ea2 | 37, 38, 39, 40, 41, 42, 43 | 0 | 未覆盖 |
|
||||
| audio_d2128_esd.prj · 0786c112bd | 44, 45, 46, 47, 48, 49, 50 | 0 | 未覆盖 |
|
||||
| audio_d2128_surge_esd.prj · 1326530805 | 51, 52, 53, 54, 55, 56, 57, 58 | 0 | 未覆盖 |
|
||||
| audio_t605a.prj · 6249e3b8e4 | 60, 61, 62, 63, 64, 65, 66 | 0 | 未覆盖 |
|
||||
| bob.prj · 1a40673ab4 | 67, 68, 69, 70, 71 | 0 | 未覆盖 |
|
||||
| bob_shield.prj · fc8d09c983 | 72, 73, 74 | 0 | 未覆盖 |
|
||||
| bob_xpon.prj · 943ccbc2f7 | 77, 78, 79 | 0 | 未覆盖 |
|
||||
| bt_rtl8761.prj · 6fdc0b230e | 80, 81, 82, 83, 84, 85, 86, 87 | 0 | 未覆盖 |
|
||||
| ca_card_high.prj · 6116093290 | 91, 92, 93, 94, 95, 96, 97 | 0 | 未覆盖 |
|
||||
| ca_card_low.prj · abd489c265 | 98, 99, 100 | 0 | 未覆盖 |
|
||||
| ca_nds_msd5s69.prj · 312be1ad3f | 101, 102, 103 | 0 | 未覆盖 |
|
||||
| dvbt_ocp.prj · 4c5e90a1be | 312, 313 | 0 | 未覆盖 |
|
||||
| dvbt_ocp_od_gate.prj · 7adda0c253 | 314, 315 | 0 | 未覆盖 |
|
||||
| ethernet_1g.prj · da6029740f | 322 | 0 | 未覆盖 |
|
||||
| ethernet_a.prj · 00d8b04fa6 | 324, 325 | 0 | 未覆盖 |
|
||||
| ethernet_a_emc.prj · ab351a144d | 326, 327 | 0 | 未覆盖 |
|
||||
| ethernet_v.prj · 1e78d4ad40 | 328, 329 | 0 | 未覆盖 |
|
||||
| ethernet_v_camera.prj · def269650f | 330, 331 | 0 | 未覆盖 |
|
||||
| fem_kct8529h.prj · 13586535a9 | 337, 338, 339, 340, 341, 342, 343 | 0 | 未覆盖 |
|
||||
| fem_sky85717.prj · 8c12fd9b42 | 353, 354, 355, 356, 357, 358, 359 | 0 | 未覆盖 |
|
||||
| ldo1117_to252.prj · 053f19cdcd | 367 | 0 | 未覆盖 |
|
||||
| ldo1117_to252_0.prj · 3c4b4ecc60 | 368 | 0 | 未覆盖 |
|
||||
| ldo1117_to252_a.prj · bbc4b07fe2 | 369 | 0 | 未覆盖 |
|
||||
| ldo1117_to252_b.prj · 1a1cdc8bfe | 370 | 0 | 未覆盖 |
|
||||
| ldo1117_to252_c.prj · 9f80a38c68 | 371 | 0 | 未覆盖 |
|
||||
| ldo1117_to252_d.prj · c5e2a54807 | 372 | 0 | 未覆盖 |
|
||||
| ldo_1117.prj · 4229b7c0a0 | 373 | 0 | 未覆盖 |
|
||||
| ldo_1117_1v5.prj · 3e886188ae | 374 | 0 | 未覆盖 |
|
||||
| ldo_1117_1v5.prj · ed5dace4ba | 375 | 0 | 未覆盖 |
|
||||
| ldo_1117_1v5.prj · 32c52f6437 | 376 | 0 | 未覆盖 |
|
||||
| ldo_1117_1v8.prj · 9ca917fe09 | 377 | 0 | 未覆盖 |
|
||||
| ldo_1117_1v8.prj · 009ab46382 | 378 | 0 | 未覆盖 |
|
||||
| ldo_1117_1v8.prj · fe1ea827e2 | 379 | 0 | 未覆盖 |
|
||||
| ldo_1117_3v3.prj · d594fc61a9 | 380 | 0 | 未覆盖 |
|
||||
| ldo_1117_3v3.prj · 56024698a7 | 381 | 0 | 未覆盖 |
|
||||
| ldo_1117_3v3.prj · 9a4d3ba5d0 | 382 | 0 | 未覆盖 |
|
||||
| ldo_1117_a.prj · 1c64ec79aa | 383 | 0 | 未覆盖 |
|
||||
| ldo_1117_b.prj · bcb4b4017a | 384 | 0 | 未覆盖 |
|
||||
| ldo_1117_c.prj · 8bcbbfbafe | 385 | 0 | 未覆盖 |
|
||||
| ldo_1117_d.prj · 546091705e | 386 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23.prj · 65da610a4c | 392, 393, 394 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_1v5.prj · 986a46e1b8 | 395, 396, 397 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_1v5_a.prj · 65333edf53 | 398, 399, 400 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_1v8.prj · 151ac70306 | 401, 402, 403 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_1v8_a.prj · d65baa1d59 | 404, 405, 406 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_1v8_b.prj · f43fc74710 | 407, 408, 409 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_2v5.prj · 3b8ebfee28 | 410, 411, 412 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_2v5_a.prj · 76a6f10c07 | 413, 414, 415 | 0 | 未覆盖 |
|
||||
| ldo_s2032_sot23_a.prj · 9afe3eed88 | 416, 417, 418 | 0 | 未覆盖 |
|
||||
| ldo_wl2811ea_sot23_1v5.prj · 03b146975a | 419, 420, 421 | 0 | 未覆盖 |
|
||||
| ldo_wl2811ea_sot23_1v8.prj · 9152c392ee | 422, 423, 424 | 0 | 未覆盖 |
|
||||
| ldo_wl2811ea_sot23_2v5.prj · b701d5b2e4 | 425, 426, 427 | 0 | 未覆盖 |
|
||||
| ldo_wl2811ea_sot23_adj.prj · c9f627eb7d | 428, 429, 430 | 0 | 未覆盖 |
|
||||
| lnb_mp8124.prj · c10e7bd2c2 | 437, 438, 439, 440, 441 | 0 | 未覆盖 |
|
||||
| lnb_mp8124_nodec.prj · 3a4685a09b | 432, 433, 434, 435, 436 | 0 | 未覆盖 |
|
||||
| lnb_tps65233.prj · 7cebbb12e6 | 442, 443, 444, 445, 446, 447, 448 | 0 | 未覆盖 |
|
||||
| mos-integrated-nmos.prj · bb34f6614a | 450 | 0 | 未覆盖 |
|
||||
| nandflash_bga63.prj · 31807ad105 | 461, 462 | 0 | 未覆盖 |
|
||||
| nandflash_tsop48.prj · 3b2f668c42 | 463 | 0 | 未覆盖 |
|
||||
| nor_flash_sop8.prj · 2aa42f724f | 464 | 0 | 未覆盖 |
|
||||
| ovp_circuit.prj · 0219c48734 | 518 | 0 | 未覆盖 |
|
||||
| pef7071_phy.prj · 008a1a6507 | 20, 21, 22, 23, 24, 25, 26, 27 | 8 | 未覆盖 |
|
||||
| rtc5638he.prj · 778fb484d5 | 344, 345, 346, 347, 348, 349, 350, 351, 352 | 0 | 未覆盖 |
|
||||
| rtl8197x.prj · 86fcfef21c | 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529 | 0 | 未覆盖 |
|
||||
| sd_card_high_cost_esd.prj · cda8210a8c | 530, 531, 532 | 0 | 未覆盖 |
|
||||
| sd_card_low_cost_esd.prj · ef0d2bb5f5 | 533, 534, 535 | 0 | 未覆盖 |
|
||||
| sensor_gc2063_sc3335_sc200ai.prj · 981cd40966 | 536, 537, 538 | 0 | 未覆盖 |
|
||||
| speaker_class-d-audio-amplifier.prj · 206b1c3026 | 539, 540, 541 | 0 | 未覆盖 |
|
||||
| spi_nand_flash.prj · fafa5d4252 | 542 | 0 | 未覆盖 |
|
||||
| t18250_in_loop_filter_shield.prj · e4f2dc6748 | 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553 | 0 | 未覆盖 |
|
||||
| telephony_interface_voice_si32192.prj · 24086dd2ab | 554, 555 | 0 | 未覆盖 |
|
||||
| telephony_interface_voice_si32280.prj · 178a144ca1 | 556, 557, 558, 559, 560, 561, 562, 563, 564 | 0 | 未覆盖 |
|
||||
| tuner+demod_dvb-c_m88rc6800.prj · 0f840071d7 | 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575 | 0 | 未覆盖 |
|
||||
| tuner_atbm253.prj · 110dc6f5c0 | 587, 588, 589, 590, 591 | 0 | 未覆盖 |
|
||||
| tuner_demod_mxl248.prj · 26594f8a75 | 577, 578, 579, 580, 581, 582, 583, 584, 585, 586 | 0 | 未覆盖 |
|
||||
| tuner_demod_mxl248.prj · e299962e81 | 592, 593, 594, 595, 596, 597, 598, 599, 600, 601 | 0 | 未覆盖 |
|
||||
| tuner_mxl608_shield.prj · cf17607708 | 602, 603, 604, 605, 606, 607, 608, 609, 610 | 0 | 未覆盖 |
|
||||
| tuner_r58150.prj · 537ed3b2c1 | 611, 612, 613, 614, 615, 616 | 0 | 未覆盖 |
|
||||
| tuner_r836.prj · cca63e4916 | 617, 618, 619, 620, 621, 622, 623 | 0 | 未覆盖 |
|
||||
| tuner_r836_a.prj · e41cc11d66 | 624, 625, 626, 627, 628, 629, 630 | 0 | 未覆盖 |
|
||||
| tuner_r836_h_cbb.prj · 882836fa9e | 631, 632, 633, 634, 635, 636, 637, 638, 639, 640 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_dip+snaky_lines.prj · 78145e0625 | 641, 642, 643, 644, 645, 646, 647, 648 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_double_tuner.prj · 662d874266 | 649, 650, 651, 652, 653, 654, 655, 656 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+gdt+filter.prj · 6f23c09e9e | 657, 658, 659, 660, 661, 662, 663, 664 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+gdt.prj · 9583c3ea34 | 679, 680, 681, 682, 683, 684, 685 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+snaky_lines.prj · e08bababf6 | 504, 505, 506, 507, 508, 509, 510 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+snaky_lines.prj · f9a967caf1 | 665, 666, 667, 668, 669, 670, 671 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+snaky_lines_a.prj · 6b2d71dc48 | 490, 491, 492, 493, 494, 495, 496 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+snaky_lines_a.prj · 5e6177a7de | 497, 498, 499, 500, 501, 502, 503 | 0 | 未覆盖 |
|
||||
| tuner_rda5815_smd+snaky_lines_a.prj · fd2fe146f9 | 672, 673, 674, 675, 676, 677, 678 | 0 | 未覆盖 |
|
||||
| tuner_t18250.prj · 72c882b7d6 | 686, 687, 688, 689, 690, 691, 692, 693, 694 | 0 | 未覆盖 |
|
||||
| wifi-8192fr-2l.prj · be5eb866d0 | 722, 723 | 0 | 未覆盖 |
|
||||
| wifi-rtl8188ftv-1t1r-2l.prj · e33a176a9b | 724, 725, 726, 727 | 0 | 未覆盖 |
|
||||
| wifi_8192er.prj · d5c71c558b | 729, 730, 731, 732 | 0 | 未覆盖 |
|
||||
| wifi_rtl8192fc.prj · d5b6659208 | 733, 734, 735, 736, 737 | 0 | 未覆盖 |
|
||||
| wifi_rtl8723du.prj · 88e300dc79 | 738, 739, 740, 741 | 0 | 未覆盖 |
|
||||
| xview.prj · 460b5b6322 | 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753 | 0 | 未覆盖 |
|
||||
@@ -0,0 +1,13 @@
|
||||
# Supervisor safety regression evidence boundaries
|
||||
|
||||
These automated tests do not open CAD or certify schematic property writes.
|
||||
|
||||
| Test | Real behavior exercised | Replaced behavior |
|
||||
| --- | --- | --- |
|
||||
| `test_mock_native_read_recovery_spends_same_project_worker_retry_budget` | Durable source hashes and attempt proof, runtime rebuild decision, shared native/worker project budget | Desktop/CAD ownership and worker failure |
|
||||
| `test_mock_different_projects_each_get_one_retry_without_accumulating_budget` | Two project identities, one rebuild per project, successful execution cannot replenish a spent budget | Desktop/CAD ownership and monitored worker execution |
|
||||
| `test_real_supervisor_process_crash_reconciles_without_replaying_mock_business` | Separate Python child, production admission and one-shot claim, process creation identity, hard `os._exit`, durable saved-result fallback, uncertainty protection at legacy/project-task evidence roots, separate cleanup child | Business replacement writes a durable witness instead of using CAD; no private desktop or CAD resources are acquired |
|
||||
|
||||
The process test covers both an unknown outcome before publication and a completed business result published before a crash. Cleanup must neither execute the witness operation again nor alter an earlier completed result. Unknown outcomes retain their original protection marker after reconciliation; proven completed outcomes do not acquire an uncertainty marker.
|
||||
|
||||
Remaining physical acceptance: inject failures in the frozen executable before and after the actual first CAD mutation, prove marker/helper durability failure prevents every production write/save path, and confirm actual CAD/desktop release and independent schematic readback. The tests above do not provide that evidence.
|
||||
|
After Width: | Height: | Size: 150 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 261 KiB |
|
After Width: | Height: | Size: 97 KiB |
|
After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 142 KiB |
|
After Width: | Height: | Size: 296 KiB |
|
After Width: | Height: | Size: 163 KiB |
|
After Width: | Height: | Size: 73 KiB |
|
After Width: | Height: | Size: 44 KiB |
|
After Width: | Height: | Size: 109 KiB |
@@ -1,8 +1,8 @@
|
||||
# DxPartReplace V1.1.2 中文使用说明
|
||||
# DxPartReplace V1.1.3 中文使用说明
|
||||
|
||||
V1.1.0 沿用原有任务视图和料号规则编辑,增加按工程拆分的任务、按位号选择的替换计划及可选 Excel 操作。
|
||||
保留原有任务视图和料号规则编辑,支持按工程拆分任务、按位号选择替换目标及可选 Excel 操作。
|
||||
|
||||
[窗口 GIF 操作教程](media/v1.1.2-window-bom-tutorial.gif):创建任务、导出/导入 BOM、错误行提示、保存、顺序执行、查看结果和开始新一轮。仅含工具窗口及其弹窗,实际 CAD 等待已压缩。
|
||||
第一次使用请先看 [README 逐步图文教程](../../README.md#图文教程),按截图完成创建任务、BOM、替换和结果查看。以下内容用于补充查询各项规则。
|
||||
|
||||
## 日常流程
|
||||
|
||||
@@ -31,9 +31,9 @@ V1.1.0 沿用原有任务视图和料号规则编辑,增加按工程拆分的
|
||||
|
||||
导入先核对所有工程归属,再按工程匹配位号;缺少清单时,按该工程 BOM 中的旧料号只读采集。不会执行替换。切换任务可查看、编辑各自预览,点击“保存本批位号计划”才一次写入所有受影响任务。保存失败全部回滚,预览期间不改已保存计划。工程、图页、UID、位号、旧料号及原 Value 用于身份校验,未知工程、重复器件或跨工程目标都会拒绝;不接受公式。布尔选项可使用 TRUE/FALSE 或 是/否。
|
||||
|
||||
开发版新增原生采集(尚未打包发布):本机有兼容的 `icdb2ascii.exe` 时,BOM 校验和位号采集优先在临时数据库副本中一次导出真实 UID、位号及物料属性,不必逐条启动 CAD 查找。工具不可用或报告不受支持且已安全清理时,自动使用原 CAD 采集。源工程变化或后台清理不确定时停止并保留诊断;完整图页查询和实际替换仍沿用原执行链。运行前请保存并关闭目标工程。详情见 [原生采集接入记录](../research/2026-10-16-native-cli-integration.md)。
|
||||
V1.1.3 已包含原生采集:本机有兼容的 `icdb2ascii.exe` 时,BOM 校验和位号采集优先在临时数据库副本中一次导出真实 UID、位号及物料属性,不必逐条启动 CAD 查找。工具不可用或报告不受支持且已安全清理时,自动使用原 CAD 采集。源工程变化或后台清理不确定时停止并保留诊断;完整图页查询和实际替换仍沿用原执行链。运行前请保存并关闭目标工程。详情见 [原生采集接入记录](../research/2026-10-16-native-cli-integration.md)。
|
||||
|
||||
BOM 首次采集或新增的位号默认“不替换”;已有计划中未列出的行保留原勾选和编辑状态,不会因后续 BOM 省略而自动停用。如果本次仅执行 BOM 列出的目标,请在预览中检查并取消其他启用行,或在 BOM 中显式填写对应的 `enabled=FALSE` 行。每个填写的位号须在当前工程清单中唯一匹配;同名位号跨图页时应填写图页或 UID。导入失败不会改变已保存计划。
|
||||
首次采集中未被 BOM 列为替换目标的位号默认“不替换”;已有计划中未列出的行保留原勾选和编辑状态,不会因后续 BOM 省略而自动停用。如果本次仅执行 BOM 列出的目标,请在预览中检查并取消其他启用行,或在 BOM 中显式填写对应的 `enabled=FALSE` 行。每个填写的位号须在当前工程清单中唯一匹配;同名位号跨图页时应填写图页或 UID。导入失败不会改变已保存计划。
|
||||
|
||||
“同步 NC”默认勾选,可以按行关闭;仅按原规则识别独立 `/NC` 标记。Value 复制选项和每行自定义 Value 延续原工具的处理方式。替换计划不允许旧、新料号相同,也不允许同批链式或循环替换。
|
||||
|
||||
|
||||
@@ -930,6 +930,7 @@ Sub QPFinishBatchItem()
|
||||
Next
|
||||
End If
|
||||
If FinalState = "restored_ok" Then
|
||||
QPMarkBatchMutation Command
|
||||
QPCloseBatchProject TargetPath, True
|
||||
QP_BatchSaveCommitted = True
|
||||
ElseIf Not QP_SessionMode Then
|
||||
@@ -975,6 +976,42 @@ Sub QPFinishBatchItem()
|
||||
QPProgress "finished", 1, 1
|
||||
End Sub
|
||||
|
||||
Sub QPMarkBatchMutation(ByRef Command)
|
||||
Dim Shell, Entry, Required, CommandLine, Key, Value, ExitCode
|
||||
Set Shell = CreateObject("WScript.Shell")
|
||||
Required = False
|
||||
For Each Entry In Shell.Environment("Process")
|
||||
If InStr(1, CStr(Entry), "DXPART_ATTEMPT_FILE=", vbTextCompare) = 1 Or _
|
||||
InStr(1, CStr(Entry), "DXPART_BATCH_SESSION_ID=", vbTextCompare) = 1 Or _
|
||||
InStr(1, CStr(Entry), "DXPART_BATCH_GENERATION=", vbTextCompare) = 1 Or _
|
||||
InStr(1, CStr(Entry), "DXPART_ATTEMPT_ID=", vbTextCompare) = 1 Then Required = True
|
||||
Next
|
||||
If Not Required Then Exit Sub
|
||||
If Len(QP_HelperExecutable) = 0 Or Not QP_Fso.FileExists(QP_HelperExecutable) Then _
|
||||
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "mutation proof helper is unavailable"
|
||||
CommandLine = Chr(34) & QP_HelperExecutable & Chr(34)
|
||||
If Len(QP_HelperScript) > 0 Then
|
||||
If Not QP_Fso.FileExists(QP_HelperScript) Then _
|
||||
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "mutation proof script is unavailable"
|
||||
CommandLine = CommandLine & " " & Chr(34) & QP_HelperScript & Chr(34)
|
||||
End If
|
||||
CommandLine = CommandLine & " --mark-batch-attempt"
|
||||
For Each Key In Array("job_id", "task_id", "project_path")
|
||||
Value = CStr(Command(Key))
|
||||
If Len(Value) = 0 Or InStr(Value, Chr(34)) > 0 Or InStr(Value, vbCr) > 0 Or InStr(Value, vbLf) > 0 Then _
|
||||
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "invalid mutation proof identity"
|
||||
Select Case Key
|
||||
Case "job_id": CommandLine = CommandLine & " --attempt-job "
|
||||
Case "task_id": CommandLine = CommandLine & " --attempt-task "
|
||||
Case "project_path": CommandLine = CommandLine & " --attempt-project "
|
||||
End Select
|
||||
CommandLine = CommandLine & Chr(34) & Value & Chr(34)
|
||||
Next
|
||||
ExitCode = Shell.Run(CommandLine, 0, True)
|
||||
If ExitCode <> 0 Then Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", _
|
||||
"durable mutation proof failed; no CAD write allowed"
|
||||
End Sub
|
||||
|
||||
Sub QPAutomaticReplaceAndRestore(ByRef Command, ByVal SnapshotPath)
|
||||
Dim ErrorNumber, ErrorSource, ErrorDescription
|
||||
QPRequireCommandSnapshot Command, SnapshotPath
|
||||
@@ -1590,6 +1627,7 @@ Sub QPRestoreAfterOperatorReplace(ByRef Command, ByVal SnapshotPath)
|
||||
QPProgress "restore_preflight", Index + 1, Records.Item("components").Count
|
||||
Next
|
||||
QPProgress "before_restore_transaction", 0, Plans.Count
|
||||
QPMarkBatchMutation Command
|
||||
QP_RestoreTransactionOpen = True
|
||||
QP_App.ActiveView.Block.StartUndoTransaction
|
||||
On Error Resume Next
|
||||
|
||||
@@ -80,6 +80,9 @@ try {
|
||||
if (-not (Test-Path -LiteralPath $executable -PathType Leaf)) {
|
||||
throw "PyInstaller did not produce $executable"
|
||||
}
|
||||
$runtimeSmokeRoot = Join-Path $repositoryRoot ("build\frozen-runtime-" + [Guid]::NewGuid().ToString("N"))
|
||||
& $PythonExecutable tools\verify_packaged_runtime.py $executable $runtimeSmokeRoot
|
||||
if ($LASTEXITCODE -ne 0) { throw "Frozen supervisor verification failed; build stopped" }
|
||||
$sourceQuickPartReplace = Join-Path $repositoryRoot "integration\dxdesigner795\QuickPartReplace.vbs"
|
||||
$packagedQuickPartReplace = Join-Path $repositoryRoot "dist\DxPartReplace\_internal\legacy-runtime\dxdesigner795\QuickPartReplace.vbs"
|
||||
if (-not (Test-Path -LiteralPath $sourceQuickPartReplace -PathType Leaf)) {
|
||||
|
||||
@@ -43,6 +43,8 @@ a = Analysis(
|
||||
"win32con",
|
||||
"win32gui",
|
||||
"win32process",
|
||||
# GetProcessTimes imports this from pywin32's C extension at runtime.
|
||||
"win32timezone",
|
||||
"pywinauto",
|
||||
"pywinauto.controls.uiawrapper",
|
||||
],
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# UTF-8
|
||||
VSVersionInfo(
|
||||
ffi=FixedFileInfo(
|
||||
filevers=(1, 1, 3, 0),
|
||||
prodvers=(1, 1, 3, 0),
|
||||
filevers=(1, 1, 4, 3),
|
||||
prodvers=(1, 1, 4, 3),
|
||||
mask=0x3f,
|
||||
flags=0x0,
|
||||
flags=0x2,
|
||||
OS=0x40004,
|
||||
fileType=0x1,
|
||||
subtype=0x0,
|
||||
@@ -15,11 +15,11 @@ VSVersionInfo(
|
||||
StringTable('040904B0', [
|
||||
StringStruct('CompanyName', 'DxProductivity'),
|
||||
StringStruct('FileDescription', 'DxDesigner Part Replacement'),
|
||||
StringStruct('FileVersion', '1.1.3.0'),
|
||||
StringStruct('FileVersion', '1.1.4.3'),
|
||||
StringStruct('InternalName', 'DxPartReplace'),
|
||||
StringStruct('OriginalFilename', 'DxPartReplace.exe'),
|
||||
StringStruct('ProductName', 'DxDesigner Part Replacement'),
|
||||
StringStruct('ProductVersion', '1.1.3')
|
||||
StringStruct('ProductVersion', '1.1.4-beta.3')
|
||||
])
|
||||
]),
|
||||
VarFileInfo([VarStruct('Translation', [1033, 1200])])
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "dx-productivity"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4b3"
|
||||
requires-python = ">=3.12,<3.13"
|
||||
dependencies = ["PySide6>=6.7,<7", "pywin32>=306", "pywinauto>=0.6.9,<1", "openpyxl>=3.1,<4"]
|
||||
|
||||
|
||||
@@ -48,12 +48,17 @@ def configure_application_identity(application: QApplication) -> None:
|
||||
def _parse_arguments(argv: Sequence[str] | None) -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(prog="DxPartReplace")
|
||||
parser.add_argument("--smoke-test", action="store_true", help=argparse.SUPPRESS)
|
||||
parser.add_argument('--batch-worker', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument("--data-root", type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument("--native-helper", action="store_true", help=argparse.SUPPRESS)
|
||||
parser.add_argument("--snapshot", type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--mark-batch-attempt', action='store_true', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--attempt-job', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--attempt-task', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--attempt-project', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--cell-catalog-worker', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument("--sceneos-cli", action="store_true", help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-action', choices=('probe', 'quit'), help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-action', choices=('probe', 'quit', 'checkpoint'), help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-evidence', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-pid', type=int, help=argparse.SUPPRESS)
|
||||
return parser.parse_args(list(argv) if argv is not None else None)
|
||||
@@ -113,9 +118,30 @@ def _flush(context: LoggingContext) -> None:
|
||||
handler.flush()
|
||||
|
||||
|
||||
def _smoke_batch_runtime(data_root: Path, context: LoggingContext) -> None:
|
||||
"""Exercise process tracking inside the frozen supervisor before releasing."""
|
||||
from dx_productivity.trial_task_center.batch_supervisor import run_supervised
|
||||
with_store = TaskStore(data_root / 'task-center.db')
|
||||
try:
|
||||
service = TaskCenterService(with_store, context, recover_on_startup=False)
|
||||
result = run_supervised(service, 'runtime_smoke', {})
|
||||
if not result['process_identity_read']:
|
||||
raise RuntimeError('Smoke worker identity was not checked')
|
||||
finally:
|
||||
with_store.close()
|
||||
context.logger.info('packaged_runtime_smoke_passed supervisor_pid=%s', result['pid'])
|
||||
|
||||
|
||||
def main(argv: Sequence[str] | None = None) -> int:
|
||||
"""Run the Qt shell, returning a conventional process status code."""
|
||||
arguments = _parse_arguments(argv)
|
||||
if arguments.batch_worker is not None:
|
||||
from dx_productivity.trial_task_center.batch_worker import run
|
||||
return run(arguments.batch_worker)
|
||||
if arguments.mark_batch_attempt:
|
||||
from dx_productivity.trial_task_center.native_helper import run_attempt_marker
|
||||
return run_attempt_marker(job_id=arguments.attempt_job, task_id=arguments.attempt_task,
|
||||
project_path=arguments.attempt_project)
|
||||
if arguments.cell_catalog_worker is not None:
|
||||
from dx_productivity.trial_task_center.cell_catalog_worker import run
|
||||
return run(arguments.cell_catalog_worker)
|
||||
@@ -141,8 +167,11 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
try:
|
||||
database_path = data_root / "task-center.db"
|
||||
_log_startup(logging_context, data_root, database_path)
|
||||
if arguments.smoke_test and bool(getattr(sys, 'frozen', False)):
|
||||
_smoke_batch_runtime(data_root, logging_context)
|
||||
store = TaskStore(database_path)
|
||||
service = TaskCenterService(store, logging_context)
|
||||
service._supervised = True
|
||||
qt_application = QApplication([sys.executable, *(argv or ())])
|
||||
configure_application_identity(qt_application)
|
||||
window = _load_window()(service)
|
||||
|
||||
@@ -41,6 +41,7 @@ def execute_batch(
|
||||
stop_event: Event,
|
||||
continue_after_backup_failure: bool = False,
|
||||
reuse_project_session: bool = True,
|
||||
runtime=None,
|
||||
) -> BatchRunResult:
|
||||
preparer = service._quick_run_preparer
|
||||
retention = maintain_quick_run_storage(preparer.data_root)
|
||||
@@ -71,6 +72,7 @@ def execute_batch(
|
||||
projects,
|
||||
rules,
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}),
|
||||
)
|
||||
metric['completed_units'] = len(session.commands)
|
||||
initial_progress = progress
|
||||
@@ -108,7 +110,8 @@ def execute_batch(
|
||||
progress.total_items,
|
||||
)
|
||||
progress = session.progress
|
||||
with backups, preparer.project_session_scope(enabled=reuse_project_session) as project_session:
|
||||
with backups, preparer.project_session_scope(enabled=reuse_project_session,
|
||||
**({'runtime': runtime} if runtime is not None else {})) as project_session:
|
||||
while not progress.queue_complete:
|
||||
if stop_event.is_set():
|
||||
backups.commit_prefetched(progress)
|
||||
@@ -137,7 +140,8 @@ def execute_batch(
|
||||
try:
|
||||
last_result = preparer.execute_step(task_id, projects, rules, copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
expected_item=expected, batch_mode=True, batch_session=session,
|
||||
**({'project_session': project_session} if project_session.enabled else {}))
|
||||
**({'project_session': project_session} if project_session.enabled else {}),
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
metric['state'] = last_result.state
|
||||
metric['completed_units'] = int(last_result.state in {'saved_ok', 'restored_ok', 'no_match'})
|
||||
finally:
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Durable write-ahead proof for one logical job in a private CAD attempt.
|
||||
|
||||
A prepared record is evidence of no write only while its complete identity still
|
||||
matches. Helpers never create records and must fail closed if publication fails.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import tempfile
|
||||
from uuid import uuid4
|
||||
|
||||
from .path_access import canonical_path, internal_path, retry_sharing_violation
|
||||
|
||||
|
||||
_IDENTITY = ('session_id', 'generation', 'attempt_id', 'task_id', 'job_id', 'project_path')
|
||||
|
||||
|
||||
def _project(value):
|
||||
return os.path.normcase(str(canonical_path(value)))
|
||||
|
||||
|
||||
def _validate(record):
|
||||
if not isinstance(record, dict) or record.get('schema_version') != 1:
|
||||
raise ValueError('invalid batch attempt schema')
|
||||
for key in ('session_id', 'task_id', 'job_id', 'project_path'):
|
||||
value = record.get(key)
|
||||
if not isinstance(value, str) or not value.strip() or any(ord(c) < 32 for c in value):
|
||||
raise ValueError(f'invalid batch attempt {key}')
|
||||
generation = record.get('generation')
|
||||
if type(generation) is not int or generation < 1:
|
||||
raise ValueError('invalid batch attempt generation')
|
||||
if not isinstance(record.get('attempt_id'), str) or not re.fullmatch(r'[0-9a-f]{32}', record['attempt_id']):
|
||||
raise ValueError('invalid batch attempt ID')
|
||||
if record.get('state') not in {'prepared', 'mutation_possible'}:
|
||||
raise ValueError('invalid batch attempt state')
|
||||
return record
|
||||
|
||||
|
||||
def _publish(path, record):
|
||||
path = internal_path(path)
|
||||
descriptor, temporary = tempfile.mkstemp(prefix=f'.{path.name}.', suffix='.tmp', dir=path.parent)
|
||||
try:
|
||||
with os.fdopen(descriptor, 'w', encoding='utf-8', newline='') as handle:
|
||||
json.dump(record, handle, ensure_ascii=False, indent=2)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
retry_sharing_violation(lambda: os.replace(temporary, path))
|
||||
# POSIX needs a directory sync for the atomic rename; Windows does not
|
||||
# expose fsync on directories through the CRT.
|
||||
if os.name != 'nt':
|
||||
directory = os.open(path.parent, os.O_RDONLY)
|
||||
try:
|
||||
os.fsync(directory)
|
||||
finally:
|
||||
os.close(directory)
|
||||
finally:
|
||||
Path(temporary).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def prepare_attempt(path, *, session_id, generation, task_id, job_id, project_path, attempt_id=None):
|
||||
record = _validate(dict(schema_version=1, session_id=str(session_id), generation=generation,
|
||||
task_id=str(task_id), job_id=str(job_id), project_path=_project(project_path),
|
||||
attempt_id=uuid4().hex if attempt_id is None else attempt_id, state='prepared'))
|
||||
retry_sharing_violation(lambda: internal_path(path).parent.mkdir(parents=True, exist_ok=True))
|
||||
_publish(path, record)
|
||||
return record
|
||||
|
||||
|
||||
def read_attempt(path):
|
||||
return _validate(json.loads(internal_path(path).read_text(encoding='utf-8')))
|
||||
|
||||
|
||||
def _bound_attempt(path=None, *, job_id=None, task_id=None, project_path=None):
|
||||
if path is None and 'DXPART_ATTEMPT_FILE' not in os.environ:
|
||||
if any(key in os.environ for key in ('DXPART_BATCH_SESSION_ID', 'DXPART_BATCH_GENERATION', 'DXPART_ATTEMPT_ID')):
|
||||
raise ValueError('missing batch attempt file')
|
||||
return None
|
||||
raw_path = os.environ.get('DXPART_ATTEMPT_FILE') if path is None else os.fspath(path)
|
||||
if not raw_path or not Path(raw_path).is_absolute():
|
||||
raise ValueError('invalid batch attempt file')
|
||||
path = internal_path(raw_path)
|
||||
job_id = str(job_id) if job_id is not None else os.environ.get('DXPART_JOB_ID')
|
||||
task_id = str(task_id) if task_id is not None else os.environ.get('DXPART_TASK_ID')
|
||||
if path.is_dir():
|
||||
if not job_id or not re.fullmatch(r'[A-Za-z0-9_-]+', job_id):
|
||||
raise ValueError('missing or unsafe batch attempt job ID')
|
||||
path = path / f'{job_id}.json'
|
||||
record = read_attempt(path)
|
||||
expected = {'session_id': os.environ.get('DXPART_BATCH_SESSION_ID'),
|
||||
'attempt_id': os.environ.get('DXPART_ATTEMPT_ID'),
|
||||
'generation': os.environ.get('DXPART_BATCH_GENERATION')}
|
||||
if not all(expected.values()) or str(record['generation']) != expected.pop('generation'):
|
||||
raise ValueError('missing or stale batch generation')
|
||||
if any(record[key] != value for key, value in expected.items()):
|
||||
raise ValueError('stale batch attempt identity')
|
||||
if not job_id or not task_id or record['job_id'] != job_id or record['task_id'] != task_id:
|
||||
raise ValueError('batch attempt task/job mismatch')
|
||||
# Optional environment bindings remain authoritative even when the caller
|
||||
# supplies explicit command fields.
|
||||
for key, value in (('DXPART_JOB_ID', job_id), ('DXPART_TASK_ID', task_id)):
|
||||
if key in os.environ and os.environ[key] != value:
|
||||
raise ValueError('batch process task/job mismatch')
|
||||
if project_path is not None and record['project_path'] != _project(project_path):
|
||||
raise ValueError('batch attempt project mismatch')
|
||||
return path, record
|
||||
|
||||
|
||||
def mark_mutation_possible(path=None, *, job_id=None, task_id=None, project_path=None):
|
||||
bound = _bound_attempt(path, job_id=job_id, task_id=task_id, project_path=project_path)
|
||||
if bound is None:
|
||||
return None
|
||||
path, record = bound
|
||||
if record['state'] != 'mutation_possible':
|
||||
record = dict(record, state='mutation_possible')
|
||||
_publish(path, record)
|
||||
return record
|
||||
|
||||
|
||||
def can_retry_attempt(path, expected):
|
||||
try:
|
||||
_validate(expected)
|
||||
actual = read_attempt(path)
|
||||
return actual['state'] == 'prepared' and all(actual[key] == expected[key] for key in _IDENTITY)
|
||||
except (OSError, ValueError, TypeError, KeyError):
|
||||
return False
|
||||
@@ -0,0 +1,88 @@
|
||||
"""Readable release history, separate from authoritative resource proofs."""
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from .logging_setup import LoggingContext, append_task_event, configure_logging
|
||||
from .path_access import internal_path
|
||||
|
||||
|
||||
def _append_line(path, line):
|
||||
path = internal_path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open('a', encoding='utf-8', newline='\n') as stream:
|
||||
stream.write(line + '\n')
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
|
||||
|
||||
def write_release_event(root, path, record, phase, *, context=None, reason='batch_end',
|
||||
elapsed=0.0, error=''):
|
||||
"""Try each diagnostic destination; failures never interrupt native cleanup."""
|
||||
cleanup = record.get('cleanup') if isinstance(record.get('cleanup'), dict) else {}
|
||||
confirmation = cleanup.get('desktop_confirmation')
|
||||
confirmation = confirmation if isinstance(confirmation, dict) else {}
|
||||
cad = record.get('cad_shutdown') if isinstance(record.get('cad_shutdown'), dict) else {}
|
||||
disappeared = confirmation.get('released')
|
||||
if phase == 'succeeded' and record.get('reconciled'):
|
||||
disappeared = True
|
||||
details = {'batch_id': record.get('session_id'), 'batch_kind': record.get('kind'),
|
||||
'generation': record.get('generation'), 'desktop': record.get('desktop'),
|
||||
'cad_pid': record.get('cad_pid'), 'cad_guid': record.get('cad_guid'),
|
||||
'reason': reason, 'result': 'releasing' if phase == 'started' else (
|
||||
'released' if phase == 'succeeded' else 'blocked'),
|
||||
'elapsed_seconds': elapsed, 'record_path': str(path),
|
||||
'desktop_handle_closed': cleanup.get('desktop_handle_closed'),
|
||||
'desktop_disappeared': disappeared, 'cad_exit_code': cad.get('cad_exit_code'),
|
||||
'quit_helper_exit_code': cad.get('quit_helper_exit_code'),
|
||||
'cad_shutdown': cad, 'shutdown_warning': record.get('shutdown_warning'),
|
||||
'retries': record.get('retries', {}), 'cleanup': cleanup, 'error': str(error)}
|
||||
title = {'started': '开始释放后台资源', 'succeeded': '后台资源释放成功',
|
||||
'failed': '后台资源释放失败,资源释放待处理'}[phase]
|
||||
purpose = {'batch_end': '批次结束', 'rebuild': '安全重建', 'manual_retry': '重新清理',
|
||||
'restart_recheck': '重启后重新检查'}.get(reason, reason)
|
||||
cad_text = f"CAD PID={details['cad_pid']}" if details['cad_pid'] else (
|
||||
'CAD 未启动' if record.get('cad_start_requested') is False else
|
||||
'CAD 启动未完成绑定' if record.get('cad_start_requested') else 'CAD 启动状态未知')
|
||||
message = (f"{title} · {purpose} · 批次 {details['batch_id']} · 代次 {details['generation']}"
|
||||
f" · 桌面 {details['desktop']} · {cad_text} · 耗时 {elapsed:.3f} 秒")
|
||||
if phase != 'started':
|
||||
label = lambda value: '已确认' if value is True else '未确认'
|
||||
message += (f" · 桌面句柄关闭:{label(details['desktop_handle_closed'])}"
|
||||
f" · 桌面消失:{label(disappeared)}")
|
||||
if cad:
|
||||
message += f" · CAD 退出码={details['cad_exit_code']},退出助手码={details['quit_helper_exit_code']}"
|
||||
if error:
|
||||
message += ' · 原因:' + str(error)
|
||||
message = message.replace('\r', '\\r').replace('\n', '\\n')
|
||||
level = 'ERROR' if phase == 'failed' else 'INFO'
|
||||
timestamp = datetime.now(timezone.utc).isoformat(timespec='milliseconds')
|
||||
errors = []
|
||||
try:
|
||||
_append_line(Path(path).with_name('resource-release.log'),
|
||||
f'{timestamp} {level} {message} | ' + json.dumps(details, ensure_ascii=False))
|
||||
except Exception as failure:
|
||||
errors.append('resource-release.log: ' + str(failure))
|
||||
try:
|
||||
context = context if isinstance(context, LoggingContext) else configure_logging(root)
|
||||
except Exception as failure:
|
||||
errors.append('application log: ' + str(failure))
|
||||
return errors
|
||||
for task_id in record.get('log_task_ids', ()):
|
||||
try:
|
||||
append_task_event(context, level=level, task_id=task_id,
|
||||
operation='resource_release_' + phase, message=message, details=details)
|
||||
except Exception as failure:
|
||||
errors.append('task ' + str(task_id) + ': ' + str(failure))
|
||||
try:
|
||||
context.logger.log(40 if phase == 'failed' else 20, '%s | %s', message,
|
||||
json.dumps(details, ensure_ascii=False))
|
||||
if errors:
|
||||
context.logger.error('resource_release_log_errors batch=%s errors=%s',
|
||||
record.get('session_id'), errors)
|
||||
for handler in context.logger.handlers:
|
||||
handler.flush()
|
||||
except Exception as failure:
|
||||
errors.append('application log: ' + str(failure))
|
||||
return errors
|
||||
@@ -0,0 +1,190 @@
|
||||
"""Persistent resource ownership, independent of project replacement results."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, retry_sharing_violation
|
||||
|
||||
|
||||
# Recovery retains owned native handles after failed cleanup. This registry is
|
||||
# never an implicit execution context and never contains COM application objects.
|
||||
LIVE_RESOURCES = {}
|
||||
|
||||
|
||||
class BatchResourceCleanupError(TaskCenterError):
|
||||
def __init__(self, message, *, completed_result=None, primary_error=None, cleanup_error=None):
|
||||
super().__init__('batch_resources_pending', message)
|
||||
self.completed_result = completed_result
|
||||
self.primary_error = primary_error
|
||||
self.cleanup_error = cleanup_error
|
||||
self.bom_report = getattr(primary_error, "bom_report", None)
|
||||
|
||||
|
||||
def write_record(path, record):
|
||||
path = internal_path(path)
|
||||
retry_sharing_violation(lambda: path.parent.mkdir(parents=True, exist_ok=True))
|
||||
temporary = path.with_name(path.name + '.' + uuid4().hex + '.tmp')
|
||||
try:
|
||||
with temporary.open('w', encoding='utf-8', newline='\n') as stream:
|
||||
json.dump(record, stream, ensure_ascii=False, indent=2)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
retry_sharing_violation(lambda: os.replace(temporary, path))
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _valid_record(record, path):
|
||||
if not isinstance(record, dict):
|
||||
return False
|
||||
session = record.get('session_id')
|
||||
processes = record.get('processes')
|
||||
return (record.get('version') == 1 and isinstance(session, str)
|
||||
and re.fullmatch('[0-9a-f]{32}', session) is not None and path.parent.name == session
|
||||
and isinstance(record.get('kind'), str) and bool(record['kind'])
|
||||
and type(record.get('generation')) is int and record['generation'] >= 1
|
||||
and isinstance(record.get('desktop'), str)
|
||||
and re.fullmatch('dx-[0-9a-f]{32}', record['desktop']) is not None
|
||||
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released'}
|
||||
and isinstance(processes, list) and all(isinstance(item, dict)
|
||||
and type(item.get('pid')) is int and item['pid'] > 0
|
||||
and 'creation_time' in item and (item['creation_time'] is None
|
||||
or isinstance(item['creation_time'], str)) for item in processes)
|
||||
and 'cad_pid' in record and (record['cad_pid'] is None
|
||||
or type(record['cad_pid']) is int and record['cad_pid'] > 0)
|
||||
and 'cad_guid' in record and (record['cad_guid'] is None
|
||||
or isinstance(record['cad_guid'], str) and bool(record['cad_guid']))
|
||||
and all(isinstance(record.get(key), dict) for key in ('checkpoints', 'retries'))
|
||||
and isinstance(record.get('timings'), list))
|
||||
|
||||
|
||||
def pending_resources(root):
|
||||
pending = []
|
||||
for path in sorted(internal_path(canonical_path(root) / 'batch-resources').glob('*/session.json')):
|
||||
path = canonical_path(path)
|
||||
try:
|
||||
record = json.loads(internal_path(path).read_text(encoding='utf-8'))
|
||||
if not _valid_record(record, path):
|
||||
raise ValueError('invalid resource ownership record')
|
||||
if record.get('state') != 'released':
|
||||
pending.append(dict(record, record_path=str(path)))
|
||||
except (OSError, ValueError, TypeError) as error:
|
||||
pending.append({'state': 'blocked', 'record_path': str(path), 'error': str(error)})
|
||||
return tuple(pending)
|
||||
|
||||
|
||||
def guard_pending_resources(root):
|
||||
if pending_resources(root):
|
||||
raise BatchResourceCleanupError('上一批资源尚未确认释放;请先重试释放资源,再开始新批次。')
|
||||
|
||||
|
||||
def process_identity(process):
|
||||
result = {'pid': process.pid, 'creation_time': None}
|
||||
handle = getattr(process, 'handle', None)
|
||||
if handle is not None:
|
||||
import win32process
|
||||
result['creation_time'] = str(win32process.GetProcessTimes(handle)['CreationTime'])
|
||||
return result
|
||||
|
||||
|
||||
def _process_gone(identity):
|
||||
import win32api
|
||||
import win32event
|
||||
import win32process
|
||||
try:
|
||||
handle = win32api.OpenProcess(0x101000, False, identity['pid'])
|
||||
except Exception as error:
|
||||
code = getattr(error, 'winerror', error.args[0] if error.args else None)
|
||||
if code == 87:
|
||||
return True
|
||||
raise
|
||||
try:
|
||||
if win32event.WaitForSingleObject(handle, 0) == 0:
|
||||
return True
|
||||
created = identity.get('creation_time')
|
||||
# A reused PID is an unrelated process and must never be terminated.
|
||||
return created is not None and str(win32process.GetProcessTimes(handle)['CreationTime']) != created
|
||||
finally:
|
||||
handle.Close()
|
||||
|
||||
|
||||
def cleanup_pending_resources(root, *, logging_context=None):
|
||||
from .isolated_desktop import _Win32Api
|
||||
from .batch_release_log import write_release_event
|
||||
for record in pending_resources(root):
|
||||
path = Path(record['record_path'])
|
||||
live = LIVE_RESOURCES.get(str(path))
|
||||
if live is not None:
|
||||
try:
|
||||
live.close(reason='manual_retry')
|
||||
except Exception:
|
||||
pass # The live owner records the failure and retains ownership.
|
||||
continue
|
||||
started = time.monotonic()
|
||||
def log(phase, error=''):
|
||||
try:
|
||||
failures = write_release_event(root, path, record, phase, context=logging_context,
|
||||
reason='restart_recheck', elapsed=time.monotonic() - started, error=error)
|
||||
except Exception as failure:
|
||||
failures = [str(failure)]
|
||||
if failures:
|
||||
record.setdefault('release_log_errors', []).extend(failures)
|
||||
try:
|
||||
diagnostic = dict(record)
|
||||
diagnostic.pop('record_path', None)
|
||||
write_record(path, diagnostic)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if not record.get('session_id'):
|
||||
continue # Corrupt evidence cannot authorize resource actions.
|
||||
log('started')
|
||||
desktop = record.get('desktop')
|
||||
if desktop:
|
||||
_Win32Api().confirm_desktop_released(desktop)
|
||||
if not all(_process_gone(item) for item in record.get('processes', ())):
|
||||
log('failed', '记录中的进程仍未确认退出')
|
||||
continue
|
||||
# After restart only confirm disappearance. Never attach to COM,
|
||||
# close a project, or terminate a PID whose handles we no longer own.
|
||||
record.pop('record_path', None)
|
||||
record = dict(record, state='released', reconciled=True, released_at=time.time())
|
||||
write_record(path, record)
|
||||
log('succeeded')
|
||||
except Exception as error:
|
||||
if record.get('session_id'):
|
||||
record.pop('record_path', None)
|
||||
log('failed', str(error))
|
||||
write_record(path, dict(record, state='blocked', error=str(error)))
|
||||
return pending_resources(root)
|
||||
|
||||
|
||||
def project_checkpoint_released(root):
|
||||
root = canonical_path(root)
|
||||
try:
|
||||
checkpoint = json.loads(internal_path(root / 'project-checkpoint.json').read_text(encoding='utf-8'))
|
||||
path = Path(checkpoint['resource_record']).resolve()
|
||||
record = json.loads(internal_path(path).read_text(encoding='utf-8'))
|
||||
key = str(root).casefold()
|
||||
return (_valid_record(record, path) and checkpoint.get('version') == 1
|
||||
and type(checkpoint.get('generation')) is int
|
||||
and isinstance(checkpoint.get('attempt_id'), str)
|
||||
and re.fullmatch('[0-9a-f]{32}', checkpoint['attempt_id']) is not None
|
||||
and type(checkpoint.get('cad_pid')) is int and checkpoint['cad_pid'] > 0
|
||||
and isinstance(checkpoint.get('cad_guid'), str) and bool(checkpoint['cad_guid'])
|
||||
and isinstance(checkpoint.get('project_path'), str)
|
||||
and Path(checkpoint['project_path']).is_absolute()
|
||||
and checkpoint['project_closed'] is True and checkpoint['worker_closed'] is True
|
||||
and checkpoint['evidence_root'] == str(root)
|
||||
and record['state'] in {'ready', 'closing', 'released'}
|
||||
and (checkpoint['session_id'], checkpoint['generation'], checkpoint['cad_pid'], checkpoint['cad_guid'])
|
||||
== (record['session_id'], record['generation'], record['cad_pid'], record['cad_guid'])
|
||||
and record['checkpoints'][key] == checkpoint['attempt_id'])
|
||||
except (OSError, ValueError, TypeError, KeyError):
|
||||
return False
|
||||
@@ -0,0 +1,407 @@
|
||||
"""One lazy private desktop/CAD owner per import or execution batch."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import time
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from .batch_resources import (BatchResourceCleanupError, LIVE_RESOURCES, cleanup_pending_resources,
|
||||
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record)
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, legacy_path
|
||||
from .isolated_desktop import PrivateDesktop
|
||||
from .operation_control import check_cancelled
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
|
||||
|
||||
class _DesktopLease:
|
||||
"""Close only this operation's processes; the batch owns the desktop itself."""
|
||||
def __init__(self, owner, evidence, *, native=False):
|
||||
self.owner, self.evidence = owner, Path(evidence)
|
||||
self.native, self.attempt_id = native, uuid4().hex
|
||||
self.generation, self.desktop = None, None
|
||||
self.processes = []
|
||||
self.cleanup_record = None
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return (self.desktop if self.desktop is not None else self.owner._ensure_desktop()).name
|
||||
|
||||
def __enter__(self):
|
||||
self.desktop = self.owner._ensure_desktop()
|
||||
self.generation = self.owner.generation
|
||||
if self.native:
|
||||
self.owner.record['native_operations'][str(self.evidence.resolve())] = {
|
||||
'binding': self.identity, 'workers_closed': False}
|
||||
self.owner._save()
|
||||
return self
|
||||
|
||||
@property
|
||||
def identity(self):
|
||||
return {'session_id': self.owner.session_id, 'generation': self.generation,
|
||||
'attempt_id': self.attempt_id}
|
||||
|
||||
def __exit__(self, error_type, error, traceback):
|
||||
try:
|
||||
self.close()
|
||||
except BatchResourceCleanupError as cleanup:
|
||||
raise BatchResourceCleanupError(cleanup.message, primary_error=error,
|
||||
cleanup_error=cleanup) from cleanup
|
||||
|
||||
def start_process(self, command, **options):
|
||||
available = self.owner.record['state'] == 'ready' or (
|
||||
not self.native and self.owner.record['state'] == 'closing')
|
||||
if self.desktop is None or self.generation != self.owner.generation or not available:
|
||||
raise WorkerMonitorError('stale or unavailable batch resource lease')
|
||||
process = self.desktop.start_process(command, **options)
|
||||
self.processes.append(process)
|
||||
self.owner._track(process)
|
||||
return process
|
||||
|
||||
def close(self):
|
||||
errors, remaining, records = [], [], []
|
||||
for process in reversed(self.processes):
|
||||
try:
|
||||
self.owner._release_process(process, desktop=self.desktop, generation=self.generation)
|
||||
records.append(getattr(process, 'cleanup_record', {}))
|
||||
except Exception as error:
|
||||
errors.append(str(error))
|
||||
remaining.append(process)
|
||||
self.processes = list(reversed(remaining))
|
||||
self.cleanup_record = {'scope': 'operation', 'processes': records,
|
||||
'workers_closed': not errors, 'desktop_retained_by_batch': True, 'errors': errors}
|
||||
if errors:
|
||||
self.owner._save(state='blocked', error='; '.join(errors))
|
||||
raise BatchResourceCleanupError('工作进程未能确认退出;批次已停止。' + '; '.join(errors))
|
||||
if self.native and self.generation == self.owner.generation:
|
||||
self.owner.record['native_operations'][str(self.evidence.resolve())] = {
|
||||
'binding': self.identity, 'workers_closed': True}
|
||||
self.owner._save()
|
||||
|
||||
|
||||
class BatchCadRuntime:
|
||||
def __init__(self, root, kind, stop_event=None, *, desktop_factory=None, cad_factory=None, session_id=None):
|
||||
self.root = canonical_path(root)
|
||||
self.kind, self.stop_event = kind, stop_event
|
||||
self.session_id = uuid4().hex if session_id is None else UUID(str(session_id)).hex
|
||||
self.path = self.root / 'batch-resources' / self.session_id / 'session.json'
|
||||
self.desktop_factory = desktop_factory or PrivateDesktop
|
||||
self.cad_factory = cad_factory
|
||||
self.desktop = self.cad = None
|
||||
self.generation = 0
|
||||
self.result = None
|
||||
self.record = {'version': 1, 'session_id': self.session_id, 'kind': kind, 'state': 'new',
|
||||
'generation': 0, 'desktop': None, 'cad_pid': None, 'cad_guid': None,
|
||||
'processes': [], 'checkpoints': {}, 'retries': {}, 'timings': [], 'native_operations': {}}
|
||||
self._retry_counts = {}
|
||||
self._project_attempts = {}
|
||||
self._logging = None
|
||||
|
||||
def bind_logs(self, context, task_ids):
|
||||
"""Keep every participating task's final release outcome visible."""
|
||||
tasks = tuple(task_ids)
|
||||
if any(not isinstance(task, str) or not task or Path(task).name != task
|
||||
or task in {'.', '..'} for task in tasks):
|
||||
raise ValueError('resource logs require safe task identifiers')
|
||||
self._logging = context
|
||||
self.record['log_task_ids'] = list(dict.fromkeys((*self.record.get('log_task_ids', ()), *tasks)))
|
||||
if self.generation:
|
||||
self._save()
|
||||
|
||||
def __enter__(self):
|
||||
guard_pending_resources(self.root)
|
||||
return self
|
||||
|
||||
def __exit__(self, error_type, error, traceback):
|
||||
try:
|
||||
self.close()
|
||||
except Exception as cleanup:
|
||||
def detail(failure):
|
||||
return {'type': type(failure).__name__, 'message': str(failure),
|
||||
'code': getattr(failure, 'code', None)}
|
||||
self.record['primary_error'] = detail(error) if error is not None else None
|
||||
self.record['cleanup_error'] = detail(cleanup)
|
||||
try:
|
||||
self._save()
|
||||
except Exception:
|
||||
pass # Journal failure must not replace either original failure.
|
||||
context = ('BOM 导入读取已停止;未生成可执行预览。'
|
||||
if self.kind == 'bom_import' else
|
||||
'已核验任务结果保留。' if self.result is not None else '本批操作已停止。')
|
||||
raise BatchResourceCleanupError('批次资源释放待处理;' + context + str(cleanup),
|
||||
completed_result=self.result, primary_error=error, cleanup_error=cleanup) from cleanup
|
||||
|
||||
def _save(self, **values):
|
||||
record = dict(self.record, **values)
|
||||
write_record(self.path, record)
|
||||
self.record = record
|
||||
|
||||
def _timing(self, phase, started, **details):
|
||||
self.record['timings'].append({'phase': phase, 'seconds': time.monotonic() - started,
|
||||
'generation': self.generation, **details})
|
||||
|
||||
def _ensure_desktop(self, abort_event=None):
|
||||
if self.record['state'] == 'blocked':
|
||||
raise BatchResourceCleanupError('当前批次资源已阻断,不能启动新的工作进程。')
|
||||
if self.desktop is None:
|
||||
check_cancelled(self.stop_event if abort_event is None else abort_event)
|
||||
self.generation += 1
|
||||
name = 'dx-' + uuid4().hex
|
||||
self._save(state='starting', generation=self.generation, desktop=name,
|
||||
processes=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
|
||||
cad_shutdown=None, cad_start_requested=False, cleanup=None)
|
||||
LIVE_RESOURCES[str(self.path)] = self
|
||||
self.desktop = self.desktop_factory(name, enabled=True)
|
||||
started = time.monotonic()
|
||||
try:
|
||||
self.desktop.__enter__()
|
||||
except Exception:
|
||||
self._save(state='blocked', error='private desktop startup failed')
|
||||
raise
|
||||
self._timing('desktop_start', started)
|
||||
self._save(state='ready')
|
||||
return self.desktop
|
||||
|
||||
def _track(self, process):
|
||||
identity = process_identity(process)
|
||||
if identity not in self.record['processes']:
|
||||
self.record['processes'].append(identity)
|
||||
self._save()
|
||||
|
||||
def _release_process(self, process, *, desktop=None, generation=None):
|
||||
process.close()
|
||||
release = getattr(desktop if desktop is not None else self.desktop, 'forget_closed_process', None)
|
||||
if release:
|
||||
release(process)
|
||||
if generation is None or generation == self.generation:
|
||||
self.record['processes'] = [item for item in self.record['processes'] if item['pid'] != process.pid]
|
||||
|
||||
def native_scope(self, evidence):
|
||||
return _DesktopLease(self, evidence, native=True)
|
||||
|
||||
def _ensure_cad(self, abort_event=None):
|
||||
if self.cad is None:
|
||||
from .isolated_cad import IsolatedCadSession
|
||||
factory = self.cad_factory or IsolatedCadSession
|
||||
lease = _DesktopLease(self, self.path.parent)
|
||||
started = time.monotonic()
|
||||
self._save(cad_start_requested=True)
|
||||
cad = factory(self.path.parent / f'generation-{self.generation + (self.desktop is None)}',
|
||||
abort_event=abort_event, desktop=lease)
|
||||
self.cad = cad # Retain cleanup ownership even if startup fails.
|
||||
try:
|
||||
cad.__enter__()
|
||||
except Exception as error:
|
||||
from .isolated_cad import PrivateCadStartupError
|
||||
if isinstance(error, PrivateCadStartupError):
|
||||
# __enter__ certified its local processes/handles closed.
|
||||
# A never-ready instance must not receive a second Quit.
|
||||
self.cad = None
|
||||
raise
|
||||
self._timing('cad_start', started)
|
||||
self._save(state='ready', cad_pid=cad.binding.pid, cad_guid=cad.binding.guid)
|
||||
return self.cad
|
||||
|
||||
def _prepare_attempts(self, root, command, attempt_id, generation):
|
||||
from .batch_attempt import prepare_attempt
|
||||
from .quick_run import _read_commands
|
||||
rows = _read_commands(internal_path(root / 'command.tsv'))
|
||||
selected_rows = [row for row in rows if row[1] == command[-1]]
|
||||
if len(selected_rows) != 1 or len({row[1] for row in rows}) != len(rows):
|
||||
raise TaskCenterError('target_identity', '批次工作命令身份缺失或重复,已停止。')
|
||||
selected = selected_rows[0]
|
||||
project = canonical_path(selected[2])
|
||||
self.record.update(active_project=str(project), active_task=selected[0], active_job=selected[1])
|
||||
directory = self.path.parent / 'attempts' / f'g{generation}' / attempt_id
|
||||
expected = []
|
||||
for row in rows:
|
||||
if canonical_path(row[2]) == project:
|
||||
path = directory / (row[1] + '.json')
|
||||
record = prepare_attempt(path, session_id=self.session_id, generation=generation,
|
||||
task_id=row[0], job_id=row[1], project_path=str(project), attempt_id=attempt_id)
|
||||
expected.append((path, record))
|
||||
self._project_attempts.setdefault(str(project).casefold(), []).extend(expected)
|
||||
return project, legacy_path(directory), expected
|
||||
|
||||
def _can_retry(self, project, expected):
|
||||
from .batch_attempt import can_retry_attempt
|
||||
key = str(project).casefold()
|
||||
return (not self._retry_counts.get(key) and expected
|
||||
and all(can_retry_attempt(path, record) for path, record in self._project_attempts.get(key, ())))
|
||||
|
||||
def _checkpoint(self, root, project, attempt_id):
|
||||
self.cad.checkpoint(root / ('cad-closed-' + attempt_id + '.json'))
|
||||
key = str(root).casefold()
|
||||
self.record['checkpoints'][key] = attempt_id
|
||||
self._save(state='ready')
|
||||
write_record(root / 'project-checkpoint.json', {'version': 1, 'resource_record': str(self.path),
|
||||
'session_id': self.session_id, 'generation': self.generation, 'attempt_id': attempt_id,
|
||||
'evidence_root': str(root), 'project_path': str(project), 'project_closed': True,
|
||||
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']})
|
||||
|
||||
def run_worker(self, command, timeout, *, abort_event=None, cooperative_abort=False):
|
||||
from .discovery_cache import project_fingerprint
|
||||
from .isolated_cad import PrivateCadStartupError, is_identity_conflict
|
||||
from .worker_monitor import run_monitored_process
|
||||
root = canonical_path(command[3])
|
||||
while True:
|
||||
check_cancelled(abort_event or self.stop_event)
|
||||
attempt_id = uuid4().hex
|
||||
# No previous empty-project certificate may authorize continuation
|
||||
# once another operation could attach or open a project.
|
||||
self.record['checkpoints'].pop(str(root).casefold(), None)
|
||||
self.record['active_attempt'] = attempt_id
|
||||
if self.generation:
|
||||
self._save()
|
||||
generation = self.generation + (self.desktop is None)
|
||||
project, directory, expected = self._prepare_attempts(root, command, attempt_id, generation)
|
||||
self._ensure_desktop(abort_event)
|
||||
before = project_fingerprint(project, self.root)
|
||||
status = internal_path(root / 'status.tsv')
|
||||
status_before = status.read_bytes() if status.exists() else None
|
||||
started = time.monotonic()
|
||||
failure = None
|
||||
try:
|
||||
cad = self._ensure_cad(abort_event)
|
||||
cad.checkpoint(root / ('cad-ready-' + attempt_id + '.json'))
|
||||
environment = {'DXPART_ATTEMPT_FILE': str(directory), 'DXPART_ATTEMPT_ID': attempt_id,
|
||||
'DXPART_BATCH_SESSION_ID': self.session_id, 'DXPART_BATCH_GENERATION': str(self.generation)}
|
||||
def start_worker(launch, **options):
|
||||
return cad.start_worker(launch, extra_environment=environment, **options)
|
||||
result = run_monitored_process(command, timeout, abort_event=abort_event,
|
||||
cooperative_abort=cooperative_abort, process_factory=start_worker)
|
||||
self._checkpoint(root, project, attempt_id)
|
||||
if result.returncode not in (2, 3):
|
||||
return result
|
||||
failure = WorkerMonitorError(result.stderr or 'pre-mutation worker startup failed')
|
||||
except (PrivateCadStartupError, WorkerMonitorError, subprocess.TimeoutExpired, OSError) as error:
|
||||
failure = error
|
||||
finally:
|
||||
self._timing('project_worker', started, project=str(project), attempt_id=attempt_id,
|
||||
task_id=self.record['active_task'], job_id=self.record['active_job'])
|
||||
# Missing proof, changed inputs/status, cancellation or a previous
|
||||
# write anywhere in this project can never authorize replay.
|
||||
after = project_fingerprint(project, self.root)
|
||||
status_after = status.read_bytes() if status.exists() else None
|
||||
# Runtime database changes require separate identity evidence. Until
|
||||
# available for a failed read, use the stricter full fingerprint.
|
||||
safe = (not is_identity_conflict(failure) and before is not None and before == after
|
||||
and status_before == status_after and self._can_retry(project, expected)
|
||||
and not internal_path(root / 'recovery.request').exists()
|
||||
and not (abort_event is not None and abort_event.is_set())
|
||||
and not (self.stop_event is not None and self.stop_event.is_set()))
|
||||
if not safe:
|
||||
raise failure
|
||||
key = str(project).casefold()
|
||||
self._retry_counts[key] = 1
|
||||
self.record['retries'][key] = {'count': 1, 'reason': str(failure), 'attempt_id': attempt_id}
|
||||
self._save(state='recycling')
|
||||
self.close(rebuilding=True) # Prove disappearance before creating another.
|
||||
self.desktop = self.cad = None
|
||||
|
||||
def recover_native_failure(self, project, evidence, reason):
|
||||
"""One read-only recovery, based on durable source and worker proof."""
|
||||
from .batch_attempt import can_retry_attempt
|
||||
project, evidence = canonical_path(project), canonical_path(evidence)
|
||||
key = str(project).casefold()
|
||||
if (self._retry_counts.get(key) or self.record['state'] == 'blocked'
|
||||
or self.stop_event is not None and self.stop_event.is_set()):
|
||||
return False
|
||||
try:
|
||||
proof = json.loads(internal_path(evidence / 'native-retry-proof.json').read_text(encoding='utf-8'))
|
||||
sources = proof['source_hashes']
|
||||
operation = self.record['native_operations'][str(evidence)]
|
||||
if (proof['version'] != 1 or proof['native_read_only'] is not True
|
||||
or operation['workers_closed'] is not True or proof['binding'] != operation['binding']
|
||||
or proof['binding']['session_id'] != self.session_id
|
||||
or proof['binding']['generation'] != self.generation
|
||||
or proof['workers_closed'] is not True or proof['project_path'] != str(project)
|
||||
or proof['evidence_root'] != str(evidence) or str(project) not in sources
|
||||
or len(sources) != 2
|
||||
or not all(canonical_path(name).is_relative_to(project.parent) for name in sources)
|
||||
or not all(hashlib.sha256(internal_path(name).read_bytes()).hexdigest() == digest
|
||||
for name, digest in sources.items())
|
||||
or not all(can_retry_attempt(path, record)
|
||||
for path, record in self._project_attempts.get(key, ()))):
|
||||
return False
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
return False
|
||||
self._retry_counts[key] = 1
|
||||
self.record['retries'][key] = {'count': 1, 'reason': str(reason), 'backend': 'native_cli'}
|
||||
self._save(state='recycling')
|
||||
self.close(rebuilding=True)
|
||||
self.desktop = self.cad = None
|
||||
self._ensure_desktop()
|
||||
return True
|
||||
|
||||
def _release_log(self, phase, *, reason, elapsed=0.0, error=''):
|
||||
from .batch_release_log import write_release_event
|
||||
try:
|
||||
errors = write_release_event(self.root, self.path, self.record, phase,
|
||||
context=self._logging, reason=reason, elapsed=elapsed, error=error)
|
||||
except Exception as failure:
|
||||
errors = [str(failure)]
|
||||
if errors:
|
||||
self.record.setdefault('release_log_errors', []).extend(errors)
|
||||
try:
|
||||
self._save()
|
||||
except Exception:
|
||||
pass # A diagnostic write cannot bypass process/desktop cleanup.
|
||||
|
||||
def close(self, *, rebuilding=False, reason=None):
|
||||
if self.desktop is None and not self.generation:
|
||||
return # A cache-only batch never acquired resources.
|
||||
if self.record['state'] == 'released':
|
||||
return
|
||||
reason = reason or ('rebuild' if rebuilding else 'batch_end')
|
||||
started = time.monotonic()
|
||||
self._release_log('started', reason=reason)
|
||||
try:
|
||||
self._close_resources(rebuilding=rebuilding)
|
||||
except Exception as error:
|
||||
self._release_log('failed', reason=reason, elapsed=time.monotonic() - started, error=error)
|
||||
raise
|
||||
self._release_log('succeeded', reason=reason, elapsed=time.monotonic() - started)
|
||||
|
||||
def _close_resources(self, *, rebuilding=False):
|
||||
started = time.monotonic()
|
||||
errors = []
|
||||
try:
|
||||
self._save(state='closing')
|
||||
except Exception as error:
|
||||
errors.append('resource journal: ' + str(error))
|
||||
if self.cad is not None:
|
||||
cad, self.cad = self.cad, None
|
||||
try:
|
||||
cad.__exit__(None, None, None)
|
||||
except Exception as error:
|
||||
if rebuilding:
|
||||
self.record['shutdown_warning'] = str(error)
|
||||
else:
|
||||
errors.append(str(error))
|
||||
finally:
|
||||
cad_record = getattr(cad, 'record', {})
|
||||
cad_record = cad_record if isinstance(cad_record, dict) else {}
|
||||
self.record['cad_shutdown'] = {key: cad_record[key] for key in (
|
||||
'state', 'clean', 'cad_exit_code', 'quit_helper_exit_code', 'cleanup_errors',
|
||||
'shutdown_seconds') if key in cad_record}
|
||||
if self.desktop is not None:
|
||||
try:
|
||||
self.desktop.close()
|
||||
except Exception as error:
|
||||
errors.append(str(error))
|
||||
self._timing('batch_release', started)
|
||||
self.record['cleanup'] = getattr(self.desktop, 'cleanup_record', None)
|
||||
if errors:
|
||||
self._save(state='blocked', error='; '.join(errors))
|
||||
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
|
||||
released_at = time.time()
|
||||
release = {'generation': self.generation, 'desktop': self.record['desktop'],
|
||||
'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid'],
|
||||
'cleanup': self.record['cleanup'], 'released_at': released_at}
|
||||
releases = [*self.record.get('generation_releases', ()), release]
|
||||
self._save(state='released', released_at=released_at, processes=[], generation_releases=releases)
|
||||
LIVE_RESOURCES.pop(str(self.path), None)
|
||||
@@ -0,0 +1,323 @@
|
||||
"""GUI-side batch admission, process supervision and resource-only reconciliation."""
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
import json
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import queue
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, legacy_path
|
||||
from .supervisor_protocol import VERSION, MessageGate, valid_journal, encode, decode, decode_error
|
||||
|
||||
_WORKER_ROOT = None
|
||||
_LAUNCH_LOCK = threading.Lock()
|
||||
_LIVE_SUPERVISORS = {}
|
||||
_WRITE_OPERATIONS = frozenset({'execute_task_queue', 'execute_quick_run_queue', 'execute_quick_run', 'recover_quick_run'})
|
||||
|
||||
|
||||
def _mark_interrupted(root, journal, reason):
|
||||
from .batch_resources import write_record
|
||||
journal.update(state='blocked', outcome='interrupted', error=str(reason))
|
||||
if journal['operation'] in _WRITE_OPERATIONS:
|
||||
# A missing final result is not proof of no mutation. Publish the same
|
||||
# protection the existing recovery UI understands, without overwriting
|
||||
# any earlier, more specific marker.
|
||||
journal['business_uncertain'] = True
|
||||
for task_id in journal['task_ids']:
|
||||
allowed = {canonical_path(root), canonical_path(Path(root) / 'tasks' / task_id)}
|
||||
actual = journal.get('uncertainty_roots', {}).get(task_id)
|
||||
roots = {canonical_path(actual)} if actual and canonical_path(actual) in allowed else allowed
|
||||
for evidence in roots:
|
||||
marker = internal_path(evidence / 'quick-run/uncertain.flag')
|
||||
marker.parent.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
with marker.open('x', encoding='utf-8') as stream:
|
||||
stream.write('reason=supervisor_interrupted; batch=' + journal['batch_id'] + '; manual reconciliation required\n')
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
except FileExistsError:
|
||||
pass
|
||||
write_record(Path(root) / 'batch-resources' / journal['batch_id'] / 'supervisor.json', journal)
|
||||
|
||||
|
||||
def _worker_context(root):
|
||||
global _WORKER_ROOT
|
||||
_WORKER_ROOT = canonical_path(root)
|
||||
|
||||
|
||||
def _file_lock(handle, acquire):
|
||||
from .task_queue import _os_lock
|
||||
_os_lock(handle, acquire)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def supervisor_lease(root):
|
||||
root = internal_path(root)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
if not _LAUNCH_LOCK.acquire(blocking=False):
|
||||
raise TaskCenterError('supervisor_busy', '另一批操作正在运行,请先安全收尾。')
|
||||
handle = None
|
||||
acquired = False
|
||||
try:
|
||||
handle = (root / '.batch-supervisor.execution.lock').open('a+b')
|
||||
if handle.seek(0, os.SEEK_END) == 0:
|
||||
handle.write(b'\0')
|
||||
handle.flush()
|
||||
try:
|
||||
_file_lock(handle, True)
|
||||
acquired = True
|
||||
except OSError as error:
|
||||
raise TaskCenterError('supervisor_busy', '另一窗口的批次监督进程尚未结束。') from error
|
||||
yield
|
||||
finally:
|
||||
if handle is not None:
|
||||
if acquired:
|
||||
_file_lock(handle, False)
|
||||
handle.close()
|
||||
_LAUNCH_LOCK.release()
|
||||
|
||||
|
||||
def assert_supervisor_idle(root):
|
||||
if _WORKER_ROOT == canonical_path(root):
|
||||
return
|
||||
with supervisor_lease(root):
|
||||
pass
|
||||
|
||||
|
||||
@contextmanager
|
||||
def mutation_lease(root):
|
||||
if _WORKER_ROOT == canonical_path(root):
|
||||
yield
|
||||
else:
|
||||
with supervisor_lease(root):
|
||||
yield
|
||||
|
||||
|
||||
def pending_supervisors(root):
|
||||
from .batch_resources import _process_gone
|
||||
pending = []
|
||||
for path in (internal_path(root) / 'batch-resources').glob('*/supervisor.json'):
|
||||
try:
|
||||
value = json.loads(path.read_text(encoding='utf-8'))
|
||||
if not valid_journal(value, path.parent.name):
|
||||
raise ValueError('invalid supervisor ownership record')
|
||||
if value.get('state') == 'finished':
|
||||
continue
|
||||
identity = value.get('process') or value.get('parent_process')
|
||||
if (value.get('state') in {'starting', 'running'} and identity and not _process_gone(identity)
|
||||
and value.get('parent_process') and not _process_gone(value['parent_process'])):
|
||||
continue
|
||||
pending.append(dict(value, record_path=str(canonical_path(path)), state='blocked'))
|
||||
except (OSError, ValueError, TypeError, KeyError) as error:
|
||||
pending.append({'state': 'blocked', 'record_path': str(canonical_path(path)), 'error': str(error)})
|
||||
return tuple(pending)
|
||||
|
||||
|
||||
def reconcile_supervisors(root):
|
||||
"""Recheck orphan ownership only; this never launches or replays a job."""
|
||||
from .batch_resources import _process_gone, pending_resources, write_record
|
||||
physical = pending_resources(root)
|
||||
for record in pending_supervisors(root):
|
||||
identity = record.get('process')
|
||||
if not record.get('batch_id') or (identity and not _process_gone(identity)):
|
||||
continue
|
||||
if record.get('outcome') not in {'interrupted', 'completed_resources_pending'}:
|
||||
_mark_interrupted(root, record, 'Supervisor ended without a reconciled final result')
|
||||
if any(item.get('session_id') == record['batch_id'] or not item.get('session_id') for item in physical):
|
||||
continue
|
||||
path = Path(record.pop('record_path'))
|
||||
outcome = 'completed' if record.get('outcome') == 'completed_resources_pending' else 'interrupted'
|
||||
record.update(state='finished', reconciled=True, outcome=outcome, reconciled_at=time.time())
|
||||
write_record(path, record)
|
||||
return pending_supervisors(root)
|
||||
|
||||
|
||||
def _command(request):
|
||||
if bool(getattr(sys, 'frozen', False)):
|
||||
return (sys.executable, '--batch-worker', str(request))
|
||||
return (sys.executable, '-m', 'dx_productivity.trial_task_center.batch_worker', '--request', str(request))
|
||||
|
||||
|
||||
def _environment():
|
||||
environment = dict(os.environ)
|
||||
if bool(getattr(sys, 'frozen', False)):
|
||||
for key in ('PYTHONPATH', 'PYTHONHOME'):
|
||||
environment.pop(key, None)
|
||||
else:
|
||||
environment['PYTHONPATH'] = str(Path(__file__).resolve().parents[2])
|
||||
return environment
|
||||
|
||||
|
||||
def _finish_response(root, journal, response, context):
|
||||
"""A supervisor exit may release its last USER reference. Never repeat work."""
|
||||
from .batch_resources import cleanup_pending_resources, write_record
|
||||
from .supervisor_result import validated_outcome, resource_pending_error
|
||||
result, error, primary = validated_outcome(journal['operation'], response)
|
||||
pending = cleanup_pending_resources(root, logging_context=context)
|
||||
related = tuple(item for item in pending if item.get('session_id') == journal['batch_id'] or not item.get('session_id'))
|
||||
journal['state'] = 'blocked' if related else 'finished'
|
||||
journal['outcome'] = ('completed_resources_pending' if related and response.get('business_succeeded')
|
||||
else response.get('status', 'interrupted'))
|
||||
journal['resource_recheck'] = {'pending': bool(related), 'checked_at': time.time()}
|
||||
write_record(Path(root) / 'batch-resources' / journal['batch_id'] / 'supervisor.json', journal)
|
||||
if related:
|
||||
raise resource_pending_error(journal['operation'], result, error, primary)
|
||||
if response.get('business_succeeded'):
|
||||
if response.get('error'):
|
||||
context.logger.info('supervisor_resource_recovery_passed batch_id=%s business_replayed=false', journal['batch_id'])
|
||||
return result
|
||||
if response.get('operation_error'):
|
||||
if error is not None and hasattr(error, 'bom_report') and not hasattr(primary, 'bom_report'):
|
||||
primary.bom_report = error.bom_report
|
||||
raise primary
|
||||
if response.get('error'):
|
||||
raise error
|
||||
raise TaskCenterError('supervisor_interrupted', '批次监督进程异常退出;未重放任务,请检查已有结果和保护记录。')
|
||||
|
||||
|
||||
def run_supervised(service, operation, arguments, *, stop_event=None, on_progress=None, on_phase=None):
|
||||
from .batch_resources import write_record, process_identity
|
||||
from .isolated_desktop import _Win32Api
|
||||
root = canonical_path(service._logging.root)
|
||||
if stop_event is not None and stop_event.is_set():
|
||||
raise TaskCenterError('operation_cancelled', '本批操作已取消,未启动监督进程。')
|
||||
with supervisor_lease(root):
|
||||
if operation == 'cleanup_batch_resources':
|
||||
for key, (owned_api, owned_process) in tuple(_LIVE_SUPERVISORS.items()):
|
||||
if key.parent.parent == root / 'batch-resources':
|
||||
owned_process.close()
|
||||
owned_api.close_pending()
|
||||
_LIVE_SUPERVISORS.pop(key)
|
||||
if operation != 'cleanup_batch_resources' and pending_supervisors(root):
|
||||
raise TaskCenterError('batch_resources_pending', '上一监督进程尚未完成资源复核,请先重新清理资源。')
|
||||
batch_id, attempt_id = uuid4().hex, uuid4().hex
|
||||
directory = root / 'batch-resources' / batch_id
|
||||
request = directory / 'request.json'
|
||||
journal = {'version': VERSION, 'batch_id': batch_id, 'attempt_id': attempt_id,
|
||||
'operation': operation, 'state': 'starting', 'started_at': time.time(),
|
||||
'task_ids': list(arguments.get('task_ids', ())) or ([arguments['task_id']] if arguments.get('task_id') else []),
|
||||
'process': None}
|
||||
if any(not isinstance(value, str) or Path(value).name != value or value in {'.', '..'}
|
||||
for value in journal['task_ids']):
|
||||
raise TaskCenterError('target_identity', '监督进程任务编号无效。')
|
||||
journal['uncertainty_roots'] = {task_id: str(canonical_path(
|
||||
service._project_task_service(task_id)._quick_run_preparer.data_root))
|
||||
for task_id in journal['task_ids'] if operation in _WRITE_OPERATIONS}
|
||||
import win32api
|
||||
parent = type('ParentIdentity', (), {'pid': os.getpid(), 'handle': win32api.GetCurrentProcess()})()
|
||||
journal['parent_process'] = process_identity(parent)
|
||||
write_record(request, {'version': VERSION, 'batch_id': batch_id, 'attempt_id': attempt_id,
|
||||
'root': str(root), 'database': str(canonical_path(service._store.path)),
|
||||
'operation': operation, 'arguments': encode(arguments), 'task_ids': journal['task_ids']})
|
||||
journal['request_sha256'] = hashlib.sha256(internal_path(request).read_bytes()).hexdigest()
|
||||
write_record(directory / 'supervisor.json', journal)
|
||||
gate, messages = MessageGate(batch_id, attempt_id), queue.Queue()
|
||||
api, process, reader = _Win32Api(), None, None
|
||||
sent_cancel, last_heartbeat = False, time.monotonic()
|
||||
transport_error = None
|
||||
def receive():
|
||||
try:
|
||||
while line := process.stdout.readline(1024 * 1024):
|
||||
try:
|
||||
messages.put(json.loads(line))
|
||||
except (ValueError, UnicodeError):
|
||||
messages.put({'type': 'invalid'})
|
||||
finally:
|
||||
messages.put(None)
|
||||
try:
|
||||
# A normal input-desktop supervisor owns its private-desktop children.
|
||||
# Only the supervisor, never the GUI, creates a private desktop.
|
||||
process = api.create_process(_command(internal_path(request)), None, legacy_path(root),
|
||||
_environment(), stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
|
||||
journal.update(state='running', process=process_identity(process))
|
||||
write_record(directory / 'supervisor.json', journal)
|
||||
reader = threading.Thread(target=receive, daemon=True)
|
||||
reader.start()
|
||||
while process.poll() is None or not messages.empty():
|
||||
if stop_event is not None and stop_event.is_set() and not sent_cancel:
|
||||
write_record(directory / 'cancel.json', {'version': VERSION, 'batch_id': batch_id, 'attempt_id': attempt_id})
|
||||
sent_cancel = True
|
||||
try:
|
||||
value = messages.get(timeout=.1)
|
||||
except queue.Empty:
|
||||
if time.monotonic() - last_heartbeat > 15:
|
||||
service._logging.logger.warning('supervisor_heartbeat_delayed batch_id=%s; no automatic replay', batch_id)
|
||||
last_heartbeat = time.monotonic()
|
||||
continue
|
||||
if value is None or not gate.accept(value):
|
||||
continue
|
||||
last_heartbeat = time.monotonic()
|
||||
if value['type'] == 'progress' and on_progress is not None:
|
||||
on_progress(value['payload'])
|
||||
elif value['type'] == 'phase' and on_phase is not None:
|
||||
payload = decode(value['payload'])
|
||||
on_phase(*payload)
|
||||
elif value['type'] == 'state':
|
||||
state = decode(value['payload'])
|
||||
service._task_queue_feedback = state['queue']
|
||||
service._batch_feedback = state['batch']
|
||||
if on_phase is not None and state['phase'] in {'recycling', 'closing'}:
|
||||
from .bom_import_state import BomImportPhase
|
||||
on_phase(BomImportPhase.RECOVERING if state['phase'] == 'recycling' else BomImportPhase.RELEASING,
|
||||
'正在安全恢复读取会话…' if state['phase'] == 'recycling' else '正在释放后台资源…')
|
||||
process.wait(timeout=15)
|
||||
except Exception as error:
|
||||
transport_error = error
|
||||
finally:
|
||||
service._task_queue_feedback = None
|
||||
service._batch_feedback = None
|
||||
try:
|
||||
if process is not None:
|
||||
process.close() # Waits, and confines termination to the owned Job.
|
||||
api.close_pending()
|
||||
except Exception as error:
|
||||
if process is not None:
|
||||
_LIVE_SUPERVISORS[directory / 'supervisor.json'] = (api, process)
|
||||
transport_error = error if transport_error is None else TaskCenterError(
|
||||
'batch_resources_pending', f'监督进程错误:{transport_error};收尾错误:{error}')
|
||||
if reader is not None:
|
||||
reader.join(3)
|
||||
try:
|
||||
try:
|
||||
response = json.loads(internal_path(directory / 'result.json').read_text(encoding='utf-8'))
|
||||
except FileNotFoundError:
|
||||
saved = json.loads(internal_path(directory / 'business-result.json').read_text(encoding='utf-8'))
|
||||
if (saved.get('version') != VERSION or saved.get('batch_id') != batch_id
|
||||
or saved.get('attempt_id') != attempt_id or not saved['payload'].get('business_succeeded')):
|
||||
raise ValueError('Untrusted saved business result')
|
||||
response = dict(saved, type='result', sequence=gate.sequence + 1)
|
||||
service._logging.logger.warning('supervisor_final_missing_saved_result_retained batch_id=%s', batch_id)
|
||||
if (response.get('version') != VERSION or response.get('batch_id') != batch_id
|
||||
or response.get('attempt_id') != attempt_id or response.get('type') != 'result'
|
||||
or type(response.get('sequence')) is not int or response['sequence'] < gate.sequence
|
||||
or type(response.get('generation')) is not int or response['generation'] < gate.generation):
|
||||
raise ValueError('stale or invalid final supervisor result')
|
||||
from .supervisor_result import validated_outcome, resource_pending_error
|
||||
decoded_result, decoded_error, decoded_primary = validated_outcome(operation, response['payload'])
|
||||
except (OSError, ValueError, KeyError, TypeError, AttributeError) as error:
|
||||
_mark_interrupted(root, journal, transport_error or error)
|
||||
# Recheck physical resources even when final business results are
|
||||
# missing; this never authorizes repeating the interrupted job.
|
||||
from .batch_resources import cleanup_pending_resources
|
||||
cleanup_pending_resources(root, logging_context=service._logging)
|
||||
failure = TaskCenterError('supervisor_interrupted',
|
||||
'批次监督进程未留下可信结果;保留保护记录,不自动重试。' +
|
||||
(f' 首发原因:{transport_error}' if transport_error else ''))
|
||||
failure.primary_error = transport_error or error
|
||||
raise failure from (transport_error or error)
|
||||
if transport_error is not None:
|
||||
if response['payload']['business_succeeded']:
|
||||
journal.update(state='blocked', outcome='completed_resources_pending', error=str(transport_error))
|
||||
write_record(directory / 'supervisor.json', journal)
|
||||
else:
|
||||
_mark_interrupted(root, journal, transport_error)
|
||||
raise resource_pending_error(operation, decoded_result, transport_error, decoded_primary)
|
||||
return _finish_response(root, journal, response['payload'], service._logging)
|
||||
@@ -0,0 +1,195 @@
|
||||
"""One request, one resource-owning supervisor process, no Qt application."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import traceback
|
||||
from uuid import UUID
|
||||
|
||||
from .path_access import internal_path, canonical_path
|
||||
from .supervisor_protocol import VERSION, OPERATIONS, valid_journal, encode, decode, encode_error
|
||||
|
||||
def _output_stream():
|
||||
if sys.stdout is not None:
|
||||
return sys.stdout, False
|
||||
# Windowed PyInstaller deliberately sets sys.stdout=None. The supervisor
|
||||
# still has the explicit inherited pipe from STARTUPINFO; duplicate it so
|
||||
# closing this Python stream cannot close an unrelated borrowed handle.
|
||||
import msvcrt
|
||||
import win32api
|
||||
import win32con
|
||||
source = win32api.GetStdHandle(win32con.STD_OUTPUT_HANDLE)
|
||||
handle = win32api.DuplicateHandle(win32api.GetCurrentProcess(), source,
|
||||
win32api.GetCurrentProcess(), 0, False, win32con.DUPLICATE_SAME_ACCESS)
|
||||
descriptor = msvcrt.open_osfhandle(handle.Detach(), os.O_WRONLY | os.O_BINARY)
|
||||
return os.fdopen(descriptor, 'w', encoding='utf-8', buffering=1), True
|
||||
|
||||
|
||||
def _admit_request(directory, request, request_bytes):
|
||||
from .batch_resources import _process_gone, process_identity
|
||||
import win32api
|
||||
deadline = time.monotonic() + 5
|
||||
while True:
|
||||
journal = json.loads(internal_path(directory / 'supervisor.json').read_text(encoding='utf-8'))
|
||||
if (not valid_journal(journal, request['batch_id'])
|
||||
or journal.get('attempt_id') != request['attempt_id']
|
||||
or journal.get('state') not in {'starting', 'running'}
|
||||
or journal.get('request_sha256') != hashlib.sha256(request_bytes).hexdigest()
|
||||
or not journal.get('parent_process', {}).get('creation_time')
|
||||
or _process_gone(journal['parent_process'])):
|
||||
raise ValueError('Stale or unowned supervisor request')
|
||||
if journal.get('process'):
|
||||
break
|
||||
if time.monotonic() >= deadline:
|
||||
raise ValueError('Supervisor launch identity was not published')
|
||||
time.sleep(.02)
|
||||
own = process_identity(type('WorkerIdentity', (), {'pid': os.getpid(), 'handle': win32api.GetCurrentProcess()})())
|
||||
if journal['process'] != own:
|
||||
raise ValueError('Supervisor request belongs to another process')
|
||||
# Atomic exclusive publication makes requests one-shot, including retained
|
||||
# requests invoked manually or racing a delayed/duplicate launch.
|
||||
with internal_path(directory / 'claim.json').open('x', encoding='utf-8') as stream:
|
||||
json.dump({'version': VERSION, 'batch_id': request['batch_id'], 'attempt_id': request['attempt_id'],
|
||||
'process': own}, stream)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
|
||||
|
||||
def run(request_path):
|
||||
from .batch_resources import write_record
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
from .batch_supervisor import _worker_context
|
||||
from .logging_setup import configure_logging
|
||||
from .service import TaskCenterService
|
||||
from .store import TaskStore
|
||||
|
||||
output, owns_output = _output_stream()
|
||||
request_path = canonical_path(request_path, strict=True)
|
||||
request_bytes = internal_path(request_path).read_bytes()
|
||||
request = json.loads(request_bytes)
|
||||
batch_id, attempt_id = request['batch_id'], request['attempt_id']
|
||||
if (request.get('version') != VERSION or UUID(batch_id).hex != batch_id or UUID(attempt_id).hex != attempt_id
|
||||
or request['operation'] not in OPERATIONS):
|
||||
raise ValueError('Invalid batch supervisor request')
|
||||
root = canonical_path(request['root'])
|
||||
directory = root / 'batch-resources' / batch_id
|
||||
if request_path != directory / 'request.json':
|
||||
raise ValueError('Supervisor request ownership path mismatch')
|
||||
_admit_request(directory, request, request_bytes)
|
||||
_worker_context(root)
|
||||
stop, finished, output_lock = threading.Event(), threading.Event(), threading.Lock()
|
||||
kind = {'inspect_bom_batch': 'bom_import', 'preview_bom_batch': 'bom_import',
|
||||
'execute_task_queue': 'task_queue', 'execute_quick_run_queue': 'quick_run_queue'}.get(request['operation'], request['operation'])
|
||||
runtime = BatchCadRuntime(root, kind, stop, session_id=batch_id)
|
||||
sequence = 0
|
||||
def emit(kind, payload=None, *, persist=False):
|
||||
nonlocal sequence
|
||||
with output_lock:
|
||||
sequence += 1
|
||||
envelope = {'version': VERSION, 'batch_id': batch_id, 'attempt_id': attempt_id,
|
||||
'generation': max(1, runtime.generation), 'sequence': sequence,
|
||||
'type': kind, 'payload': payload}
|
||||
if persist:
|
||||
write_record(directory / 'result.json', envelope)
|
||||
try:
|
||||
output.write(json.dumps(envelope, ensure_ascii=True, allow_nan=False) + '\n')
|
||||
output.flush()
|
||||
except (BrokenPipeError, OSError):
|
||||
stop.set()
|
||||
def monitor():
|
||||
while not finished.is_set():
|
||||
try:
|
||||
cancel = internal_path(directory / 'cancel.json')
|
||||
if cancel.is_file():
|
||||
value = json.loads(cancel.read_text(encoding='utf-8'))
|
||||
if value == {'version': VERSION, 'batch_id': batch_id, 'attempt_id': attempt_id}:
|
||||
stop.set()
|
||||
emit('heartbeat')
|
||||
feedback = getattr(service, '_task_queue_feedback', None)
|
||||
emit('state', encode({'phase': runtime.record['state'], 'queue': feedback,
|
||||
'batch': service.batch_feedback()}))
|
||||
except Exception:
|
||||
stop.set() # Lost control evidence cannot authorize new work.
|
||||
finished.wait(1)
|
||||
context = configure_logging(root, log_path=directory / 'supervisor.log')
|
||||
store = TaskStore(Path(request['database']))
|
||||
service = TaskCenterService(store, context, recover_on_startup=False)
|
||||
thread = threading.Thread(target=monitor, daemon=True)
|
||||
thread.start()
|
||||
response = {'business_succeeded': False, 'status': 'error'}
|
||||
try:
|
||||
arguments = decode(request['arguments'])
|
||||
operation = request['operation']
|
||||
def progress(message):
|
||||
emit('progress', str(message))
|
||||
def phase(value, message):
|
||||
emit('phase', encode((value, str(message))))
|
||||
if operation == 'cleanup_batch_resources':
|
||||
result = service.cleanup_batch_resources()
|
||||
response.update(status='ok', business_succeeded=True, result=encode(result))
|
||||
else:
|
||||
with runtime:
|
||||
runtime.bind_logs(context, request['task_ids'])
|
||||
try:
|
||||
if operation == 'probe':
|
||||
result = {'pid': os.getpid(), 'resource_state': 'not_acquired'}
|
||||
elif operation == 'runtime_smoke':
|
||||
with runtime.native_scope(directory / 'runtime-smoke') as lease:
|
||||
child = lease.start_process((str(Path(os.environ['SystemRoot'])/'System32/cmd.exe'), '/d', '/c', 'exit', '0'))
|
||||
if not runtime.record['processes'][0]['creation_time'] or child.wait(timeout=15):
|
||||
raise RuntimeError('Supervisor process-identity smoke failed')
|
||||
result = {'pid': os.getpid(), 'process_identity_read': True}
|
||||
elif operation == 'execute_task_queue':
|
||||
from .task_queue import execute_task_queue
|
||||
result = execute_task_queue(service, **arguments, stop_event=stop, runtime=runtime)
|
||||
elif operation in {'inspect_bom_batch', 'preview_bom_batch'}:
|
||||
controls = {'stop_event': stop, 'on_progress': progress, 'runtime': runtime}
|
||||
if operation == 'inspect_bom_batch':
|
||||
controls['on_phase'] = phase
|
||||
result = getattr(service, operation)(**arguments, **controls)
|
||||
elif operation == 'collect_target_inventory':
|
||||
result = service.collect_target_inventory(**arguments, stop_event=stop, on_progress=progress, runtime=runtime)
|
||||
elif operation == 'execute_quick_run_queue':
|
||||
result = service.execute_quick_run_queue(**arguments, stop_event=stop, runtime=runtime)
|
||||
elif operation == 'target_cell_choices':
|
||||
result = service.target_cell_choices(**arguments)
|
||||
else:
|
||||
result = getattr(service, operation)(**arguments, runtime=runtime)
|
||||
runtime.result = result
|
||||
response.update(status='ok', business_succeeded=True, result=encode(result))
|
||||
write_record(directory / 'business-result.json', {'version': VERSION, 'batch_id': batch_id,
|
||||
'attempt_id': attempt_id, 'generation': max(1, runtime.generation), 'payload': response})
|
||||
if operation == 'inspect_bom_batch':
|
||||
from .bom_import_state import BomImportPhase
|
||||
phase(BomImportPhase.RELEASING, '本批位号核验结束,正在释放后台资源…')
|
||||
except Exception as error:
|
||||
response['operation_error'] = encode_error(error)
|
||||
raise
|
||||
except Exception as error:
|
||||
context.report_exception('batch_supervisor', error)
|
||||
response.update(status='error', error=encode_error(error))
|
||||
finally:
|
||||
finished.set()
|
||||
thread.join(3)
|
||||
store.close()
|
||||
emit('result', response, persist=True)
|
||||
if owns_output:
|
||||
output.close()
|
||||
return 0
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--request', required=True, type=Path)
|
||||
args = parser.parse_args()
|
||||
return run(args.request)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
@@ -55,11 +55,11 @@ def _assert_editable(service, task_id):
|
||||
raise TaskCenterError('target_plan_locked', f'工程任务当前不能修改 BOM 计划:{context.task.name}')
|
||||
|
||||
|
||||
def preview_batch(service, task_id, path, *, stop_event=None, on_progress=None):
|
||||
def preview_batch(service, task_id, path, *, stop_event=None, on_progress=None, runtime=None):
|
||||
"""Strict adapter to the same validation engine used by the import report."""
|
||||
from .bom_import_report import inspect_batch
|
||||
|
||||
report = inspect_batch(service, task_id, path, stop_event=stop_event, on_progress=on_progress)
|
||||
report = inspect_batch(service, task_id, path, stop_event=stop_event, on_progress=on_progress, runtime=runtime)
|
||||
if report.preview is not None:
|
||||
return report.preview
|
||||
failed = next((row for row in report.rows if row.status == 'failed'), None)
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
"""Read-only, exhaustive BOM validation with an all-or-nothing preview."""
|
||||
import inspect
|
||||
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
from zipfile import BadZipFile
|
||||
@@ -8,6 +10,7 @@ from openpyxl import load_workbook
|
||||
from .bom_batch import BomBatchPreview, BomProjectPreview, _assert_editable, _identity, _scope
|
||||
from .bom_plan import BomMatcher, _normal_headers, _text, parse_bom_row
|
||||
from .errors import TaskCenterError
|
||||
from .project_discovery import ReleasedProjectReadError
|
||||
from .bom_import_state import BomImportPhase
|
||||
from .store_target_plan import merge_target_inventory
|
||||
from .target_plan import ProjectIdentityResolver, compile_target_rules
|
||||
@@ -70,6 +73,15 @@ class BomImportReport:
|
||||
return sum(project.status == 'unchecked' for project in self.projects)
|
||||
|
||||
|
||||
class BomInspectionError(TaskCenterError):
|
||||
"""Stop unsafe collection while retaining the complete, read-only row report."""
|
||||
|
||||
def __init__(self, cause: TaskCenterError, report: BomImportReport):
|
||||
super().__init__(cause.code, cause.message, error_id=cause.error_id,
|
||||
traceback_detail=cause.traceback_detail)
|
||||
self.bom_report = report
|
||||
|
||||
|
||||
def _report(rows, preview=None, errors=()):
|
||||
grouped = {}
|
||||
for row in rows:
|
||||
@@ -111,12 +123,30 @@ def _check_record_conflicts(rows, records, indices):
|
||||
rows[index] = replace(rows[index], status='failed', reason=reason)
|
||||
|
||||
|
||||
def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None, on_phase=None):
|
||||
def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None, on_phase=None, runtime=None):
|
||||
"""Validate all independent rows; never persist inventory or return partial plans.
|
||||
|
||||
Operational failures (including cancellation and native cleanup) propagate so
|
||||
callers cannot accidentally continue opening CAD after an unsafe failure.
|
||||
"""
|
||||
if runtime is None:
|
||||
from .batch_runtime import BatchCadRuntime, BatchResourceCleanupError
|
||||
report = None
|
||||
try:
|
||||
with BatchCadRuntime(service._logging.root, 'bom_import', stop_event) as owned_runtime:
|
||||
try:
|
||||
report = inspect_batch(service, task_id, path, stop_event=stop_event,
|
||||
on_progress=on_progress, on_phase=on_phase, runtime=owned_runtime)
|
||||
except BomInspectionError as error:
|
||||
report = error.bom_report
|
||||
raise
|
||||
owned_runtime.result = report
|
||||
return report
|
||||
except BatchResourceCleanupError as error:
|
||||
error.bom_report = replace(report, preview=None) if report is not None else None
|
||||
if isinstance(error.completed_result, BomImportReport):
|
||||
error.completed_result = replace(error.completed_result, preview=None)
|
||||
raise
|
||||
def check_cancelled():
|
||||
if stop_event is not None and stop_event.is_set():
|
||||
raise TaskCenterError('operation_cancelled', 'BOM 导入已取消')
|
||||
@@ -130,6 +160,7 @@ def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None,
|
||||
check_cancelled()
|
||||
notify('正在读取并校验 BOM Excel 表格…', BomImportPhase.READING)
|
||||
scope = _scope(service._store, task_id)
|
||||
runtime.bind_logs(service._logging, tuple(task.task_id for task, _ in scope))
|
||||
routing = ProjectIdentityResolver()
|
||||
owners = {routing.project(project.path): (task, project) for task, project in scope}
|
||||
if len(owners) != len(scope):
|
||||
@@ -233,13 +264,34 @@ def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None,
|
||||
inventory = tuple(row.target for row in baseline)
|
||||
else:
|
||||
progress(f'「{task.name}」正在获取位号库存,仅采集本表涉及的旧料号及已有计划料号')
|
||||
controls = {}
|
||||
collector = service.for_project_task(task.task_id)._quick_run_preparer.collect_inventory
|
||||
controls = ({'runtime': runtime} if any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in inspect.signature(collector).parameters.values()) else {})
|
||||
if stop_event is not None:
|
||||
controls['stop_event'] = stop_event
|
||||
if on_progress is not None:
|
||||
controls['on_progress'] = progress
|
||||
inventory = tuple(service.for_project_task(task.task_id)._quick_run_preparer.collect_inventory(
|
||||
project.path, tuple(dict.fromkeys((*known, *requested))), **controls))
|
||||
try:
|
||||
inventory = tuple(collector(
|
||||
project.path, tuple(dict.fromkeys((*known, *requested))), **controls))
|
||||
except TaskCenterError as error:
|
||||
if error.code == 'operation_cancelled':
|
||||
raise
|
||||
reason = f'工程采集失败:{error.message}(错误编号:{error.error_id})'
|
||||
for index in indices:
|
||||
rows[index] = replace(rows[index], status='failed', reason=reason)
|
||||
# Continue only with positive session-release evidence, not
|
||||
# merely a shared error code or an absent uncertainty marker.
|
||||
if isinstance(error, ReleasedProjectReadError):
|
||||
progress(f'「{task.name}」采集失败,已记录;继续核验其他工程')
|
||||
continue
|
||||
for index, row in enumerate(rows):
|
||||
if row.status == 'unchecked':
|
||||
rows[index] = replace(row, reason='前序工程采集异常,已停止后续核验')
|
||||
report = _report(rows, errors=(
|
||||
f'已停止批量采集:{task.name};{error.message}(错误编号:{error.error_id})',))
|
||||
raise BomInspectionError(error, report) from error
|
||||
check_cancelled()
|
||||
progress(f'「{task.name}」正在匹配 {len(indices)} 行 BOM 与工程位号…', BomImportPhase.MATCHING)
|
||||
merged = merge_target_inventory(baseline, inventory)
|
||||
|
||||
@@ -17,6 +17,8 @@ class BomImportPhase(Enum):
|
||||
VALIDATING = 'validating'
|
||||
INVENTORY = 'inventory'
|
||||
MATCHING = 'matching'
|
||||
RECOVERING = 'recovering'
|
||||
RELEASING = 'releasing'
|
||||
CANCELLING = 'cancelling'
|
||||
PREVIEW_READY = 'preview_ready'
|
||||
REJECTED = 'rejected'
|
||||
@@ -30,6 +32,11 @@ _NEXT = {
|
||||
BomImportPhase.INVENTORY: {BomImportPhase.INVENTORY, BomImportPhase.MATCHING},
|
||||
BomImportPhase.MATCHING: {BomImportPhase.MATCHING, BomImportPhase.INVENTORY},
|
||||
}
|
||||
_RECOVERY_PHASES = {BomImportPhase.RECOVERING, BomImportPhase.RELEASING}
|
||||
for _phase in tuple(_NEXT):
|
||||
_NEXT[_phase] |= _RECOVERY_PHASES
|
||||
for _phase in _RECOVERY_PHASES:
|
||||
_NEXT[_phase] = _RECOVERY_PHASES | {BomImportPhase.INVENTORY, BomImportPhase.MATCHING}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -90,8 +97,12 @@ class BomImportMachine:
|
||||
return False
|
||||
cancelled_error = isinstance(error, TaskCenterError) and error.code == 'operation_cancelled'
|
||||
if error is not None and not cancelled_error:
|
||||
from .bom_import_report import BomImportReport
|
||||
reason = error.message if isinstance(error, TaskCenterError) else str(error)
|
||||
self.state = replace(self.state, phase=BomImportPhase.FAILED, report=None,
|
||||
report = getattr(error, 'bom_report', None)
|
||||
if not isinstance(report, BomImportReport):
|
||||
report = None
|
||||
self.state = replace(self.state, phase=BomImportPhase.FAILED, report=report,
|
||||
message=f'本次 BOM 导入未完成:{reason[:240]};原有预览和编辑保留。')
|
||||
elif cancelled_error or self.state.phase is BomImportPhase.CANCELLING:
|
||||
self.state = replace(self.state, phase=BomImportPhase.CANCELLED, report=None,
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
"""Allowlisted, bounded support export; never packages project or CAD inputs."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
import zipfile
|
||||
|
||||
from .path_access import canonical_path, internal_path
|
||||
|
||||
MAX_FILE_BYTES = 4 * 1024 * 1024
|
||||
MAX_BUNDLE_BYTES = 32 * 1024 * 1024
|
||||
MAX_FILES = 512
|
||||
_BATCH_NAMES = {'session.json', 'supervisor.json', 'request.json', 'result.json',
|
||||
'business-result.json', 'supervisor.log', 'resource-release.log', 'cancel.json'}
|
||||
_QUICK_NAMES = {'private-desktop.json', 'checkpoint.json', 'status.json',
|
||||
'uncertain.flag', 'resolved-uncertain.flag', 'protection.json', 'cleanup.json',
|
||||
'project-session.json', 'project-checkpoint.json', 'status.tsv'}
|
||||
_SECRET_KEY = re.compile(r'(?:password|passwd|(?:client[_-]?)?secret|credentials?|authorization|api[_-]?key|access[_-]?token|refresh[_-]?token|token)', re.I)
|
||||
_USER_KEY = re.compile(r'(?:user(?:[_-]?name)?|login[_-]?name)', re.I)
|
||||
_SECRET_TEXT = re.compile(r'''(?ix)(\b(?:password|passwd|(?:client[_-]?)?secret|credentials?|authorization|api[_-]?key|access[_-]?token|refresh[_-]?token|token)\b["']?\s*[=:]\s*)(?:"[^"\r\n]*"|'[^'\r\n]*'|[^\s,;}]+)''')
|
||||
|
||||
|
||||
def _scrub_text(text):
|
||||
# Scrub other users as well as the exporting user's home, including JSON
|
||||
# escaped Windows paths. Preserve suffixes useful for fault diagnosis.
|
||||
text = re.sub(r'(?i)(?:[A-Z]:[\\/]+Users[\\/]+|/Users/|/home/)[^\\/\s"\']+', '<USER_HOME>', text)
|
||||
home = str(Path.home())
|
||||
for prefix in {home, home.replace('\\', '/'), home.replace('\\', '\\\\')}:
|
||||
text = re.sub(re.escape(prefix), '<USER_HOME>', text, flags=re.I)
|
||||
username = os.environ.get('USERNAME') or os.environ.get('USER')
|
||||
if username and len(username) > 2:
|
||||
text = re.sub(r'(?<![\w])' + re.escape(username) + r'(?![\w])', '<USER>', text, flags=re.I)
|
||||
text = re.sub(r'(?i)\bBearer\s+[A-Za-z0-9._~+/-]+=*', 'Bearer <REDACTED>', text)
|
||||
text = _SECRET_TEXT.sub(r'\1<REDACTED>', text)
|
||||
text = re.sub(r'''(?ix)(\b(?:user(?:[_-]?name)?|login[_-]?name)\b["']?\s*[=:]\s*)(?:"[^"\r\n]*"|'[^'\r\n]*'|[^\s,;}]+)''', r'\1<USER>', text)
|
||||
text = re.sub(r'(?i)(https?://)[^\s/@:]+:[^\s/@]+@', r'\1<REDACTED>@', text)
|
||||
return text
|
||||
|
||||
|
||||
def _scrub_json(value):
|
||||
if isinstance(value, dict):
|
||||
return {_scrub_text(str(key)): ('<REDACTED>' if _SECRET_KEY.fullmatch(str(key))
|
||||
else '<USER>' if _USER_KEY.fullmatch(str(key)) else _scrub_json(item))
|
||||
for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_scrub_json(item) for item in value]
|
||||
return _scrub_text(value) if isinstance(value, str) else value
|
||||
|
||||
|
||||
def _allowed(parts):
|
||||
if len(parts) == 2 and parts[0] == 'logs':
|
||||
return bool(re.fullmatch(r'[^/]+\.log(?:\.\d+)?', parts[-1]))
|
||||
if len(parts) == 3 and parts[0] == 'batch-resources':
|
||||
return parts[-1] in _BATCH_NAMES
|
||||
if len(parts) >= 5 and parts[0] == 'batch-resources' and parts[2] == 'attempts':
|
||||
return parts[-1].endswith('.json')
|
||||
if len(parts) >= 2 and parts[0] == 'attempts':
|
||||
return parts[-1].endswith('.json')
|
||||
if len(parts) == 3 and parts[0] == 'tasks':
|
||||
return bool(re.fullmatch(r'events\.jsonl(?:\.\d+)?', parts[-1]))
|
||||
if len(parts) >= 2 and parts[0] in {'queue', 'queues', 'task-queues'}:
|
||||
return parts[-1].endswith('.json')
|
||||
if len(parts) >= 2 and any(value in parts for value in ('quick-run', 'quickrun', 'quick-run-history')):
|
||||
return parts[-1] in _QUICK_NAMES
|
||||
return False
|
||||
|
||||
|
||||
def export_diagnostic_bundle(data_root, destination: Path) -> Path:
|
||||
"""Write a new ZIP, excluding links, binaries, inputs and credential configs.
|
||||
|
||||
This is a point-in-time best-effort read: active writers may produce partial
|
||||
log lines. Unsupported/unreadable items are documented, never dereferenced.
|
||||
"""
|
||||
root, destination = canonical_path(data_root), canonical_path(destination)
|
||||
if destination.suffix.lower() != '.zip':
|
||||
raise ValueError('diagnostic destination must have a .zip suffix')
|
||||
manifest = {'version': 1, 'included': [], 'omissions': [], 'unknowns': [
|
||||
'External desktop handle holders are unknown; empty window/thread lists do not prove release.',
|
||||
'Active logs and metadata are point-in-time reads; no source CAD, libraries, BOM, database or environment configuration is included.'],
|
||||
'limits': {'max_file_bytes': MAX_FILE_BYTES, 'max_bundle_bytes': MAX_BUNDLE_BYTES, 'max_files': MAX_FILES}}
|
||||
total = 0
|
||||
physical = internal_path(root)
|
||||
# os.walk does not descend symlinks; explicitly prune Windows junctions too.
|
||||
with zipfile.ZipFile(internal_path(destination), 'x', compression=zipfile.ZIP_DEFLATED) as archive:
|
||||
for directory, folders, files in os.walk(physical, followlinks=False):
|
||||
base = Path(directory)
|
||||
safe_folders = []
|
||||
for name in sorted(folders):
|
||||
entry = base / name
|
||||
relative = entry.relative_to(physical).as_posix()
|
||||
info = entry.lstat()
|
||||
if stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 0x400:
|
||||
manifest['omissions'].append({'path': _scrub_text(relative), 'reason': 'link_or_reparse_point'})
|
||||
elif base == physical and name not in {'logs', 'batch-resources', 'attempts', 'tasks', 'queue', 'queues', 'task-queues', 'quick-run', 'quick-run-history'}:
|
||||
manifest['omissions'].append({'path': _scrub_text(relative), 'reason': 'outside_allowlist'})
|
||||
else:
|
||||
safe_folders.append(name)
|
||||
folders[:] = safe_folders
|
||||
for name in sorted(files):
|
||||
path = base / name
|
||||
relative = path.relative_to(physical)
|
||||
label = _scrub_text(relative.as_posix())
|
||||
reason = None
|
||||
try:
|
||||
info = path.lstat()
|
||||
if not _allowed(relative.parts):
|
||||
reason = 'outside_allowlist'
|
||||
elif stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 0x400 or info.st_nlink != 1:
|
||||
reason = 'link_or_reparse_point'
|
||||
elif not stat.S_ISREG(info.st_mode):
|
||||
reason = 'not_regular_file'
|
||||
elif info.st_size > MAX_FILE_BYTES or total + info.st_size > MAX_BUNDLE_BYTES or len(manifest['included']) >= MAX_FILES:
|
||||
reason = 'size_or_count_limit'
|
||||
else:
|
||||
with path.open('rb') as source:
|
||||
payload = source.read(MAX_FILE_BYTES + 1)
|
||||
if len(payload) > MAX_FILE_BYTES or b'\x00' in payload:
|
||||
reason = 'binary_or_size_limit'
|
||||
else:
|
||||
text = payload.decode('utf-8-sig')
|
||||
if relative.suffix == '.json':
|
||||
text = json.dumps(_scrub_json(json.loads(text)), ensure_ascii=False, indent=2)
|
||||
elif '.jsonl' in relative.name:
|
||||
text = '\n'.join(json.dumps(_scrub_json(json.loads(line)), ensure_ascii=False)
|
||||
for line in text.splitlines() if line.strip()) + '\n'
|
||||
else:
|
||||
text = _scrub_text(text)
|
||||
archive.writestr(label, text)
|
||||
total += len(payload)
|
||||
manifest['included'].append(label)
|
||||
except (OSError, UnicodeError, ValueError):
|
||||
reason = 'unreadable_or_invalid_text'
|
||||
if reason:
|
||||
manifest['omissions'].append({'path': label, 'reason': reason})
|
||||
archive.writestr('manifest.json', json.dumps(manifest, ensure_ascii=False, indent=2))
|
||||
return destination
|
||||
@@ -130,6 +130,66 @@ class DiscoveryCache:
|
||||
def invalidate(self, project: Path):
|
||||
with self.lock:
|
||||
self._entries.pop(_key(project), None)
|
||||
self._disk_path(project).unlink(missing_ok=True)
|
||||
|
||||
def _disk_path(self, project):
|
||||
from .path_access import internal_path
|
||||
return internal_path(Path(self.data_root) / 'discovery-cache' /
|
||||
(hashlib.sha256(_key(project).encode('utf-8')).hexdigest() + '.json'))
|
||||
|
||||
def _disk_entry(self, project, stamp):
|
||||
import json
|
||||
from uuid import uuid4
|
||||
from .supervisor_protocol import decode
|
||||
path = self._disk_path(project)
|
||||
if stamp is None or not path.exists():
|
||||
return None
|
||||
try:
|
||||
if path.stat().st_size > 32 * 1024 * 1024:
|
||||
raise ValueError('inventory cache exceeds size limit')
|
||||
record = json.loads(path.read_text(encoding='utf-8'))
|
||||
checksum = record.pop('record_sha256')
|
||||
if (record.get('version') != 2 or not isinstance(checksum, str)
|
||||
or checksum != hashlib.sha256(json.dumps(record, ensure_ascii=True, sort_keys=True).encode()).hexdigest()
|
||||
or type(record.get('collect_components')) is not bool
|
||||
or not isinstance(record.get('parts'), list)
|
||||
or not all(isinstance(part, str) for part in record['parts'])):
|
||||
raise ValueError('invalid discovery cache coverage or identity')
|
||||
payload = json.dumps(record['discovery'], ensure_ascii=True, sort_keys=True).encode('utf-8')
|
||||
if record['payload_sha256'] != hashlib.sha256(payload).hexdigest():
|
||||
raise ValueError('invalid discovery cache')
|
||||
if record['project'] != _key(project) or record['fingerprint'] != hashlib.sha256(repr(stamp).encode()).hexdigest():
|
||||
return None
|
||||
discovery = decode(record['discovery'])
|
||||
from .quick_run_protocol import ProjectDiscovery
|
||||
if not isinstance(discovery, ProjectDiscovery):
|
||||
raise ValueError('unexpected discovery cache type')
|
||||
_discovery_identity(discovery)
|
||||
return stamp, frozenset(record['parts']), record['collect_components'], discovery
|
||||
except (OSError, ValueError, TypeError, KeyError):
|
||||
try:
|
||||
path.replace(path.with_name(path.name + '.bad-' + uuid4().hex))
|
||||
except OSError:
|
||||
pass # A bad cache is never authoritative, even if quarantine fails.
|
||||
return None
|
||||
|
||||
def _save_disk(self, project, stamp, requested, components, discovery):
|
||||
import json
|
||||
from .batch_resources import write_record
|
||||
from .supervisor_protocol import encode
|
||||
try:
|
||||
payload = encode(discovery)
|
||||
record = {'version': 2, 'project': _key(project),
|
||||
'fingerprint': hashlib.sha256(repr(stamp).encode()).hexdigest(), 'parts': sorted(requested),
|
||||
'collect_components': components, 'discovery': payload,
|
||||
'payload_sha256': hashlib.sha256(json.dumps(payload, ensure_ascii=True, sort_keys=True).encode()).hexdigest()}
|
||||
record['record_sha256'] = hashlib.sha256(json.dumps(record, ensure_ascii=True, sort_keys=True).encode()).hexdigest()
|
||||
write_record(self._disk_path(project), record)
|
||||
files = sorted(self._disk_path(project).parent.glob('*.json'), key=lambda path: path.stat().st_mtime_ns, reverse=True)
|
||||
for stale in files[self.capacity:]:
|
||||
stale.unlink(missing_ok=True)
|
||||
except (OSError, ValueError, TypeError):
|
||||
pass # Cache persistence is optional, unlike execution write-ahead proof.
|
||||
|
||||
def discover(self, project, parts, collect_components, loader, *, stop_event=None):
|
||||
with self.lock:
|
||||
@@ -140,7 +200,7 @@ class DiscoveryCache:
|
||||
key = _key(project)
|
||||
requested = frozenset(part.casefold() for part in parts)
|
||||
before = fingerprint()
|
||||
entry = self._entries.pop(key, None)
|
||||
entry = self._entries.pop(key, None) or self._disk_entry(project, before)
|
||||
if entry is not None:
|
||||
stamp, covered, has_components, discovery = entry
|
||||
if (before is not None and before == stamp and requested <= covered
|
||||
@@ -197,6 +257,7 @@ class DiscoveryCache:
|
||||
and (len(getattr(discovery, 'components', ())) + len(sheets)
|
||||
+ sum(len(parts) for parts in matches.values())) <= 100000):
|
||||
self._entries[key] = (after, requested, collect_components, deepcopy(discovery))
|
||||
self._save_disk(project, after, requested, collect_components, discovery)
|
||||
while len(self._entries) > self.capacity:
|
||||
self._entries.popitem(last=False)
|
||||
return discovery
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from pathlib import Path
|
||||
|
||||
from .errors import TaskCenterError
|
||||
@@ -58,11 +59,15 @@ def read_component_inventory(path: Path, project_path: Path,
|
||||
|
||||
|
||||
def collect_target_inventory(preparer, project_path: Path,
|
||||
old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None) -> tuple[ComponentTarget, ...]:
|
||||
old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None, runtime=None) -> tuple[ComponentTarget, ...]:
|
||||
if not old_part_numbers:
|
||||
raise TaskCenterError("target_inventory", "at least one old Part Number is required")
|
||||
from .operation_control import check_cancelled, control_options
|
||||
check_cancelled(stop_event)
|
||||
rows = preparer.collect_inventory(project_path, old_part_numbers, **control_options(stop_event, on_progress))
|
||||
runtime_options = ({'runtime': runtime} if runtime is not None and any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in inspect.signature(preparer.collect_inventory).parameters.values()) else {})
|
||||
rows = preparer.collect_inventory(project_path, old_part_numbers, **runtime_options,
|
||||
**control_options(stop_event, on_progress))
|
||||
check_cancelled(stop_event)
|
||||
return rows
|
||||
@@ -12,13 +12,43 @@ from uuid import uuid4
|
||||
from .cad_identity import CadBinding, current_desktop_name, verify_application, project_file_path
|
||||
from .isolated_desktop import PrivateDesktop
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
from .path_access import internal_path, canonical_path, retry_sharing_violation
|
||||
|
||||
|
||||
class PrivateCadStartupError(WorkerMonitorError):
|
||||
"""The owned desktop was released before any project worker was launched."""
|
||||
|
||||
|
||||
class CadIdentityConflict(WorkerMonitorError, ValueError):
|
||||
"""An invalid CAD ownership certificate can never authorize automatic replay."""
|
||||
|
||||
|
||||
class PrivateCadIdentityError(PrivateCadStartupError, CadIdentityConflict):
|
||||
"""Startup ownership failed, and its local resources were released."""
|
||||
|
||||
|
||||
PRIVATE_CAD_IDENTITY_EXIT_CODE = 4
|
||||
|
||||
|
||||
def is_identity_conflict(error):
|
||||
"""Keep the no-retry decision when cleanup or evidence I/O wraps the cause."""
|
||||
pending, seen = [error], set()
|
||||
while pending:
|
||||
item = pending.pop()
|
||||
if item is None or id(item) in seen:
|
||||
continue
|
||||
seen.add(id(item))
|
||||
# The legacy VBS launcher transports these ownership failures as exit
|
||||
# 2 plus text. Treat only its explicit identity diagnostics as conflicts.
|
||||
if (isinstance(item, CadIdentityConflict) or any(message in str(item).casefold()
|
||||
for message in ('incomplete private cad identity', 'private cad guid mismatch'))):
|
||||
return True
|
||||
pending.extend((item.__cause__, item.__context__))
|
||||
return False
|
||||
|
||||
|
||||
def _helper_command(action, evidence, pid):
|
||||
evidence = internal_path(evidence)
|
||||
if getattr(sys, 'frozen', False):
|
||||
return (sys.executable, '--private-cad-action', action, '--private-cad-evidence', str(evidence),
|
||||
'--private-cad-pid', str(pid))
|
||||
@@ -36,15 +66,7 @@ def _replace_record(temporary, target):
|
||||
# Windows readers can temporarily deny delete access to the old JSON.
|
||||
# Keep the complete old record until replacement succeeds, with a finite
|
||||
# retry budget so persistent permissions failures still reach the caller.
|
||||
deadline = time.monotonic() + 1.0
|
||||
while True:
|
||||
try:
|
||||
temporary.replace(target)
|
||||
return
|
||||
except PermissionError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise
|
||||
time.sleep(0.02)
|
||||
retry_sharing_violation(lambda: internal_path(temporary).replace(internal_path(target)), target=target)
|
||||
|
||||
|
||||
def _designer_executable():
|
||||
@@ -65,10 +87,11 @@ def _designer_executable():
|
||||
|
||||
class IsolatedCadSession:
|
||||
"""The supervisor never attaches to COM on the user's input desktop."""
|
||||
def __init__(self, root, *, abort_event=None):
|
||||
self.root = Path(root).resolve()
|
||||
def __init__(self, root, *, abort_event=None, desktop=None):
|
||||
self.root = canonical_path(root)
|
||||
self.token = uuid4().hex
|
||||
self.desktop = PrivateDesktop('dx-' + self.token, enabled=True)
|
||||
self.borrowed_desktop = desktop is not None
|
||||
self.desktop = desktop if desktop is not None else PrivateDesktop('dx-' + self.token, enabled=True)
|
||||
self.cad = None
|
||||
self.binding = None
|
||||
self.started = time.monotonic()
|
||||
@@ -77,12 +100,12 @@ class IsolatedCadSession:
|
||||
|
||||
def _record(self, **values):
|
||||
self.record.update(values)
|
||||
self.root.mkdir(parents=True, exist_ok=True)
|
||||
target = self.root / 'private-desktop.json'
|
||||
internal_path(self.root).mkdir(parents=True, exist_ok=True)
|
||||
target = internal_path(self.root / 'private-desktop.json')
|
||||
temporary = target.with_suffix('.tmp')
|
||||
temporary.write_text(json.dumps(self.record, ensure_ascii=False, indent=2), encoding='utf-8')
|
||||
_replace_record(temporary, target)
|
||||
history = self.root / 'private-desktop-sessions'
|
||||
history = internal_path(self.root / 'private-desktop-sessions')
|
||||
history.mkdir(exist_ok=True)
|
||||
archived = history / (self.token + '.json')
|
||||
pending = archived.with_suffix('.tmp')
|
||||
@@ -105,18 +128,27 @@ class IsolatedCadSession:
|
||||
deadline = time.monotonic() + 120
|
||||
while probe.poll() is None:
|
||||
self._check_abort()
|
||||
cad_exit = self.cad.poll()
|
||||
if cad_exit is not None:
|
||||
raise WorkerMonitorError(f'owned private CAD exited during startup: exit code {cad_exit}')
|
||||
if time.monotonic() >= deadline:
|
||||
raise WorkerMonitorError('private CAD bootstrap timed out')
|
||||
time.sleep(0.05)
|
||||
code = probe.poll()
|
||||
if code:
|
||||
raise WorkerMonitorError('private CAD startup identity failed: ' + probe.stderr.read().decode('utf-8', errors='replace'))
|
||||
failure_type = CadIdentityConflict if code == PRIVATE_CAD_IDENTITY_EXIT_CODE else WorkerMonitorError
|
||||
raise failure_type('private CAD startup failed: ' + probe.stderr.read().decode('utf-8', errors='replace'))
|
||||
finally:
|
||||
probe.close()
|
||||
result = json.loads(bootstrap.read_text(encoding='utf-8'))
|
||||
self.binding = CadBinding(**result)
|
||||
if (self.binding.pid, self.binding.desktop, self.binding.token) != (self.cad.pid, self.desktop.name, self.token):
|
||||
raise WorkerMonitorError('private CAD bootstrap returned a foreign identity')
|
||||
try:
|
||||
result = json.loads(internal_path(bootstrap).read_text(encoding='utf-8'))
|
||||
binding = CadBinding(**result)
|
||||
if (type(binding.pid) is not int or not isinstance(binding.guid, str) or not binding.guid
|
||||
or (binding.pid, binding.desktop, binding.token) != (self.cad.pid, self.desktop.name, self.token)):
|
||||
raise ValueError('private CAD bootstrap returned a foreign identity')
|
||||
except (OSError, ValueError, TypeError) as failure:
|
||||
raise CadIdentityConflict('private CAD bootstrap identity evidence is invalid: ' + str(failure)) from failure
|
||||
self.binding = binding
|
||||
self._record(state='ready', guid=self.binding.guid, startup_seconds=time.monotonic() - self.started)
|
||||
return self
|
||||
except Exception as error:
|
||||
@@ -126,16 +158,51 @@ class IsolatedCadSession:
|
||||
self._uncertain('startup cleanup failed: ' + str(failure))
|
||||
raise WorkerMonitorError('private CAD startup cleanup requires reconciliation: ' + str(failure)) from error
|
||||
self._record(state='released', clean=False, startup_error=str(error))
|
||||
if is_identity_conflict(error):
|
||||
raise PrivateCadIdentityError(str(error)) from error
|
||||
raise PrivateCadStartupError(str(error)) from error
|
||||
|
||||
def _check_abort(self):
|
||||
if self.abort_event is not None and self.abort_event.is_set():
|
||||
raise WorkerMonitorError('private CAD startup was cancelled before execution')
|
||||
|
||||
def start_worker(self, command, **options):
|
||||
def start_worker(self, command, *, extra_environment=None, **options):
|
||||
if self.binding is None or self.cad.poll() is not None:
|
||||
raise WorkerMonitorError('owned private CAD is unavailable')
|
||||
return self.desktop.start_process(command, env=_child_environment(self.binding.environment()), **options)
|
||||
environment = self.binding.environment()
|
||||
environment.update(extra_environment or {})
|
||||
# The verified CAD binding cannot be overridden by attempt metadata.
|
||||
binding = self.binding.environment()
|
||||
for key in ('DXPART_DESKTOP', 'DXPART_CAD_PID', 'DXPART_CAD_GUID', 'DXPART_SESSION_TOKEN'):
|
||||
environment[key] = binding[key]
|
||||
return self.desktop.start_process(command, env=_child_environment(environment), **options)
|
||||
|
||||
def checkpoint(self, evidence):
|
||||
"""Certify an idle, identity-verified CAD before crossing project boundaries."""
|
||||
if self.binding is None or self.cad.poll() is not None:
|
||||
raise WorkerMonitorError('owned CAD unavailable at project checkpoint')
|
||||
evidence = Path(evidence)
|
||||
internal_path(evidence.parent).mkdir(parents=True, exist_ok=True)
|
||||
process = self.start_worker(_helper_command('checkpoint', evidence, self.cad.pid),
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
try:
|
||||
code = process.wait(timeout=20)
|
||||
if code:
|
||||
failure_type = CadIdentityConflict if code == PRIVATE_CAD_IDENTITY_EXIT_CODE else WorkerMonitorError
|
||||
raise failure_type('project close checkpoint failed: ' +
|
||||
process.stderr.read().decode('utf-8', errors='replace'))
|
||||
try:
|
||||
result = json.loads(internal_path(evidence).read_text(encoding='utf-8'))
|
||||
if (not isinstance(result, dict) or result.get('binding') != asdict(self.binding)
|
||||
or result.get('project_path') != '' or result.get('project_closed') is not True):
|
||||
raise ValueError('project close checkpoint identity mismatch')
|
||||
except (OSError, ValueError, TypeError) as failure:
|
||||
raise CadIdentityConflict('project close checkpoint identity evidence is invalid: ' + str(failure)) from failure
|
||||
if self.cad.poll() is not None:
|
||||
raise WorkerMonitorError('owned CAD exited during project checkpoint')
|
||||
finally:
|
||||
process.close()
|
||||
return result
|
||||
|
||||
def __exit__(self, error_type, error, traceback):
|
||||
closing_started = time.monotonic()
|
||||
@@ -174,7 +241,9 @@ class IsolatedCadSession:
|
||||
finally:
|
||||
try:
|
||||
self.desktop.close()
|
||||
self._record(state='released', clean=error is None and cleanup_error is None,
|
||||
self._record(state='session_closed' if self.borrowed_desktop else 'released',
|
||||
desktop_retained_by_batch=self.borrowed_desktop,
|
||||
clean=error is None and cleanup_error is None,
|
||||
shutdown_seconds=time.monotonic() - closing_started,
|
||||
desktop_cleanup=getattr(self.desktop, 'cleanup_record', None), **exit_codes)
|
||||
except Exception as failure:
|
||||
@@ -190,12 +259,17 @@ class IsolatedCadSession:
|
||||
self._record(state='uncertain', error=message, **details)
|
||||
|
||||
|
||||
def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abort=False):
|
||||
def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abort=False, runtime=None):
|
||||
from .worker_monitor import run_monitored_process
|
||||
if runtime is not None:
|
||||
return runtime.run_worker(command, timeout, abort_event=abort_event,
|
||||
cooperative_abort=cooperative_abort)
|
||||
try:
|
||||
with IsolatedCadSession(Path(command[3]), abort_event=abort_event) as session:
|
||||
return run_monitored_process(command, timeout, abort_event=abort_event, process_factory=session.start_worker,
|
||||
**({'cooperative_abort': True} if cooperative_abort else {}))
|
||||
except CadIdentityConflict:
|
||||
raise
|
||||
except PrivateCadStartupError as error:
|
||||
# The existing handoff recognizes exit=2 with unchanged status as a
|
||||
# retryable pre-mutation startup failure, without an uncertainty lock.
|
||||
@@ -203,13 +277,21 @@ def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abor
|
||||
|
||||
|
||||
def native_action(action, evidence, expected_pid):
|
||||
try:
|
||||
return _run_native_action(action, evidence, expected_pid)
|
||||
except CadIdentityConflict as error:
|
||||
print(str(error), file=sys.stderr)
|
||||
return PRIVATE_CAD_IDENTITY_EXIT_CODE
|
||||
|
||||
|
||||
def _run_native_action(action, evidence, expected_pid):
|
||||
"""Runs in a child created with lpDesktop, never in a Python thread switch."""
|
||||
if action != 'probe':
|
||||
return _native_action(action, evidence, expected_pid)
|
||||
# COM startup can block inside a native call, so keep diagnostics outside
|
||||
# the redirected pipes and retain them when the supervisor ends the child.
|
||||
import faulthandler
|
||||
with Path(str(evidence) + '.fault.txt').open('a', encoding='utf-8') as trace:
|
||||
with internal_path(str(evidence) + '.fault.txt').open('a', encoding='utf-8') as trace:
|
||||
faulthandler.dump_traceback_later(20, repeat=True, file=trace)
|
||||
try:
|
||||
return _native_action(action, evidence, expected_pid)
|
||||
@@ -228,10 +310,10 @@ def _native_action(action, evidence, expected_pid):
|
||||
desktop = os.environ.get('DXPART_DESKTOP', '')
|
||||
token = os.environ.get('DXPART_SESSION_TOKEN', '')
|
||||
if not desktop.startswith('dx-') or current_desktop_name() != desktop:
|
||||
raise ValueError('CAD bootstrap is not on the requested private desktop')
|
||||
raise CadIdentityConflict('CAD bootstrap is not on the requested private desktop')
|
||||
expected_pid = int(expected_pid)
|
||||
if expected_pid <= 0 or str(expected_pid) != os.environ.get('DXPART_CAD_PID'):
|
||||
raise ValueError('invalid CAD bootstrap PID')
|
||||
raise CadIdentityConflict('invalid CAD bootstrap PID')
|
||||
deadline = time.monotonic() + 60
|
||||
while True:
|
||||
try:
|
||||
@@ -250,16 +332,29 @@ def _native_action(action, evidence, expected_pid):
|
||||
# ROT is scoped to this desktop. Certify the freshly launched instance
|
||||
# here; every subsequent VBS/helper attachment checks its GUID.
|
||||
binding = CadBinding(desktop, expected_pid, str(app.Guid()), token)
|
||||
verify_application(app, binding, require_view=False)
|
||||
Path(evidence).write_text(json.dumps(asdict(binding)), encoding='utf-8')
|
||||
elif action == 'quit':
|
||||
binding = CadBinding.from_environment()
|
||||
try:
|
||||
verify_application(app, binding, require_view=False)
|
||||
except ValueError as error:
|
||||
raise CadIdentityConflict(str(error)) from error
|
||||
internal_path(evidence).write_text(json.dumps(asdict(binding)), encoding='utf-8')
|
||||
elif action in {'quit', 'checkpoint'}:
|
||||
try:
|
||||
binding = CadBinding.from_environment()
|
||||
if binding is None:
|
||||
raise ValueError('missing private CAD identity')
|
||||
verify_application(app, binding, require_view=False)
|
||||
opened_project = project_file_path(app)
|
||||
except ValueError as error:
|
||||
raise CadIdentityConflict(str(error)) from error
|
||||
# ActiveViewHandle can retain a destroyed/reused HWND after closing
|
||||
# the last project. The private ROT/GUID binds this empty instance.
|
||||
verify_application(app, binding, require_view=False)
|
||||
if project_file_path(app):
|
||||
raise ValueError('owned CAD still has a project open; no implicit save or discard')
|
||||
app.Quit()
|
||||
if opened_project:
|
||||
raise CadIdentityConflict('owned CAD still has a project open; no implicit save or discard')
|
||||
if action == 'quit':
|
||||
app.Quit()
|
||||
else:
|
||||
from .batch_resources import write_record
|
||||
write_record(evidence, {'binding': asdict(binding), 'project_path': '', 'project_closed': True})
|
||||
else:
|
||||
raise ValueError('unknown private CAD action')
|
||||
except BaseException as failure:
|
||||
|
||||
@@ -8,6 +8,7 @@ from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from contextlib import ExitStack
|
||||
from copy import deepcopy
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
import math
|
||||
@@ -24,6 +25,19 @@ from .errors import TaskCenterError
|
||||
DESKTOP_RELEASE_TIMEOUT_SECONDS = 5.0
|
||||
|
||||
|
||||
def _native_handle(handle):
|
||||
try:
|
||||
return int(handle) if handle is not None else None
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _resource_event(operation, handle, success, error=None):
|
||||
return {'operation': operation, 'handle': _native_handle(handle), 'success': success,
|
||||
'pid': os.getpid(), 'thread_id': get_native_id(), 'time_ns': time.time_ns(),
|
||||
'error': str(error) if error is not None else None}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class StartupInfo:
|
||||
lpDesktop: str
|
||||
@@ -74,9 +88,40 @@ class _Win32Api:
|
||||
self.desktop_release_check['caller_desktop'] = current_desktop_name()
|
||||
except Exception as error:
|
||||
self.desktop_release_check['caller_desktop_error'] = str(error)
|
||||
try:
|
||||
self.desktop_release_check['remaining_windows'] = self._desktop_windows(name)
|
||||
except Exception as error:
|
||||
self.desktop_release_check['window_query_error'] = str(error)
|
||||
raise TaskCenterError('private_desktop_cleanup', 'owned private desktop still exists after cleanup')
|
||||
time.sleep(min(0.05, max(0.0, deadline - time.monotonic())))
|
||||
|
||||
def _desktop_windows(self, name):
|
||||
"""Diagnose residue without switching desktops or terminating foreign owners."""
|
||||
import win32gui
|
||||
|
||||
desktop = self.service.OpenDesktop(name, 0, False, 1) # DESKTOP_READOBJECTS
|
||||
windows = []
|
||||
try:
|
||||
user = ctypes.WinDLL('user32', use_last_error=True)
|
||||
callback_type = ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
|
||||
enumerate_windows = user.EnumDesktopWindows
|
||||
enumerate_windows.argtypes = [wintypes.HANDLE, callback_type, wintypes.LPARAM]
|
||||
enumerate_windows.restype = wintypes.BOOL
|
||||
handles = []
|
||||
callback = callback_type(lambda handle, _: handles.append(handle) or True)
|
||||
# An empty desktop can return FALSE with no last error. Our
|
||||
# callback always continues, so only a nonzero error is a failure.
|
||||
ctypes.set_last_error(0)
|
||||
if not enumerate_windows(int(desktop), callback, 0) and ctypes.get_last_error():
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
for handle in handles:
|
||||
thread, pid = self.process.GetWindowThreadProcessId(handle)
|
||||
windows.append({'hwnd': int(handle), 'pid': pid, 'thread_id': thread,
|
||||
'class': win32gui.GetClassName(handle)})
|
||||
finally:
|
||||
desktop.CloseDesktop()
|
||||
return windows
|
||||
|
||||
def create_process(self, command: Sequence[str], desktop: str, cwd: Path | None, env: Mapping[str, str] | None,
|
||||
*, stdout=None, stderr=None):
|
||||
startup = self.process.STARTUPINFO()
|
||||
@@ -404,6 +449,8 @@ class PrivateDesktop:
|
||||
self._processes: list[subprocess.Popen[str]] = []
|
||||
self._api = api
|
||||
self.cleanup_record = None
|
||||
self._desktop_handle_history = []
|
||||
self._process_history = []
|
||||
|
||||
def __enter__(self) -> "PrivateDesktop":
|
||||
if not self.enabled:
|
||||
@@ -420,7 +467,12 @@ class PrivateDesktop:
|
||||
if os.name != "nt" and self._api is None:
|
||||
return
|
||||
self._api = self._api or _Win32Api()
|
||||
self._handle = self._api.create_desktop(self.name)
|
||||
try:
|
||||
self._handle = self._api.create_desktop(self.name)
|
||||
except Exception as error:
|
||||
self._desktop_handle_history.append(_resource_event('create', None, False, error))
|
||||
raise
|
||||
self._desktop_handle_history.append(_resource_event('create', self._handle, self._handle is not None))
|
||||
|
||||
def start_process(
|
||||
self,
|
||||
@@ -440,8 +492,31 @@ class PrivateDesktop:
|
||||
options = {} if stdout is None and stderr is None else {'stdout': stdout, 'stderr': stderr}
|
||||
process = self._api.create_process(command, build_startup_info(self.name).lpDesktop, cwd, env, **options)
|
||||
self._processes.append(process)
|
||||
self._process_history.append({'pid': getattr(process, 'pid', None),
|
||||
'owner_pid': os.getpid(), 'owner_thread_id': get_native_id(),
|
||||
'started_time_ns': time.time_ns(), 'process_handle': _native_handle(getattr(process, 'handle', None)),
|
||||
'job_handle': _native_handle(getattr(process, 'job', None)), 'forgotten': False})
|
||||
return process
|
||||
|
||||
def _remember_process_cleanup(self, process, *, forgotten=False):
|
||||
for record in reversed(self._process_history):
|
||||
if record['pid'] == getattr(process, 'pid', None) and not record['forgotten']:
|
||||
record['cleanup'] = deepcopy(getattr(process, 'cleanup_record', {}))
|
||||
record['forgotten'] = forgotten
|
||||
record['observed_time_ns'] = time.time_ns()
|
||||
break
|
||||
|
||||
def forget_closed_process(self, process):
|
||||
"""Drop a worker only after its owning wrapper confirmed all cleanup."""
|
||||
record = getattr(process, 'cleanup_record', {})
|
||||
if (record.get('process_handle_closed') is not True
|
||||
or record.get('job_handle_closed') is not True
|
||||
or record.get('unconfirmed_descendant_handles', 0)
|
||||
or record.get('errors') or process.poll() is None):
|
||||
raise OSError('cannot forget a worker with unconfirmed owned resources')
|
||||
self._processes = [item for item in self._processes if item is not process]
|
||||
self._remember_process_cleanup(process, forgotten=True)
|
||||
|
||||
def close(self) -> None:
|
||||
errors, remaining = [], []
|
||||
self.cleanup_record = {'desktop': self.name, 'processes': []}
|
||||
@@ -456,6 +531,7 @@ class PrivateDesktop:
|
||||
failed = True
|
||||
self.cleanup_record['processes'].append(getattr(process, 'cleanup_record', {
|
||||
'pid': getattr(process, 'pid', None), 'exit_code': getattr(process, 'returncode', None)}))
|
||||
self._remember_process_cleanup(process)
|
||||
if failed:
|
||||
remaining.append(process)
|
||||
self._processes = list(reversed(remaining))
|
||||
@@ -467,11 +543,15 @@ class PrivateDesktop:
|
||||
errors.append(error)
|
||||
close_desktop = getattr(self._handle, "CloseDesktop", None)
|
||||
if close_desktop:
|
||||
event = _resource_event('close', self._handle, False)
|
||||
try:
|
||||
close_desktop()
|
||||
event['success'] = True
|
||||
self._handle = None
|
||||
except Exception as error:
|
||||
errors.append(error)
|
||||
event['error'] = str(error)
|
||||
self._desktop_handle_history.append(event)
|
||||
confirm = getattr(self._api, 'confirm_desktop_released', None)
|
||||
if confirm and self._handle is None:
|
||||
try:
|
||||
@@ -480,5 +560,50 @@ class PrivateDesktop:
|
||||
errors.append(error)
|
||||
self.cleanup_record.update(desktop_handle_closed=self._handle is None,
|
||||
desktop_confirmation=getattr(self._api, 'desktop_release_check', None),
|
||||
desktop_handle_history=deepcopy(self._desktop_handle_history),
|
||||
process_history=deepcopy(self._process_history),
|
||||
diagnostic_scope={'external_handle_holders': 'unknown',
|
||||
'thread_desktops': 'unknown unless observed through residual windows; no exhaustive holder scan',
|
||||
'empty_windows_prove_no_holders': False},
|
||||
errors=[str(error) for error in errors])
|
||||
if errors:
|
||||
self.cleanup_record['owned_resource_diagnostics'] = self._owned_resource_diagnostics()
|
||||
_raise_cleanup_errors(errors)
|
||||
|
||||
def _owned_resource_diagnostics(self):
|
||||
"""Bounded queries of our handles only; never terminate/query foreign PIDs.
|
||||
|
||||
GetThreadDesktop can introduce borrowed USER references. Do not run a
|
||||
global thread scan in this long-lived owner while retrying release.
|
||||
Window observations identify only window-owning threads, not every
|
||||
thread or an external OpenDesktop handle holder.
|
||||
"""
|
||||
resources = []
|
||||
for process in self._processes:
|
||||
item = {'pid': getattr(process, 'pid', None),
|
||||
'process_handle': _native_handle(getattr(process, 'handle', None)),
|
||||
'job_handle': _native_handle(getattr(process, 'job', None)),
|
||||
'thread_handle': _native_handle(getattr(process, 'thread', None)),
|
||||
'cleanup': deepcopy(getattr(process, 'cleanup_record', {}))}
|
||||
try:
|
||||
item['exit_code'] = process.poll()
|
||||
except Exception as error:
|
||||
item['poll_error'] = str(error)
|
||||
job = getattr(process, 'job', None)
|
||||
job_api = getattr(process, '_job_api', None)
|
||||
if job is not None and job_api is not None:
|
||||
try:
|
||||
item['job_accounting'] = job_api.QueryInformationJobObject(
|
||||
job, job_api.JobObjectBasicAccountingInformation)
|
||||
except Exception as error:
|
||||
item['job_query_error'] = str(error)
|
||||
resources.append(item)
|
||||
confirmation = getattr(self._api, 'desktop_release_check', None) or {}
|
||||
return {'owned_processes': resources,
|
||||
'pending_launch_cleanup_count': len(getattr(self._api, '_pending_cleanup', [])),
|
||||
'residual_windows': deepcopy(confirmation.get('remaining_windows')),
|
||||
'window_query_error': confirmation.get('window_query_error'),
|
||||
'external_handle_holders': 'unknown',
|
||||
'thread_desktops': {'observations': deepcopy(confirmation.get('remaining_windows', [])),
|
||||
'complete': False, 'unobserved_threads': 'unknown'},
|
||||
'empty_windows_prove_no_holders': False}
|
||||
@@ -11,6 +11,7 @@ import tempfile
|
||||
from threading import RLock
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, retry_sharing_violation, strip_extended_prefix
|
||||
|
||||
|
||||
def _short_path(path: Path) -> str | None:
|
||||
@@ -18,8 +19,8 @@ def _short_path(path: Path) -> str | None:
|
||||
function.argtypes = [ctypes.c_wchar_p, ctypes.c_wchar_p, ctypes.c_uint]
|
||||
function.restype = ctypes.c_uint
|
||||
buffer = ctypes.create_unicode_buffer(32768)
|
||||
count = function(str(path), buffer, len(buffer))
|
||||
return buffer.value if count and count < len(buffer) else None
|
||||
count = function(str(internal_path(path)), buffer, len(buffer))
|
||||
return strip_extended_prefix(buffer.value) if count and count < len(buffer) else None
|
||||
|
||||
|
||||
def _fits(root) -> bool:
|
||||
@@ -34,12 +35,13 @@ class _AliasPool:
|
||||
self.lock = RLock()
|
||||
|
||||
def alias(self, root: Path) -> Path:
|
||||
root = canonical_path(root, strict=True)
|
||||
import _winapi
|
||||
with self.lock:
|
||||
if self.directory is None:
|
||||
local = Path(os.environ.get('LOCALAPPDATA') or tempfile.gettempdir())
|
||||
parent = local / 'DxProductivity' / 'path-links'
|
||||
parent.mkdir(parents=True, exist_ok=True)
|
||||
internal_path(parent).mkdir(parents=True, exist_ok=True)
|
||||
self.directory = Path(tempfile.mkdtemp(prefix='dxp-', dir=parent))
|
||||
key = hashlib.sha256(os.path.normcase(str(root)).encode('utf-8')).hexdigest()[:20]
|
||||
link = self.directory / key
|
||||
@@ -50,7 +52,7 @@ class _AliasPool:
|
||||
# junction contains no copies and never changes the target.
|
||||
_winapi.CreateJunction(str(root), str(link))
|
||||
self.links[link] = root
|
||||
if not link.is_junction() or link.resolve(strict=True) != root:
|
||||
if not link.is_junction() or canonical_path(link, strict=True) != root:
|
||||
raise OSError('compatibility directory no longer points to the confirmed evidence')
|
||||
return link
|
||||
|
||||
@@ -60,7 +62,7 @@ class _AliasPool:
|
||||
for link, target in tuple(self.links.items()):
|
||||
try:
|
||||
if link.is_junction():
|
||||
if link.resolve() != target:
|
||||
if canonical_path(link) != target:
|
||||
raise OSError('refusing to remove a redirected compatibility directory')
|
||||
link.rmdir() # Remove only the link, never the target tree.
|
||||
elif link.exists():
|
||||
@@ -88,11 +90,12 @@ atexit.register(cleanup_worker_aliases)
|
||||
|
||||
|
||||
def worker_paths(script: Path, root: Path) -> tuple[str, str]:
|
||||
script, root = canonical_path(script), canonical_path(root)
|
||||
if os.name == 'nt':
|
||||
_prepare_project_access(root)
|
||||
if os.name != 'nt' or _fits(root):
|
||||
return str(script), str(root)
|
||||
canonical = root.resolve(strict=True)
|
||||
canonical = canonical_path(internal_path(root), strict=True)
|
||||
short = _short_path(canonical)
|
||||
if short and _fits(short):
|
||||
return str(Path(short) / script.name), short
|
||||
@@ -106,7 +109,7 @@ def worker_paths(script: Path, root: Path) -> tuple[str, str]:
|
||||
|
||||
def _prepare_project_access(root: Path):
|
||||
"""Keep command identities canonical while giving legacy CAD short entry paths."""
|
||||
command = root / 'command.tsv'
|
||||
command = internal_path(root / 'command.tsv')
|
||||
if not command.is_file():
|
||||
return
|
||||
rows = []
|
||||
@@ -117,32 +120,32 @@ def _prepare_project_access(root: Path):
|
||||
if not value or value in seen:
|
||||
continue
|
||||
seen.add(value)
|
||||
project = Path(value).resolve(strict=False)
|
||||
if (_fits(project.parent) and len(str(project)) < 260) or not project.is_file():
|
||||
project = canonical_path(internal_path(value))
|
||||
if (_fits(project.parent) and len(str(project)) < 260) or not internal_path(project).is_file():
|
||||
continue
|
||||
short = _short_path(project)
|
||||
if short and _fits(Path(short).parent) and len(short) < 260:
|
||||
access = Path(short)
|
||||
else:
|
||||
access = _aliases.alias(project.parent) / project.name
|
||||
if not _fits(access.parent) or len(str(access)) >= 260 or access.resolve(strict=True) != project:
|
||||
if not _fits(access.parent) or len(str(access)) >= 260 or canonical_path(access, strict=True) != project:
|
||||
raise TaskCenterError('quick_run_path_length', '工程短路径入口不能与原工程身份对应,未启动 CAD 操作。')
|
||||
pair = (str(project), str(access))
|
||||
if pair not in rows:
|
||||
rows.append(pair)
|
||||
target = root / 'project-access.tsv'
|
||||
target = internal_path(root / 'project-access.tsv')
|
||||
if rows or target.exists():
|
||||
temporary = target.with_suffix('.tmp')
|
||||
with temporary.open('w', encoding='utf-8', newline='') as stream:
|
||||
writer = csv.writer(stream, delimiter='\t', lineterminator='\n')
|
||||
writer.writerow(('project_path', 'access_path'))
|
||||
writer.writerows(rows)
|
||||
temporary.replace(target)
|
||||
retry_sharing_violation(lambda: temporary.replace(target))
|
||||
|
||||
|
||||
def validate_project_access(root: Path):
|
||||
"""Reject disappeared/redirected entry paths before and after a native worker."""
|
||||
target = Path(root) / 'project-access.tsv'
|
||||
target = internal_path(Path(root) / 'project-access.tsv')
|
||||
if not target.is_file():
|
||||
return
|
||||
with target.open(encoding='utf-8-sig', newline='') as stream:
|
||||
@@ -151,7 +154,7 @@ def validate_project_access(root: Path):
|
||||
raise TaskCenterError('target_identity', '工程路径入口清单格式已改变')
|
||||
for row in reader:
|
||||
try:
|
||||
same = Path(row['access_path']).resolve(strict=True) == Path(row['project_path']).resolve(strict=True)
|
||||
same = canonical_path(internal_path(row['access_path']), strict=True) == canonical_path(internal_path(row['project_path']), strict=True)
|
||||
except (OSError, ValueError, KeyError):
|
||||
same = False
|
||||
if not same:
|
||||
|
||||
@@ -18,6 +18,7 @@ import traceback
|
||||
from typing import Any, Mapping
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import internal_path
|
||||
from .retention import rotate_text_log
|
||||
|
||||
|
||||
@@ -75,24 +76,28 @@ class LoggingContext:
|
||||
return reported
|
||||
|
||||
|
||||
def configure_logging(root: Path) -> LoggingContext:
|
||||
def configure_logging(root: Path, *, log_path: Path | None = None) -> LoggingContext:
|
||||
"""Create storage directories and a UTF-8 rotating application log handler."""
|
||||
storage_root = Path(root).resolve()
|
||||
log_path = storage_root / "logs" / "application.log"
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
(storage_root / "tasks").mkdir(parents=True, exist_ok=True)
|
||||
from .path_access import canonical_path, internal_path
|
||||
storage_root = canonical_path(root)
|
||||
separate_log = log_path is not None
|
||||
log_path = canonical_path(log_path) if separate_log else storage_root / "logs" / "application.log"
|
||||
internal_path(log_path.parent).mkdir(parents=True, exist_ok=True)
|
||||
internal_path(storage_root / "tasks").mkdir(parents=True, exist_ok=True)
|
||||
logger_name = "dx_productivity.trial_task_center." + hashlib.sha256(
|
||||
str(storage_root.resolve()).encode("utf-8")
|
||||
).hexdigest()[:16]
|
||||
if separate_log:
|
||||
logger_name += '.' + hashlib.sha256(str(log_path).encode('utf-8')).hexdigest()[:12]
|
||||
logger = logging.getLogger(logger_name)
|
||||
logger.setLevel(logging.DEBUG)
|
||||
logger.propagate = False
|
||||
if not any(
|
||||
isinstance(handler, RotatingFileHandler) and Path(handler.baseFilename) == log_path
|
||||
isinstance(handler, RotatingFileHandler) and canonical_path(handler.baseFilename) == log_path
|
||||
for handler in logger.handlers
|
||||
):
|
||||
handler = RotatingFileHandler(
|
||||
log_path,
|
||||
internal_path(log_path),
|
||||
maxBytes=_MAX_LOG_BYTES,
|
||||
backupCount=_BACKUP_COUNT,
|
||||
encoding="utf-8",
|
||||
@@ -151,11 +156,11 @@ def append_task_event(
|
||||
encoded = json.dumps(event, ensure_ascii=False, separators=(",", ":"), allow_nan=False) + "\n"
|
||||
try:
|
||||
rotate_text_log(
|
||||
path,
|
||||
internal_path(path),
|
||||
max_bytes=_MAX_EVENT_LOG_BYTES,
|
||||
backup_count=_EVENT_BACKUP_COUNT,
|
||||
)
|
||||
with path.open("a", encoding="utf-8", newline="\n") as handle:
|
||||
with internal_path(path).open("a", encoding="utf-8", newline="\n") as handle:
|
||||
handle.write(encoded)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
@@ -183,7 +188,7 @@ def append_task_event(
|
||||
|
||||
def _event_file_stamp(path):
|
||||
try:
|
||||
info = path.stat()
|
||||
info = internal_path(path).stat()
|
||||
return info.st_dev, info.st_ino, info.st_size, info.st_mtime_ns, info.st_ctime_ns
|
||||
except OSError:
|
||||
return None
|
||||
@@ -226,15 +231,15 @@ def _task_directory(context: LoggingContext, task_id: str) -> Path:
|
||||
if not task_id or Path(task_id).name != task_id or task_id in {".", ".."}:
|
||||
raise TaskCenterError("task_id", "task event requires a safe task identifier")
|
||||
path = context.root / "tasks" / task_id
|
||||
if not path.is_dir():
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
if not internal_path(path).is_dir():
|
||||
internal_path(path).mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def _next_event_sequence(path: Path) -> int:
|
||||
try:
|
||||
for candidate in (path, path.with_name(f"{path.name}.1")):
|
||||
if not candidate.exists():
|
||||
if not internal_path(candidate).exists():
|
||||
continue
|
||||
lines = _tail_utf8_lines(candidate, 1)
|
||||
if lines:
|
||||
@@ -248,7 +253,7 @@ def _next_event_sequence(path: Path) -> int:
|
||||
def _tail_utf8_lines(path: Path, limit: int) -> tuple[str, ...]:
|
||||
if limit < 1:
|
||||
return ()
|
||||
with Path(path).open("rb") as handle:
|
||||
with internal_path(Path(path)).open("rb") as handle:
|
||||
handle.seek(0, os.SEEK_END)
|
||||
cursor = handle.tell()
|
||||
chunks = []
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
import argparse
|
||||
import faulthandler
|
||||
import ctypes
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -13,6 +14,48 @@ import time
|
||||
import traceback
|
||||
|
||||
|
||||
def snapshot_attempt(snapshot):
|
||||
"""Bind batch evidence to the actual snapshot and its confirmed queue row."""
|
||||
from .batch_attempt import _bound_attempt
|
||||
if 'DXPART_ATTEMPT_FILE' not in os.environ:
|
||||
return _bound_attempt()
|
||||
snapshot = Path(snapshot)
|
||||
metadata, _ = read_snapshot(snapshot)
|
||||
if not snapshot.name.startswith('snapshot-') or snapshot.suffix != '.tsv':
|
||||
raise ValueError('batch snapshot has no job identity')
|
||||
job_id = snapshot.stem[len('snapshot-'):]
|
||||
with (snapshot.parent / 'command.tsv').open(encoding='utf-8-sig', newline='') as handle:
|
||||
commands = [row for row in csv.DictReader(handle, delimiter='\t') if row.get('job_id') == job_id]
|
||||
if len(commands) != 1:
|
||||
raise ValueError('batch snapshot must match one confirmed command')
|
||||
command = commands[0]
|
||||
if os.path.normcase(os.path.abspath(command['project_path'])) != os.path.normcase(os.path.abspath(metadata[1])):
|
||||
raise ValueError('batch command and snapshot projects differ')
|
||||
return _bound_attempt(job_id=job_id, task_id=command['task_id'], project_path=metadata[1])
|
||||
|
||||
|
||||
def mark_snapshot_mutation(snapshot):
|
||||
from .batch_attempt import mark_mutation_possible
|
||||
bound = snapshot_attempt(snapshot)
|
||||
if bound is None:
|
||||
return None
|
||||
path, record = bound
|
||||
return mark_mutation_possible(path, job_id=record['job_id'], task_id=record['task_id'],
|
||||
project_path=record['project_path'])
|
||||
|
||||
|
||||
def run_attempt_marker(*, job_id, task_id, project_path):
|
||||
from .batch_attempt import mark_mutation_possible
|
||||
if not job_id or not task_id or not project_path:
|
||||
return 2
|
||||
try:
|
||||
mark_mutation_possible(job_id=job_id, task_id=task_id, project_path=project_path)
|
||||
return 0
|
||||
except (OSError, ValueError, KeyError, TypeError) as error:
|
||||
print(f'Batch mutation proof failed: {error}', file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def decode(value):
|
||||
characters = []
|
||||
index = 0
|
||||
@@ -542,6 +585,7 @@ def run_native(snapshot, report):
|
||||
slot_count = int(binding.get("slot_count", report.get("slot_count", 1)))
|
||||
if slot_count < 1:
|
||||
raise ValueError("Recorded replacement has no valid slots")
|
||||
mark_snapshot_mutation(snapshot)
|
||||
report["promoted_pins"] = promote_pin_numbers(app, api, report["replaced_uids"], slot_count,
|
||||
lambda phase: _promotion_phase(snapshot, report, phase))
|
||||
report["already_replaced"] = True
|
||||
@@ -706,6 +750,7 @@ def run_native(snapshot, report):
|
||||
cell_catalog=cell_proof)
|
||||
report["stage"] = "native_replace_started"
|
||||
save_report(snapshot, report)
|
||||
mark_snapshot_mutation(snapshot)
|
||||
click(dlg_item(dialog, 1695))
|
||||
|
||||
replacement_poll_count = 0
|
||||
@@ -820,6 +865,15 @@ def run(snapshot):
|
||||
watchdog.daemon = True
|
||||
watchdog.start()
|
||||
try:
|
||||
bound = snapshot_attempt(snapshot)
|
||||
if bound is not None:
|
||||
_, current = bound
|
||||
identity = {key: current[key] for key in
|
||||
('session_id', 'generation', 'attempt_id', 'task_id', 'job_id', 'project_path')}
|
||||
if report.get('batch_attempt') != identity:
|
||||
report.clear()
|
||||
report.update(snapshot=str(snapshot), ok=False, stage='preflight')
|
||||
report['batch_attempt'] = identity
|
||||
run_native(snapshot, report)
|
||||
report["ok"] = True
|
||||
return 0
|
||||
|
||||
@@ -7,31 +7,42 @@ import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, retry_sharing_violation
|
||||
from .isolated_desktop import PrivateDesktop
|
||||
from .native_inventory_protocol import NativeInventoryUnsupported, read_ascii_discovery, read_native_text
|
||||
from .operation_control import check_cancelled, report_progress
|
||||
|
||||
|
||||
_RETRY_NATIVE_READ = object()
|
||||
_NATIVE_CRASH_CODES = frozenset({0xC0000005, 0xC000001D, 0xC0000094,
|
||||
0xC00000FD, 0xC0000374, 0xC0000409})
|
||||
|
||||
|
||||
class _NativeInventoryProcessError(NativeInventoryUnsupported):
|
||||
"""A launch, watchdog or explicit crash failure, separate from invalid input."""
|
||||
|
||||
|
||||
def _hash(path):
|
||||
with path.open('rb') as stream:
|
||||
with internal_path(path).open('rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
|
||||
|
||||
def _verify_sources(hashes):
|
||||
try:
|
||||
if any(path.resolve(strict=True) != path or _hash(path) != digest for path, digest in hashes.items()):
|
||||
if any(canonical_path(internal_path(path), strict=True) != path or _hash(path) != digest for path, digest in hashes.items()):
|
||||
raise OSError('source identity changed')
|
||||
except OSError as error:
|
||||
raise TaskCenterError('target_identity', '原生采集期间源工程或数据库已变化或无法核验,请重新导入') from error
|
||||
|
||||
|
||||
def _project_options(project):
|
||||
text = read_native_text(project)
|
||||
text = read_native_text(internal_path(project))
|
||||
def option(name, default=None):
|
||||
values = set(re.findall(r'^KEY ' + name + r' "([^"\r\n]*)"\s*$', text, re.MULTILINE))
|
||||
if not values and default is not None:
|
||||
@@ -45,8 +56,8 @@ def _project_options(project):
|
||||
snapshot = option('FrontEndSnapshot', 'DxD')
|
||||
if snapshot.casefold() != 'dxd':
|
||||
raise NativeInventoryUnsupported('当前工程不是已验证的 DxD 数据库快照')
|
||||
database = (project.parent / option('iCDBDir') / 'icdb.dat').resolve(strict=True)
|
||||
if not database.is_relative_to(project.parent) or not database.is_file():
|
||||
database = canonical_path(internal_path(project.parent / option('iCDBDir') / 'icdb.dat'), strict=True)
|
||||
if not database.is_relative_to(project.parent) or not internal_path(database).is_file():
|
||||
raise NativeInventoryUnsupported('数据库不在当前工程目录内')
|
||||
return text, block, snapshot, database
|
||||
|
||||
@@ -71,27 +82,33 @@ class NativeInventoryReader:
|
||||
raise NativeInventoryUnsupported('本机未安装完整的原生器件清单工具')
|
||||
return tools
|
||||
|
||||
def read(self, project, old_part_numbers, *, stop_event=None, on_progress=None):
|
||||
def read(self, project, old_part_numbers, *, stop_event=None, on_progress=None, runtime=None):
|
||||
check_cancelled(stop_event)
|
||||
try:
|
||||
project = Path(project).resolve(strict=True)
|
||||
project = canonical_path(internal_path(project), strict=True)
|
||||
text, block, snapshot, database = _project_options(project)
|
||||
tools = self._tools()
|
||||
except (NativeInventoryUnsupported, OSError, ImportError, UnicodeError, TaskCenterError) as error:
|
||||
report_progress(on_progress, f'使用 CAD 核验位号:{str(error)[:160]}')
|
||||
return None
|
||||
return self._read(project, text, block, snapshot, database, tools, old_part_numbers,
|
||||
stop_event=stop_event, on_progress=on_progress)
|
||||
for attempt in range(2):
|
||||
result = self._read(project, text, block, snapshot, database, tools, old_part_numbers,
|
||||
stop_event=stop_event, on_progress=on_progress, runtime=runtime,
|
||||
allow_retry=runtime is not None and attempt == 0)
|
||||
if result is not _RETRY_NATIVE_READ:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _read(self, project, text, block, snapshot, database, tools, old_part_numbers, *, stop_event, on_progress):
|
||||
def _read(self, project, text, block, snapshot, database, tools, old_part_numbers, *, stop_event, on_progress, runtime=None, allow_retry=False):
|
||||
token = uuid4().hex
|
||||
evidence = self.data_root / 'quick-run-history' / ('native-inventory-' + token)
|
||||
evidence.mkdir(parents=True, exist_ok=False)
|
||||
evidence = internal_path(self.data_root / 'quick-run-history' / ('native-inventory-' + token))
|
||||
retry_sharing_violation(lambda: evidence.mkdir(parents=True, exist_ok=False))
|
||||
base = Path(self.temporary_root or Path(os.environ.get('LOCALAPPDATA') or tempfile.gettempdir())
|
||||
/ 'DxProductivity' / 'native-inventory').resolve()
|
||||
base.mkdir(parents=True, exist_ok=True)
|
||||
workspace = Path(tempfile.mkdtemp(prefix='dx-cli-', dir=base)).resolve()
|
||||
desktop = self.desktop_factory('dx-cli-' + token, enabled=True)
|
||||
desktop = (runtime.native_scope(canonical_path(evidence)) if runtime is not None
|
||||
else self.desktop_factory('dx-cli-' + token, enabled=True))
|
||||
record = {'backend': 'native_cli', 'project': str(project), 'state': 'reading',
|
||||
'workspace': str(workspace), 'tools': []}
|
||||
started = time.monotonic()
|
||||
@@ -100,12 +117,12 @@ class NativeInventoryReader:
|
||||
source_hashes = {}
|
||||
try:
|
||||
source_hashes = {project: _hash(project), database: _hash(database)}
|
||||
if read_native_text(project) != text:
|
||||
if read_native_text(internal_path(project)) != text:
|
||||
raise TaskCenterError('target_identity', '创建原生采集快照时工程配置已变化,请重新导入')
|
||||
source = workspace / 'input.prj'
|
||||
target_database = workspace / 'database'
|
||||
target_database.mkdir()
|
||||
shutil.copyfile(database, target_database / 'icdb.dat')
|
||||
shutil.copyfile(internal_path(database), target_database / 'icdb.dat')
|
||||
# No server address files or running database directory are copied.
|
||||
text = re.sub(r'^KEY iCDBDir "[^"\r\n]*"\s*$',
|
||||
'KEY iCDBDir "database"', text, flags=re.MULTILINE)
|
||||
@@ -130,17 +147,21 @@ class NativeInventoryReader:
|
||||
self._run_tool(desktop, command, workspace, environment, output,
|
||||
record, stop_event)
|
||||
finally:
|
||||
primary_error = sys.exception()
|
||||
try:
|
||||
desktop.close()
|
||||
clean = True
|
||||
except Exception as error:
|
||||
record.update(state='uncertain', cleanup_error=str(error))
|
||||
record.update(state='uncertain', cleanup_error=str(error),
|
||||
primary_error=str(primary_error) if primary_error is not None else None)
|
||||
try:
|
||||
(evidence / 'uncertain.flag').write_text(str(error), encoding='utf-8')
|
||||
except OSError as marker_error:
|
||||
record['uncertain_marker_error'] = str(marker_error)
|
||||
raise TaskCenterError('inventory_cleanup',
|
||||
f'原生采集进程未能确认安全释放,请保留诊断记录:{evidence}') from error
|
||||
failure = TaskCenterError('inventory_cleanup',
|
||||
f'原生采集进程未能确认安全释放,请保留诊断记录:{canonical_path(evidence)}')
|
||||
failure.primary_error, failure.cleanup_error = primary_error, error
|
||||
raise failure from error
|
||||
_verify_sources(source_hashes)
|
||||
check_cancelled(stop_event)
|
||||
result = read_ascii_discovery(project, output / 'scope.txt', old_part_numbers,
|
||||
@@ -151,9 +172,40 @@ class NativeInventoryReader:
|
||||
if record['state'] == 'uncertain':
|
||||
raise TaskCenterError('inventory_cleanup', f'原生采集清理状态不确定:{evidence}') from error
|
||||
_verify_sources(source_hashes)
|
||||
record.update(state='fallback', reason=str(error))
|
||||
report_progress(on_progress, f'原生清单未能完整核对,改用 CAD 采集:{str(error)[:160]}')
|
||||
record.update(source_hashes_unchanged=True,
|
||||
source_hashes={str(path): digest for path, digest in source_hashes.items()})
|
||||
retry = False
|
||||
cleanup = getattr(desktop, 'cleanup_record', None)
|
||||
session_failure = runtime is not None and isinstance(error, _NativeInventoryProcessError)
|
||||
if (session_failure
|
||||
and clean and isinstance(cleanup, dict) and cleanup.get('workers_closed') is True):
|
||||
check_cancelled(stop_event)
|
||||
from .batch_resources import write_record
|
||||
write_record(evidence / 'native-retry-proof.json', {
|
||||
'version': 1, 'native_read_only': True, 'project_path': str(project),
|
||||
'source_hashes': {str(path): digest for path, digest in source_hashes.items()},
|
||||
'workers_closed': True, 'evidence_root': str(canonical_path(evidence)),
|
||||
'binding': dict(desktop.identity),
|
||||
})
|
||||
_verify_sources(source_hashes)
|
||||
if allow_retry:
|
||||
retry = runtime.recover_native_failure(project, evidence, str(error))
|
||||
if session_failure and not retry:
|
||||
record.update(state='session_failed', reason=str(error))
|
||||
raise TaskCenterError('inventory_session',
|
||||
f'原生读取会话异常,未能完成安全重试,批次已停止。请保留诊断记录后排查:{evidence};{error}') from error
|
||||
record.update(state='retrying' if retry else 'fallback', reason=str(error))
|
||||
if retry:
|
||||
result = _RETRY_NATIVE_READ
|
||||
report_progress(on_progress, '原生读取会话已安全释放,正在重新核验一次…')
|
||||
else:
|
||||
report_progress(on_progress, f'原生清单未能完整核对,改用 CAD 采集:{str(error)[:160]}')
|
||||
except TaskCenterError as error:
|
||||
if record['state'] == 'uncertain':
|
||||
try:
|
||||
(evidence / 'uncertain.flag').write_text(error.message, encoding='utf-8')
|
||||
except OSError as marker_error:
|
||||
record['uncertain_marker_error'] = str(marker_error)
|
||||
if record['state'] != 'uncertain':
|
||||
record.update(state='cancelled' if error.code == 'operation_cancelled' else 'failed', reason=error.message)
|
||||
raise
|
||||
@@ -181,19 +233,36 @@ class NativeInventoryReader:
|
||||
started = time.monotonic()
|
||||
metric = {'name': name, 'report': report_name, 'exit_code': None}
|
||||
record['tools'].append(metric)
|
||||
process = None
|
||||
try:
|
||||
with (output / (log_name + '.stdout.txt')).open('wb') as stdout, (
|
||||
output / (log_name + '.stderr.txt')).open('wb') as stderr:
|
||||
process = desktop.start_process(command, cwd=workspace, env=environment,
|
||||
stdout=stdout, stderr=stderr)
|
||||
try:
|
||||
process = desktop.start_process(command, cwd=workspace, env=environment,
|
||||
stdout=stdout, stderr=stderr)
|
||||
except OSError as error:
|
||||
raise _NativeInventoryProcessError(f'{name} 启动失败:{error}') from error
|
||||
deadline = started + self.timeout
|
||||
while process.poll() is None:
|
||||
check_cancelled(stop_event)
|
||||
if time.monotonic() >= deadline:
|
||||
raise NativeInventoryUnsupported(f'{name} 原生读取超时')
|
||||
raise _NativeInventoryProcessError(f'{name} 原生读取超时')
|
||||
time.sleep(0.05)
|
||||
metric['exit_code'] = process.poll()
|
||||
if metric['exit_code'] & 0xFFFFFFFF in _NATIVE_CRASH_CODES:
|
||||
raise _NativeInventoryProcessError(f'{name} 原生读取进程崩溃:{metric["exit_code"]}')
|
||||
if metric['exit_code'] != 0:
|
||||
raise NativeInventoryUnsupported(f'{name} 返回退出码 {metric["exit_code"]}')
|
||||
finally:
|
||||
primary_error = sys.exception()
|
||||
metric['seconds'] = time.monotonic() - started
|
||||
if process is not None and callable(getattr(process, 'close', None)):
|
||||
try:
|
||||
process.close()
|
||||
except Exception as error:
|
||||
record.update(state='uncertain', cleanup_error=str(error),
|
||||
primary_error=str(primary_error) if primary_error is not None else None)
|
||||
failure = TaskCenterError('inventory_cleanup',
|
||||
f'原生读取进程及其子进程未能确认退出:{error}')
|
||||
failure.primary_error, failure.cleanup_error = primary_error, error
|
||||
raise failure from error
|
||||
@@ -41,6 +41,7 @@ def _effective_properties(text):
|
||||
if len(re.findall(r'^Config .+', text, re.MULTILINE)) != 1:
|
||||
raise NativeInventoryUnsupported('展开报告必须具有唯一配置')
|
||||
result, attributes, paths = {}, {}, []
|
||||
inconsistent_refs = set()
|
||||
symbol = None
|
||||
|
||||
def finish():
|
||||
@@ -54,7 +55,7 @@ def _effective_properties(text):
|
||||
if uid in result or paths != ['']:
|
||||
raise NativeInventoryUnsupported('展开报告含重复或嵌套实例,需由 CAD 核验')
|
||||
if ref != attributes.get('Ref Designator', ''):
|
||||
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
|
||||
inconsistent_refs.add(uid)
|
||||
result[uid] = dict(attributes)
|
||||
if len(result) > 100000:
|
||||
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
|
||||
@@ -76,10 +77,10 @@ def _effective_properties(text):
|
||||
raise NativeInventoryUnsupported('展开报告的器件属性不唯一')
|
||||
attributes[key] = value or ''
|
||||
finish()
|
||||
return result
|
||||
return result, inconsistent_refs
|
||||
|
||||
|
||||
def _parse_scope(text, requested, effective):
|
||||
def _parse_scope(text, requested, effective, inconsistent_refs):
|
||||
blocks, sheets, targets = set(), set(), set()
|
||||
seen_uids = set()
|
||||
block, symbol, attributes = '', None, {}
|
||||
@@ -100,6 +101,11 @@ def _parse_scope(text, requested, effective):
|
||||
if 'Value' in attributes and 'Value' not in values:
|
||||
raise NativeInventoryUnsupported('展开报告缺少 Value 生效属性')
|
||||
if values.get('Part Number', '').casefold() in requested:
|
||||
# A library reference override on an unrelated part cannot affect
|
||||
# the requested identities. Still validate every UID/config/path,
|
||||
# and require CAD whenever the ambiguous component is requested.
|
||||
if uid in inconsistent_refs:
|
||||
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
|
||||
ref, part = values.get('Ref Designator', ''), values['Part Number']
|
||||
if not ref or 'Ref Designator' not in attributes:
|
||||
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
|
||||
@@ -146,9 +152,9 @@ def read_ascii_discovery(project, report_path, old_part_numbers, *, expected_roo
|
||||
_require_report(text, 'HIER BLOCKS')
|
||||
if effective_report_path is None:
|
||||
raise NativeInventoryUnsupported('缺少实例生效属性报告,需由 CAD 核验')
|
||||
effective = _effective_properties(read_native_text(effective_report_path))
|
||||
effective, inconsistent_refs = _effective_properties(read_native_text(effective_report_path))
|
||||
requested = {part.casefold() for part in old_part_numbers}
|
||||
blocks, available, targets = _parse_scope(text, requested, effective)
|
||||
blocks, available, targets = _parse_scope(text, requested, effective, inconsistent_refs)
|
||||
if expected_root is not None and expected_root not in blocks:
|
||||
raise NativeInventoryUnsupported('原生全工程报告与工程根块身份不一致')
|
||||
components = tuple(ComponentTarget(Path(project), *row) for row in sorted(targets))
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Filesystem access paths never replace canonical project/resource identities."""
|
||||
from __future__ import annotations
|
||||
|
||||
import ntpath
|
||||
import os
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
|
||||
def strip_extended_prefix(value):
|
||||
value = os.fspath(value)
|
||||
if value.startswith('\\\\?\\UNC\\'):
|
||||
return '\\\\' + value[8:]
|
||||
if value.startswith('\\\\?\\'):
|
||||
return value[4:]
|
||||
return value
|
||||
|
||||
|
||||
def windows_extended_path(value):
|
||||
value = os.fspath(value)
|
||||
if value.startswith('\\\\?\\'):
|
||||
return value
|
||||
value = ntpath.abspath(value)
|
||||
return '\\\\?\\UNC\\' + value[2:] if value.startswith('\\\\') else '\\\\?\\' + value
|
||||
|
||||
|
||||
def canonical_path(value, *, strict=False):
|
||||
return Path(strip_extended_prefix(internal_path(value).resolve(strict=strict)))
|
||||
|
||||
|
||||
def internal_path(value):
|
||||
"""Use Unicode extended Windows paths only at internal filesystem boundaries."""
|
||||
return Path(windows_extended_path(value)) if os.name == 'nt' else Path(value)
|
||||
|
||||
|
||||
def _sharing_denial(error, target):
|
||||
if getattr(error, 'winerror', None) in (32, 33):
|
||||
return True
|
||||
if os.name != 'nt' or getattr(error, 'winerror', None) != 5 or target is None:
|
||||
return False
|
||||
# MoveFileEx can report ACCESS_DENIED for a reader denying delete sharing.
|
||||
# Confirm that specific condition, rather than retrying arbitrary ACL errors.
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
kernel = ctypes.WinDLL('kernel32', use_last_error=True)
|
||||
create = kernel.CreateFileW
|
||||
create.argtypes = (wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD, ctypes.c_void_p,
|
||||
wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE)
|
||||
create.restype = wintypes.HANDLE
|
||||
handle = create(str(internal_path(target)), 0x10000, 7, None, 3, 0, None)
|
||||
if handle == ctypes.c_void_p(-1).value:
|
||||
return ctypes.get_last_error() in (32, 33)
|
||||
close = kernel.CloseHandle
|
||||
close.argtypes = (wintypes.HANDLE,)
|
||||
close(handle)
|
||||
return False
|
||||
|
||||
|
||||
def retry_sharing_violation(operation, *, target=None, sleep=time.sleep):
|
||||
"""At most three attempts and 150ms delay; access denial is never replayed."""
|
||||
for attempt in range(3):
|
||||
try:
|
||||
return operation()
|
||||
except OSError as error:
|
||||
if attempt == 2 or not _sharing_denial(error, target):
|
||||
raise
|
||||
sleep(.05 * (attempt + 1))
|
||||
|
||||
|
||||
def legacy_path(value):
|
||||
"""Give CAD/WSH a short entry while retaining canonical identity elsewhere."""
|
||||
from .legacy_paths import _aliases, _fits, _short_path
|
||||
path = canonical_path(value, strict=True)
|
||||
if os.name != 'nt' or (_fits(path.parent) and len(str(path)) < 260):
|
||||
return path
|
||||
short = _short_path(internal_path(path))
|
||||
if short and _fits(Path(short).parent) and len(short) < 260:
|
||||
access = Path(short)
|
||||
elif path.is_dir():
|
||||
access = _aliases.alias(path)
|
||||
else:
|
||||
access = _aliases.alias(path.parent) / path.name
|
||||
if canonical_path(access, strict=True) != path:
|
||||
raise OSError('compatibility entry does not identify the canonical path')
|
||||
return access
|
||||
@@ -3,16 +3,42 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, retry_sharing_violation
|
||||
from .quick_run_protocol import ProjectDiscovery, _encode_vbs_field
|
||||
from .operation_control import check_cancelled, report_progress
|
||||
|
||||
|
||||
class ReleasedProjectReadError(TaskCenterError):
|
||||
"""Project read failed, with positive evidence that its CAD session released."""
|
||||
|
||||
|
||||
def _read_failure(root, message):
|
||||
error_type = TaskCenterError
|
||||
if (root / 'project-checkpoint.json').exists():
|
||||
from .batch_runtime import project_checkpoint_released
|
||||
if project_checkpoint_released(root):
|
||||
error_type = ReleasedProjectReadError
|
||||
return error_type('quick_run_sheet_discovery', message)
|
||||
try:
|
||||
record = json.loads((root / 'private-desktop.json').read_text(encoding='utf-8'))
|
||||
cleanup = record['desktop_cleanup']
|
||||
if (record['state'] == 'released' and record['clean'] is True
|
||||
and record['cad_exit_code'] == 0 and record['quit_helper_exit_code'] == 0
|
||||
and cleanup['desktop_handle_closed'] is True and cleanup['errors'] == []
|
||||
and cleanup['desktop_confirmation']['released'] is True):
|
||||
error_type = ReleasedProjectReadError
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
pass # Absence of an uncertainty marker is never release evidence.
|
||||
return error_type('quick_run_sheet_discovery', message)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DiscoveryRuntime:
|
||||
"""Worker and evidence adapters shared with the execution entry point."""
|
||||
@@ -39,13 +65,13 @@ def discover_project(
|
||||
) -> ProjectDiscovery:
|
||||
"""Enumerate sheets and requested source parts while the owned project is open."""
|
||||
check_cancelled(stop_event)
|
||||
project = project_path.resolve(strict=True)
|
||||
project = canonical_path(internal_path(project_path), strict=True)
|
||||
token = uuid4().hex
|
||||
job_id = "discover_" + token
|
||||
root = data_root / "quick-run-history" / ("sheet-discovery-" + token)
|
||||
root.mkdir(parents=True, exist_ok=False)
|
||||
root = internal_path(data_root / "quick-run-history" / ("sheet-discovery-" + token))
|
||||
retry_sharing_violation(lambda: root.mkdir(parents=True, exist_ok=False))
|
||||
script = root / "QuickPartReplace.vbs"
|
||||
shutil.copyfile(script_source, script)
|
||||
shutil.copyfile(internal_path(script_source), script)
|
||||
(root / "command.tsv").write_text(
|
||||
"task_id\tjob_id\tproject_path\told_part\tnew_part\tvalue_mode\r\n"
|
||||
f"sheet_discovery\t{job_id}\t{project}\tDISCOVERY_OLD\tDISCOVERY_NEW\tpreserve\r\n",
|
||||
@@ -68,7 +94,7 @@ def discover_project(
|
||||
)
|
||||
if collect_components:
|
||||
(root / "component-inventory.request").write_text(job_id, encoding="ascii")
|
||||
script_argument, root_argument = runtime.worker_paths(script, root)
|
||||
script_argument, root_argument = runtime.worker_paths(canonical_path(script), canonical_path(root))
|
||||
command = (
|
||||
str(runtime.cscript_resolver()),
|
||||
"//nologo",
|
||||
@@ -83,18 +109,23 @@ def discover_project(
|
||||
completed = runtime.process_runner(command, runtime.timeout)
|
||||
except (OSError, subprocess.SubprocessError) as error:
|
||||
_require_released_reader(root)
|
||||
check_cancelled(stop_event)
|
||||
raise TaskCenterError(
|
||||
"quick_run_sheet_discovery",
|
||||
failure = _read_failure(
|
||||
root,
|
||||
f"枚举工程图页时执行器未能安全结束:{project};{error}",
|
||||
) from error
|
||||
)
|
||||
if isinstance(failure, ReleasedProjectReadError):
|
||||
check_cancelled(stop_event)
|
||||
raise failure from error
|
||||
_require_released_reader(root)
|
||||
check_cancelled(stop_event)
|
||||
report_progress(on_progress, f'读取会话已结束,正在核对位号清单:{project.name}')
|
||||
(root / "worker-output.txt").write_text(completed.stdout + "\n" + completed.stderr, encoding="utf-8")
|
||||
if completed.returncode != 0:
|
||||
detail = completed.stderr.strip() or completed.stdout.strip() or "sheet discovery failed"
|
||||
raise TaskCenterError("quick_run_sheet_discovery", runtime.failure_message(detail))
|
||||
failure = _read_failure(root, runtime.failure_message(detail))
|
||||
if isinstance(failure, ReleasedProjectReadError):
|
||||
check_cancelled(stop_event)
|
||||
raise failure
|
||||
check_cancelled(stop_event)
|
||||
report_progress(on_progress, f'读取会话已结束,正在核对位号清单:{project.name}')
|
||||
sheets = runtime.read_sheets(root / "sheets.tsv", project)
|
||||
matches = runtime.read_matches(root / "inventory.tsv", project, sheets)
|
||||
if collect_components:
|
||||
@@ -107,6 +138,10 @@ def discover_project(
|
||||
|
||||
|
||||
def _require_released_reader(root):
|
||||
if (root / 'project-checkpoint.json').exists():
|
||||
from .batch_runtime import project_checkpoint_released
|
||||
if not project_checkpoint_released(root):
|
||||
raise TaskCenterError('inventory_cleanup', f'工程读取关闭检查点无效,请保留诊断记录:{root}')
|
||||
if (root / 'uncertain.flag').exists():
|
||||
raise TaskCenterError('inventory_cleanup', f'采集会话未能正常收尾,请检查诊断记录后重试:{root}')
|
||||
|
||||
@@ -33,7 +33,7 @@ class ProjectSessionWorker:
|
||||
The VBS worker owns CAD. This driver never attaches to COM or terminates CAD.
|
||||
A separate monitor thread retains the existing idle watchdog and worker job.
|
||||
"""
|
||||
def __init__(self, project: Path, root: Path, *, private_desktop=False):
|
||||
def __init__(self, project: Path, root: Path, *, private_desktop=False, runtime=None):
|
||||
self.project, self.root = Path(project), Path(root).resolve()
|
||||
self.token = uuid4().hex
|
||||
self._done, self._abort = Event(), Event()
|
||||
@@ -46,12 +46,13 @@ class ProjectSessionWorker:
|
||||
self._timeout = 150
|
||||
self.starts = 0
|
||||
self.private_desktop = private_desktop
|
||||
self.runtime = runtime
|
||||
|
||||
def _monitor(self, command, timeout):
|
||||
try:
|
||||
if self.private_desktop:
|
||||
if self.private_desktop or self.runtime is not None:
|
||||
from .isolated_cad import run_isolated_process
|
||||
self._result = run_isolated_process(command, timeout, abort_event=self._abort)
|
||||
self._result = run_isolated_process(command, timeout, abort_event=self._abort, **({'runtime': self.runtime} if self.runtime is not None else {}))
|
||||
else:
|
||||
self._result = run_monitored_process(command, timeout, abort_event=self._abort)
|
||||
except Exception as error:
|
||||
@@ -130,10 +131,11 @@ class ProjectSessionWorker:
|
||||
|
||||
class ProjectSessionScope:
|
||||
"""One serial task can visit projects, but can hold only one worker at a time."""
|
||||
def __init__(self, fallback, *, enabled, private_desktop=False):
|
||||
def __init__(self, fallback, *, enabled, private_desktop=False, runtime=None):
|
||||
self.fallback, self.enabled = fallback, enabled
|
||||
self.worker = None
|
||||
self.private_desktop = private_desktop
|
||||
self.runtime = runtime
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
@@ -143,7 +145,7 @@ class ProjectSessionScope:
|
||||
return self.fallback(command, timeout)
|
||||
self.prepare(project)
|
||||
if self.worker is None:
|
||||
self.worker = ProjectSessionWorker(Path(project), Path(command[3]), private_desktop=self.private_desktop)
|
||||
self.worker = ProjectSessionWorker(Path(project), Path(command[3]), private_desktop=self.private_desktop, runtime=self.runtime)
|
||||
return self.worker.run_step(command, timeout)
|
||||
|
||||
def prepare(self, project):
|
||||
|
||||
@@ -11,19 +11,20 @@ import stat
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, strip_extended_prefix
|
||||
from .models import TaskRecord, TaskStatus
|
||||
from .scanner import _is_schematic_project, _reject_reparse
|
||||
from .store import TaskStore
|
||||
|
||||
|
||||
def _validated_root(root: Path, excluded_roots: tuple[Path, ...]) -> tuple[Path, tuple[Path, ...]]:
|
||||
root = Path(os.path.abspath(root))
|
||||
root = Path(os.path.abspath(strip_extended_prefix(root)))
|
||||
for ancestor in (*reversed(root.parents), root):
|
||||
_reject_reparse(ancestor.lstat(), ancestor)
|
||||
root = root.resolve(strict=True)
|
||||
if not root.is_dir():
|
||||
_reject_reparse(internal_path(ancestor).lstat(), ancestor)
|
||||
root = canonical_path(root, strict=True)
|
||||
if not internal_path(root).is_dir():
|
||||
raise TaskCenterError("discovery_root", "project root is not a directory")
|
||||
excluded = tuple(Path(path).resolve(strict=False) for path in excluded_roots)
|
||||
excluded = tuple(canonical_path(path) for path in excluded_roots)
|
||||
if any(root.is_relative_to(blocked) for blocked in excluded):
|
||||
raise TaskCenterError("discovery_root", "project root is inside an excluded directory")
|
||||
return root, excluded
|
||||
@@ -49,9 +50,9 @@ def scan_project_directory(root: Path, *, excluded_roots: tuple[Path, ...] = (),
|
||||
while pending:
|
||||
check_cancelled()
|
||||
directory = pending.pop()
|
||||
_reject_reparse(directory.lstat(), directory)
|
||||
_reject_reparse(internal_path(directory).lstat(), directory)
|
||||
children = []
|
||||
with os.scandir(directory) as entries:
|
||||
with os.scandir(internal_path(directory)) as entries:
|
||||
for entry in entries:
|
||||
check_cancelled()
|
||||
children.append(entry)
|
||||
@@ -59,7 +60,7 @@ def scan_project_directory(root: Path, *, excluded_roots: tuple[Path, ...] = (),
|
||||
folders: list[Path] = []
|
||||
for entry in sorted(children, key=lambda item: item.name.casefold()):
|
||||
check_cancelled()
|
||||
path = Path(entry.path)
|
||||
path = Path(strip_extended_prefix(entry.path))
|
||||
if any(path.is_relative_to(blocked) for blocked in excluded):
|
||||
continue
|
||||
details = entry.stat(follow_symlinks=False)
|
||||
@@ -67,7 +68,7 @@ def scan_project_directory(root: Path, *, excluded_roots: tuple[Path, ...] = (),
|
||||
if stat.S_ISDIR(details.st_mode):
|
||||
folders.append(path)
|
||||
elif stat.S_ISREG(details.st_mode) and path.suffix.casefold() == ".prj" and _is_schematic_project(path):
|
||||
projects.append(path.resolve(strict=True))
|
||||
projects.append(canonical_path(path, strict=True))
|
||||
found.extend(projects)
|
||||
visited += 1
|
||||
if on_progress is not None:
|
||||
@@ -82,15 +83,15 @@ def _validate_selected(root: Path, selected_paths: tuple[Path, ...], excluded: t
|
||||
selected: set[Path] = set()
|
||||
ancestor_projects: dict[Path, bool] = {}
|
||||
for candidate in selected_paths:
|
||||
path = Path(os.path.abspath(candidate))
|
||||
path = Path(os.path.abspath(strip_extended_prefix(candidate)))
|
||||
if not path.is_relative_to(root) or any(path.is_relative_to(blocked) for blocked in excluded):
|
||||
raise TaskCenterError("project_selection", "selected project is outside discovery results")
|
||||
relative = path.relative_to(root)
|
||||
cursor = root
|
||||
for part in relative.parts:
|
||||
cursor /= part
|
||||
_reject_reparse(cursor.lstat(), cursor)
|
||||
if not path.is_file() or path.suffix.casefold() != ".prj" or not _is_schematic_project(path):
|
||||
_reject_reparse(internal_path(cursor).lstat(), cursor)
|
||||
if not internal_path(path).is_file() or path.suffix.casefold() != ".prj" or not _is_schematic_project(path):
|
||||
raise TaskCenterError("project_selection", "selected file is not a schematic project")
|
||||
# A project in any ancestor directory stops discovery below that directory.
|
||||
directory = path.parent
|
||||
@@ -98,9 +99,9 @@ def _validate_selected(root: Path, selected_paths: tuple[Path, ...], excluded: t
|
||||
directory = directory.parent
|
||||
if directory not in ancestor_projects:
|
||||
has_project = False
|
||||
with os.scandir(directory) as entries:
|
||||
with os.scandir(internal_path(directory)) as entries:
|
||||
for entry in entries:
|
||||
sibling = Path(entry.path)
|
||||
sibling = Path(strip_extended_prefix(entry.path))
|
||||
if sibling.suffix.casefold() != ".prj" or any(sibling.is_relative_to(b) for b in excluded):
|
||||
continue
|
||||
details = entry.stat(follow_symlinks=False)
|
||||
@@ -111,7 +112,7 @@ def _validate_selected(root: Path, selected_paths: tuple[Path, ...], excluded: t
|
||||
ancestor_projects[directory] = has_project
|
||||
if ancestor_projects[directory]:
|
||||
raise TaskCenterError("project_selection", "selected project is hidden by an ancestor project")
|
||||
selected.add(path.resolve(strict=True))
|
||||
selected.add(canonical_path(path, strict=True))
|
||||
return tuple(sorted(selected, key=lambda path: str(path).casefold()))
|
||||
|
||||
|
||||
@@ -120,7 +121,7 @@ def create_project_tasks(store: TaskStore, root: Path, *, selected_paths: tuple[
|
||||
root, excluded = _validated_root(root, excluded_roots)
|
||||
discovered = (scan_project_directory(root, excluded_roots=excluded) if selected_paths is None
|
||||
else _validate_selected(root, selected_paths, excluded))
|
||||
active_paths = {project.path.resolve(strict=False) for task in store.list_tasks()
|
||||
active_paths = {canonical_path(project.path) for task in store.list_tasks()
|
||||
if task.status not in {TaskStatus.DELETED, TaskStatus.COMPLETED}
|
||||
for project in store.list_projects(task.task_id)}
|
||||
pending = [path for path in discovered if path not in active_paths]
|
||||
|
||||
@@ -7,6 +7,7 @@ from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from functools import lru_cache
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -19,6 +20,7 @@ from typing import Protocol
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import internal_path
|
||||
from .models import ProjectRecord, ProjectStatus, RuleRecord
|
||||
from .retention import maintain_quick_run_storage, rotate_text_log
|
||||
from .worker_runtime import copy_worker_script
|
||||
@@ -111,22 +113,22 @@ class QuickRunPreparer:
|
||||
self._validate_fields(selected_projects, enabled_rules)
|
||||
_validate_target_scope(selected_projects, enabled_rules, sheet_targets)
|
||||
run_token = _safe_run_token(self._run_token_factory())
|
||||
maintain_quick_run_storage(self._data_root)
|
||||
maintain_quick_run_storage(_io_path(self._data_root))
|
||||
|
||||
root = self._data_root / "quick-run"
|
||||
script_path = root / "QuickPartReplace.vbs"
|
||||
command_path = root / "command.tsv"
|
||||
status_path = root / "status.tsv"
|
||||
log_path = root / "quick-part-replace.log"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
_io_path(root).mkdir(parents=True, exist_ok=True)
|
||||
_assert_no_pending_status(status_path, log_path)
|
||||
copy_worker_script(source, script_path)
|
||||
if sheet_targets is not None:
|
||||
manifest_path = root / "sheets.tsv"
|
||||
manifest_path.write_text(_sheet_manifest_text(selected_projects, sheet_targets), encoding="utf-8", newline="")
|
||||
_io_path(manifest_path).write_text(_sheet_manifest_text(selected_projects, sheet_targets), encoding="utf-8", newline="")
|
||||
else:
|
||||
(root / "sheets.tsv").unlink(missing_ok=True)
|
||||
command_path.write_text(
|
||||
(_io_path(root / "sheets.tsv")).unlink(missing_ok=True)
|
||||
_io_path(command_path).write_text(
|
||||
_command_text(
|
||||
task_id,
|
||||
run_token,
|
||||
@@ -143,7 +145,7 @@ class QuickRunPreparer:
|
||||
discovery_matches,
|
||||
discovery_failures,
|
||||
)
|
||||
status_path.write_text(
|
||||
_io_path(status_path).write_text(
|
||||
initial_status,
|
||||
encoding="utf-8",
|
||||
newline="\n",
|
||||
@@ -156,7 +158,7 @@ class QuickRunPreparer:
|
||||
root = self._data_root / "quick-run"
|
||||
command_path = root / "command.tsv"
|
||||
status_path = root / "status.tsv"
|
||||
if not command_path.is_file() or not status_path.is_file():
|
||||
if not _io_path(command_path).is_file() or not _io_path(status_path).is_file():
|
||||
return None
|
||||
commands = _read_commands(command_path)
|
||||
if {command_task_id for command_task_id, *_ in commands} != {task_id}:
|
||||
@@ -171,6 +173,7 @@ class QuickRunPreparer:
|
||||
*,
|
||||
copy_part_number_to_value: bool = False,
|
||||
before_sheet_discovery: Callable[[Path], None] | None = None,
|
||||
runtime=None,
|
||||
) -> QuickRunResult:
|
||||
"""Create a page-addressed queue after read-only sheet discovery."""
|
||||
all_selected_projects = _selected_projects(projects)
|
||||
@@ -201,7 +204,8 @@ class QuickRunPreparer:
|
||||
before_sheet_discovery(project.path)
|
||||
if self._sheet_discoverer is None:
|
||||
discovery = self.discover_project(project.path, old_part_numbers,
|
||||
**({'collect_components': True} if targeted else {}))
|
||||
**({'collect_components': True} if targeted else {}),
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
if targeted:
|
||||
from .inventory import validate_target_inventory
|
||||
validate_target_inventory(project.path, enabled_rules, discovery.components)
|
||||
@@ -217,7 +221,7 @@ class QuickRunPreparer:
|
||||
if discovery_matches is not None:
|
||||
discovery_matches[project_key] = {_MISSING_PROJECT_SHEET: frozenset()}
|
||||
except TaskCenterError:
|
||||
if project.path.exists():
|
||||
if _io_path(project.path).exists():
|
||||
raise
|
||||
project_key = _path_key(project.path)
|
||||
targets = (_MISSING_PROJECT_SHEET,)
|
||||
@@ -244,6 +248,7 @@ class QuickRunPreparer:
|
||||
*,
|
||||
copy_part_number_to_value: bool = False,
|
||||
before_sheet_discovery: Callable[[Path], None] | None = None,
|
||||
runtime=None,
|
||||
) -> QuickRunBatchSession:
|
||||
enabled_rules = tuple(rule for rule in rules if rule.enabled)
|
||||
handoff = self.ensure_batch_handoff(
|
||||
@@ -252,6 +257,7 @@ class QuickRunPreparer:
|
||||
rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value,
|
||||
before_sheet_discovery=before_sheet_discovery,
|
||||
**({'runtime': runtime} if runtime is not None else {}),
|
||||
)
|
||||
source = self._required_script_source()
|
||||
if source.resolve() != handoff.script_path.resolve():
|
||||
@@ -272,7 +278,7 @@ class QuickRunPreparer:
|
||||
project_keys_by_index = tuple(_path_key(command[2]) for command in commands)
|
||||
status_stat = _required_stat(handoff.status_path, "quick_run_status", "cannot inspect quick-run status")
|
||||
manifest_path = handoff.command_path.with_name("sheets.tsv")
|
||||
command_lines = handoff.command_path.read_text(encoding='utf-8-sig').splitlines()
|
||||
command_lines = _io_path(handoff.command_path).read_text(encoding='utf-8-sig').splitlines()
|
||||
job_column = command_lines[0].split('\t').index('job_id')
|
||||
target_command_rows = ({line.split('\t')[job_column]: line for line in command_lines[1:] if line}
|
||||
if {'target_scope', 'target_cell'} & set(command_lines[0].split('\t')) else {})
|
||||
@@ -301,13 +307,13 @@ class QuickRunPreparer:
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def project_session_scope(self, *, enabled=True):
|
||||
def project_session_scope(self, *, enabled=True, runtime=None):
|
||||
"""Retain only the built-in supervised worker; injected workers stay compatible."""
|
||||
from .project_session import ProjectSessionScope
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
try:
|
||||
with ProjectSessionScope(self._process_runner,
|
||||
enabled=enabled and self._process_runner is _run_process, private_desktop=True) as scope:
|
||||
enabled=enabled and self._process_runner is _run_process, private_desktop=True, runtime=runtime) as scope:
|
||||
yield scope
|
||||
except (WorkerMonitorError, subprocess.TimeoutExpired) as error:
|
||||
_write_uncertain_flag(self._data_root / 'quick-run/uncertain.flag', 'project_session_cleanup')
|
||||
@@ -325,17 +331,19 @@ class QuickRunPreparer:
|
||||
batch_mode: bool = False,
|
||||
batch_session: QuickRunBatchSession | None = None,
|
||||
project_session=None,
|
||||
runtime=None,
|
||||
) -> QuickRunStepResult:
|
||||
"""Prepare or resume a queue, then execute one replacement rule end to end."""
|
||||
uncertain_path = self._data_root / "quick-run" / "uncertain.flag"
|
||||
if uncertain_path.exists():
|
||||
if _io_path(uncertain_path).exists():
|
||||
raise TaskCenterError("quick_run_uncertain", "a previous quick-run worker timed out or returned an uncorrelated status; reconcile the project before retrying")
|
||||
if self._process_runner is _run_process and not batch_mode:
|
||||
from .single_step import prepare_private_single_step
|
||||
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value)
|
||||
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
batch_mode = True
|
||||
current_root = self._data_root / "quick-run"
|
||||
if batch_session is None and self.evidence_path(task_id) == current_root and (current_root / "status.tsv").is_file():
|
||||
if batch_session is None and self.evidence_path(task_id) == current_root and (_io_path(current_root / "status.tsv")).is_file():
|
||||
recorded = _read_status_details(current_root / "status.tsv")
|
||||
latest_states = {job_id: state for job_id, state, _, _ in recorded}
|
||||
if any(state == BATCH_SAVED_UNVERIFIED_STATE for state in latest_states.values()):
|
||||
@@ -367,7 +375,8 @@ class QuickRunPreparer:
|
||||
if handoff is None:
|
||||
if any(getattr(rule, "scope", "all") == "targets" for rule in enabled_rules):
|
||||
handoff = self.ensure_batch_handoff(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
copy_part_number_to_value=copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
else:
|
||||
handoff = self.prepare(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
@@ -398,15 +407,17 @@ class QuickRunPreparer:
|
||||
if batch_mode:
|
||||
if batch_session is not None:
|
||||
current_command_stamp = _write_current_command(batch_session, selected_command)
|
||||
batch_request.write_text(selected_command[1], encoding="ascii")
|
||||
_io_path(batch_request).write_text(selected_command[1], encoding="ascii")
|
||||
else:
|
||||
batch_request.unlink(missing_ok=True)
|
||||
_io_path(batch_request).unlink(missing_ok=True)
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
|
||||
self.invalidate_discovery(Path(selected_command[2]))
|
||||
try:
|
||||
if project_session is not None:
|
||||
completed = project_session.run_step(Path(selected_command[2]), command, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
elif runtime is not None and self._process_runner is _run_process:
|
||||
completed = _run_process(command, IDLE_PROGRESS_TIMEOUT_SECONDS, runtime=runtime)
|
||||
else:
|
||||
completed = self._process_runner(command, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
except subprocess.TimeoutExpired as error:
|
||||
@@ -530,7 +541,7 @@ class QuickRunPreparer:
|
||||
raise TaskCenterError("quick_run_execution", f"unexpected quick-run state: {state}")
|
||||
if batch_mode and state == "saved_ok":
|
||||
try:
|
||||
handoff.command_path.with_name("current-state.tsv").unlink(missing_ok=True)
|
||||
_io_path(handoff.command_path.with_name("current-state.tsv")).unlink(missing_ok=True)
|
||||
except OSError as error:
|
||||
_append_log(
|
||||
handoff.log_path,
|
||||
@@ -602,7 +613,7 @@ class QuickRunPreparer:
|
||||
root = self._data_root / "quick-run"
|
||||
command_path = root / "command.tsv"
|
||||
status_path = root / "status.tsv"
|
||||
if not command_path.is_file() or not status_path.is_file():
|
||||
if not _io_path(command_path).is_file() or not _io_path(status_path).is_file():
|
||||
return None
|
||||
try:
|
||||
commands = _read_commands(command_path)
|
||||
@@ -624,7 +635,7 @@ class QuickRunPreparer:
|
||||
_archive_obsolete_handoff(self._data_root, root)
|
||||
return None
|
||||
if _queue_value_mode(command_path) != value_mode:
|
||||
if statuses or any(root.glob("snapshot-*")) or _queue_has_mutation_evidence(root):
|
||||
if statuses or any(_io_path(root).glob("snapshot-*")) or _queue_has_mutation_evidence(root):
|
||||
raise TaskCenterError("quick_run_pending", "recorded Value policy differs; reconcile the old queue before creating a new task")
|
||||
_archive_obsolete_handoff(self._data_root, root)
|
||||
return None
|
||||
@@ -681,10 +692,14 @@ class QuickRunPreparer:
|
||||
self._discovery_cache.invalidate(project_path)
|
||||
|
||||
def discover_project(self, project_path: Path, old_part_numbers: tuple[str, ...], *,
|
||||
collect_components: bool = False, stop_event=None, on_progress=None) -> ProjectDiscovery:
|
||||
collect_components: bool = False, stop_event=None, on_progress=None, runtime=None) -> ProjectDiscovery:
|
||||
from .operation_control import check_cancelled, control_options, report_progress
|
||||
check_cancelled(stop_event)
|
||||
controls = control_options(stop_event, on_progress)
|
||||
if runtime is not None and any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in inspect.signature(self._discover_project_sheets).parameters.values()):
|
||||
controls['runtime'] = runtime
|
||||
attempt = 0
|
||||
def load():
|
||||
nonlocal attempt
|
||||
@@ -700,14 +715,15 @@ class QuickRunPreparer:
|
||||
Path(project_path), old_part_numbers, collect_components, load,
|
||||
**({'stop_event': stop_event} if stop_event is not None else {}))
|
||||
|
||||
def collect_inventory(self, project_path: Path, old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None) -> tuple:
|
||||
def collect_inventory(self, project_path: Path, old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None, runtime=None) -> tuple:
|
||||
"""Collect validated component targets, retaining reusable sheet evidence."""
|
||||
if not old_part_numbers:
|
||||
raise TaskCenterError("target_inventory", "at least one old Part Number is required")
|
||||
requested = {part.casefold() for part in old_part_numbers}
|
||||
from .operation_control import control_options
|
||||
return tuple(component for component in self.discover_project(
|
||||
project_path, old_part_numbers, collect_components=True, **control_options(stop_event, on_progress)).components
|
||||
project_path, old_part_numbers, collect_components=True, **control_options(stop_event, on_progress),
|
||||
**({'runtime': runtime} if runtime is not None else {})).components
|
||||
if component.old_part_number.casefold() in requested)
|
||||
|
||||
def _discover_project_sheets(
|
||||
@@ -718,6 +734,7 @@ class QuickRunPreparer:
|
||||
collect_components: bool = False,
|
||||
stop_event=None,
|
||||
on_progress=None,
|
||||
runtime=None,
|
||||
) -> _ProjectDiscovery:
|
||||
"""Enumerate sheets and requested source parts while the owned project is open."""
|
||||
from .project_discovery import DiscoveryRuntime, discover_project
|
||||
@@ -726,14 +743,16 @@ class QuickRunPreparer:
|
||||
if collect_components and self._process_runner is _run_process:
|
||||
from .native_inventory import NativeInventoryReader
|
||||
native = NativeInventoryReader(self._data_root).read(
|
||||
project_path, old_part_numbers, stop_event=stop_event, on_progress=on_progress)
|
||||
project_path, old_part_numbers, stop_event=stop_event, on_progress=on_progress,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
if native is not None:
|
||||
return native
|
||||
|
||||
process_runner = self._process_runner
|
||||
if stop_event is not None and process_runner is _run_process:
|
||||
if (stop_event is not None or runtime is not None) and process_runner is _run_process:
|
||||
def process_runner(command, timeout):
|
||||
return _run_process(command, timeout, abort_event=stop_event)
|
||||
return _run_process(command, timeout, abort_event=stop_event,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
|
||||
return discover_project(project_path, old_part_numbers,
|
||||
data_root=self._data_root, script_source=self._required_script_source(),
|
||||
@@ -754,7 +773,7 @@ class QuickRunPreparer:
|
||||
return self._data_root
|
||||
|
||||
def _required_script_source(self) -> Path:
|
||||
if not self._script_source.is_file():
|
||||
if not _io_path(self._script_source).is_file():
|
||||
raise TaskCenterError("quick_run_script_missing", "QuickPartReplace.vbs runtime resource is missing")
|
||||
return self._script_source
|
||||
|
||||
@@ -1056,10 +1075,10 @@ def _assert_no_pending_status(status_path: Path, log_path: Path) -> None:
|
||||
|
||||
|
||||
def _read_status(status_path: Path) -> tuple[tuple[str, str], ...]:
|
||||
if not status_path.exists():
|
||||
if not _io_path(status_path).exists():
|
||||
return ()
|
||||
try:
|
||||
lines = status_path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(status_path).read_text(encoding="utf-8-sig").splitlines()
|
||||
except (OSError, UnicodeError) as error:
|
||||
raise TaskCenterError("quick_run_status", "cannot read quick-run status") from error
|
||||
if not lines or lines[0].split("\t") != ["job_id", "state", "code", "message"]:
|
||||
@@ -1080,17 +1099,17 @@ def _read_status(status_path: Path) -> tuple[tuple[str, str], ...]:
|
||||
|
||||
|
||||
def _append_log(log_path: Path, entry: str, *, check_rotation: bool = True) -> None:
|
||||
if not log_path.parent.is_dir():
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not _io_path(log_path.parent).is_dir():
|
||||
_io_path(log_path.parent).mkdir(parents=True, exist_ok=True)
|
||||
if check_rotation:
|
||||
rotate_text_log(log_path)
|
||||
with log_path.open("a", encoding="utf-8", newline="\n") as handle:
|
||||
rotate_text_log(_io_path(log_path))
|
||||
with _io_path(log_path).open("a", encoding="utf-8", newline="\n") as handle:
|
||||
handle.write(entry + "\n")
|
||||
|
||||
|
||||
def _write_uncertain_flag(path: Path, reason: str = "manual_reconciliation_required") -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8", newline="\n") as handle:
|
||||
_io_path(path.parent).mkdir(parents=True, exist_ok=True)
|
||||
with _io_path(path).open("w", encoding="utf-8", newline="\n") as handle:
|
||||
handle.write(f"reason={_log_text(reason)}; manual reconciliation required\n")
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
@@ -1102,14 +1121,14 @@ def _states_are_complete(statuses: tuple[tuple[str, str, str, str], ...]) -> boo
|
||||
|
||||
|
||||
def _queue_has_mutation_evidence(root: Path) -> bool:
|
||||
if (root / "uncertain.flag").exists():
|
||||
if (_io_path(root / "uncertain.flag")).exists():
|
||||
return True
|
||||
for report_path in root.glob("snapshot-*.tsv.native.json"):
|
||||
for report_path in _io_path(root).glob("snapshot-*.tsv.native.json"):
|
||||
rollback_marker = Path(str(report_path)[:-len(".json")] + ".rolled-back")
|
||||
if rollback_marker.is_file():
|
||||
if _io_path(rollback_marker).is_file():
|
||||
continue
|
||||
try:
|
||||
report = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
report = json.loads(_io_path(report_path).read_text(encoding="utf-8"))
|
||||
except (OSError, UnicodeError, json.JSONDecodeError):
|
||||
return True
|
||||
if report.get("ok") is True or report.get("stage") in {"native_replace_started", "native_replace_applied", "pin_promotion_started", "native_replace_verified"}:
|
||||
@@ -1122,12 +1141,12 @@ def _rollback_marker_verified(root: Path, job_id: str) -> bool:
|
||||
snapshot = root / f"snapshot-{job_id}.tsv"
|
||||
report_path = root / f"snapshot-{job_id}.tsv.native.json"
|
||||
try:
|
||||
report = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
report = json.loads(_io_path(report_path).read_text(encoding="utf-8"))
|
||||
return (
|
||||
snapshot.is_file()
|
||||
_io_path(snapshot).is_file()
|
||||
and isinstance(report, dict)
|
||||
and report.get("stage") in {"native_replace_started", "native_replace_applied", "pin_promotion_started", "native_replace_verified"}
|
||||
and marker.read_text(encoding="utf-8-sig").strip() == "native replacement rollback verified"
|
||||
and _io_path(marker).read_text(encoding="utf-8-sig").strip() == "native replacement rollback verified"
|
||||
)
|
||||
except (OSError, UnicodeError, json.JSONDecodeError):
|
||||
return False
|
||||
@@ -1138,7 +1157,7 @@ def _recover_pre_mutation_hard_failures(
|
||||
commands: tuple[tuple[str, str, str, str, str], ...],
|
||||
statuses: tuple[tuple[str, str, str, str], ...],
|
||||
) -> tuple[tuple[str, str, str, str], ...]:
|
||||
if (root / "uncertain.flag").exists():
|
||||
if (_io_path(root / "uncertain.flag")).exists():
|
||||
return statuses
|
||||
command_ids = {job_id for _, job_id, *_ in commands}
|
||||
final_states = {job_id: state for job_id, state, _, _ in statuses}
|
||||
@@ -1158,7 +1177,7 @@ def _recover_pre_mutation_hard_failures(
|
||||
try:
|
||||
safe_job_id = _safe_run_token(job_id)
|
||||
report_path = root / f"snapshot-{safe_job_id}.tsv.native.json"
|
||||
report = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
report = json.loads(_io_path(report_path).read_text(encoding="utf-8"))
|
||||
except (TaskCenterError, OSError, UnicodeError, json.JSONDecodeError):
|
||||
continue
|
||||
if (
|
||||
@@ -1186,26 +1205,26 @@ def _write_status_details(path: Path, statuses: tuple[tuple[str, str, str, str],
|
||||
text = "job_id\tstate\tcode\tmessage\r\n"
|
||||
text += "".join("\t".join(_encode_vbs_field(field) for field in entry) + "\r\n" for entry in statuses)
|
||||
try:
|
||||
temporary.write_text(text, encoding="utf-8", newline="")
|
||||
os.replace(temporary, path)
|
||||
_io_path(temporary).write_text(text, encoding="utf-8", newline="")
|
||||
os.replace(_io_path(temporary), _io_path(path))
|
||||
except OSError as error:
|
||||
temporary.unlink(missing_ok=True)
|
||||
_io_path(temporary).unlink(missing_ok=True)
|
||||
raise TaskCenterError("quick_run_status", "cannot recover pre-mutation quick-run status") from error
|
||||
|
||||
|
||||
def _archive_obsolete_handoff(data_root: Path, root: Path) -> Path:
|
||||
timestamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
|
||||
destination = data_root / "quick-run-history" / f"obsolete-{timestamp}-{uuid4().hex[:8]}"
|
||||
destination.mkdir(parents=True, exist_ok=False)
|
||||
for entry in tuple(root.iterdir()):
|
||||
shutil.move(str(entry), destination / entry.name)
|
||||
_io_path(destination).mkdir(parents=True, exist_ok=False)
|
||||
for entry in tuple(_io_path(root).iterdir()):
|
||||
shutil.move(str(_io_path(entry)), _io_path(destination / entry.name))
|
||||
_append_log(destination / "quick-part-replace.log", "quick_run_obsolete_queue_archived")
|
||||
return destination
|
||||
|
||||
|
||||
def _read_commands(path: Path) -> tuple[tuple[str, str, str, str, str], ...]:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
except (OSError, UnicodeError) as error:
|
||||
raise TaskCenterError("quick_run_command", "cannot read quick-run command") from error
|
||||
if not lines:
|
||||
@@ -1230,7 +1249,7 @@ def _read_commands(path: Path) -> tuple[tuple[str, str, str, str, str], ...]:
|
||||
|
||||
def _read_command_sheet_targets(path: Path) -> dict[str, tuple[str, str]] | None:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
header = lines[0].split("\t")
|
||||
names = ("job_id", "schematic_name", "sheet_number")
|
||||
present = tuple(name in header for name in names[1:])
|
||||
@@ -1260,7 +1279,7 @@ def _read_sheet_manifest(
|
||||
expected_projects: tuple[ProjectRecord, ...] | None = None,
|
||||
) -> dict[str, tuple[tuple[str, str], ...]]:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
if not lines or lines[0].split("\t") != ["project_path", "schematic_name", "sheet_number"]:
|
||||
raise ValueError("invalid manifest header")
|
||||
result: dict[str, list[tuple[str, str]]] = {}
|
||||
@@ -1301,7 +1320,7 @@ def _read_discovery_inventory(
|
||||
sheets: tuple[tuple[str, str], ...],
|
||||
) -> dict[tuple[str, str], frozenset[str]]:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
if not lines or lines[0].split("\t") != ["project_path", "schematic_name", "sheet_number", "old_part"]:
|
||||
raise ValueError("invalid inventory header")
|
||||
expected_project = _path_key(project_path)
|
||||
@@ -1371,7 +1390,7 @@ def _initial_discovery_status(
|
||||
return rows[0] + "\r\n"
|
||||
commands = _read_commands(command_path)
|
||||
sheets = _read_command_sheet_targets(command_path)
|
||||
targeted = "target_scope" in command_path.read_text(encoding="utf-8-sig").splitlines()[0].split("\t")
|
||||
targeted = "target_scope" in _io_path(command_path).read_text(encoding="utf-8-sig").splitlines()[0].split("\t")
|
||||
if sheets is None:
|
||||
raise TaskCenterError("quick_run_sheet_discovery", "discovery inventory requires page-addressed commands")
|
||||
possible_by_sheet: dict[tuple[str, str, str], set[str]] = {}
|
||||
@@ -1417,12 +1436,12 @@ def _queue_matches_inputs(
|
||||
expected_cells = {str(getattr(rule, "target_cell", "")) for rule in rules}
|
||||
if actual_cells != expected_cells:
|
||||
return False
|
||||
if "target_scope" in command_path.read_text(encoding="utf-8-sig").splitlines()[0].split("\t"):
|
||||
if "target_scope" in _io_path(command_path).read_text(encoding="utf-8-sig").splitlines()[0].split("\t"):
|
||||
sheets = _read_command_sheet_targets(command_path)
|
||||
manifest = _read_sheet_manifest(command_path.with_name("sheets.tsv"), projects) if sheets is not None else None
|
||||
expected = _command_text("expected", "expected", projects, rules, base_mode,
|
||||
sheet_targets=manifest).splitlines()
|
||||
actual = command_path.read_text(encoding="utf-8-sig").splitlines()
|
||||
actual = _io_path(command_path).read_text(encoding="utf-8-sig").splitlines()
|
||||
return (expected[0] == actual[0] and len(expected) == len(actual) and
|
||||
all(expected_row.split("\t")[2:] == actual_row.split("\t")[2:]
|
||||
for expected_row, actual_row in zip(expected[1:], actual[1:])))
|
||||
@@ -1458,10 +1477,10 @@ def _queue_matches_inputs(
|
||||
|
||||
|
||||
def _read_status_details(path: Path) -> tuple[tuple[str, str, str, str], ...]:
|
||||
if not path.exists():
|
||||
if not _io_path(path).exists():
|
||||
return ()
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
except (OSError, UnicodeError) as error:
|
||||
raise TaskCenterError("quick_run_status", "cannot read quick-run status") from error
|
||||
if not lines or lines[0].split("\t") != ["job_id", "state", "code", "message"]:
|
||||
@@ -1484,7 +1503,7 @@ def _open_status_for_read(path: Path):
|
||||
deadline = time.monotonic() + 1.0
|
||||
while True:
|
||||
try:
|
||||
return path.open('rb')
|
||||
return _io_path(path).open('rb')
|
||||
except PermissionError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise
|
||||
@@ -1526,7 +1545,7 @@ def _read_appended_status_details(
|
||||
|
||||
def _required_stat(path: Path, code: str, message: str) -> os.stat_result:
|
||||
try:
|
||||
return path.stat()
|
||||
return _io_path(path).stat()
|
||||
except OSError as error:
|
||||
raise TaskCenterError(code, message) from error
|
||||
|
||||
@@ -1575,7 +1594,7 @@ def _write_current_command(
|
||||
state = session.latest.get(job_id)
|
||||
state_path = path.with_name("current-state.tsv")
|
||||
if state is None:
|
||||
state_path.unlink(missing_ok=True)
|
||||
_io_path(state_path).unlink(missing_ok=True)
|
||||
else:
|
||||
_write_atomic_queue_text(state_path,
|
||||
"job_id\tstate\tcode\tmessage\r\n" + "\t".join(
|
||||
@@ -1619,7 +1638,7 @@ def _write_current_command(
|
||||
)
|
||||
if session.next_index == 0 or (previous_state is not None and previous_state[0] != "no_match"):
|
||||
try:
|
||||
state_path.unlink(missing_ok=True)
|
||||
_io_path(state_path).unlink(missing_ok=True)
|
||||
except OSError as error:
|
||||
raise TaskCenterError("quick_run_status", "cannot clear the previous batch state") from error
|
||||
else:
|
||||
@@ -1639,21 +1658,21 @@ def _write_current_command(
|
||||
def _write_atomic_queue_text(path: Path, text: str, code: str, message: str) -> None:
|
||||
temporary = path.with_name(f".{path.name}.{uuid4().hex}.tmp")
|
||||
try:
|
||||
with temporary.open("w", encoding="utf-8", newline="") as handle:
|
||||
with _io_path(temporary).open("w", encoding="utf-8", newline="") as handle:
|
||||
handle.write(text)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
deadline = time.monotonic() + 1.0
|
||||
while True:
|
||||
try:
|
||||
os.replace(temporary, path)
|
||||
os.replace(_io_path(temporary), _io_path(path))
|
||||
break
|
||||
except PermissionError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise
|
||||
time.sleep(.02)
|
||||
except OSError as error:
|
||||
temporary.unlink(missing_ok=True)
|
||||
_io_path(temporary).unlink(missing_ok=True)
|
||||
raise TaskCenterError(code, message) from error
|
||||
|
||||
|
||||
@@ -1734,7 +1753,7 @@ def _resolve_cscript(windows_root: Path | None = None, is_64_bit: bool | None =
|
||||
if is_64_bit is None:
|
||||
is_64_bit = sys.maxsize > 2**32 or bool(os.environ.get("PROCESSOR_ARCHITEW6432"))
|
||||
candidate = windows / ("SysWOW64" if is_64_bit else "System32") / "cscript.exe"
|
||||
if candidate.is_file():
|
||||
if _io_path(candidate).is_file():
|
||||
return candidate
|
||||
raise TaskCenterError("quick_run_cscript", "Windows Script Host cscript.exe was not found")
|
||||
|
||||
@@ -1751,12 +1770,13 @@ def _cached_native_helper_launch_arguments(frozen: bool, executable: str) -> tup
|
||||
return (resolved_executable, str(Path(__file__).with_name("application.py")))
|
||||
|
||||
|
||||
def _run_process(command: tuple[str, ...], timeout: int, *, abort_event=None) -> subprocess.CompletedProcess[str]:
|
||||
def _run_process(command: tuple[str, ...], timeout: int, *, abort_event=None, runtime=None) -> subprocess.CompletedProcess[str]:
|
||||
from .isolated_cad import run_isolated_process
|
||||
from .legacy_paths import validate_project_access
|
||||
|
||||
validate_project_access(Path(command[3]))
|
||||
result = run_isolated_process(command, timeout,
|
||||
**({'runtime': runtime} if runtime is not None else {}),
|
||||
**({'abort_event': abort_event, 'cooperative_abort': True} if abort_event is not None else {}))
|
||||
validate_project_access(Path(command[3]))
|
||||
return result
|
||||
@@ -1799,7 +1819,7 @@ def _value_mode_matches_option(recorded_mode: str, enabled: bool) -> bool:
|
||||
|
||||
def _queue_value_mode(path: Path) -> str:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
header = lines[0].split("\t")
|
||||
if "value_mode" not in header:
|
||||
return "legacy_part_label"
|
||||
@@ -1817,7 +1837,7 @@ def _queue_value_mode(path: Path) -> str:
|
||||
|
||||
def _read_command_cells(path: Path) -> dict[str, str]:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
header = lines[0].split("\t")
|
||||
if "target_cell" not in header:
|
||||
return {}
|
||||
@@ -1842,7 +1862,7 @@ def _read_command_cells(path: Path) -> dict[str, str]:
|
||||
|
||||
def _read_command_options(path: Path) -> dict[str, tuple[str, str, bool, str]]:
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8-sig").splitlines()
|
||||
lines = _io_path(path).read_text(encoding="utf-8-sig").splitlines()
|
||||
if not lines:
|
||||
raise ValueError("empty queue")
|
||||
header = lines[0].split("\t")
|
||||
@@ -1958,3 +1978,9 @@ def _validate_target_scope(projects, rules, sheet_targets) -> None:
|
||||
for target in rule.targets:
|
||||
if (_path_key(target.project_path), target.schematic_name, target.sheet_number) not in available:
|
||||
raise TaskCenterError("target_identity", "selected target sheet is missing from discovery")
|
||||
|
||||
|
||||
def _io_path(path):
|
||||
# Snapshot parsers also accept an immutable cached reader. Preserve that
|
||||
# protocol; converting it to Path would bypass the coherent read snapshot.
|
||||
return internal_path(path) if isinstance(path, (str, os.PathLike)) else path
|
||||
@@ -4,6 +4,7 @@ from dataclasses import dataclass, replace
|
||||
import json
|
||||
from pathlib import Path
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path
|
||||
from .quick_run_protocol import QuickRunProgress
|
||||
|
||||
|
||||
@@ -19,9 +20,9 @@ def failed_native_result_uncertain(root, commands, statuses, *, files=None):
|
||||
continue
|
||||
path = root / f'snapshot-{job}.tsv.native.json'
|
||||
try:
|
||||
report = json.loads(files.read(path) if files is not None else path.read_text(encoding='utf-8-sig'))
|
||||
report = json.loads(files.read(path) if files is not None else internal_path(path).read_text(encoding='utf-8-sig'))
|
||||
except (OSError, UnicodeError, json.JSONDecodeError):
|
||||
if job in awaiting or path.exists():
|
||||
if job in awaiting or internal_path(path).exists():
|
||||
return True
|
||||
continue
|
||||
if not isinstance(report, dict):
|
||||
@@ -56,7 +57,7 @@ class _ReadFiles:
|
||||
def stamp(path):
|
||||
self.checkpoint()
|
||||
try:
|
||||
stat = path.stat()
|
||||
stat = internal_path(path).stat()
|
||||
return (stat.st_dev, stat.st_ino, stat.st_size, stat.st_mtime_ns, stat.st_ctime_ns)
|
||||
except OSError:
|
||||
return None
|
||||
@@ -64,7 +65,7 @@ class _ReadFiles:
|
||||
try:
|
||||
# Include sidecar evidence used by hard-failure recovery as well as
|
||||
# command/status and flags; creation/removal changes the root stamp.
|
||||
entries = tuple(root.iterdir())
|
||||
entries = tuple(internal_path(root).iterdir())
|
||||
except OSError:
|
||||
entries = ()
|
||||
result = {path.name: stamp(path) for path in entries}
|
||||
@@ -85,7 +86,7 @@ class _ReadFiles:
|
||||
self.checkpoint()
|
||||
if path not in self.texts:
|
||||
chunks = []
|
||||
with path.open('r', encoding='utf-8-sig') as source:
|
||||
with internal_path(path).open('r', encoding='utf-8-sig') as source:
|
||||
while True:
|
||||
self.checkpoint()
|
||||
chunk = source.read(65536)
|
||||
@@ -110,7 +111,7 @@ class _CachedPath:
|
||||
|
||||
def exists(self):
|
||||
self.files.checkpoint()
|
||||
return self.path.exists()
|
||||
return internal_path(self.path).exists()
|
||||
|
||||
|
||||
def read_snapshot(preparer, task_id, projects, rules, *, copy_part_number_to_value=False,
|
||||
@@ -148,12 +149,13 @@ def _read_snapshot_once(preparer, task_id, projects, rules, *, copy_part_number_
|
||||
def candidates():
|
||||
yield current
|
||||
files.checkpoint()
|
||||
yield from sorted((path for path in history.glob('obsolete-*') if path.is_dir()), reverse=True)
|
||||
yield from sorted((canonical_path(path) for path in internal_path(history).glob('obsolete-*')
|
||||
if internal_path(path).is_dir()), reverse=True)
|
||||
for candidate in candidates():
|
||||
files.checkpoint()
|
||||
files.track_root(candidate)
|
||||
command = candidate / 'command.tsv'
|
||||
if not command.is_file() or not (candidate / 'status.tsv').is_file():
|
||||
if not internal_path(command).is_file() or not (internal_path(candidate / 'status.tsv')).is_file():
|
||||
continue
|
||||
try:
|
||||
parsed = api._read_commands(files.path(command))
|
||||
@@ -168,7 +170,7 @@ def _read_snapshot_once(preparer, task_id, projects, rules, *, copy_part_number_
|
||||
if root is None:
|
||||
files.checkpoint()
|
||||
return QuickRunSnapshot(progress, False, None, False, 'selection')
|
||||
interrupted = root == current and (root / 'uncertain.flag').is_file()
|
||||
interrupted = root == current and (internal_path(root / 'uncertain.flag')).is_file()
|
||||
phase = 'selection'
|
||||
try:
|
||||
details = api._read_status_details(files.path(root / 'status.tsv'))
|
||||
|
||||
@@ -105,7 +105,7 @@ def require_current_cell_binding(snapshot, binding, project, metadata):
|
||||
return current
|
||||
|
||||
|
||||
def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=False, confirmed=False):
|
||||
def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=False, confirmed=False, runtime=None):
|
||||
if not confirmed:
|
||||
raise TaskCenterError('quick_run_recovery_confirmation', '核验恢复需要明确确认,不能作为自动重试执行。')
|
||||
maintain_quick_run_storage(preparer._data_root)
|
||||
@@ -227,7 +227,11 @@ def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=Fal
|
||||
launch = (str(preparer._cscript_resolver()), '//nologo', script_argument, root_argument,
|
||||
*_native_helper_launch_arguments(), job_id)
|
||||
try:
|
||||
result = preparer._process_runner(launch, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
from .quick_run import _run_process
|
||||
if runtime is not None and preparer._process_runner is _run_process:
|
||||
result = _run_process(launch, IDLE_PROGRESS_TIMEOUT_SECONDS, runtime=runtime)
|
||||
else:
|
||||
result = preparer._process_runner(launch, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
except (OSError, subprocess.SubprocessError) as error:
|
||||
raise TaskCenterError('quick_run_recovery', '恢复执行未能确认结束,原保护锁仍在;不要重新替换或删除证据。') from error
|
||||
(audit / 'worker-output.txt').write_text(result.stdout + '\n' + result.stderr, encoding='utf-8')
|
||||
|
||||
@@ -56,10 +56,13 @@ class TrialRunner:
|
||||
logging_context: LoggingContext,
|
||||
*,
|
||||
probe_source_resolver: Callable[[TaskRecord, ProjectRecord], Path] | None = None,
|
||||
recover_on_startup: bool = True,
|
||||
) -> None:
|
||||
self._store = store
|
||||
self._logging = logging_context
|
||||
self._probe_source_resolver = probe_source_resolver or self._default_probe_source
|
||||
if not recover_on_startup:
|
||||
return # Per-batch processes must not replay application-start recovery.
|
||||
migration_recoveries = self._store.pending_migration_recoveries()
|
||||
for recovery in migration_recoveries:
|
||||
self._event_transition(
|
||||
|
||||
@@ -8,10 +8,11 @@ import re
|
||||
import stat
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, strip_extended_prefix
|
||||
|
||||
|
||||
def _path_identity(path: Path) -> str:
|
||||
normalized = os.path.normpath(str(path))
|
||||
normalized = os.path.normpath(strip_extended_prefix(path))
|
||||
return os.path.normcase(normalized) if os.name == "nt" else normalized
|
||||
|
||||
|
||||
@@ -22,7 +23,7 @@ _ICDB_DIRECTORY = re.compile(rb'^\s*KEY\s+iCDBDir\s+"[^"]+"\s*$', re.IGNORECASE
|
||||
|
||||
|
||||
def _is_schematic_project(path: Path) -> bool:
|
||||
with path.open("rb") as project_file:
|
||||
with internal_path(path).open("rb") as project_file:
|
||||
contents = project_file.read(_PROJECT_HEADER_LIMIT)
|
||||
return bool(_DXD_FRONT_END.search(contents) and _ICDB_DIRECTORY.search(contents))
|
||||
|
||||
@@ -46,13 +47,13 @@ def _lexical_path(path: Path) -> str:
|
||||
|
||||
def scan_projects(root: Path, *, excluded_roots: tuple[Path, ...] = ()) -> tuple[Path, ...]:
|
||||
"""Discover DxDesigner schematic projects without selecting or modifying them."""
|
||||
root = Path(root)
|
||||
root = Path(strip_extended_prefix(root))
|
||||
try:
|
||||
if not root.exists() or not root.is_dir():
|
||||
if not internal_path(root).exists() or not internal_path(root).is_dir():
|
||||
raise TaskCenterError("discovery_root", f"discovery root is not a directory: {root}")
|
||||
root_details = root.lstat()
|
||||
root_details = internal_path(root).lstat()
|
||||
_reject_reparse(root_details, root)
|
||||
canonical_root = root.resolve()
|
||||
canonical_root = canonical_path(root)
|
||||
excluded = tuple(_lexical_path(Path(path)) for path in excluded_roots)
|
||||
root_key = _lexical_path(root)
|
||||
if any(root_key == blocked or root_key.startswith(blocked + os.sep) for blocked in excluded):
|
||||
@@ -66,10 +67,10 @@ def scan_projects(root: Path, *, excluded_roots: tuple[Path, ...] = ()) -> tuple
|
||||
visited: set[tuple[int, int]] = set()
|
||||
|
||||
def read_entries(directory: Path) -> tuple[tuple[Path, os.stat_result], ...]:
|
||||
with os.scandir(directory) as entries:
|
||||
with os.scandir(internal_path(directory)) as entries:
|
||||
result: list[tuple[Path, os.stat_result]] = []
|
||||
for entry in sorted(entries, key=lambda item: item.name.casefold()):
|
||||
candidate = Path(entry.path)
|
||||
candidate = Path(strip_extended_prefix(entry.path))
|
||||
if is_excluded(candidate):
|
||||
continue
|
||||
entry_details = entry.stat(follow_symlinks=False)
|
||||
@@ -84,7 +85,7 @@ def scan_projects(root: Path, *, excluded_roots: tuple[Path, ...] = ()) -> tuple
|
||||
and candidate.suffix.casefold() == ".prj"
|
||||
and _is_schematic_project(candidate)
|
||||
):
|
||||
canonical = candidate.resolve()
|
||||
canonical = canonical_path(candidate)
|
||||
candidates.setdefault(_path_identity(canonical), canonical)
|
||||
|
||||
if candidates:
|
||||
@@ -94,10 +95,10 @@ def scan_projects(root: Path, *, excluded_roots: tuple[Path, ...] = ()) -> tuple
|
||||
directory: Path,
|
||||
directory_entries: tuple[tuple[Path, os.stat_result], ...] | None = None,
|
||||
) -> None:
|
||||
canonical_directory = directory.resolve()
|
||||
canonical_directory = canonical_path(directory)
|
||||
if not _contained(canonical_root, canonical_directory):
|
||||
raise TaskCenterError("discovery_containment", f"discovery path escapes its root: {directory}")
|
||||
details = directory.lstat()
|
||||
details = internal_path(directory).lstat()
|
||||
_reject_reparse(details, directory)
|
||||
identity = details.st_dev, details.st_ino
|
||||
if identity in visited:
|
||||
@@ -112,7 +113,7 @@ def scan_projects(root: Path, *, excluded_roots: tuple[Path, ...] = ()) -> tuple
|
||||
and candidate.suffix.casefold() == ".prj"
|
||||
and _is_schematic_project(candidate)
|
||||
):
|
||||
canonical = candidate.resolve()
|
||||
canonical = canonical_path(candidate)
|
||||
if not _contained(canonical_root, canonical):
|
||||
raise TaskCenterError("discovery_containment", f"project path escapes its root: {candidate}")
|
||||
candidates.setdefault(_path_identity(canonical), canonical)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from contextlib import contextmanager
|
||||
from contextlib import contextmanager, ExitStack
|
||||
from datetime import datetime
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -35,12 +35,26 @@ _QUICK_RUN_THREAD_LOCK = threading.Lock()
|
||||
|
||||
@contextmanager
|
||||
def _quick_run_lease(root: Path):
|
||||
from .batch_supervisor import mutation_lease
|
||||
try:
|
||||
with mutation_lease(root):
|
||||
with _quick_run_lease_owned(root):
|
||||
yield
|
||||
except TaskCenterError as error:
|
||||
if error.code == 'supervisor_busy':
|
||||
raise TaskCenterError('quick_run_busy', '另一批操作正在执行,请先安全暂停。') from error
|
||||
raise
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _quick_run_lease_owned(root: Path):
|
||||
if not _QUICK_RUN_THREAD_LOCK.acquire(blocking=False):
|
||||
raise TaskCenterError("quick_run_busy", "another quick-run operation is already executing")
|
||||
handle = None
|
||||
locked = False
|
||||
try:
|
||||
lock_path = root / ".quick-run.execution.lock"
|
||||
from .path_access import internal_path
|
||||
lock_path = internal_path(root / ".quick-run.execution.lock")
|
||||
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
handle = lock_path.open("a+b")
|
||||
if handle.seek(0, os.SEEK_END) == 0:
|
||||
@@ -95,12 +109,29 @@ class TaskCenterService:
|
||||
probe_source_resolver: Callable[[TaskRecord, ProjectRecord], Path] | None = None,
|
||||
quick_run_preparer: QuickRunPreparer | None = None,
|
||||
runner: TrialRunner | None = None,
|
||||
supervised: bool = False,
|
||||
recover_on_startup: bool = True,
|
||||
) -> None:
|
||||
self._store = store
|
||||
self._logging = logging_context
|
||||
self._path_opener = path_opener
|
||||
self._clock = clock
|
||||
self._runner = runner or TrialRunner(store, logging_context, probe_source_resolver=probe_source_resolver)
|
||||
self._supervised = supervised
|
||||
with ExitStack() as startup:
|
||||
if runner is None and recover_on_startup:
|
||||
from .batch_supervisor import mutation_lease, pending_supervisors
|
||||
try:
|
||||
startup.enter_context(mutation_lease(logging_context.root))
|
||||
except TaskCenterError as error:
|
||||
if error.code != 'supervisor_busy':
|
||||
raise
|
||||
recover_on_startup = False
|
||||
if recover_on_startup and pending_supervisors(logging_context.root):
|
||||
recover_on_startup = False
|
||||
# Keep admission for the whole recovery, not just an idle probe:
|
||||
# another window must not start a batch while startup rewrites state.
|
||||
self._runner = runner or TrialRunner(store, logging_context, probe_source_resolver=probe_source_resolver,
|
||||
**({'recover_on_startup': False} if not recover_on_startup else {}))
|
||||
self._quick_run_preparer = quick_run_preparer or QuickRunPreparer(logging_context.root)
|
||||
self._batch_feedback = None
|
||||
self._project_facades: dict[str, TaskCenterService] = {}
|
||||
@@ -142,10 +173,14 @@ class TaskCenterService:
|
||||
return self._operation("export_bom_template", task_id,
|
||||
lambda: export_template(self._store, task_id, path))
|
||||
|
||||
def preview_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None):
|
||||
def preview_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, runtime=None):
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('preview_bom_batch', {'task_id': task_id, 'path': path},
|
||||
stop_event=stop_event, on_progress=on_progress)
|
||||
from .bom_batch import preview_batch
|
||||
return self._idle_quick_run_operation('preview_bom_batch', task_id,
|
||||
lambda: preview_batch(self, task_id, path, **control_options(stop_event, on_progress)))
|
||||
lambda: preview_batch(self, task_id, path, **control_options(stop_event, on_progress),
|
||||
**({'runtime': runtime} if runtime is not None else {})))
|
||||
|
||||
def save_bom_batch(self, preview, rows_by_task):
|
||||
from .bom_batch import save_batch
|
||||
@@ -186,6 +221,7 @@ class TaskCenterService:
|
||||
# the database. A facade shares the initialized runner instead.
|
||||
child = TaskCenterService(self._store, self._logging, path_opener=self._path_opener,
|
||||
clock=self._clock, quick_run_preparer=child_preparer, runner=self._runner)
|
||||
child._supervised = self._supervised
|
||||
self._project_facades[task_id] = child
|
||||
return self._project_facades[task_id]
|
||||
|
||||
@@ -360,6 +396,8 @@ class TaskCenterService:
|
||||
|
||||
def target_cell_choices(self, task_id: str, new_part_number: str) -> tuple[str, ...]:
|
||||
"""Read candidates for the captured batch; reject a changed project scope."""
|
||||
if self._supervised:
|
||||
return self._supervise('target_cell_choices', {'task_id': task_id, 'new_part_number': new_part_number})
|
||||
from .cell_catalog import read_project_cell_choices
|
||||
part_number = new_part_number.strip()
|
||||
projects = self.target_cell_projects(task_id)
|
||||
@@ -388,10 +426,18 @@ class TaskCenterService:
|
||||
return self._operation("list_projects", task_id, lambda: self._store.list_projects(task_id))
|
||||
|
||||
def collect_target_inventory(self, task_id: str, old_part_numbers: tuple[str, ...] | None = None,
|
||||
*, stop_event=None, on_progress=None):
|
||||
*, stop_event=None, on_progress=None, runtime=None):
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('collect_target_inventory', {'task_id': task_id, 'old_part_numbers': old_part_numbers},
|
||||
stop_event=stop_event, on_progress=on_progress)
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('collect_inventory', lambda owned:
|
||||
self.collect_target_inventory(task_id, old_part_numbers,
|
||||
stop_event=stop_event, on_progress=on_progress, runtime=owned), stop_event, task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.collect_target_inventory(task_id, old_part_numbers, **control_options(stop_event, on_progress))
|
||||
return delegated.collect_target_inventory(task_id, old_part_numbers, runtime=runtime,
|
||||
**control_options(stop_event, on_progress))
|
||||
def collect():
|
||||
from .bom_batch import _assert_editable
|
||||
_assert_editable(self, task_id)
|
||||
@@ -409,7 +455,7 @@ class TaskCenterService:
|
||||
project_path=str(project[0].path)) as metric:
|
||||
report_progress(on_progress, f'正在采集工程:{project[0].display_name}')
|
||||
rows = collect_target_inventory(self._quick_run_preparer, project[0].path, parts,
|
||||
**control_options(stop_event, on_progress))
|
||||
runtime=runtime, **control_options(stop_event, on_progress))
|
||||
check_cancelled(stop_event)
|
||||
metric['completed_units'] = len(rows)
|
||||
with self._store.transaction():
|
||||
@@ -555,11 +601,16 @@ class TaskCenterService:
|
||||
|
||||
return self._operation("expected_probe_path", task_id, expected_path)
|
||||
|
||||
def prepare_quick_run(self, task_id: str) -> QuickRunResult:
|
||||
def prepare_quick_run(self, task_id: str, *, runtime=None) -> QuickRunResult:
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('prepare_quick_run', {'task_id': task_id})
|
||||
"""Create the legacy VBS handoff without starting a task-center trial."""
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('prepare_quick_run', lambda owned:
|
||||
self.prepare_quick_run(task_id, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.prepare_quick_run(task_id)
|
||||
return delegated.prepare_quick_run(task_id, runtime=runtime)
|
||||
def prepare() -> QuickRunResult:
|
||||
task = self._store.get_task(task_id)
|
||||
preparer = (self._quick_run_preparer.ensure_batch_handoff
|
||||
@@ -569,6 +620,7 @@ class TaskCenterService:
|
||||
self._store.list_projects(task_id),
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
**({'runtime': runtime} if task.target_plan_mode else {}),
|
||||
)
|
||||
append_task_event(
|
||||
self._logging,
|
||||
@@ -624,11 +676,16 @@ class TaskCenterService:
|
||||
with _quick_run_lease(self._logging.root):
|
||||
return self._operation("set_value_policy", task_id, update)
|
||||
|
||||
def execute_quick_run(self, task_id: str, *, expected_item: tuple[str, str, str, str] | None = None) -> QuickRunStepResult:
|
||||
def execute_quick_run(self, task_id: str, *, expected_item: tuple[str, str, str, str] | None = None, runtime=None) -> QuickRunStepResult:
|
||||
"""Execute one replacement rule through restoration and verification."""
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('execute_quick_run', {'task_id': task_id, 'expected_item': expected_item})
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('quick_run_step', lambda owned:
|
||||
self.execute_quick_run(task_id, expected_item=expected_item, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.execute_quick_run(task_id, expected_item=expected_item)
|
||||
return delegated.execute_quick_run(task_id, expected_item=expected_item, runtime=runtime)
|
||||
def execute() -> QuickRunStepResult:
|
||||
task = self._store.get_task(task_id)
|
||||
result = self._quick_run_preparer.execute_step(
|
||||
@@ -637,6 +694,7 @@ class TaskCenterService:
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
**({"expected_item": expected_item} if expected_item is not None else {}),
|
||||
runtime=runtime,
|
||||
)
|
||||
self._synchronize_quick_run_progress(task_id)
|
||||
append_task_event(
|
||||
@@ -663,12 +721,26 @@ class TaskCenterService:
|
||||
stop_event: threading.Event,
|
||||
continue_after_backup_failure: bool = False,
|
||||
reuse_project_session: bool = True,
|
||||
runtime=None,
|
||||
):
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('execute_quick_run_queue', {'task_id': task_id, 'expected_item': expected_item,
|
||||
'continue_after_backup_failure': continue_after_backup_failure,
|
||||
'reuse_project_session': reuse_project_session}, stop_event=stop_event)
|
||||
if runtime is None:
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(self._logging.root, 'quick_run_queue', stop_event) as owned_runtime:
|
||||
owned_runtime.bind_logs(self._logging, (task_id,))
|
||||
result = self.execute_quick_run_queue(task_id, expected_item=expected_item,
|
||||
stop_event=stop_event, continue_after_backup_failure=continue_after_backup_failure,
|
||||
reuse_project_session=reuse_project_session, runtime=owned_runtime)
|
||||
owned_runtime.result = result
|
||||
return result
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.execute_quick_run_queue(task_id, expected_item=expected_item,
|
||||
stop_event=stop_event, continue_after_backup_failure=continue_after_backup_failure,
|
||||
reuse_project_session=reuse_project_session)
|
||||
reuse_project_session=reuse_project_session, runtime=runtime)
|
||||
from .batch import execute_batch
|
||||
from .task_queue import assert_task_queue_idle
|
||||
|
||||
@@ -690,6 +762,7 @@ class TaskCenterService:
|
||||
stop_event=stop_event,
|
||||
continue_after_backup_failure=continue_after_backup_failure,
|
||||
reuse_project_session=reuse_project_session,
|
||||
runtime=runtime,
|
||||
),
|
||||
)
|
||||
metric['state'] = 'paused' if result.paused else 'failed' if result.progress.failed_items else 'completed'
|
||||
@@ -700,17 +773,24 @@ class TaskCenterService:
|
||||
if result.paused:
|
||||
self._pause_quick_run_queue(task_id, 'queue paused at item boundary')
|
||||
return result
|
||||
except Exception:
|
||||
except Exception as error:
|
||||
if getattr(error, 'code', None) == 'batch_resources_pending':
|
||||
raise
|
||||
self._pause_quick_run_queue(task_id, 'queue stopped after execution error')
|
||||
raise
|
||||
finally:
|
||||
self._batch_feedback = None
|
||||
|
||||
def prepare_failed_quick_run(self, task_id: str) -> QuickRunProgress:
|
||||
def prepare_failed_quick_run(self, task_id: str, *, runtime=None) -> QuickRunProgress:
|
||||
"""Reset only failed projects and discard the finished queue handoff."""
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('prepare_failed_quick_run', {'task_id': task_id})
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('prepare_failed_quick_run', lambda owned:
|
||||
self.prepare_failed_quick_run(task_id, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.prepare_failed_quick_run(task_id)
|
||||
return delegated.prepare_failed_quick_run(task_id, runtime=runtime)
|
||||
def prepare_retry() -> QuickRunProgress:
|
||||
task = self._store.get_task(task_id)
|
||||
if self._quick_run_preparer.has_interrupted(task_id):
|
||||
@@ -756,6 +836,7 @@ class TaskCenterService:
|
||||
self._store.list_projects(task_id),
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
runtime=runtime,
|
||||
)
|
||||
return self._quick_run_preparer.progress(
|
||||
task_id,
|
||||
@@ -775,10 +856,45 @@ class TaskCenterService:
|
||||
return self._idle_quick_run_operation('start_new_round', task_id,
|
||||
lambda: start_new_round(self, task_id))
|
||||
|
||||
def inspect_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, on_phase=None):
|
||||
def _owned_cad_operation(self, kind, action, stop_event=None, *, task_id=None):
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(self._logging.root, kind, stop_event) as runtime:
|
||||
runtime.bind_logs(self._logging, (task_id,) if task_id is not None else ())
|
||||
result = action(runtime)
|
||||
runtime.result = result
|
||||
return result
|
||||
|
||||
def pending_batch_resources(self):
|
||||
from .batch_runtime import pending_resources
|
||||
from .batch_supervisor import pending_supervisors
|
||||
return (*pending_resources(self._logging.root), *pending_supervisors(self._logging.root))
|
||||
|
||||
def export_diagnostics(self, destination: Path) -> Path:
|
||||
from .diagnostics_bundle import export_diagnostic_bundle
|
||||
return self._operation('export_diagnostics', None,
|
||||
lambda: export_diagnostic_bundle(self._logging.root, destination))
|
||||
|
||||
def cleanup_batch_resources(self):
|
||||
"""Retry owned-resource release without opening or changing any project."""
|
||||
if self._supervised:
|
||||
result = self._supervise('cleanup_batch_resources', {})
|
||||
from .batch_supervisor import reconcile_supervisors
|
||||
reconcile_supervisors(self._logging.root)
|
||||
return self.pending_batch_resources()
|
||||
from .batch_runtime import cleanup_pending_resources
|
||||
from .task_queue import assert_task_queue_idle
|
||||
assert_task_queue_idle(self._logging.root)
|
||||
with _quick_run_lease(self._logging.root):
|
||||
return self._operation('cleanup_batch_resources', None,
|
||||
lambda: cleanup_pending_resources(self._logging.root, logging_context=self._logging))
|
||||
|
||||
def inspect_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, on_phase=None, runtime=None):
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('inspect_bom_batch', {'task_id': task_id, 'path': path},
|
||||
stop_event=stop_event, on_progress=on_progress, on_phase=on_phase)
|
||||
from .bom_import_report import inspect_batch
|
||||
return self._idle_quick_run_operation('inspect_bom_batch', task_id,
|
||||
lambda: inspect_batch(self, task_id, path, on_phase=on_phase,
|
||||
lambda: inspect_batch(self, task_id, path, on_phase=on_phase, runtime=runtime,
|
||||
**control_options(stop_event, on_progress)))
|
||||
|
||||
def _pause_quick_run_queue(self, task_id: str, message: str) -> None:
|
||||
@@ -835,10 +951,15 @@ class TaskCenterService:
|
||||
return delegated.has_interrupted_quick_run(task_id)
|
||||
return self._quick_run_preparer.has_interrupted(task_id)
|
||||
|
||||
def recover_quick_run(self, task_id: str, *, confirmed: bool = False) -> QuickRunStepResult:
|
||||
def recover_quick_run(self, task_id: str, *, confirmed: bool = False, runtime=None) -> QuickRunStepResult:
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('recover_quick_run', {'task_id': task_id, 'confirmed': confirmed})
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('recover_quick_run', lambda owned:
|
||||
self.recover_quick_run(task_id, confirmed=confirmed, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.recover_quick_run(task_id, confirmed=confirmed)
|
||||
return delegated.recover_quick_run(task_id, confirmed=confirmed, runtime=runtime)
|
||||
from .recovery import recover
|
||||
|
||||
def reconcile() -> QuickRunStepResult:
|
||||
@@ -852,7 +973,7 @@ class TaskCenterService:
|
||||
raise TaskCenterError("quick_run_recovery", "工程包含手动执行阶段或失败记录,不能混用自动队列恢复。")
|
||||
result = recover(self._quick_run_preparer, task_id, projects,
|
||||
self._execution_rules(task_id), copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
confirmed=confirmed)
|
||||
confirmed=confirmed, runtime=runtime)
|
||||
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED}:
|
||||
self._store.set_task_status(task_id, TaskStatus.READY, summary="interrupted quick-run reconciled")
|
||||
self._synchronize_quick_run_progress(task_id)
|
||||
@@ -991,6 +1112,11 @@ class TaskCenterService:
|
||||
self._logging.logger.info("operation_complete=%s task_id=%s at=%s", name, task_id or "", completed_timestamp)
|
||||
return result
|
||||
|
||||
def _supervise(self, operation, arguments, **controls):
|
||||
from .batch_supervisor import run_supervised
|
||||
return self._operation(operation, arguments.get('task_id'),
|
||||
lambda: run_supervised(self, operation, arguments, **controls))
|
||||
|
||||
def _idle_quick_run_operation(
|
||||
self,
|
||||
name: str,
|
||||
@@ -1025,6 +1151,8 @@ class TaskCenterService:
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
)
|
||||
if self._supervised:
|
||||
return progress # Only the batch process persists live execution state.
|
||||
by_path = {os.path.normcase(os.path.abspath(str(project.path))): project.project_id for project in projects}
|
||||
project_by_id = {project.project_id: project for project in projects}
|
||||
summary_by_id = {
|
||||
|
||||
@@ -5,7 +5,19 @@ from .errors import TaskCenterError
|
||||
from .schematic_backup import create_schematic_backup
|
||||
|
||||
|
||||
def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_number_to_value):
|
||||
def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_number_to_value, *, runtime=None):
|
||||
from .quick_run import (_queue_value_mode, _value_mode, _read_status_details,
|
||||
_queue_has_mutation_evidence)
|
||||
root = preparer.data_root / 'quick-run'
|
||||
command, status = root / 'command.tsv', root / 'status.tsv'
|
||||
# Reject a frozen-policy conflict before even opening the project tree for
|
||||
# backup. This preflight only reads evidence and cannot archive/rewrite it.
|
||||
if (command.is_file() and status.is_file()
|
||||
and _queue_value_mode(command) != _value_mode(copy_part_number_to_value)
|
||||
and (_read_status_details(status) or any(root.glob('snapshot-*'))
|
||||
or _queue_has_mutation_evidence(root))):
|
||||
raise TaskCenterError('quick_run_pending',
|
||||
'recorded Value policy differs; reconcile the old queue before creating a new task')
|
||||
progress = preparer.progress(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
if progress.queue_complete or progress.next_project_path is None:
|
||||
@@ -15,4 +27,5 @@ def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_nu
|
||||
create_schematic_backup(preparer.data_root, progress.next_project_path,
|
||||
preparer.data_root / 'quick-run-history' / ('batch-' + uuid4().hex))
|
||||
preparer.ensure_batch_handoff(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
copy_part_number_to_value=copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
@@ -125,11 +125,12 @@ class TaskStore:
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self.path = Path(path)
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
from .path_access import internal_path
|
||||
internal_path(self.path.parent).mkdir(parents=True, exist_ok=True)
|
||||
self._lock = threading.RLock()
|
||||
self._transaction_depth = 0
|
||||
self._savepoint_index = 0
|
||||
self._connection = sqlite3.connect(self.path, isolation_level=None, check_same_thread=False)
|
||||
self._connection = sqlite3.connect(internal_path(self.path), isolation_level=None, check_same_thread=False)
|
||||
self._connection.row_factory = sqlite3.Row
|
||||
with self._lock:
|
||||
self._connection.execute("PRAGMA foreign_keys = ON")
|
||||
@@ -418,6 +419,18 @@ class TaskStore:
|
||||
raise TaskCenterError("task_running", "pause a running task before deleting it")
|
||||
return self.set_task_status(task_id, TaskStatus.DELETED)
|
||||
|
||||
def archive_tasks(self, task_ids: Sequence[str]) -> tuple[TaskRecord, ...]:
|
||||
"""Move a safe close-session task set to the recycle bin in one commit."""
|
||||
with self.transaction():
|
||||
archived = []
|
||||
for task_id in task_ids:
|
||||
task = self.get_task(task_id)
|
||||
if task.status in {TaskStatus.RUNNING, TaskStatus.DELETED}:
|
||||
raise TaskCenterError("archive_task_state", "task cannot be archived in its current state")
|
||||
archived.append(self.set_task_status(task_id, TaskStatus.DELETED,
|
||||
error_id=task.error_id, summary=task.summary))
|
||||
return tuple(archived)
|
||||
|
||||
def record_quick_run_progress(
|
||||
self,
|
||||
task_id: str,
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Versioned, allowlisted JSON values exchanged with the batch supervisor."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import fields, is_dataclass
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
from .errors import TaskCenterError
|
||||
|
||||
VERSION = 1
|
||||
OPERATIONS = frozenset({'probe', 'runtime_smoke', 'inspect_bom_batch', 'preview_bom_batch',
|
||||
'collect_target_inventory', 'execute_task_queue', 'execute_quick_run_queue', 'execute_quick_run',
|
||||
'prepare_quick_run', 'prepare_failed_quick_run', 'recover_quick_run', 'cleanup_batch_resources',
|
||||
'target_cell_choices'})
|
||||
|
||||
|
||||
def valid_journal(value, batch_id):
|
||||
def identity(item):
|
||||
return (isinstance(item, dict) and type(item.get('pid')) is int and item['pid'] > 0
|
||||
and isinstance(item.get('creation_time'), str) and bool(item['creation_time']))
|
||||
return (isinstance(value, dict) and value.get('version') == VERSION and value.get('batch_id') == batch_id
|
||||
and isinstance(batch_id, str) and re.fullmatch('[0-9a-f]{32}', batch_id) is not None
|
||||
and isinstance(value.get('attempt_id'), str) and re.fullmatch('[0-9a-f]{32}', value['attempt_id']) is not None
|
||||
and value.get('operation') in OPERATIONS and value.get('state') in {'starting', 'running', 'blocked', 'finished'}
|
||||
and isinstance(value.get('task_ids'), list) and all(isinstance(v, str) and v and Path(v).name == v
|
||||
and v not in {'.', '..'} for v in value['task_ids'])
|
||||
and identity(value.get('parent_process')) and (value.get('process') is None or identity(value['process']))
|
||||
and isinstance(value.get('request_sha256'), str) and re.fullmatch('[0-9a-f]{64}', value['request_sha256']) is not None)
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _types():
|
||||
from . import models, target_plan, quick_run_protocol, bom_batch, bom_import_report, task_queue, bom_import_state, batch
|
||||
modules = (models, target_plan, quick_run_protocol, bom_batch, bom_import_report, task_queue, bom_import_state, batch)
|
||||
return {value.__module__ + '.' + value.__name__: value
|
||||
for module in modules for value in vars(module).values()
|
||||
if isinstance(value, type) and value.__module__ == module.__name__
|
||||
and (is_dataclass(value) or issubclass(value, Enum))}
|
||||
|
||||
|
||||
def encode(value):
|
||||
if isinstance(value, Enum):
|
||||
return {'type': 'enum', 'name': type(value).__module__ + '.' + type(value).__name__, 'value': value.value}
|
||||
if value is None or type(value) in (str, int, float, bool):
|
||||
return value
|
||||
if isinstance(value, Path):
|
||||
return {'type': 'path', 'value': str(value)}
|
||||
if isinstance(value, datetime):
|
||||
return {'type': 'datetime', 'value': value.isoformat()}
|
||||
if type(value) in (tuple, list, frozenset):
|
||||
return {'type': type(value).__name__, 'items': [encode(item) for item in value]}
|
||||
if isinstance(value, dict):
|
||||
return {'type': 'dict', 'items': [[encode(key), encode(item)] for key, item in value.items()]}
|
||||
if is_dataclass(value):
|
||||
name = type(value).__module__ + '.' + type(value).__name__
|
||||
if name in _types():
|
||||
return {'type': 'object', 'name': name,
|
||||
'fields': {field.name: encode(getattr(value, field.name)) for field in fields(value)}}
|
||||
raise ValueError('Unsupported supervisor value: ' + type(value).__name__)
|
||||
|
||||
|
||||
def decode(value):
|
||||
if value is None or type(value) in (str, int, float, bool):
|
||||
return value
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError('Invalid supervisor value')
|
||||
kind = value.get('type')
|
||||
if kind == 'path':
|
||||
return Path(value['value'])
|
||||
if kind == 'datetime':
|
||||
return datetime.fromisoformat(value['value'])
|
||||
if kind in ('tuple', 'list', 'frozenset'):
|
||||
return {'tuple': tuple, 'list': list, 'frozenset': frozenset}[kind](decode(v) for v in value['items'])
|
||||
if kind == 'dict':
|
||||
return {decode(k): decode(v) for k, v in value['items']}
|
||||
cls = _types().get(value.get('name'))
|
||||
if cls is None:
|
||||
raise ValueError('Unknown supervisor type')
|
||||
if kind == 'enum' and issubclass(cls, Enum):
|
||||
return cls(value['value'])
|
||||
if kind == 'object' and is_dataclass(cls):
|
||||
if set(value['fields']) != {f.name for f in fields(cls)}:
|
||||
raise ValueError('Supervisor record fields do not match schema')
|
||||
return cls(**{key: decode(v) for key, v in value['fields'].items()})
|
||||
raise ValueError('Invalid supervisor type')
|
||||
|
||||
|
||||
def encode_error(error, *, depth=0):
|
||||
result = {'code': getattr(error, 'code', 'supervisor_operation'),
|
||||
'message': getattr(error, 'message', str(error)),
|
||||
'error_id': getattr(error, 'error_id', None)}
|
||||
for key in ('bom_report', 'completed_result'):
|
||||
if getattr(error, key, None) is not None:
|
||||
result[key] = encode(getattr(error, key))
|
||||
if depth < 3:
|
||||
for key in ('primary_error', 'cleanup_error'):
|
||||
other = getattr(error, key, None)
|
||||
if other is not None and other is not error:
|
||||
result[key] = encode_error(other, depth=depth + 1)
|
||||
return result
|
||||
|
||||
|
||||
def decode_error(value):
|
||||
error = TaskCenterError(value['code'], value['message'], error_id=value.get('error_id'))
|
||||
for key in ('bom_report', 'completed_result'):
|
||||
if key in value:
|
||||
setattr(error, key, decode(value[key]))
|
||||
for key in ('primary_error', 'cleanup_error'):
|
||||
if key in value:
|
||||
setattr(error, key, decode_error(value[key]))
|
||||
return error
|
||||
|
||||
|
||||
class MessageGate:
|
||||
def __init__(self, batch_id, attempt_id):
|
||||
self.batch_id, self.attempt_id = batch_id, attempt_id
|
||||
self.sequence, self.generation = 0, 1
|
||||
|
||||
def accept(self, value):
|
||||
if (not isinstance(value, dict) or value.get('version') != VERSION
|
||||
or value.get('batch_id') != self.batch_id or value.get('attempt_id') != self.attempt_id
|
||||
or type(value.get('sequence')) is not int or value['sequence'] <= self.sequence
|
||||
or type(value.get('generation')) is not int or value['generation'] < self.generation
|
||||
or value.get('type') not in {'progress', 'phase', 'heartbeat', 'result', 'state'}):
|
||||
return False
|
||||
self.sequence, self.generation = value['sequence'], value['generation']
|
||||
return True
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Validate business evidence before changing any supervisor safety gate."""
|
||||
from .supervisor_protocol import decode, decode_error
|
||||
|
||||
|
||||
def validated_outcome(operation, body):
|
||||
from .batch import BatchRunResult
|
||||
from .bom_import_report import BomImportReport
|
||||
from .bom_batch import BomBatchPreview
|
||||
from .quick_run_protocol import QuickRunResult, QuickRunStepResult, QuickRunProgress
|
||||
from .target_plan import TargetPlanRow
|
||||
from .task_queue import TaskQueueResult
|
||||
if (not isinstance(body, dict) or type(body.get('business_succeeded')) is not bool
|
||||
or body.get('status') not in {'ok', 'error'}):
|
||||
raise ValueError('Invalid supervisor business outcome')
|
||||
result = decode(body['result']) if body['business_succeeded'] else None
|
||||
expected = {'inspect_bom_batch': BomImportReport, 'preview_bom_batch': BomBatchPreview,
|
||||
'execute_task_queue': TaskQueueResult, 'execute_quick_run_queue': BatchRunResult,
|
||||
'execute_quick_run': QuickRunStepResult, 'recover_quick_run': QuickRunStepResult,
|
||||
'prepare_quick_run': QuickRunResult, 'prepare_failed_quick_run': QuickRunProgress,
|
||||
'collect_target_inventory': tuple, 'target_cell_choices': tuple,
|
||||
'cleanup_batch_resources': tuple, 'probe': dict, 'runtime_smoke': dict}
|
||||
if body['business_succeeded'] and not isinstance(result, expected[operation]):
|
||||
raise ValueError('Supervisor result type does not match requested operation')
|
||||
if body['business_succeeded'] and operation == 'collect_target_inventory' and not all(isinstance(v, TargetPlanRow) for v in result):
|
||||
raise ValueError('Invalid inventory plan result')
|
||||
errors = []
|
||||
for name in ('error', 'operation_error'):
|
||||
value = body.get(name)
|
||||
if value is not None and (not isinstance(value, dict) or not isinstance(value.get('code'), str)
|
||||
or not isinstance(value.get('message'), str)):
|
||||
raise ValueError('Invalid supervisor failure')
|
||||
errors.append(decode_error(value) if value is not None else None)
|
||||
if not body['business_succeeded'] and not any(errors):
|
||||
raise ValueError('Failed supervisor outcome has no failure evidence')
|
||||
return result, *errors
|
||||
|
||||
|
||||
def resource_pending_error(operation, result, error, primary_error=None):
|
||||
from dataclasses import replace
|
||||
from .batch_resources import BatchResourceCleanupError
|
||||
from .bom_import_report import BomImportReport
|
||||
if isinstance(result, BomImportReport):
|
||||
result = replace(result, preview=None)
|
||||
failure = BatchResourceCleanupError('后台资源仍待清理;报告和已保存核验结果保留。' + (str(error) if error else ''),
|
||||
completed_result=result, primary_error=primary_error, cleanup_error=error)
|
||||
if isinstance(result, BomImportReport):
|
||||
failure.bom_report = result
|
||||
elif getattr(error, 'bom_report', None) is not None:
|
||||
failure.bom_report = replace(error.bom_report, preview=None)
|
||||
return failure
|
||||
@@ -0,0 +1,57 @@
|
||||
"""Recoverable task-list archival performed only during an opted-in normal close."""
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .models import ProjectStatus, TaskStatus
|
||||
from .path_access import internal_path
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ArchiveReport:
|
||||
archived_ids: tuple[str, ...]
|
||||
retained: tuple[tuple[str, str], ...] = ()
|
||||
|
||||
@property
|
||||
def retained_ids(self):
|
||||
return tuple(task_id for task_id, _ in self.retained)
|
||||
|
||||
|
||||
def archive_completed_session(service):
|
||||
"""Soft-delete safe tasks atomically; preserve recovery tasks and all files."""
|
||||
return service._idle_quick_run_operation('archive_completed_session', None,
|
||||
lambda: _archive(service))
|
||||
|
||||
|
||||
def _archive(service):
|
||||
store = service._store
|
||||
pending = service.pending_batch_resources()
|
||||
owned = set()
|
||||
unknown = False
|
||||
for record in pending:
|
||||
task_ids = record.get('log_task_ids') or record.get('task_ids')
|
||||
if not isinstance(task_ids, (list, tuple)) or not task_ids or any(
|
||||
not isinstance(task_id, str) or not task_id for task_id in task_ids):
|
||||
unknown = True
|
||||
else:
|
||||
owned.update(task_ids)
|
||||
retained, safe = [], []
|
||||
with store.transaction():
|
||||
for task in store.list_tasks():
|
||||
reason = None
|
||||
if unknown or task.task_id in owned:
|
||||
reason = '后台资源尚未确认释放;重新打开后使用“重新清理资源”。'
|
||||
elif task.status is TaskStatus.RUNNING:
|
||||
reason = '任务仍在执行状态;重新打开后暂停或核验恢复。'
|
||||
elif any(project.status in {ProjectStatus.CAPTURING_BEFORE, ProjectStatus.AWAITING_OPERATOR,
|
||||
ProjectStatus.CAPTURING_AFTER, ProjectStatus.VERIFYING}
|
||||
for project in store.list_projects(task.task_id)):
|
||||
reason = '人工流程尚未结束;重新打开后完成核验。'
|
||||
elif (service.has_interrupted_quick_run(task.task_id) or
|
||||
internal_path(service._project_task_service(task.task_id)._quick_run_preparer.data_root
|
||||
/ 'quick-run' / 'uncertain.flag').exists()):
|
||||
reason = '存在中断、保护或待核验结果;重新打开后使用“核验并恢复”。'
|
||||
if reason:
|
||||
retained.append((task.task_id, f'{task.name}:{reason}'))
|
||||
else:
|
||||
safe.append(task.task_id)
|
||||
store.archive_tasks(safe)
|
||||
return ArchiveReport(tuple(safe), tuple(retained))
|
||||
@@ -3,6 +3,7 @@
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import asdict, dataclass, is_dataclass
|
||||
import hashlib
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -46,6 +47,8 @@ def _os_lock(handle, acquire):
|
||||
|
||||
|
||||
def _lock_file(root: Path):
|
||||
from .path_access import internal_path
|
||||
root = internal_path(root)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
handle = (root / '.task-queue.execution.lock').open('a+b')
|
||||
if handle.seek(0, os.SEEK_END) == 0:
|
||||
@@ -56,6 +59,13 @@ def _lock_file(root: Path):
|
||||
|
||||
def assert_task_queue_idle(root: Path) -> None:
|
||||
"""Reject external mutations while a queue owns CAD; its runner may call services."""
|
||||
from .batch_supervisor import assert_supervisor_idle
|
||||
try:
|
||||
assert_supervisor_idle(root)
|
||||
except TaskCenterError as error:
|
||||
if error.code != 'supervisor_busy':
|
||||
raise
|
||||
raise TaskCenterError('quick_run_busy', '另一批操作正在执行,请先安全暂停。') from error
|
||||
if _QUEUE_OWNER == get_ident():
|
||||
return
|
||||
if _QUEUE_LOCK.locked():
|
||||
@@ -119,8 +129,17 @@ def _save(path: Path, state: dict) -> None:
|
||||
|
||||
|
||||
def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
queue_id: str | None = None) -> TaskQueueResult:
|
||||
queue_id: str | None = None, runtime=None) -> TaskQueueResult:
|
||||
"""Execute the frozen order; uncertain state stops before the next project."""
|
||||
if runtime is None and getattr(service, '_supervised', False):
|
||||
return service._supervise('execute_task_queue', {'task_ids': task_ids, 'queue_id': queue_id}, stop_event=stop_event)
|
||||
if runtime is None:
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(service._logging.root, 'task_queue', stop_event) as owned_runtime:
|
||||
result = execute_task_queue(service, task_ids, stop_event=stop_event,
|
||||
queue_id=queue_id, runtime=owned_runtime)
|
||||
owned_runtime.result = result
|
||||
return result
|
||||
ids = tuple(task_ids)
|
||||
if not ids or len(set(ids)) != len(ids):
|
||||
raise TaskCenterError('task_queue_selection', '请选择不同的单工程任务。')
|
||||
@@ -136,6 +155,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
project_paths.append(str(projects[0].path.resolve(strict=False)).casefold())
|
||||
if len(set(project_paths)) != len(ids):
|
||||
raise TaskCenterError('task_queue_selection', '不能在同一队列重复处理同一个工程。')
|
||||
runtime.bind_logs(service._logging, ids)
|
||||
hashes = {task_id: _plan_hash(child, task_id) for task_id, child in children.items()}
|
||||
token = queue_id or uuid4().hex
|
||||
if len(token) != 32 or any(char not in '0123456789abcdef' for char in token):
|
||||
@@ -183,8 +203,13 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
expected = (str(progress.next_project_path), progress.next_old_part_number,
|
||||
progress.next_new_part_number,
|
||||
'part_label' if child.get_task(task_id).copy_part_number_to_value else 'preserve')
|
||||
result = child.execute_quick_run_queue(task_id, expected_item=expected,
|
||||
stop_event=stop_event)
|
||||
execute = child.execute_quick_run_queue
|
||||
parameters = inspect.signature(execute).parameters.values()
|
||||
runtime_options = ({'runtime': runtime} if any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in parameters) else {})
|
||||
result = execute(task_id, expected_item=expected, stop_event=stop_event,
|
||||
**runtime_options)
|
||||
if child.has_interrupted_quick_run(task_id) or (
|
||||
result.last_result is not None and result.last_result.state == 'batch_saved_unverified'
|
||||
):
|
||||
|
||||
@@ -4,7 +4,7 @@ from threading import Event
|
||||
from shiboken6 import isValid
|
||||
import time
|
||||
from PySide6.QtCore import QElapsedTimer, QSettings, QSignalBlocker, QThreadPool, QTimer, Qt
|
||||
from PySide6.QtWidgets import (QApplication, QDialog, QHBoxLayout, QMainWindow, QMessageBox,
|
||||
from PySide6.QtWidgets import (QApplication, QCheckBox, QDialog, QFileDialog, QHBoxLayout, QMainWindow, QMessageBox,
|
||||
QPushButton, QLabel, QSplitter, QTabWidget, QVBoxLayout, QWidget)
|
||||
from ..errors import TaskCenterError
|
||||
from ..models import TaskStatus
|
||||
@@ -12,7 +12,7 @@ from ..version import VERSION
|
||||
from .dialogs import _TaskDialog, _MappingDialog
|
||||
from .execution_state import execution_state
|
||||
from .mapping_table import MappingTable
|
||||
from .messages import ERROR_TEXT, _format_live_log_line
|
||||
from .messages import ERROR_TEXT, _format_live_log_line, format_operation_error
|
||||
from .project_plan import DirectoryTaskDialog, ProjectPlanPanel
|
||||
from .project_summary import ProjectSummary
|
||||
from .run_panel import RunPanel
|
||||
@@ -31,6 +31,8 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
self._close_plan_confirmer = close_plan_confirmer or self._confirm_close_plans
|
||||
self._close_pending = False
|
||||
self._close_saving = False
|
||||
self._close_archived = False
|
||||
self._archive_close_notice = ''
|
||||
self._quick_run_confirmer = quick_run_confirmer or self._confirm_quick_run
|
||||
self._task_id = None
|
||||
self._context = None
|
||||
@@ -88,13 +90,21 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
self.run_panel.stop_queue_button.clicked.connect(self._pause_queue)
|
||||
self.run_panel.retry_button.clicked.connect(self._retry_failed)
|
||||
self.run_panel.recover_button.clicked.connect(self._recover_quick_run)
|
||||
self.run_panel.cleanup_resources_button.clicked.connect(self._cleanup_batch_resources)
|
||||
self.run_panel.diagnostics_button.clicked.connect(self._export_diagnostics)
|
||||
self.run_panel.new_round_button.clicked.connect(self._start_new_round)
|
||||
self.run_panel.history_button.clicked.connect(self._open_round_history)
|
||||
self.run_panel.copy_value_check.toggled.connect(self._value_option_changed)
|
||||
self.run_panel.reset_layout_button.clicked.connect(self._reset_layout)
|
||||
self.run_panel.toggle_log_button.toggled.connect(self._log_visibility_changed)
|
||||
self.archive_on_close_check = QCheckBox('正常关闭时归档并清空当前任务列表(可从回收站恢复)',
|
||||
objectName='archiveOnNormalClose')
|
||||
self.archive_on_close_check.setToolTip('默认关闭。仅归档安全任务,保留计划、日志和文件;待核验或资源未释放的任务保留在当前列表。')
|
||||
self._build_layout()
|
||||
self._layout_settings = QSettings(str(service.layout_settings_path), QSettings.Format.IniFormat)
|
||||
self.archive_on_close_check.setChecked(self._layout_settings.value(
|
||||
'tasks/archive_on_close', False, type=bool))
|
||||
self.archive_on_close_check.toggled.connect(self._set_archive_on_close)
|
||||
sort_index = self.task_list.sort.findData(self._layout_settings.value('tasks/sort', 'execution'))
|
||||
if sort_index >= 0:
|
||||
self.task_list.sort.setCurrentIndex(sort_index)
|
||||
@@ -105,6 +115,8 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
widget.restoreState(saved)
|
||||
self._refresh_task_list()
|
||||
self._update_gates()
|
||||
if getattr(service, '_supervised', False):
|
||||
QTimer.singleShot(0, self._startup_resource_recheck)
|
||||
|
||||
@property
|
||||
def _loading(self):
|
||||
@@ -143,6 +155,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
details_layout = QVBoxLayout(details)
|
||||
details_layout.setContentsMargins(0, 0, 0, 0)
|
||||
details_layout.addWidget(self.run_panel.copy_value_check)
|
||||
details_layout.addWidget(self.archive_on_close_check)
|
||||
details_layout.addWidget(self.detail_splitter, 1)
|
||||
self.workspace_splitter = QSplitter(Qt.Orientation.Horizontal)
|
||||
self.workspace_splitter.setChildrenCollapsible(False)
|
||||
@@ -176,6 +189,20 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
box.exec()
|
||||
return 'save' if box.clickedButton() is save else 'discard' if box.clickedButton() is discard else 'cancel'
|
||||
|
||||
def _set_archive_on_close(self, enabled):
|
||||
self._layout_settings.setValue('tasks/archive_on_close', bool(enabled))
|
||||
self._layout_settings.sync()
|
||||
|
||||
def _session_archived(self, report):
|
||||
self._close_archived = True
|
||||
self._close_pending = True
|
||||
if report.retained:
|
||||
self._archive_close_notice = ('以下任务未归档,重新打开后仍保留在任务列表,可继续恢复:\n'
|
||||
+ '\n'.join(reason for _, reason in report.retained))
|
||||
self._set_operator_message(self._archive_close_notice, 'warning')
|
||||
if self.isVisible() and QApplication.platformName().casefold() != 'offscreen':
|
||||
QMessageBox.warning(self, '已保留待恢复任务', self._archive_close_notice)
|
||||
|
||||
def closeEvent(self, event):
|
||||
if self._close_saving:
|
||||
event.ignore()
|
||||
@@ -205,6 +232,12 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
event.ignore()
|
||||
return
|
||||
self.plan_panel.discard_all()
|
||||
if self.archive_on_close_check.isChecked() and not self._close_archived:
|
||||
from ..task_archive import archive_completed_session
|
||||
event.ignore()
|
||||
self._close_saving = True
|
||||
self._run_async(lambda: archive_completed_session(self.service), self._session_archived)
|
||||
return
|
||||
for name, widget in self._layout_widgets().items():
|
||||
self._layout_settings.setValue(f'single_project/{name}', widget.saveState())
|
||||
self._layout_settings.setValue('tasks/sort', self.task_list.sort.currentData())
|
||||
@@ -285,15 +318,18 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
|
||||
def _update_gates(self):
|
||||
busy = bool(self._workers) or self._loading
|
||||
resources_pending = bool(self.service.pending_batch_resources())
|
||||
self.run_panel.cleanup_resources_button.setVisible(resources_pending)
|
||||
self.run_panel.cleanup_resources_button.setEnabled(resources_pending and not busy)
|
||||
state = execution_state(self._context, busy=busy, dirty=self.plan_panel.table.dirty)
|
||||
group_unsaved = self._group_has_unsaved_plans() if state.editable or state.new_round else False
|
||||
context = self._context
|
||||
self.run_panel.quick_run_button.setText(state.execute_text)
|
||||
self.run_panel.quick_run_button.setEnabled(state.execute)
|
||||
self.run_panel.quick_run_button.setEnabled(state.execute and not resources_pending)
|
||||
self.run_panel.retry_button.setVisible(bool(context and context.phase == 'retry'))
|
||||
self.run_panel.retry_button.setEnabled(state.retry)
|
||||
self.run_panel.retry_button.setEnabled(state.retry and not resources_pending)
|
||||
self.run_panel.recover_button.setVisible(bool(context and context.interrupted))
|
||||
self.run_panel.recover_button.setEnabled(state.recover)
|
||||
self.run_panel.recover_button.setEnabled(state.recover and not resources_pending)
|
||||
self.run_panel.new_round_button.setVisible(bool(context and context.task.status is TaskStatus.COMPLETED))
|
||||
self.run_panel.new_round_button.setEnabled(state.new_round and not group_unsaved)
|
||||
self.run_panel.history_button.setVisible(bool(context and
|
||||
@@ -305,6 +341,9 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
rule_editable = state.editable and not group_unsaved
|
||||
self.mapping_table.set_rules_editable(rule_editable)
|
||||
self.plan_panel.set_plan_editable(state.editable, busy=busy)
|
||||
if resources_pending:
|
||||
self.plan_panel.collect_button.setEnabled(False)
|
||||
self.plan_panel.import_bom_button.setEnabled(False)
|
||||
self.add_mapping_button.setEnabled(rule_editable)
|
||||
selected = self.mapping_table.current_rule_id() is not None
|
||||
for button in (self.edit_mapping_button, self.delete_mapping_button,
|
||||
@@ -331,7 +370,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
and t.status is not TaskStatus.DELETED}
|
||||
group_dirty = self.plan_panel.has_unsaved_plan(group_ids)
|
||||
group_active = any(t.status in {TaskStatus.RUNNING, TaskStatus.PAUSED} for t in self._tasks if t.task_id in group_ids)
|
||||
self.task_list.queue_button.setEnabled(grouped and not busy and not group_dirty and not bool(context.interrupted))
|
||||
self.task_list.queue_button.setEnabled(grouped and not busy and not group_dirty and not bool(context.interrupted) and not resources_pending)
|
||||
for button in (self.task_list.up_button, self.task_list.down_button):
|
||||
button.setEnabled(grouped and not busy and not group_active
|
||||
and self.task_list.sort.currentData() == 'execution'
|
||||
@@ -342,7 +381,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
if self._loading and not self._message:
|
||||
self.run_panel.operator_band.setText('正在读取任务,可继续选择其他任务。')
|
||||
if not busy:
|
||||
self.run_panel.execution_status.setText('任务读取失败' if read_failed else
|
||||
self.run_panel.execution_status.setText('资源待清理' if resources_pending else '任务读取失败' if read_failed else
|
||||
'结果待核验' if context and context.interrupted else '空闲')
|
||||
|
||||
def _group_has_unsaved_plans(self):
|
||||
@@ -573,6 +612,36 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
self._set_operator_message(f"任务「{name}」{'已暂停' if result.paused else '已结束'}:已保存 {p.saved_items},失败 {p.failed_items},未命中 {p.no_match_items}。",
|
||||
'warning' if result.paused or p.failed_items else 'success')
|
||||
|
||||
def _cleanup_batch_resources(self):
|
||||
if not self.run_panel.cleanup_resources_button.isEnabled():
|
||||
return
|
||||
def done(pending):
|
||||
self._set_operator_message(
|
||||
'后台资源仍待清理,请保留诊断记录后重试。' if pending else '后台资源已释放,可以继续。',
|
||||
'warning' if pending else 'success')
|
||||
self._update_gates()
|
||||
self._run_async(self.service.cleanup_batch_resources, done,
|
||||
self.run_panel.cleanup_resources_button)
|
||||
|
||||
def _startup_resource_recheck(self):
|
||||
if self._closed:
|
||||
return
|
||||
if self._workers or self._loading:
|
||||
QTimer.singleShot(250, self._startup_resource_recheck)
|
||||
return
|
||||
if self.service.pending_batch_resources():
|
||||
self._update_gates()
|
||||
self._cleanup_batch_resources() # Resource-only; never resumes a queue.
|
||||
|
||||
def _export_diagnostics(self):
|
||||
from pathlib import Path
|
||||
name = 'DxPartReplace-诊断-' + time.strftime('%Y%m%d-%H%M%S') + '.zip'
|
||||
value, _ = QFileDialog.getSaveFileName(self, '导出脱敏诊断包', name, 'ZIP 文件 (*.zip)')
|
||||
if value:
|
||||
self._run_async(lambda: self.service.export_diagnostics(Path(value)),
|
||||
lambda path: self._set_operator_message(f'脱敏诊断包已导出:{path}', 'success'),
|
||||
self.run_panel.diagnostics_button)
|
||||
|
||||
def _retry_failed(self):
|
||||
if self.run_panel.retry_button.isEnabled():
|
||||
task_id = self._task_id
|
||||
@@ -765,7 +834,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
|
||||
def _show_error(self, error, *, owner_task_id=None, **_unused):
|
||||
stable = error if isinstance(error, TaskCenterError) else self.service.report_ui_error(error)
|
||||
message = f'{ERROR_TEXT.get(stable.code, "操作未完成")}\n{stable.message}\n错误编号:{stable.error_id}'
|
||||
message = format_operation_error(stable)
|
||||
if owner_task_id:
|
||||
name = next((t.name for t in self._tasks if t.task_id == owner_task_id), owner_task_id)
|
||||
message = f'任务「{name}」:{message}'
|
||||
|
||||
@@ -4,6 +4,10 @@ import json
|
||||
import re
|
||||
|
||||
ERROR_TEXT = {
|
||||
"batch_resources_pending": "后台资源尚未确认释放,请使用“重新清理资源”后再开始下一批。",
|
||||
"supervisor_busy": "另一批操作正在执行,请先安全收尾。",
|
||||
"supervisor_interrupted": "批次监督连接异常,保护记录已保留,没有自动重放替换。",
|
||||
"inventory_session": "原生读取会话异常,安全重试未完成,已停止本批处理。请查看诊断记录并排除原因后重试。",
|
||||
"task_required": "请先选择任务。",
|
||||
"directory_required": "请先选择工程目录。",
|
||||
"discovery_root_locked": "旧目录已有执行记录,不能在同一任务中更换目录;请新建任务。",
|
||||
@@ -48,6 +52,28 @@ ERROR_TEXT = {
|
||||
"task_name": "任务名称为空或已经存在。",
|
||||
}
|
||||
|
||||
|
||||
def format_operation_error(error):
|
||||
title = ERROR_TEXT.get(error.code, '操作未完成')
|
||||
report = getattr(error, 'bom_report', None)
|
||||
if report is not None:
|
||||
title = 'BOM 导入未完成,校验报告已保留。' + title
|
||||
result = getattr(error, 'completed_result', None)
|
||||
if (getattr(result, 'state', None) == 'saved_ok'
|
||||
or getattr(getattr(result, 'progress', None), 'saved_items', 0)
|
||||
or any(getattr(entry, 'state', None) == 'completed' for entry in getattr(result, 'entries', ()))):
|
||||
title += ' 已保存核验的结果保留,不会重新替换。'
|
||||
lines = [title]
|
||||
for label, name, limit in (('首次错误', 'primary_error', 180), ('收尾错误', 'cleanup_error', 140)):
|
||||
failure = getattr(error, name, None)
|
||||
if failure is not None:
|
||||
text = getattr(failure, 'message', str(failure)).replace('\n', ' ')
|
||||
lines.append(f'{label}:{text[:limit]}' + ('…' if len(text) > limit else ''))
|
||||
if len(lines) == 1:
|
||||
lines.append(error.message[:260])
|
||||
lines.append('错误编号:' + error.error_id)
|
||||
return '\n'.join(lines)
|
||||
|
||||
DIALOG_ERROR_TEXT = {
|
||||
"duplicate_source": "该旧料号已经存在,请修改现有规则或输入其他旧料号。",
|
||||
"mapping_chain": "目标料号同时是另一条规则的旧料号,请拆分任务后再执行。",
|
||||
|
||||
@@ -25,6 +25,10 @@ class RunPanel(QWidget):
|
||||
self.stop_queue_button = QPushButton('完成当前项后暂停', objectName='stopQueueButton')
|
||||
self.retry_button = QPushButton('重试失败项', objectName='retryTaskItems')
|
||||
self.recover_button = QPushButton('核验并恢复', objectName='recoverQuickRunButton')
|
||||
self.cleanup_resources_button = QPushButton('重新清理资源', objectName='cleanupBatchResources')
|
||||
self.cleanup_resources_button.setToolTip('只释放本程序拥有的后台资源,不执行工程替换。')
|
||||
self.diagnostics_button = QPushButton('导出诊断包', objectName='exportDiagnosticBundle')
|
||||
self.diagnostics_button.setToolTip('导出脱敏日志与资源记录,不包含工程、中心库、BOM 或任务数据库。')
|
||||
self.new_round_button = QPushButton('开始新一轮', objectName='startNewRoundButton')
|
||||
self.history_button = QPushButton('查看轮次证据', objectName='roundHistoryButton')
|
||||
self.copy_value_check = QCheckBox('Value 策略:替换后将 Part Label 写入 Value(逐行自定义优先)', objectName='copyPartNumberToValue')
|
||||
@@ -37,7 +41,7 @@ class RunPanel(QWidget):
|
||||
self.log_dialog = None
|
||||
self.toggle_log_button.toggled.connect(self._toggle_log)
|
||||
buttons = QHBoxLayout()
|
||||
for button in (self.quick_run_button, self.retry_button, self.recover_button,
|
||||
for button in (self.quick_run_button, self.retry_button, self.recover_button, self.cleanup_resources_button, self.diagnostics_button,
|
||||
self.new_round_button, self.history_button, self.stop_queue_button):
|
||||
buttons.addWidget(button)
|
||||
buttons.addStretch()
|
||||
|
||||
@@ -1 +1 @@
|
||||
VERSION = "1.1.3"
|
||||
VERSION = "1.1.4-beta.3"
|
||||
@@ -2,9 +2,11 @@
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
|
||||
from .path_access import internal_path
|
||||
|
||||
|
||||
def copy_worker_script(source: Path, destination: Path) -> None:
|
||||
source, destination = Path(source), Path(destination)
|
||||
source, destination = internal_path(source), internal_path(destination)
|
||||
if source.resolve() == destination.resolve():
|
||||
return
|
||||
shutil.copyfile(source, destination)
|
||||
|
||||
@@ -33,6 +33,42 @@ def test_bom_resource_verifier_accepts_cli_only_reads_without_cad(native_bom_evi
|
||||
assert verifier.verify_bom_resources(root)=={'cad_sessions':0,'ready_native_cli_reads':2}
|
||||
|
||||
|
||||
def test_desktop_observer_discovers_batch_resource_records(tmp_path, monkeypatch):
|
||||
monkeypatch.syspath_prepend(str(Path(__file__).resolve().parents[2] / 'tools'))
|
||||
from private_desktop_observer import PrivateDesktopObserver
|
||||
|
||||
path = tmp_path / 'data/batch-resources/batch/session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(json.dumps({'desktop': 'dx-observer-test', 'cad_pid': None, 'state': 'ready'}))
|
||||
observer = PrivateDesktopObserver(tmp_path)
|
||||
observer._sample()
|
||||
assert observer.records['dx-observer-test']['evidence_path'] == str(path.relative_to(tmp_path))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault', [None, 'pending', 'unbound', 'workers_open'])
|
||||
def test_bom_resource_verifier_checks_batch_release_and_operation_binding(native_bom_evidence, fault):
|
||||
verifier, root, rows = native_bom_evidence
|
||||
operations = {}
|
||||
for i, row in enumerate(rows):
|
||||
path = root / f'data/tasks/{i}/quick-run-history/native-inventory-ok/native-inventory.json'
|
||||
row['cleanup'].update(scope='operation', workers_closed=True, desktop_retained_by_batch=True)
|
||||
path.write_text(json.dumps(row))
|
||||
operations[str(path.parent.resolve())] = {'workers_closed': fault != 'workers_open',
|
||||
'binding': {'session_id': 'batch', 'generation': 1, 'attempt_id': str(i)}}
|
||||
if fault == 'unbound':
|
||||
operations.clear()
|
||||
path = root / 'data/batch-resources/batch/session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(json.dumps(dict(state='blocked' if fault == 'pending' else 'released',
|
||||
session_id='batch', generation=1, native_operations=operations,
|
||||
cleanup=dict(desktop_handle_closed=True, desktop_confirmation={'released':True}, errors=[]))))
|
||||
if fault:
|
||||
with pytest.raises(AssertionError):
|
||||
verifier.verify_bom_resources(root)
|
||||
else:
|
||||
assert verifier.verify_bom_resources(root)['ready_native_cli_reads'] == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault',['uncertain','fallback_active_job','fallback_retained_workspace'])
|
||||
def test_bom_resource_verifier_rejects_bad_earlier_attempt_despite_clean_retries(native_bom_evidence,fault):
|
||||
verifier,root,rows=native_bom_evidence
|
||||
|
||||
@@ -4,6 +4,8 @@ from pathlib import Path
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import json
|
||||
import pytest
|
||||
|
||||
|
||||
def test_default_data_root_uses_windows_known_folder_without_environment(tmp_path: Path, monkeypatch) -> None:
|
||||
@@ -189,3 +191,30 @@ def test_icon_uses_package_location_for_frozen_script_entrypoint(tmp_path, monke
|
||||
assert not qapp.windowIcon().isNull()
|
||||
assert not qapp.windowIcon().pixmap(32, 32).isNull()
|
||||
assert qapp.applicationVersion() == application.VERSION
|
||||
|
||||
|
||||
@pytest.mark.parametrize('missing_dependency', [False, True])
|
||||
def test_runtime_smoke_tracks_real_process_and_releases_on_failure(tmp_path, monkeypatch, missing_dependency):
|
||||
from dx_productivity.trial_task_center.application import _smoke_batch_runtime
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
import win32process
|
||||
|
||||
if missing_dependency:
|
||||
def fail(_handle):
|
||||
raise ModuleNotFoundError("No module named 'win32timezone'")
|
||||
monkeypatch.setattr(win32process, 'GetProcessTimes', fail)
|
||||
with pytest.raises(ModuleNotFoundError, match='win32timezone'):
|
||||
_smoke_batch_runtime(tmp_path, configure_logging(tmp_path))
|
||||
else:
|
||||
_smoke_batch_runtime(tmp_path, configure_logging(tmp_path))
|
||||
paths = list((tmp_path / 'batch-resources').glob('*/session.json'))
|
||||
if missing_dependency:
|
||||
# The GUI now verifies supervisor identity before launching anything.
|
||||
assert not paths
|
||||
return
|
||||
assert len(paths) == 1
|
||||
record = json.loads(paths[0].read_text(encoding='utf-8'))
|
||||
assert record['state'] == 'released' and record['cad_pid'] is None
|
||||
assert record['cleanup']['desktop_handle_closed']
|
||||
assert record['cleanup']['desktop_confirmation']['released']
|
||||
assert not record['cleanup']['errors']
|
||||
@@ -355,6 +355,7 @@ def test_batch_clears_worker_current_state_after_saved_completion(tmp_path):
|
||||
|
||||
def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp_path, monkeypatch):
|
||||
import dx_productivity.trial_task_center.batch as batch_module
|
||||
import dx_productivity.trial_task_center.quick_run as quick_run_module
|
||||
|
||||
project_count = 20
|
||||
sheet_count = 5
|
||||
@@ -396,6 +397,15 @@ def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp
|
||||
|
||||
initial = service.quick_run_progress(task.task_id)
|
||||
expected = (str(initial.next_project_path), 'OLD-00', 'NEW-00', 'preserve')
|
||||
reads = {'_read_commands': 0, '_read_status_details': 0}
|
||||
def observe(name):
|
||||
original = getattr(quick_run_module, name)
|
||||
def counted(*args, **kwargs):
|
||||
reads[name] += 1
|
||||
return original(*args, **kwargs)
|
||||
monkeypatch.setattr(quick_run_module, name, counted)
|
||||
for name in reads:
|
||||
observe(name)
|
||||
tracemalloc.start()
|
||||
started = time.perf_counter()
|
||||
result = service.execute_quick_run_queue(task.task_id, expected_item=expected, stop_event=Event())
|
||||
@@ -410,13 +420,17 @@ def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp
|
||||
'sheets_per_project': sheet_count, 'rules_per_sheet': rule_count,
|
||||
'steps': total, 'saved_steps': saved, 'elapsed_seconds': elapsed,
|
||||
'peak_bytes': peak, 'steps_per_second': total / elapsed,
|
||||
'full_queue_parses': reads,
|
||||
}, indent=2), encoding='utf-8')
|
||||
assert result.progress.queue_complete
|
||||
assert result.progress.total_items == total
|
||||
assert result.progress.saved_items == saved
|
||||
assert result.progress.no_match_items == total - saved
|
||||
assert len(calls) == total and len(set(calls)) == total
|
||||
assert elapsed < 30
|
||||
# Wall-clock budgets depend on the machine and tracemalloc overhead. Guard
|
||||
# the repeated full-file work directly; keep timings as benchmark evidence.
|
||||
assert reads['_read_commands'] <= 2
|
||||
assert reads['_read_status_details'] <= 2
|
||||
assert peak < 128 * 1024 * 1024
|
||||
|
||||
|
||||
|
||||
@@ -360,16 +360,15 @@ def test_execute_quick_step_launches_automatic_runtime_and_reports_completion(tm
|
||||
|
||||
result = preparer.execute_step("task", (project,), (rule,))
|
||||
|
||||
assert calls == [
|
||||
((
|
||||
"C:\\Windows\\SysWOW64\\cscript.exe",
|
||||
"//nologo",
|
||||
str(result.script_path),
|
||||
str(tmp_path / "quick-run"),
|
||||
*_native_helper_launch_arguments(),
|
||||
_command_field(result.command_path, "job_id"),
|
||||
), 150)
|
||||
]
|
||||
assert len(calls) == 1
|
||||
command, timeout = calls[0]
|
||||
assert timeout == 150
|
||||
assert command[:2] == ("C:\\Windows\\SysWOW64\\cscript.exe", "//nologo")
|
||||
# Long evidence paths legitimately use 8.3 names or a managed alias.
|
||||
assert Path(command[2]).resolve() == result.script_path.resolve()
|
||||
assert Path(command[3]).resolve() == (tmp_path / 'quick-run').resolve()
|
||||
assert command[4:] == (*_native_helper_launch_arguments(),
|
||||
_command_field(result.command_path, 'job_id'))
|
||||
assert result.state == "restored_ok"
|
||||
assert result.message == "Replaced and verified 2 current-sheet matches."
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
from pathlib import Path
|
||||
from dx_productivity.trial_task_center.path_access import strip_extended_prefix
|
||||
import pytest
|
||||
from dx_productivity.trial_task_center.models import ProjectRecord, ProjectStatus, RuleRecord
|
||||
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer
|
||||
@@ -21,7 +22,7 @@ def test_snapshot_reads_each_file_once_and_uses_same_status(tmp_path, monkeypatc
|
||||
opens = []
|
||||
original = Path.open
|
||||
def tracked(path, *args, **kwargs):
|
||||
if path.parent == handoff.command_path.parent:
|
||||
if Path(strip_extended_prefix(path)).parent == handoff.command_path.parent:
|
||||
opens.append(path.name)
|
||||
return original(path, *args, **kwargs)
|
||||
monkeypatch.setattr(Path, 'open', tracked)
|
||||
@@ -48,7 +49,7 @@ def test_snapshot_cancellation_stops_before_next_file_and_never_writes(tmp_path,
|
||||
def tracked(path, *args, **kwargs):
|
||||
opened.append(path.name)
|
||||
result = original(path, *args, **kwargs)
|
||||
if path == handoff.command_path:
|
||||
if Path(strip_extended_prefix(path)) == handoff.command_path:
|
||||
control.cancelled = True
|
||||
return result
|
||||
monkeypatch.setattr(Path, 'open', tracked)
|
||||
@@ -91,7 +92,7 @@ def test_snapshot_retries_when_uncertain_flag_is_created_during_status_read(tmp_
|
||||
injected = False
|
||||
def racing_open(path, *args, **kwargs):
|
||||
nonlocal injected
|
||||
if path == handoff.status_path and args and args[0] == 'r' and not injected:
|
||||
if Path(strip_extended_prefix(path)) == handoff.status_path and args and args[0] == 'r' and not injected:
|
||||
injected = True
|
||||
(path.parent / 'uncertain.flag').write_text('uncertain', encoding='utf-8')
|
||||
return original(path, *args, **kwargs)
|
||||
@@ -108,7 +109,7 @@ def test_snapshot_retries_command_and_status_as_one_generation(tmp_path, monkeyp
|
||||
injected = False
|
||||
def racing_open(path, *args, **kwargs):
|
||||
nonlocal injected
|
||||
if path == handoff.status_path and args and args[0] == 'r' and not injected:
|
||||
if Path(strip_extended_prefix(path)) == handoff.status_path and args and args[0] == 'r' and not injected:
|
||||
injected = True
|
||||
handoff.command_path.write_text(command.replace(job, 'new-generation'), encoding='utf-8')
|
||||
handoff.status_path.write_text('job_id\tstate\tcode\tmessage\nnew-generation\tno_match\t\tno match\n', encoding='utf-8')
|
||||
@@ -126,7 +127,7 @@ def test_snapshot_continuous_changes_block_continuation_after_bounded_retries(tm
|
||||
reads = 0
|
||||
def racing_open(path, *args, **kwargs):
|
||||
nonlocal reads
|
||||
if path == handoff.status_path and args and args[0] == 'r':
|
||||
if Path(strip_extended_prefix(path)) == handoff.status_path and args and args[0] == 'r':
|
||||
reads += 1
|
||||
(path.parent / 'changing.txt').write_text('x' * reads, encoding='utf-8')
|
||||
return original(path, *args, **kwargs)
|
||||
|
||||
@@ -7,6 +7,7 @@ import subprocess
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.path_access import strip_extended_prefix
|
||||
from dx_productivity.trial_task_center.scanner import scan_projects
|
||||
|
||||
|
||||
@@ -58,7 +59,7 @@ def test_scan_projects_stops_at_selected_project_directory_when_it_contains_prj_
|
||||
real_scandir = os.scandir
|
||||
|
||||
def reject_nested(path: str | bytes | int) -> object:
|
||||
if Path(path) != root:
|
||||
if Path(strip_extended_prefix(path)) != root:
|
||||
raise AssertionError(f"unexpected nested scan: {path}")
|
||||
return real_scandir(path)
|
||||
|
||||
@@ -84,7 +85,7 @@ def test_scan_projects_reads_batch_root_once_before_recursive_fallback(
|
||||
|
||||
def count_root(path: str | bytes | int) -> object:
|
||||
nonlocal root_reads
|
||||
if Path(path) == root:
|
||||
if Path(strip_extended_prefix(path)) == root:
|
||||
root_reads += 1
|
||||
return real_scandir(path)
|
||||
|
||||
@@ -138,7 +139,7 @@ def test_scan_projects_turns_inaccessible_subtree_errors_into_explicit_task_erro
|
||||
real_scandir = __import__("os").scandir
|
||||
|
||||
def deny(path: str | bytes | int) -> object:
|
||||
if Path(path) == denied:
|
||||
if Path(strip_extended_prefix(path)) == denied:
|
||||
raise PermissionError("denied subtree")
|
||||
return real_scandir(path)
|
||||
|
||||
|
||||
@@ -78,14 +78,19 @@ def test_result_for_other_queued_project_is_rejected(tmp_path):
|
||||
assert (tmp_path / 'quick-run/uncertain.flag').exists()
|
||||
|
||||
|
||||
def test_policy_change_cannot_reinterpret_partial_queue(tmp_path):
|
||||
def test_policy_change_cannot_reinterpret_partial_queue(tmp_path, monkeypatch):
|
||||
projects, rules = inputs()
|
||||
preparer = QuickRunPreparer(tmp_path, script_source=SOURCE)
|
||||
result = preparer.prepare('task', projects, rules)
|
||||
job = result.command_path.read_text(encoding='utf-8').splitlines()[1].split('\t')[1]
|
||||
result.status_path.write_text(f'job_id\tstate\tcode\tmessage\n{job}\tawaiting_native_replace\t\tcaptured\n', encoding='utf-8')
|
||||
before = {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
from dx_productivity.trial_task_center import single_step
|
||||
monkeypatch.setattr(single_step, 'create_schematic_backup',
|
||||
lambda *a, **k: pytest.fail('frozen policy must be rejected before backup or CAD'))
|
||||
with pytest.raises(TaskCenterError, match='quick_run_pending'):
|
||||
preparer.execute_step('task', projects, rules, copy_part_number_to_value=True)
|
||||
assert before == {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
|
||||
|
||||
def test_service_freezes_setting_after_queue_is_prepared(tmp_path):
|
||||
@@ -189,7 +194,7 @@ def test_serial_three_project_two_rule_queue_pins_each_job(tmp_path):
|
||||
assert preparer.progress('task', projects, rules).queue_complete
|
||||
|
||||
|
||||
def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_path):
|
||||
def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_path, monkeypatch):
|
||||
projects, rules = inputs()
|
||||
preparer = QuickRunPreparer(tmp_path, script_source=SOURCE)
|
||||
result = preparer.prepare('task', projects, rules)
|
||||
@@ -200,8 +205,13 @@ def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_pa
|
||||
encoding='utf-8',
|
||||
)
|
||||
(result.command_path.parent / 'snapshot-old.tsv').write_text('retained evidence', encoding='utf-8')
|
||||
before = {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
from dx_productivity.trial_task_center import single_step
|
||||
monkeypatch.setattr(single_step, 'create_schematic_backup',
|
||||
lambda *a, **k: pytest.fail('legacy snapshot must be checked before backup or CAD'))
|
||||
with pytest.raises(TaskCenterError, match='quick_run_pending'):
|
||||
preparer.execute_step('task', projects, rules)
|
||||
assert before == {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
|
||||
|
||||
def test_frozen_launcher_reserves_helper_script_slot(monkeypatch, tmp_path):
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
import json
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import batch_attempt as attempts
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def prepared(tmp_path, monkeypatch):
|
||||
path = tmp_path / 'job.json'
|
||||
record = attempts.prepare_attempt(path, session_id='session-a', generation=1,
|
||||
task_id='task-a', job_id='job-a', project_path=tmp_path / 'board.prj')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(path))
|
||||
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
|
||||
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_ID', record['attempt_id'])
|
||||
monkeypatch.setenv('DXPART_TASK_ID', 'task-a')
|
||||
monkeypatch.setenv('DXPART_JOB_ID', 'job-a')
|
||||
return path, record
|
||||
|
||||
|
||||
def test_only_current_prepared_attempt_can_retry(prepared):
|
||||
path, expected = prepared
|
||||
assert attempts.can_retry_attempt(path, expected)
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
assert attempts.read_attempt(path)['state'] == 'mutation_possible'
|
||||
assert not attempts.can_retry_attempt(path, expected)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value', [('session_id', 'old'), ('generation', 2),
|
||||
('attempt_id', '0' * 32), ('task_id', 'old'), ('job_id', 'old'),
|
||||
('project_path', 'other.prj')])
|
||||
def test_retry_rejects_stale_identity(prepared, field, value):
|
||||
path, expected = prepared
|
||||
assert not attempts.can_retry_attempt(path, dict(expected, **{field: value}))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('contents', [None, '{truncated', '{}', '[]'])
|
||||
def test_unreadable_or_incomplete_proof_never_allows_retry(prepared, contents):
|
||||
path, expected = prepared
|
||||
if contents is None:
|
||||
path.unlink()
|
||||
else:
|
||||
path.write_text(contents)
|
||||
assert not attempts.can_retry_attempt(path, expected)
|
||||
with pytest.raises((OSError, ValueError)):
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('name,value', [('DXPART_BATCH_SESSION_ID', 'old'),
|
||||
('DXPART_BATCH_GENERATION', '2'), ('DXPART_ATTEMPT_ID', '0' * 32),
|
||||
('DXPART_TASK_ID', 'old'), ('DXPART_JOB_ID', 'old')])
|
||||
def test_marker_blocks_stale_process_identity(prepared, monkeypatch, name, value):
|
||||
path, _ = prepared
|
||||
monkeypatch.setenv(name, value)
|
||||
with pytest.raises(ValueError):
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_marker_fsync_failure_blocks_caller_without_publishing_mutation(prepared, monkeypatch):
|
||||
path, _ = prepared
|
||||
def fail(_):
|
||||
raise OSError('durability failed')
|
||||
monkeypatch.setattr(os, 'fsync', fail)
|
||||
with pytest.raises(OSError, match='durability failed'):
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
assert json.loads(path.read_text())['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_legacy_no_environment_remains_supported(monkeypatch):
|
||||
for name in tuple(os.environ):
|
||||
if name.startswith('DXPART_') and ('ATTEMPT' in name or 'BATCH_' in name):
|
||||
monkeypatch.delenv(name)
|
||||
assert attempts.mark_mutation_possible() is None
|
||||
|
||||
|
||||
def test_blank_batch_path_is_not_legacy(monkeypatch):
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', '')
|
||||
with pytest.raises(ValueError):
|
||||
attempts.mark_mutation_possible()
|
||||
|
||||
|
||||
def test_preparation_is_durable_before_return(tmp_path, monkeypatch):
|
||||
fsynced = []
|
||||
real_fsync = os.fsync
|
||||
def fsync(fd):
|
||||
fsynced.append(True)
|
||||
return real_fsync(fd)
|
||||
monkeypatch.setattr(os, 'fsync', fsync)
|
||||
path = tmp_path / 'attempt.json'
|
||||
first = attempts.prepare_attempt(path, session_id='s', generation=1,
|
||||
task_id='t', job_id='j', project_path=tmp_path / 'board.prj')
|
||||
assert fsynced and attempts.read_attempt(path)['state'] == 'prepared'
|
||||
second = attempts.prepare_attempt(tmp_path / 'second.json', session_id='s', generation=1,
|
||||
task_id='t', job_id='j', project_path=tmp_path / 'board.prj')
|
||||
assert first['attempt_id'] != second['attempt_id']
|
||||
|
||||
|
||||
def test_reused_worker_marks_each_logical_job(tmp_path, monkeypatch):
|
||||
directory = tmp_path / 'attempt'
|
||||
for job in ('job-a', 'job-b'):
|
||||
attempts.prepare_attempt(directory / f'{job}.json', session_id='session-a', generation=1,
|
||||
task_id='task-a', job_id=job, project_path=tmp_path / 'board.prj', attempt_id='a' * 32)
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(directory))
|
||||
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
|
||||
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_ID', 'a' * 32)
|
||||
for job in ('job-a', 'job-b'):
|
||||
attempts.mark_mutation_possible(job_id=job, task_id='task-a', project_path=tmp_path / 'board.prj')
|
||||
assert attempts.read_attempt(directory / f'{job}.json')['state'] == 'mutation_possible'
|
||||
|
||||
|
||||
def test_native_guard_binds_snapshot_to_command_and_current_attempt(prepared, tmp_path):
|
||||
from dx_productivity.trial_task_center.native_helper import mark_snapshot_mutation
|
||||
path, _ = prepared
|
||||
snapshot = tmp_path / 'snapshot-job-a.tsv'
|
||||
snapshot.write_text(f'M\t{tmp_path / "board.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
|
||||
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
|
||||
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
|
||||
assert mark_snapshot_mutation(snapshot)['state'] == 'mutation_possible'
|
||||
assert attempts.read_attempt(path)['state'] == 'mutation_possible'
|
||||
|
||||
|
||||
def test_native_guard_rejects_other_project_before_marking(prepared, tmp_path):
|
||||
from dx_productivity.trial_task_center.native_helper import mark_snapshot_mutation
|
||||
path, _ = prepared
|
||||
snapshot = tmp_path / 'snapshot-job-a.tsv'
|
||||
snapshot.write_text(f'M\t{tmp_path / "other.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
|
||||
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
|
||||
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
|
||||
with pytest.raises(ValueError):
|
||||
mark_snapshot_mutation(snapshot)
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_marker_cli_fails_closed_on_stale_job(prepared):
|
||||
from dx_productivity.trial_task_center.application import main
|
||||
path, record = prepared
|
||||
assert main(['--mark-batch-attempt', '--attempt-job', 'other', '--attempt-task', 'task-a',
|
||||
'--attempt-project', record['project_path']]) == 1
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_native_failure_does_not_reuse_previous_attempt_transaction(prepared, tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import native_helper as native
|
||||
path, _ = prepared
|
||||
snapshot = tmp_path / 'snapshot-job-a.tsv'
|
||||
snapshot.write_text(f'M\t{tmp_path / "board.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
|
||||
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
|
||||
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
|
||||
report_path = tmp_path / 'snapshot-job-a.tsv.native.json'
|
||||
report_path.write_text(json.dumps({'stage': 'native_replace_started',
|
||||
'transaction': {'old': 'unrelated'}, 'batch_attempt': {'attempt_id': 'old'}}))
|
||||
def unavailable_cad(snapshot, report):
|
||||
raise RuntimeError('CAD unavailable before mutation')
|
||||
monkeypatch.setattr(native, 'run_native', unavailable_cad)
|
||||
assert native.run(snapshot) == 1
|
||||
report = json.loads(report_path.read_text())
|
||||
assert report['stage'] == 'preflight' and 'transaction' not in report
|
||||
assert report['batch_attempt']['job_id'] == 'job-a'
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
def test_attempt_write_handles_long_unicode_evidence_and_keeps_canonical_identity(tmp_path, monkeypatch):
|
||||
from pathlib import Path
|
||||
directory = tmp_path / ('long-' + 'x' * 110) / ('more-' + 'y' * 80)
|
||||
path = directory / 'attempt.json'
|
||||
real_mkdir = Path.mkdir
|
||||
def strict_mkdir(target, *args, **kwargs):
|
||||
if len(str(target)) >= 260 and not str(target).startswith(chr(92) * 2 + '?'):
|
||||
raise OSError('legacy MAX_PATH')
|
||||
return real_mkdir(target, *args, **kwargs)
|
||||
monkeypatch.setattr(Path, 'mkdir', strict_mkdir)
|
||||
project = tmp_path / 'board.prj'
|
||||
record = attempts.prepare_attempt(path, session_id='session', generation=1,
|
||||
task_id='task', job_id='job', project_path=project)
|
||||
assert record['project_path'] == os.path.normcase(str(project.resolve()))
|
||||
assert attempts.can_retry_attempt(path, record)
|
||||
assert not str(record['project_path']).startswith(chr(92) * 2 + '?')
|
||||
|
||||
|
||||
def test_publication_retries_sharing_violation_only_and_keeps_original_record(tmp_path, monkeypatch):
|
||||
path = tmp_path / 'attempt.json'
|
||||
first = attempts.prepare_attempt(path, session_id='session', generation=1,
|
||||
task_id='task', job_id='job', project_path=tmp_path / 'board.prj')
|
||||
calls = []
|
||||
def fail(source, target):
|
||||
calls.append((source, target))
|
||||
error = PermissionError('sharing violation')
|
||||
error.winerror = 32
|
||||
raise error
|
||||
monkeypatch.setattr(os, 'replace', fail)
|
||||
with pytest.raises(PermissionError):
|
||||
attempts.prepare_attempt(path, session_id='session', generation=2,
|
||||
task_id='task', job_id='job', project_path=tmp_path / 'board.prj')
|
||||
assert len(calls) == 3
|
||||
assert attempts.read_attempt(path) == first
|
||||
assert list(tmp_path.iterdir()) == [path]
|
||||
@@ -0,0 +1,60 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from test_accuracy_vbs_guards import run_vbs
|
||||
from dx_productivity.trial_task_center.batch_attempt import prepare_attempt
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault', ['', 'missing', 'stale', 'wrong-project', 'missing-path'])
|
||||
def test_vbs_guard_prevents_property_write_without_durable_current_proof(tmp_path, monkeypatch, fault):
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
monkeypatch.setenv('PYTHONPATH', str(root / 'src'))
|
||||
directory = tmp_path / 'attempt'
|
||||
record_path = directory / 'job-a.json'
|
||||
record = prepare_attempt(record_path, session_id='session-a', generation=1,
|
||||
task_id='task-a', job_id='job-a', project_path=tmp_path / 'board.prj')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(directory))
|
||||
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
|
||||
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_ID', '0' * 32 if fault == 'stale' else record['attempt_id'])
|
||||
if fault == 'missing':
|
||||
record_path.unlink()
|
||||
if fault == 'missing-path':
|
||||
monkeypatch.delenv('DXPART_ATTEMPT_FILE')
|
||||
monkeypatch.delenv('DXPART_BATCH_GENERATION')
|
||||
monkeypatch.delenv('DXPART_ATTEMPT_ID')
|
||||
project = tmp_path / ('other.prj' if fault == 'wrong-project' else 'board.prj')
|
||||
output = run_vbs(tmp_path, ['QPMarkBatchMutation'], '''
|
||||
Dim Command, Attr, Failure
|
||||
Set Command = CreateObject("Scripting.Dictionary")
|
||||
Command.Add "job_id", "job-a": Command.Add "task_id", "task-a"
|
||||
Command.Add "project_path", "''' + str(project) + '''"
|
||||
Set Attr = New PropertySink
|
||||
On Error Resume Next
|
||||
GuardedWrite Command, Attr
|
||||
Failure = Err.Number
|
||||
On Error GoTo 0
|
||||
WScript.Echo CStr(Failure <> 0)
|
||||
WScript.Echo Attr.Value
|
||||
''', '''
|
||||
Dim QP_HelperExecutable, QP_HelperScript, QP_Fso
|
||||
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
|
||||
QP_HelperExecutable = "''' + sys.executable + '''"
|
||||
QP_HelperScript = "''' + str(root / 'src/dx_productivity/trial_task_center/application.py') + '''"
|
||||
Class PropertySink
|
||||
Public Value
|
||||
Private Sub Class_Initialize
|
||||
Value = "unchanged"
|
||||
End Sub
|
||||
End Class
|
||||
Sub GuardedWrite(ByRef Command, ByRef Attr)
|
||||
QPMarkBatchMutation Command
|
||||
Attr.Value = "changed"
|
||||
End Sub
|
||||
''')
|
||||
assert output.splitlines() == (['False', 'changed'] if not fault else ['True', 'unchanged'])
|
||||
if not fault:
|
||||
assert json.loads(record_path.read_text())['state'] == 'mutation_possible'
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Release diagnostics remain visible without changing cleanup decisions."""
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import batch_resources, batch_runtime
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.ui.messages import _format_task_log_lines
|
||||
from test_batch_runtime import Desktop
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_desktop():
|
||||
Desktop.created = []
|
||||
Desktop.fail_release = False
|
||||
|
||||
|
||||
def events(root, task='one'):
|
||||
return [json.loads(line) for line in (root / 'tasks' / task / 'events.jsonl')
|
||||
.read_text(encoding='utf-8').splitlines()]
|
||||
|
||||
|
||||
def runtime(root):
|
||||
result = batch_runtime.BatchCadRuntime(root, 'task_queue', desktop_factory=Desktop)
|
||||
result.bind_logs(configure_logging(root), ('one', 'two'))
|
||||
return result
|
||||
|
||||
|
||||
def test_batch_release_is_readable_and_visible_for_each_participating_task(tmp_path):
|
||||
with runtime(tmp_path) as owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
log = owner.path.with_name('resource-release.log').read_text(encoding='utf-8')
|
||||
assert '开始释放' in log and '资源释放成功' in log
|
||||
assert owner.session_id in log and owner.record['desktop'] in log
|
||||
assert 'CAD 未启动' in log
|
||||
for task in ('one', 'two'):
|
||||
rows = events(tmp_path, task)
|
||||
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_succeeded']
|
||||
details = rows[-1]['details']
|
||||
assert details['batch_id'] == owner.session_id and details['generation'] == 1
|
||||
assert details['desktop_handle_closed'] is True and details['desktop_disappeared'] is True
|
||||
assert details['elapsed_seconds'] >= 0 and details['record_path'] == str(owner.path)
|
||||
assert '资源释放成功' in '\n'.join(_format_task_log_lines(tuple(json.dumps(row) for row in rows)))
|
||||
assert '资源释放成功' in (tmp_path / 'logs/application.log').read_text(encoding='utf-8')
|
||||
|
||||
|
||||
def test_failed_release_and_manual_retry_keep_both_results_without_replay(tmp_path):
|
||||
Desktop.fail_release = True
|
||||
owner = runtime(tmp_path)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError):
|
||||
with owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
owner.result = 'saved and verified'
|
||||
failed = events(tmp_path)[-1]
|
||||
assert failed['operation'] == 'resource_release_failed'
|
||||
assert 'outside owner retains desktop' in failed['message']
|
||||
assert failed['details']['desktop_disappeared'] is False
|
||||
assert failed['details']['result'] == 'blocked'
|
||||
Desktop.fail_release = False
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
|
||||
rows = events(tmp_path)
|
||||
assert [row['sequence'] for row in rows] == [1, 2, 3, 4]
|
||||
assert rows[-1]['operation'] == 'resource_release_succeeded'
|
||||
assert rows[-1]['details']['reason'] == 'manual_retry'
|
||||
assert owner.result == 'saved and verified' and len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_release_log_failure_does_not_skip_cleanup_or_falsify_its_result(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_release_log
|
||||
def denied(*_args):
|
||||
raise OSError('release log locked')
|
||||
monkeypatch.setattr(batch_release_log, '_append_line', denied)
|
||||
with runtime(tmp_path) as owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
assert Desktop.created[0].released
|
||||
assert owner.record['state'] == 'released' and not batch_resources.pending_resources(tmp_path)
|
||||
persisted = json.loads(owner.path.read_text(encoding='utf-8'))
|
||||
assert any('release log locked' in entry for entry in persisted['release_log_errors'])
|
||||
assert events(tmp_path)[-1]['operation'] == 'resource_release_succeeded'
|
||||
|
||||
|
||||
def test_restart_recheck_logs_original_batch_and_tasks(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import isolated_desktop
|
||||
Desktop.fail_release = True
|
||||
owner = runtime(tmp_path)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError):
|
||||
with owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
batch_resources.LIVE_RESOURCES.pop(str(owner.path))
|
||||
class Api:
|
||||
def confirm_desktop_released(self, name):
|
||||
assert name == owner.record['desktop']
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path, logging_context=configure_logging(tmp_path)) == ()
|
||||
row = events(tmp_path)[-1]
|
||||
assert row['operation'] == 'resource_release_succeeded'
|
||||
assert row['details']['reason'] == 'restart_recheck'
|
||||
assert row['details']['batch_id'] == owner.session_id
|
||||
assert row['details']['desktop_disappeared'] is True
|
||||
assert len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_cached_batch_has_no_misleading_release_success(tmp_path):
|
||||
with runtime(tmp_path) as owner:
|
||||
pass
|
||||
assert not owner.path.exists()
|
||||
assert not owner.path.with_name('resource-release.log').exists()
|
||||
assert not (tmp_path / 'tasks/one/events.jsonl').exists()
|
||||
|
||||
|
||||
def test_cad_exit_and_cleanup_evidence_are_logged(tmp_path):
|
||||
with runtime(tmp_path) as owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
class Cad:
|
||||
record = {'state': 'session_closed', 'clean': True,
|
||||
'cad_exit_code': 0, 'quit_helper_exit_code': 0, 'shutdown_seconds': .01}
|
||||
def __exit__(self, *_):
|
||||
pass
|
||||
owner.cad = Cad()
|
||||
owner._save(cad_pid=123, cad_guid='owned')
|
||||
event = events(tmp_path)[-1]
|
||||
assert event['details']['cad_pid'] == 123
|
||||
assert event['details']['cad_exit_code'] == event['details']['quit_helper_exit_code'] == 0
|
||||
assert 'CAD 退出码=0' in event['message']
|
||||
|
||||
|
||||
def test_release_logger_crash_cannot_hide_final_cleanup_failure(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_release_log
|
||||
def crash(*_args, **_options):
|
||||
raise OSError('diagnostic unavailable')
|
||||
monkeypatch.setattr(batch_release_log, 'write_release_event', crash)
|
||||
Desktop.fail_release = True
|
||||
owner = runtime(tmp_path)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError):
|
||||
with owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
assert batch_resources.pending_resources(tmp_path)
|
||||
Desktop.fail_release = False
|
||||
batch_resources.LIVE_RESOURCES.pop(str(owner.path))
|
||||
from dx_productivity.trial_task_center import isolated_desktop
|
||||
class Api:
|
||||
def confirm_desktop_released(self, _name):
|
||||
pass
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
|
||||
record = json.loads(owner.path.read_text())
|
||||
assert record['state'] == 'released' and record['release_log_errors']
|
||||
@@ -0,0 +1,608 @@
|
||||
"""Batch resources are lazy, shared, and independently blocked after failed release."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
|
||||
|
||||
class Desktop:
|
||||
created = []
|
||||
fail_release = False
|
||||
|
||||
def __init__(self, name, **_options):
|
||||
self.name = name
|
||||
self.processes = []
|
||||
self.released = False
|
||||
self.cleanup_record = None
|
||||
|
||||
def __enter__(self):
|
||||
self.created.append(self)
|
||||
return self
|
||||
|
||||
def start_process(self, command, **_options):
|
||||
process = SimpleNamespace(pid=100 + len(self.processes), closed=False,
|
||||
cleanup_record={}, returncode=0)
|
||||
def close():
|
||||
process.closed = True
|
||||
process.cleanup_record = {'pid': process.pid, 'process_handle_closed': True,
|
||||
'job_handle_closed': True, 'errors': []}
|
||||
process.close = close
|
||||
process.poll = lambda: process.returncode
|
||||
self.processes.append(process)
|
||||
return process
|
||||
|
||||
def close(self):
|
||||
for process in self.processes:
|
||||
process.close()
|
||||
self.cleanup_record = {'desktop_handle_closed': True,
|
||||
'desktop_confirmation': {'released': not self.fail_release}, 'errors': []}
|
||||
if self.fail_release:
|
||||
raise OSError('outside owner retains desktop')
|
||||
self.released = True
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_desktop():
|
||||
Desktop.created = []
|
||||
Desktop.fail_release = False
|
||||
|
||||
|
||||
def test_cache_only_batch_does_not_create_desktop_or_cad(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop):
|
||||
pass
|
||||
assert not Desktop.created
|
||||
|
||||
|
||||
def test_native_operations_share_one_desktop_but_release_each_worker(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
names = []
|
||||
for name in ('first', 'second'):
|
||||
with runtime.native_scope(tmp_path / name) as scope:
|
||||
names.append(scope.name)
|
||||
process = scope.start_process(('inventory.exe',))
|
||||
assert not process.closed
|
||||
assert process.closed and not Desktop.created[0].released
|
||||
assert names[0] == names[1]
|
||||
assert len(Desktop.created) == 1 and Desktop.created[0].released
|
||||
|
||||
|
||||
def test_failed_final_release_blocks_next_batch_and_resource_cleanup_never_replays(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import (
|
||||
BatchCadRuntime, cleanup_pending_resources, guard_pending_resources, pending_resources)
|
||||
Desktop.fail_release = True
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
pass
|
||||
runtime.result = 'verified completed task'
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
assert caught.value.completed_result == 'verified completed task'
|
||||
assert pending_resources(tmp_path)
|
||||
with pytest.raises(TaskCenterError):
|
||||
guard_pending_resources(tmp_path)
|
||||
assert not list(tmp_path.rglob('uncertain.flag'))
|
||||
Desktop.fail_release = False
|
||||
assert cleanup_pending_resources(tmp_path) == ()
|
||||
guard_pending_resources(tmp_path)
|
||||
assert len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_cleanup_failure_takes_priority_over_cancel(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
Desktop.fail_release = True
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
pass
|
||||
raise TaskCenterError('operation_cancelled', 'cancel requested')
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
Desktop.fail_release = False
|
||||
from dx_productivity.trial_task_center.batch_runtime import cleanup_pending_resources
|
||||
cleanup_pending_resources(tmp_path)
|
||||
|
||||
|
||||
class Cad:
|
||||
created = []
|
||||
|
||||
def __init__(self, root, *, abort_event, desktop):
|
||||
self.root, self.desktop, self.abort_event = root, desktop, abort_event
|
||||
self.binding = SimpleNamespace(pid=500, guid='owned')
|
||||
self.closed = False
|
||||
|
||||
def __enter__(self):
|
||||
self.desktop.__enter__()
|
||||
self.created.append(self)
|
||||
return self
|
||||
|
||||
def checkpoint(self, evidence):
|
||||
assert not self.closed
|
||||
|
||||
def start_worker(self, *args, **kwargs):
|
||||
raise AssertionError('monitor is injected')
|
||||
|
||||
def __exit__(self, *_exc):
|
||||
self.closed = True
|
||||
self.desktop.close()
|
||||
|
||||
|
||||
def command_for(tmp_path, name='one'):
|
||||
root = tmp_path / 'data' / name
|
||||
root.mkdir(parents=True)
|
||||
project = tmp_path / ('project-' + name) / 'test.prj'
|
||||
project.parent.mkdir()
|
||||
project.write_text('project identity')
|
||||
(root / 'command.tsv').write_text('task_id\tjob_id\tproject_path\told_part\tnew_part\n'
|
||||
f'task\tjob\t{project}\told\tnew\n')
|
||||
return ('cscript', '//nologo', 'worker.vbs', str(root), 'python', 'job'), project
|
||||
|
||||
|
||||
def worker_runtime(tmp_path, monkeypatch, monitor):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
from dx_productivity.trial_task_center import worker_monitor
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', monitor)
|
||||
Cad.created = []
|
||||
return BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=Desktop, cad_factory=Cad)
|
||||
|
||||
|
||||
def test_two_projects_reuse_cad_and_produce_distinct_close_checkpoints(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
|
||||
one, _ = command_for(tmp_path)
|
||||
two, _ = command_for(tmp_path, 'two')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with runtime:
|
||||
runtime.run_worker(one, 5)
|
||||
assert project_checkpoint_released(Path(one[3]))
|
||||
runtime.run_worker(two, 5)
|
||||
assert project_checkpoint_released(Path(two[3]))
|
||||
assert len(Cad.created) == len(Desktop.created) == 1
|
||||
assert Cad.created[0].closed and Desktop.created[0].released
|
||||
|
||||
|
||||
@pytest.mark.parametrize('second_fails', [False, True])
|
||||
def test_prewrite_failure_releases_old_desktop_and_retries_only_once(tmp_path, monkeypatch, second_fails):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, _ = command_for(tmp_path)
|
||||
calls = []
|
||||
def monitor(command, *args, **kwargs):
|
||||
calls.append(len(calls))
|
||||
if len(calls) == 2:
|
||||
assert Desktop.created[0].released
|
||||
return subprocess.CompletedProcess(command, 2 if len(calls) == 1 or second_fails else 0, '', 'startup')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
if second_fails:
|
||||
with pytest.raises(WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
else:
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(calls) == len(Desktop.created) == len(Cad.created) == 2
|
||||
assert runtime.generation == 2
|
||||
releases = json.loads(runtime.path.read_text())['generation_releases']
|
||||
assert [release['generation'] for release in releases] == [1, 2]
|
||||
assert [release['desktop'] for release in releases] == [desktop.name for desktop in Desktop.created]
|
||||
assert all(release['cad_pid'] == 500 and release['cad_guid'] == 'owned'
|
||||
and release['cleanup']['desktop_confirmation']['released'] is True
|
||||
and release['released_at'] > 0 for release in releases)
|
||||
|
||||
|
||||
def test_possible_mutation_prevents_automatic_replay(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, _ = command_for(tmp_path)
|
||||
calls = []
|
||||
def monitor(*args, **kwargs):
|
||||
calls.append(1)
|
||||
path = next((runtime.path.parent / 'attempts').rglob('job.json'))
|
||||
record = json.loads(path.read_text())
|
||||
record['state'] = 'mutation_possible'
|
||||
path.write_text(json.dumps(record))
|
||||
raise WorkerMonitorError('lost connection after possible write')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime, pytest.raises(WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(calls) == len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_previous_checkpoint_invalidated_before_new_worker(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, _ = command_for(tmp_path)
|
||||
calls = []
|
||||
def monitor(*args, **kwargs):
|
||||
assert not project_checkpoint_released(Path(command[3]))
|
||||
calls.append(1)
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
assert project_checkpoint_released(Path(command[3]))
|
||||
def fail_checkpoint(*args):
|
||||
raise WorkerMonitorError('cannot certify empty project')
|
||||
runtime.cad.checkpoint = fail_checkpoint
|
||||
# No retry after evidence has been marked as possibly modified.
|
||||
path = next((runtime.path.parent / 'attempts').rglob('job.json'))
|
||||
record = json.loads(path.read_text())
|
||||
record['state'] = 'mutation_possible'
|
||||
path.write_text(json.dumps(record))
|
||||
with pytest.raises(WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert not project_checkpoint_released(Path(command[3]))
|
||||
|
||||
|
||||
def test_attempt_publication_failure_never_launches_worker(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_attempt
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda *a, **k: pytest.fail('must not launch'))
|
||||
def fail(*args, **kwargs):
|
||||
raise OSError('disk unavailable')
|
||||
monkeypatch.setattr(batch_attempt, 'prepare_attempt', fail)
|
||||
with runtime, pytest.raises(OSError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert not Cad.created
|
||||
assert not Desktop.created
|
||||
from dx_productivity.trial_task_center.batch_resources import pending_resources
|
||||
assert pending_resources(runtime.root) == ()
|
||||
|
||||
|
||||
def test_boundary_pause_does_not_abort_current_cad_startup(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from threading import Event
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
pause, worker_abort = Event(), Event()
|
||||
runtime.stop_event = pause
|
||||
original_enter = Cad.__enter__
|
||||
def enter(cad):
|
||||
pause.set() # User requests pause after the current task has begun.
|
||||
assert cad.abort_event is worker_abort
|
||||
assert not cad.abort_event.is_set()
|
||||
return original_enter(cad)
|
||||
monkeypatch.setattr(Cad, '__enter__', enter)
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5, abort_event=worker_abort)
|
||||
assert not list(tmp_path.rglob('uncertain.flag'))
|
||||
|
||||
|
||||
def test_cleanup_still_runs_when_resource_journal_write_fails(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_runtime
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'read'):
|
||||
pass
|
||||
real_write = batch_runtime.write_record
|
||||
def failing_write(path, record):
|
||||
if record['state'] == 'closing':
|
||||
raise OSError('journal unavailable')
|
||||
return real_write(path, record)
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', failing_write)
|
||||
assert Desktop.created[0].released
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', real_write)
|
||||
runtime.close()
|
||||
|
||||
|
||||
def test_real_startup_failure_with_released_lease_can_retry(tmp_path, monkeypatch):
|
||||
import io
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center import isolated_cad, worker_monitor
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
command, _ = command_for(tmp_path)
|
||||
starts = []
|
||||
def executable():
|
||||
starts.append(1)
|
||||
if len(starts) == 1:
|
||||
raise worker_monitor.WorkerMonitorError('registry temporarily unavailable')
|
||||
return 'cad.exe'
|
||||
class BootstrapDesktop(Desktop):
|
||||
def start_process(self, command, **options):
|
||||
process = super().start_process(command, **options)
|
||||
process.stderr = io.BytesIO()
|
||||
process.wait = lambda timeout: process.returncode
|
||||
if command[0] == 'cad.exe':
|
||||
self.cad = process
|
||||
process.returncode = None
|
||||
else:
|
||||
action, evidence, pid = command
|
||||
environment = options['env']
|
||||
binding = {'desktop': self.name, 'pid': pid, 'guid': 'owned',
|
||||
'token': environment['DXPART_SESSION_TOKEN']}
|
||||
if action == 'probe':
|
||||
Path(evidence).write_text(json.dumps(binding))
|
||||
elif action == 'checkpoint':
|
||||
Path(evidence).write_text(json.dumps({'binding': binding, 'project_path': '', 'project_closed': True}))
|
||||
elif action == 'quit':
|
||||
self.cad.returncode = 0
|
||||
return process
|
||||
monkeypatch.setattr(isolated_cad, '_designer_executable', executable)
|
||||
monkeypatch.setattr(isolated_cad, '_helper_command', lambda *args: args)
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', lambda *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=BootstrapDesktop) as runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(starts) == len(Desktop.created) == 2
|
||||
assert all(desktop.released for desktop in Desktop.created)
|
||||
|
||||
|
||||
def test_native_rebuild_requires_strict_source_and_worker_proof(tmp_path):
|
||||
import hashlib
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
command, project = command_for(tmp_path)
|
||||
database = project.parent / 'database.dat'
|
||||
database.write_bytes(b'database')
|
||||
evidence = Path(command[3]) / 'native'
|
||||
evidence.mkdir()
|
||||
proof = {'version': 1, 'native_read_only': True, 'workers_closed': True,
|
||||
'project_path': str(project.resolve()), 'evidence_root': str(evidence.resolve()),
|
||||
'source_hashes': {str(path.resolve()): hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
for path in (project, database)}}
|
||||
path = evidence / 'native-retry-proof.json'
|
||||
path.write_text(json.dumps(proof))
|
||||
with BatchCadRuntime(tmp_path / 'data', 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(evidence) as scope:
|
||||
pass
|
||||
proof['binding'] = scope.identity
|
||||
path.write_text(json.dumps(proof))
|
||||
database.write_bytes(b'changed externally')
|
||||
assert not runtime.recover_native_failure(project, evidence, 'timeout')
|
||||
database.write_bytes(b'database')
|
||||
assert runtime.recover_native_failure(project, evidence, 'timeout')
|
||||
assert Desktop.created[0].released
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
with pytest.raises(WorkerMonitorError, match='stale'):
|
||||
scope.start_process(('late-old-worker.exe',))
|
||||
assert scope.identity['generation'] == 1 and runtime.generation == 2
|
||||
scope.close() # A late close cannot grant the old operation a new identity.
|
||||
assert str(evidence.resolve()) not in runtime.record['native_operations']
|
||||
assert not runtime.recover_native_failure(project, evidence, 'another timeout')
|
||||
assert len(Desktop.created) == 2
|
||||
|
||||
|
||||
def test_mock_native_read_recovery_spends_same_project_worker_retry_budget(tmp_path, monkeypatch):
|
||||
"""Real durable proof/runtime logic; desktop, CAD and worker failure are fakes."""
|
||||
import hashlib
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, project = command_for(tmp_path)
|
||||
database = project.parent / 'database.dat'
|
||||
database.write_bytes(b'unchanged database')
|
||||
evidence = Path(command[3]) / 'native'
|
||||
evidence.mkdir()
|
||||
calls = []
|
||||
def fail_worker(*args, **kwargs):
|
||||
calls.append(1)
|
||||
raise WorkerMonitorError('prewrite failure after native recovery')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, fail_worker)
|
||||
with runtime:
|
||||
with runtime.native_scope(evidence) as scope:
|
||||
pass
|
||||
proof = {'version': 1, 'native_read_only': True, 'workers_closed': True,
|
||||
'binding': scope.identity, 'project_path': str(project.resolve()),
|
||||
'evidence_root': str(evidence.resolve()),
|
||||
'source_hashes': {str(path.resolve()): hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
for path in (project, database)}}
|
||||
(evidence / 'native-retry-proof.json').write_text(json.dumps(proof))
|
||||
assert runtime.recover_native_failure(project, evidence, 'read timeout')
|
||||
assert len(Desktop.created) == 2 and Desktop.created[0].released
|
||||
with pytest.raises(WorkerMonitorError, match='after native recovery'):
|
||||
runtime.run_worker(command, 5)
|
||||
assert calls == [1]
|
||||
assert len(Desktop.created) == 2
|
||||
assert not runtime.recover_native_failure(project, evidence, 'another read timeout')
|
||||
assert all(desktop.released for desktop in Desktop.created)
|
||||
assert runtime.record['retries'][str(project.resolve()).casefold()]['count'] == 1
|
||||
|
||||
|
||||
def test_mock_different_projects_each_get_one_retry_without_accumulating_budget(tmp_path, monkeypatch):
|
||||
"""Real per-project accounting; fake CAD/desktop and unchanged prewrite failures."""
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
one, project_one = command_for(tmp_path)
|
||||
two, project_two = command_for(tmp_path, 'two')
|
||||
calls = {one[3]: 0, two[3]: 0}
|
||||
def monitor(command, *args, **kwargs):
|
||||
calls[command[3]] += 1
|
||||
if calls[command[3]] == 2:
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
raise WorkerMonitorError('unchanged prewrite failure')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
runtime.run_worker(one, 5)
|
||||
assert len(Desktop.created) == 2
|
||||
runtime.run_worker(two, 5)
|
||||
assert len(Desktop.created) == 3 # Each project gets its own single rebuild.
|
||||
for command in (one, two):
|
||||
with pytest.raises(WorkerMonitorError, match='prewrite failure'):
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(Desktop.created) == 3 # Success cannot replenish either budget.
|
||||
assert calls == {one[3]: 3, two[3]: 3}
|
||||
assert set(runtime.record['retries']) == {
|
||||
str(project_one.resolve()).casefold(), str(project_two.resolve()).casefold()}
|
||||
assert all(value['count'] == 1 for value in runtime.record['retries'].values())
|
||||
assert all(desktop.released for desktop in Desktop.created)
|
||||
|
||||
|
||||
def test_release_publication_failure_can_be_reconciled_without_replaying(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_runtime
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
real_write = batch_runtime.write_record
|
||||
def failing_write(path, record):
|
||||
if record['state'] == 'released':
|
||||
raise OSError('cannot publish release')
|
||||
return real_write(path, record)
|
||||
with pytest.raises(TaskCenterError):
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'read'):
|
||||
pass
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', failing_write)
|
||||
assert Desktop.created[0].released and batch_runtime.pending_resources(tmp_path)
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', real_write)
|
||||
assert batch_runtime.cleanup_pending_resources(tmp_path) == ()
|
||||
assert len(Desktop.created) == 1
|
||||
releases = json.loads(runtime.path.read_text())['generation_releases']
|
||||
assert len(releases) == 1 and releases[0]['generation'] == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize('record', [
|
||||
{'version': 1, 'session_id': 'old', 'state': 'blocked'},
|
||||
{'version': 1, 'session_id': 'old', 'state': 'released'},
|
||||
[],
|
||||
])
|
||||
def test_unknown_resource_evidence_never_authorizes_cleanup(tmp_path, record):
|
||||
from dx_productivity.trial_task_center.batch_resources import cleanup_pending_resources, pending_resources
|
||||
path = tmp_path / 'batch-resources' / 'old' / 'session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(json.dumps(record))
|
||||
assert pending_resources(tmp_path)
|
||||
assert cleanup_pending_resources(tmp_path)
|
||||
assert json.loads(path.read_text()) == record
|
||||
|
||||
|
||||
def test_stale_generation_and_unknown_checkpoint_schema_are_rejected(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
path = Path(command[3]) / 'project-checkpoint.json'
|
||||
checkpoint = json.loads(path.read_text())
|
||||
assert project_checkpoint_released(Path(command[3]))
|
||||
for changes in ({'version': 999}, {'generation': 999}, {'attempt_id': 'old'}, {'cad_guid': 'foreign'}):
|
||||
path.write_text(json.dumps(dict(checkpoint, **changes)))
|
||||
assert not project_checkpoint_released(Path(command[3]))
|
||||
|
||||
|
||||
def test_restart_reconciles_only_confirmed_disappearance_without_replacement(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources, isolated_desktop
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
Desktop.fail_release = True
|
||||
runtime = BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
with pytest.raises(TaskCenterError):
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'read'):
|
||||
pass
|
||||
# Simulate restarting the UI: there are no retained owned handles to act on.
|
||||
batch_resources.LIVE_RESOURCES.pop(str(runtime.path))
|
||||
present = [True]
|
||||
class Api:
|
||||
def confirm_desktop_released(self, name):
|
||||
assert name == runtime.record['desktop']
|
||||
if present[0]:
|
||||
raise OSError('desktop still held by external thread')
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path)
|
||||
present[0] = False
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
|
||||
assert len(Desktop.created) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize('conflict', [
|
||||
'bootstrap_binding', 'bootstrap_json', 'bootstrap_schema', 'helper_identity',
|
||||
'checkpoint_binding', 'checkpoint_json', 'worker_identity',
|
||||
])
|
||||
def test_identity_conflict_never_rebuilds_or_launches_another_worker(tmp_path, monkeypatch, conflict):
|
||||
import io
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center import isolated_cad, worker_monitor
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
command, _ = command_for(tmp_path)
|
||||
workers = []
|
||||
|
||||
class IdentityDesktop(Desktop):
|
||||
def start_process(self, launch, **options):
|
||||
process = super().start_process(launch, **options)
|
||||
process.stderr = io.BytesIO(b'CAD COM GUID does not match its owned session')
|
||||
process.wait = lambda timeout: process.returncode
|
||||
if launch[0] == 'cad.exe':
|
||||
self.cad = process
|
||||
process.returncode = None
|
||||
return process
|
||||
action, evidence, pid = launch
|
||||
binding = {'desktop': self.name, 'pid': pid, 'guid': 'owned',
|
||||
'token': options['env']['DXPART_SESSION_TOKEN']}
|
||||
if action == 'probe':
|
||||
if conflict == 'helper_identity':
|
||||
process.returncode = 4 # Private helper identity-conflict exit.
|
||||
return process
|
||||
if conflict == 'bootstrap_binding':
|
||||
binding['pid'] = pid + 1
|
||||
if conflict == 'bootstrap_schema':
|
||||
binding.pop('token')
|
||||
Path(evidence).write_text('{' if conflict == 'bootstrap_json' else json.dumps(binding))
|
||||
elif action == 'checkpoint':
|
||||
closing = Path(evidence).name.startswith('cad-closed-')
|
||||
if closing and conflict == 'checkpoint_binding':
|
||||
binding['guid'] = 'foreign'
|
||||
record = {'binding': binding, 'project_path': '', 'project_closed': True}
|
||||
Path(evidence).write_text('{' if closing and conflict == 'checkpoint_json' else json.dumps(record))
|
||||
elif action == 'quit':
|
||||
self.cad.returncode = 0
|
||||
return process
|
||||
|
||||
monkeypatch.setattr(isolated_cad, '_designer_executable', lambda: 'cad.exe')
|
||||
monkeypatch.setattr(isolated_cad, '_helper_command', lambda *args: args)
|
||||
def monitor(*args, **kwargs):
|
||||
workers.append(1)
|
||||
if conflict == 'worker_identity':
|
||||
return subprocess.CompletedProcess(command, 2, '', 'private CAD GUID mismatch; no project was changed')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', monitor)
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=IdentityDesktop)
|
||||
with runtime, pytest.raises(worker_monitor.WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(Desktop.created) == 1
|
||||
assert len(workers) == (1 if conflict.startswith('checkpoint') or conflict == 'worker_identity' else 0)
|
||||
assert runtime.generation == 1 and not runtime.record['retries']
|
||||
assert Desktop.created[0].released
|
||||
|
||||
|
||||
def test_request_session_identity_is_uuid_bound(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
session = '12345678-1234-5678-1234-567812345678'
|
||||
runtime = BatchCadRuntime(tmp_path, 'bom_import', session_id=session)
|
||||
assert runtime.session_id == session.replace('-', '')
|
||||
with pytest.raises(ValueError):
|
||||
BatchCadRuntime(tmp_path, 'bom_import', session_id='../escape')
|
||||
|
||||
|
||||
def test_cleanup_preserves_primary_bom_error_and_report(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
primary = TaskCenterError('bom_inspection', 'read failed')
|
||||
primary.bom_report = object()
|
||||
Desktop.fail_release = True
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
pass
|
||||
raise primary
|
||||
error = caught.value
|
||||
assert error.primary_error is primary
|
||||
assert error.cleanup_error is not None
|
||||
assert error.bom_report is primary.bom_report
|
||||
assert 'BOM' in error.message and '已核验任务结果保留' not in error.message
|
||||
assert runtime.record['primary_error']['code'] == primary.code
|
||||
assert runtime.record['cleanup_error']
|
||||
Desktop.fail_release = False
|
||||
from dx_productivity.trial_task_center.batch_resources import cleanup_pending_resources
|
||||
cleanup_pending_resources(tmp_path)
|
||||
|
||||
|
||||
def test_attempts_live_under_short_session_root(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
assert list((runtime.path.parent / 'attempts').rglob('job.json'))
|
||||
assert not (Path(command[3]) / 'batch-attempts').exists()
|
||||
@@ -0,0 +1,122 @@
|
||||
"""Batch entry points own resource lifetime independently of task outcomes."""
|
||||
import sys
|
||||
from types import SimpleNamespace, ModuleType
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import task_queue
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from test_task_queue import Service
|
||||
from test_bom_batch import batch, records_for
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def runtime_module(monkeypatch):
|
||||
module = ModuleType('dx_productivity.trial_task_center.batch_runtime')
|
||||
scopes = []
|
||||
class CleanupError(TaskCenterError):
|
||||
def __init__(self):
|
||||
super().__init__('batch_resources_pending', 'resource cleanup blocked')
|
||||
class Runtime:
|
||||
result = None
|
||||
fail_cleanup = False
|
||||
def __init__(self, root, kind, stop_event=None):
|
||||
self.root, self.kind = root, kind
|
||||
self.closed = False
|
||||
scopes.append(self)
|
||||
def __enter__(self):
|
||||
return self
|
||||
def bind_logs(self, context, task_ids):
|
||||
self.log_task_ids = tuple(task_ids)
|
||||
def __exit__(self, *args):
|
||||
self.closed = True
|
||||
if self.fail_cleanup:
|
||||
error = CleanupError()
|
||||
error.completed_result = self.result
|
||||
raise error
|
||||
module.BatchCadRuntime = Runtime
|
||||
module.BatchResourceCleanupError = CleanupError
|
||||
monkeypatch.setitem(sys.modules, module.__name__, module)
|
||||
return Runtime, scopes
|
||||
|
||||
|
||||
def test_task_queue_owns_one_runtime_and_passes_it_to_every_child(tmp_path, runtime_module):
|
||||
Runtime, scopes = runtime_module
|
||||
service = Service(tmp_path)
|
||||
original = service.for_project_task
|
||||
seen = []
|
||||
def child(task_id):
|
||||
result = original(task_id)
|
||||
execute = result.execute_quick_run_queue
|
||||
def run(task_id, *, runtime, **options):
|
||||
assert not runtime.closed
|
||||
seen.append(runtime)
|
||||
return execute(task_id, **options)
|
||||
result.execute_quick_run_queue = run
|
||||
return result
|
||||
service.for_project_task = child
|
||||
result = task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event())
|
||||
assert [entry.state for entry in result.entries] == ['completed', 'completed']
|
||||
assert len(scopes) == 1 and scopes[0].closed
|
||||
assert seen == [scopes[0], scopes[0]]
|
||||
assert scopes[0].log_task_ids == ('A', 'B')
|
||||
|
||||
|
||||
def test_final_cleanup_failure_preserves_completed_tasks_and_resume_does_not_repeat(tmp_path, runtime_module):
|
||||
Runtime, scopes = runtime_module
|
||||
Runtime.fail_cleanup = True
|
||||
service = Service(tmp_path)
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event())
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
result = caught.value.completed_result
|
||||
assert [entry.state for entry in result.entries] == ['completed', 'completed']
|
||||
Runtime.fail_cleanup = False
|
||||
task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event(), queue_id=result.queue_id)
|
||||
assert service.calls == [('start', 'A'), ('end', 'A'), ('start', 'B'), ('end', 'B')]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('current', [True, False])
|
||||
def test_reader_requires_current_batch_project_checkpoint(tmp_path, runtime_module, current):
|
||||
from dx_productivity.trial_task_center.project_discovery import (
|
||||
_read_failure, _require_released_reader, ReleasedProjectReadError)
|
||||
module = sys.modules['dx_productivity.trial_task_center.batch_runtime']
|
||||
module.project_checkpoint_released = lambda root: current
|
||||
(tmp_path / 'project-checkpoint.json').write_text('{}')
|
||||
if current:
|
||||
_require_released_reader(tmp_path)
|
||||
assert isinstance(_read_failure(tmp_path, 'read failed'), ReleasedProjectReadError)
|
||||
else:
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
_require_released_reader(tmp_path)
|
||||
assert caught.value.code == 'inventory_cleanup'
|
||||
assert not isinstance(_read_failure(tmp_path, 'read failed'), ReleasedProjectReadError)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('read_error', [False, True])
|
||||
def test_bom_cleanup_failure_preserves_completed_or_partial_report(batch, tmp_path, runtime_module, read_error):
|
||||
from dx_productivity.trial_task_center.bom_import_report import inspect_batch
|
||||
from dx_productivity.trial_task_center.bom_plan import write_bom_template
|
||||
Runtime, scopes = runtime_module
|
||||
Runtime.fail_cleanup = True
|
||||
service, store, tasks, targets, calls = batch
|
||||
if read_error:
|
||||
def fail(*args, **kwargs):
|
||||
raise TaskCenterError('inventory_cleanup', 'reader did not close')
|
||||
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = fail
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
inspect_batch(service, tasks[0].task_id,
|
||||
write_bom_template(tmp_path / 'cleanup.xlsx', records_for(tasks, targets)))
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
report = caught.value.bom_report
|
||||
assert report is not None and report.total_rows == 2
|
||||
assert report.failed_rows == (1 if read_error else 0)
|
||||
assert report.passed_rows == (0 if read_error else 2)
|
||||
assert report.preview is None
|
||||
original = caught.value.__context__.bom_report if read_error else scopes[0].result
|
||||
assert report is not original
|
||||
assert (original.preview is not None) == (not read_error)
|
||||
if caught.value.completed_result is not None:
|
||||
assert caught.value.completed_result.preview is None
|
||||
assert len(scopes) == 1 and scopes[0].closed
|
||||
@@ -0,0 +1,58 @@
|
||||
"""An external Windows reference blocks a batch without replaying its results."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.batch_runtime import (
|
||||
BatchCadRuntime, BatchResourceCleanupError, cleanup_pending_resources, guard_pending_resources)
|
||||
from dx_productivity.trial_task_center.cad_identity import current_desktop_name
|
||||
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api, build_startup_info
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop object lifetime')
|
||||
def test_external_reference_preserves_completed_result_until_resource_only_cleanup(tmp_path):
|
||||
before = current_desktop_name()
|
||||
runtime = BatchCadRuntime(tmp_path, 'execution')
|
||||
runtime.bind_logs(configure_logging(tmp_path), ('windows-resource-test',))
|
||||
external = None
|
||||
try:
|
||||
with pytest.raises(BatchResourceCleanupError) as caught:
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
|
||||
# The test owns this independent actor, outside runtime's
|
||||
# process/job list. The runtime must leave it alive.
|
||||
external = _Win32Api().create_process((sys.executable, '-u', '-c',
|
||||
'import win32service,time,sys; '
|
||||
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000); '
|
||||
'd.SetThreadDesktop();print("ready",flush=True);time.sleep(30)', scope.name),
|
||||
build_startup_info(scope.name).lpDesktop, None, None,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
assert external.stdout.readline() == b'ready\r\n'
|
||||
runtime.result = 'already saved and independently verified'
|
||||
assert caught.value.completed_result == runtime.result
|
||||
assert external.poll() is None
|
||||
assert json.loads(runtime.path.read_text())['state'] == 'blocked'
|
||||
log = runtime.path.with_name('resource-release.log').read_text(encoding='utf-8')
|
||||
assert '资源释放失败' in log and '桌面消失:未确认' in log
|
||||
with pytest.raises(BatchResourceCleanupError):
|
||||
guard_pending_resources(tmp_path)
|
||||
assert not list(Path(tmp_path).rglob('uncertain.flag'))
|
||||
assert current_desktop_name() == before
|
||||
finally:
|
||||
if external is not None:
|
||||
external.close() # Only the test closes its own independent actor.
|
||||
assert cleanup_pending_resources(tmp_path) == ()
|
||||
assert json.loads(runtime.path.read_text())['state'] == 'released'
|
||||
assert runtime.result == 'already saved and independently verified'
|
||||
assert runtime.generation == 1
|
||||
rows = [json.loads(line) for line in (tmp_path / 'tasks/windows-resource-test/events.jsonl')
|
||||
.read_text(encoding='utf-8').splitlines()]
|
||||
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_failed',
|
||||
'resource_release_started', 'resource_release_succeeded']
|
||||
assert rows[-1]['details']['desktop_disappeared'] is True
|
||||
guard_pending_resources(tmp_path)
|
||||