Compare commits

...
21 Commits
Author SHA1 Message Date
阿白叔 46bcafd7a3 docs(release): record Beta package verification and release manifest 2026-10-21 20:00:23 +08:00
阿白叔 e46982af6c release: prepare V1.1.4 Beta 1 and label release channel 2026-10-21 19:54:02 +08:00
阿白叔 2a9398ab50 feat(logs): retain resource release history in batch and task logs 2026-10-21 18:01:59 +08:00
阿白叔 faad50df80 docs: document batch resource safety and measured acceptance results 2026-10-21 15:55:18 +08:00
阿白叔 74c1cb752c test: compare canonical worker paths when Windows uses short aliases 2026-10-21 15:53:51 +08:00
阿白叔 a8358d90d7 fix(runtime): block identity replay and retain generation release proof 2026-10-21 15:45:09 +08:00
阿白叔 486ea05ad5 test(runtime): verify external desktop references block batch cleanup safely 2026-10-21 15:36:52 +08:00
阿白叔 4b78ca0f27 test: bound thousand-step queue work independently of machine timing 2026-10-21 15:33:31 +08:00
阿白叔 0151552f55 fix(safety): reject frozen queue policy changes before project backup 2026-10-21 15:26:55 +08:00
阿白叔 2c88be99ce fix: stop batch when native session recovery is exhausted 2026-10-21 15:26:11 +08:00
阿白叔 a011e8eab6 test: replay reported BOM against captured native inventories 2026-10-21 15:14:26 +08:00
阿白叔 9a3dff2f14 test: record paired live batch runtime acceptance 2026-10-21 15:13:14 +08:00
阿白叔 bdb096fb88 fix: retain diagnostic BOM reports without actionable cleanup previews 2026-10-21 15:00:53 +08:00
阿白叔 89c7c603f6 feat(runtime): reuse private desktop and CAD per batch with safe rebuilds 2026-10-21 15:00:17 +08:00
阿白叔 681403a4d2 fix: bind native retry evidence to current batch operation 2026-10-21 14:51:32 +08:00
阿白叔 a94276494a fix: guard standalone CAD entries and preserve cleanup diagnostics 2026-10-21 14:49:14 +08:00
阿白叔 00861a59d6 feat: share batch runtime across queues and expose resource cleanup 2026-10-21 14:30:01 +08:00
阿白叔 832ab5d563 feat: persist batch attempt proof before CAD mutations 2026-10-21 14:28:25 +08:00
阿白叔 e15ee259f8 tools: trace private desktop thread owners using documented Win32 APIs 2026-10-21 13:42:22 +08:00
阿白叔 e4ef6a533e fix: avoid unrelated BOM inventory fallback and retain interrupted import reports 2026-10-21 11:45:04 +08:00
阿白叔 3b3b648580 docs: add a 13-step illustrated README tutorial 2026-10-20 08:30:17 +08:00
79 changed files with 4280 additions and 176 deletions

No files matched your search

+133 -44
View File
@@ -4,72 +4,162 @@
面向 **Windows + DxDesigner / Expedition Enterprise 7.9.5** 的器件批量替换工具。一个工程对应一个任务,可以只替换同料号下的部分位号;支持直接编辑和 Excel BOM 导入,同批工程按顺序执行。
**最新版本:V1.1.3** · [下载便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.3/DxPartReplace-1.1.3.zip) · [版本说明与附件](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/tag/v1.1.3) · [完整使用说明](docs/user/v1.1.0-project-tasks.md)
**最新版本:V1.1.4 Beta 1(预发布)** · [下载便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.4-beta.1/DxPartReplace-1.1.4-beta.1.zip) · [版本说明与附件](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/tag/v1.1.4-beta.1) · [完整使用说明](docs/user/v1.1.0-project-tasks.md)
所有发布均标记为 **Beta 预发布**,旧版本标签与下载链接保持可用。
本版提供[目标封装选择](docs/user/target-cell-selection.md):目标封装可留空,此时工具不主动覆写封装,可后续与中心库同步;需要指定时,在料号规则中读取并选择合法封装。BOM 保持原 11 列。
## 窗口操作教程
## 图文教程
下面的 GIF 录制于 V1.1.2,操作步骤仍适用于 V1.1.3,图标与版本号以新版为准。包含创建任务、BOM 导入、错误行提示、保存、顺序执行、查看结果和开始新一轮。共 20 个步骤、47.5 秒;仅记录工具及其弹窗,CAD 等待已压缩,Excel 填表示例在录制之外完成。
按下面 **13 步**,完成“创建任务 → 填 BOM → 导入校验 → 保存计划 → 替换 → 查看结果”。本例有两个工程 `Audio_D2128`、`Audio_T605A`,各自只替换 `RV05`,保留同料号下的 `RV11`。
![V1.1.2 窗口操作教程](docs/user/media/v1.1.2-window-bom-tutorial.gif)
截图来自真实窗口:起始页为 V1.1.3,后续操作图取自已验收的 V1.1.2 业务流程,操作位置相同。点击图片可以放大查看;BOM 填写图为示意。
如果图片没有自动播放,可以[单独打开 GIF](docs/user/media/v1.1.2-window-bom-tutorial.gif)。
### 第 1 步:解压并打开应用
## 开始使用
[下载 V1.1.4 Beta 1 便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.4-beta.1/DxPartReplace-1.1.4-beta.1.zip),**完整解压**,双击 `DxPartReplace.exe`。保留它与 `_internal` 的相对位置,无需安装 Python;本机需要可用的 DxDesigner、许可证和中央库。
1. 下载并完整解压便携包,保留 `DxPartReplace.exe` 和 `_internal` 的相对位置。**便携包无需安装 Python**,本机仍需安装可用的 DxDesigner、许可证和中央库。
2. 双击 `DxPartReplace.exe`。首次运行会自动创建同目录的 `data`,保存任务、日志和执行证据。
3. 点击左侧 **新建**,在弹窗填写工程根目录,点击 **读取工程**,勾选需要处理的 `.prj`,再点击 **创建任务**。每个工程自动创建一个任务。
打开后看到下图主窗口。首次使用请准备完整的工程副本,执行前保存并关闭待处理工程。
首次使用请在完整工程副本上验证映射与库配置。执行前保存并关闭待处理工程;工具会启动自己的后台 CAD,主桌面仍可查看其他文件。
![第 1 步:打开应用,进入任务主窗口](docs/user/media/tutorial/01-open.png)
## 方式一:通过 BOM 批量配置
### 第 2 步:新建任务,读取工程目录
### 1. 导出本批模板
点击左下方 **新建**。在弹窗中填写工程根目录,或点击 **选择目录**,再点击 **读取工程**。
选择任务,打开顶部 **本批位号计划**,点击 **BOM 模板**。只要已导入工程,就能导出同批所有工程的一张空白表,**不需要先采集位号或填写料号规则**。
**完成标志:** 弹窗列表显示该目录下找到的 `.prj` 工程。
工程名称、工程路径自动填写。工程路径用于唯一分配任务,请保持原样;同一工程有多条替换时,可以复制该工程行。
![第 2 步:填写根目录,点击读取工程](docs/user/media/tutorial/02-read-projects.png)
### 2. 在 Excel 填写替换项
### 第 3 步:勾选工程,创建任务
保留原模板的 11 列表头,按需要填写:
勾选这次要处理的工程,点击右下角 **创建任务**。本例勾选两个工程。
| 列名 | 填写内容 |
**完成标志:** 左侧出现两个任务,**一个工程对应一个任务**。
![第 3 步:勾选两个工程,点击创建任务](docs/user/media/tutorial/03-create-tasks.png)
### 第 4 步:导出本批 BOM 模板
选中左侧任意一个任务,切换到顶部 **本批位号计划**,点击 **BOM 模板**,选择文件保存位置。
**完成标志:** 导出的 Excel 包含本批所有工程,工程名称和完整路径已经填写好。此时不需要先采集位号,也不需要先添加料号规则。
![第 4 步:切换本批位号计划,导出 BOM 模板](docs/user/media/tutorial/04-export-template.png)
### 第 5 步:在 Excel 中填写替换项
打开刚导出的模板,**保留全部 11 列表头以及原工程路径**。同一工程需要填写多条替换时,复制该工程行后再填写。
本例在两个工程行中分别填写相同的旧、新料号和 `RV05`:
![第 5 步:BOM 填写示意,两个工程各替换 RV05](docs/user/media/tutorial/05-fill-bom.png)
| 要填写的列 | 填法 |
| --- | --- |
| `old_part_number` | 工程当前的旧料号 |
| `new_part_number` | 中央库中的目标新料号 |
| `refdes` | 要处理的位号,例如 `RV05`;一行可以写多个位号,以逗号、分号、空格或顿号分隔 |
| `enabled` | `TRUE` / `是` 表示替换,`FALSE` / `否` 表示保留 |
| `sync_nc` | 是否同步 NC,默认开启,可逐行关闭 |
| `custom_value` | 可选的自定义 Value;填写后优先写入,留空沿用任务的 Value 策略 |
| `old_part_number` | 工程当前的旧料号,本例为 `4100-CA0000-T7` |
| `new_part_number` | 中央库中的目标新料号,本例为 `4100-CA0000-T0` |
| `refdes` | 要替换的位号,本例为 `RV05`;多个位号可用逗号、分号、空格或顿号分隔 |
| `enabled` | `TRUE` / `是` 表示替换,`FALSE` / `否` 表示保留;留空默认替换,建议显式填写 |
| `sync_nc` | 是否同步 NC,默认开启;按需要填 `TRUE` 或 `FALSE` |
| `custom_value` | 可选的自定义 Value;留空时使用任务的 Value 策略 |
例如,首次配置的旧料号同时对应 `RV05`、`RV11`,只想替换 `RV05`,可以只填写 `RV05` 的启用项;首次采集的其余位号默认不替换。
料号按**文本**填写,不要使用公式。相同位号出现在多个图页时,补填 `schematic_name`、`sheet_number` 或 `uid` 消除歧义。旧新相同、链式或循环替换会被拒绝。
**再次导入时,BOM 中未列出的既有计划行会保留原勾选和编辑状态。** 如果这轮只想执行 BOM 列出的目标,请在预览中检查其他启用行,并明确取消勾选,或在 BOM 中填写对应的 `enabled=FALSE` 行。
**本例首次导入只列出 RV05,采集到的 RV11 默认保留。再次导入时,未列出的既有计划行会保留原勾选;需要停用时,请取消勾选或明确填写 `enabled=FALSE`。**
同名位号分布在不同图页时,补填 `schematic_name`、`sheet_number` 或 `uid` 消除歧义。`enabled` 留空默认替换,`sync_nc` 留空默认开启;建议显式填写。料号按文本填写;导入不接受公式,也不接受旧新相同、链式或循环替换。
填完后保存 Excel 文件。
### 3. 导入并检查报告
### 第 6 步:导入填写好的 BOM
点击 **导入 BOM**,选择填好的文件。工具先校验表格格式、工程归属,以及能从表中确定的重复声明和料号冲突;通过后,只核验本表涉及的工程。已有位号库存覆盖旧料号时直接匹配;缺少库存时优先通过原生 CLI 采集,不可用时才回退到后台 CAD 只读采集。
返回应用,在 **本批位号计划** 点击 **导入 BOM**,选择刚保存的 Excel,等待校验结果。
新一轮导入开始后,旧报告会清除,界面显示读表、获取库存、匹配位号等进度;取消选择文件则保留原报告。导入失败或取消会保留已有预览和手动编辑。保存和执行前仍会核验工程。
导入会检查表格和工程归属,并匹配相关工程的位号。缺少库存时优先调用原生 CLI 采集,不可用时回退到后台 CAD 只读采集。**导入不会执行替换,也不会直接保存计划。**
- 校验报告显示通过、失败、未检查的行数,并标明 Excel 行号及原因。
- 有错误时,整批不会生成新的可保存预览,也不会部分写入计划。修正后重新导入。
- 全部通过后,先得到预览;可以切换任务,检查或修改各工程的位号计划。
![第 6 步:点击导入 BOM,选择填写好的 Excel](docs/user/media/tutorial/06-import-bom.png)
### 4. 保存并执行
### 第 7 步:检查导入报告,有错误先修正
确认预览后点击 **保存本批位号计划**;普通单工程编辑时按钮显示 **保存位号计划**。
点击校验状态条中的 **详情**。报告会显示通过、失败、未检查的行数,以及 Excel 行号和错误原因。
点击 **执行当前任务** 处理当前工程,或点击左侧 **顺序执行本批任务** 处理同批工程。确认范围后,工具逐项执行原生 Replace Part、保存、关闭并重新打开核验。
下图是故意填入不存在位号的示例:**通过 2 行、失败 1 行**,错误定位到 Excel 第 4 行。按报告修正 Excel、保存文件,再次点击 **导入 BOM**。
**左侧搜索和显示排序只影响浏览,不改变本批执行范围。** 调整执行顺序请使用上移、下移任务按钮。
**有错误时不会部分保存。全部通过后再继续下一步。** 新一次导入开始后旧报告会清除;导入失败仍保留已有预览和手动编辑,取消文件选择则保留原报告。
## 方式二:直接在界面编辑
![第 7 步:查看校验详情,按 Excel 行号修正错误](docs/user/media/tutorial/07-import-errors.png)
### 第 8 步:核对每个工程的预览,保存计划
报告全部通过后,关闭详情,依次选中左侧各任务,核对:
- 需要替换的 `RV05` 已勾选,新料号正确。
- 需要保留的 `RV11` 未勾选。
- 自定义 Value、同步 NC 等逐行设置符合预期。
确认后点击 **保存本批位号计划**;单工程编辑或保存完成后,按钮显示 **保存位号计划**。
**完成标志:** 底部提示计划保存完成。下图为保存后的状态,只有 RV05 启用。
![第 8 步:检查替换勾选和新料号,保存本批位号计划](docs/user/media/tutorial/08-review-save.png)
### 第 9 步:按任务确认 Value 与封装设置
选中一个任务,按需设置顶部 **Value 策略**:勾选后,将替换后的 **Part Label 写入 Value**;逐行自定义 Value 优先。处理多个工程时,分别确认各任务的开关。
本例开启该选项。**复制的是 Part Label,不是把新料号直接写进 Value。**
![第 9 步:按当前任务设置 Part Label 写入 Value](docs/user/media/tutorial/09-value-policy.png)
目标封装可留空,工具不主动覆写 Cell Name,可后续与中心库同步。需要本轮指定时,在 **本批料号规则** 中编辑对应旧、新料号规则,读取并选择合法封装,核对计划中的选择后保存。详见[封装操作说明](docs/user/target-cell-selection.md)。
### 第 10 步:选择执行范围,确认后开始
只处理选中工程,点击底部 **执行当前任务**;处理同批工程,点击左侧 **顺序执行本批任务**。本例使用后者。
在确认弹窗中核对工程数量和执行范围,确认无误再点击 **Yes / 是**。
![第 10 步:核对两个工程的执行范围,确认顺序执行](docs/user/media/tutorial/10-confirm-execution.png)
**搜索和显示排序只影响浏览,不改变本批执行范围。** 调整实际执行顺序,请使用左下方 **上移任务 / 下移任务**。
### 第 11 步:等待后台执行和保存重开核验
工具按工程顺序执行替换,保存、关闭并重新打开核验。左侧显示各任务状态,底部显示当前阶段和进度。
![第 11 步:观察任务状态和底部执行进度](docs/user/media/tutorial/11-running.png)
运行期间可以在主桌面查看其他文件,请勿打开或修改本次正在处理的同一工程。需要暂停时,点击 **完成当前项后暂停**,等待当前步骤安全收尾。
**完成标志:** 任务显示已完成,底部提示本批执行结束。还要打开结果核对,不能只看进度条。
### 第 12 步:查看实际修改结果与日志
选中要检查的任务,点击右下角 **替换结果 / 日志**,查看 **位号替换结果**。
本例应看到:`RV05` 的旧料号为 `4100-CA0000-T7`,回读料号为 `4100-CA0000-T0`,核验状态为 **已保存并核验**。
![第 12 步:查看位号、旧料号、实际回读料号和核验状态](docs/user/media/tutorial/12-results.png)
需要排查原因时,切到 **完整日志**,查看具体阶段和耗时;可用 **上一页日志 / 最新日志** 翻页。失败、未命中或未核验不会被当作已保存成功。
![第 12 步补充:完整日志显示各阶段和处理信息](docs/user/media/tutorial/12-complete-log.png)
### 第 13 步:需要继续替换时,开始新一轮
已完成任务可以点击底部 **开始新一轮**:
![第 13 步入口:任务完成后,底部出现开始新一轮](docs/user/media/tutorial/13-new-round-entry.png)
核对弹窗中的任务范围,再确认。
![第 13 步:核对新一轮范围及旧证据归档说明](docs/user/media/tutorial/13-new-round.png)
**完成标志:** 已完成任务恢复为待配置,旧计划、命令、快照和日志归档保留。接下来按工程**当前料号**重新填 BOM 或修改规则,重新采集、保存后执行;旧轮次通过 **查看轮次证据** 打开。
出现状态不确定或恢复提示时,先使用 **核验并恢复**,按[恢复说明](docs/user/interrupted-recovery.md)处理。
## 不使用 Excel:直接在界面编辑
1. 在 **本批料号规则** 添加旧、新料号,按需配置 Value 与同步 NC;规则提供采集条件和默认值。
2. 切到 **本批位号计划**,点击 **采集所有匹配位号**。
@@ -78,7 +168,7 @@
两种方式可以配合使用,不必为了编辑而先导出 Excel。切换任务会保留编辑草稿;关闭时有未保存内容会提示保存全部、放弃或取消。
## Value、暂停与执行结果
## Value、NC 与日志选项
- 顶部 **Value 策略** 开关控制当前任务是否将替换后的 Part Label 写入 Value;逐行自定义 Value 优先。处理多个任务时,分别确认各任务的开关。
- **同步 NC** 是逐行选项,默认开启,可以关闭。
@@ -86,12 +176,6 @@
- 点击 **替换结果 / 日志** 查看图页、位号、旧料号、实际回读料号和核验状态。失败、未命中、未核验不会显示为已保存成功;完整日志支持翻页。
- 日志和 BOM 详情按需打开,关闭后释放窗口;数据和证据仍保留。
## 完成后再执行一轮
已完成任务可以点击 **开始新一轮**。工具归档旧计划、命令、快照与执行证据,清空过期位号基线,将同批已完成任务恢复为待配置。
接着按工程**当前料号**重新填 BOM 或修改规则,重新采集、保存并执行。旧轮次可通过 **查看轮次证据** 打开。出现状态不确定或恢复提示时,先使用 **核验并恢复**,不要直接重复执行。
## 更新程序与保留数据
先关闭旧程序并备份 `data`。完整解压新包,用新 EXE 和 `_internal` 配合原 `data` 使用;不要混用不同版本的依赖目录,也不要删除原数据。
@@ -110,6 +194,10 @@ DxPartReplace/
## 验证范围与已知限制
V1.1.4 Beta 1 新增批次后台资源管理:一次 BOM 导入或顺序执行共用一个私有桌面,CAD 按需启动并在同批工程间复用,批次结束统一退出。若提示 **资源释放待处理**,已保存核验的结果保留,请点击 **重新清理资源**;该操作只清理后台资源,不会再次替换。旧的工程保护锁仍按“核验并恢复”处理。设计、异常边界与实机结果见[批次运行时验收记录](docs/testing/2026-10-21-batch-private-runtime.md)。
资源释放会追加日志:在 **替换结果 / 日志 → 完整日志** 查看本批开始释放、释放成功/失败及重新清理的结果。独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`,记录批次、桌面、CAD PID、退出码、释放耗时、句柄关闭和桌面消失的分别确认,以及失败原因;同时保留 `session.json` 的原始释放证明。重启后重新检查也会追加记录,不覆盖原失败日志。
V1.1.3 的业务代码沿用本机已验收候选:两工程实际替换和独立回读、362 字符工程路径、BOM 失败/成功导入、Part Label 写入 Value、可选封装、旧版任务数据副本、新轮次和自有后台资源释放。新增图标及版本元数据另做冻结包检查。详见[验收报告](docs/testing/2026-10-19-release-preparation.md)。
当前按 **evaluation-only(评估试用)** 交付。不同机器、中央库和 CAD 版本仍需用工程副本确认兼容性;替换回读通过不代表已通过 Packager、ERC 或网表检查。
@@ -128,6 +216,7 @@ py -3.12 -m venv .venv
源码默认数据目录为 `%LOCALAPPDATA%\DxProductivity\trial-task-center`,可用 `DX_TASK_CENTER_HOME` 指定独立目录。运行回归:`python -m pytest -q`;构建使用 `packaging/Build-DxPartReplace.ps1`。
- [历史窗口 GIF(补充演示,录制于 V1.1.2)](docs/user/media/v1.1.2-window-bom-tutorial.gif)
- [完整桌面使用说明](docs/user/v1.1.0-project-tasks.md)
- [中断核验与恢复](docs/user/interrupted-recovery.md)
- [Value 与属性继承](docs/user/value-copy-and-layout.md)
+44
View File
@@ -0,0 +1,44 @@
{
"version": "1.1.4-beta.1",
"release_tag": "v1.1.4-beta.1",
"classification": "beta",
"prerelease": true,
"code_commit": "e46982af6c3ea145cfa04486a54227ca02ecdd89",
"application_entry_matches_source": true,
"vbs_match_source": {
"QuickPartReplace.vbs": true,
"NativeSnapshotReader.vbs": true
},
"data_directory_excluded": true,
"gif_recorded_version": "1.1.2",
"acceptance_report": "docs/testing/2026-10-21-batch-private-runtime.md",
"build_log": "build/release-1.1.4-beta.1-build.log",
"zip_sha256": "acf9f13617b5c25ddfe4c9e4df90481f6ca7838329bca1c201e7c1fdfe27d4a1",
"zip_bytes": 60266953,
"file_count": 396,
"extracted_zip_smoke": true,
"exe_sha256": "c0a43c302568206c9bbfc02203daded1df8e19df1d2083c2f0961eddd213a900",
"build_verification": {
"passed": 822,
"warnings": 1,
"seconds": 211.19
},
"frozen_helper_verification": {
"passed": true,
"version": "1.1.4-beta.1",
"windows_numeric_version": "1.1.4.1",
"windows_prerelease_flag": true,
"frozen_marker_written": true,
"stale_marker_rejected": true,
"commands": [
"probe",
"checkpoint",
"quit"
],
"opened_projects": [],
"resources_released": true,
"observer_errors": []
},
"business_module_count": 84,
"business_modules_all_match_source": true
}
+32
View File
@@ -0,0 +1,32 @@
# V1.1.4 Beta 1 构建与发布记录
源提交:`e46982af6c3ea145cfa04486a54227ca02ecdd89`。本次后续提交只补充构建记录,程序代码不变。版本显示为 `1.1.4-beta.1`,Python 包元数据为等价的 `1.1.4b1`;Windows 数字版本为 `1.1.4.1`,设置 `VS_FF_PRERELEASE`。
## 本次检查
| 项目 | 结果 |
| --- | --- |
| 标准打包检查 | 822 passed,211.19 秒;1 条已知 pywinauto COM 模式提示 |
| 冻结代码对应源码 | 84 个业务模块及应用入口编译码全部一致,两份 VBS SHA256 一致 |
| ZIP 解压 | 完整性检查、逐文件校验和解压后的 EXE 启动/干净退出通过 |
| 版本信息 | 数字版本 1.1.4.1 与 Windows 预发布标志通过 |
| 冻结保护标记 CLI | 当前尝试写入成功;旧代次被拒绝且原记录不变 |
| 冻结 CAD 助手 | 实际调用 ZIP 解压 EXE 的 probe → checkpoint → quit,通过 |
| CAD 与桌面释放 | CAD/退出助手码均为 0,批次 released,pending 为空,释放日志保留 |
| 主桌面 | Default 保持不变,无自有 CAD 前台/主桌面污染,无观察错误 |
| 用户数据 | 包内不含运行数据目录、任务数据库、现场日志或工单附件 |
本轮冻结助手验证没有打开任何工程,也没有重复替换;真实两工程替换范围和限制沿用[已记录的源码实机验收](../testing/2026-10-21-batch-private-runtime-live.md)。
初次便携包暂存被隐私目录检查阻止:仓库公开的 `docs/testing/data` 内有三份已提交的验收 JSON,被宽泛的 `data` 名称检查误判。保留失败暂存,调整为明确禁止包根运行数据和所有数据库/日志文件,在新暂存目录重新生成并通过;未放行用户数据。
## 附件
- `DxPartReplace-1.1.4-beta.1.zip`:60,266,953 字节;包含程序、完整离线文档、GIF 和空 BOM 模板。
- `BOM-template.xlsx`、`window-tutorial-1.1.4-beta.1.gif`、`release-manifest-1.1.4-beta.1.json`、`SHA256SUMS-1.1.4-beta.1.txt`。
- ZIP SHA256:`acf9f13617b5c25ddfe4c9e4df90481f6ca7838329bca1c201e7c1fdfe27d4a1`。
- [紧凑构建记录](v1.1.4-beta.1-build.json);完整逐文件清单在发布附件的 manifest。
新发布与四个旧发布都设为 Beta 预发布。旧发布只增加 Beta 标题并调整 prerelease 状态,逐一确认原标签、正文和附件 ID/名称/大小不变。新附件先上传到草稿,逐项回下载核对 SHA256,再转为公开 Beta。
本机原始证据位于 `.worktrees/batch-runtime/build/release-1.1.4-beta.1`,构建日志为同工作树 `build/release-1.1.4-beta.1-build.log`。
+31
View File
@@ -0,0 +1,31 @@
# DxPartReplace V1.1.4 Beta 1
发布日期:2026-10-21。**Beta 预发布版**。此前发布也统一标记为 Beta,原标签、附件及下载地址保留。
## 下载与升级
完整解压 `DxPartReplace-1.1.4-beta.1.zip`,运行 `DxPartReplace.exe`。无需安装 Python;本机需要可用的 DxDesigner / Expedition Enterprise 7.9.5、许可证和中央库。
升级前关闭旧程序并备份 `data`,保留原数据,配套更新 EXE 与 `_internal`。不要让不同版本同时打开同一数据目录;历史保护锁不会自动解除。
[中文图文教程](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/README.md)。包内包含离线图文教程、GIF、默认 11 列 BOM 模板、构建清单和 SHA256 校验。教程图片及 GIF 沿用已验收的旧版录制,新增的“重新清理资源”与释放日志以本版界面为准。
## 本版更新
- 每次 BOM 导入和顺序执行分别作为一个批次,共用一个私有桌面;CAD 按需启动,在批内跨工程复用,批末统一退出。
- 工程之间确认上一工程已关闭,重新核验路径、中心库、图页及目标身份;保留保存、关闭、重开回读流程。
- 首次可能写入前持久化保护标记。只有明确证明尚未修改且旧资源已释放时,允许重建重试一次;身份冲突、已修改或结果不确定时不自动重放。
- 释放失败保留已保存并核验的任务结果,阻止新批次,提供“重新清理资源”;清理操作不重复替换,重启后仍检查未收尾批次。
- 释放日志显示开始、成功、失败和重新检查,包含耗时、CAD退出码、句柄关闭与桌面消失的分别确认、失败原因。同批各任务“完整日志”可见,独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`。
- 修复 BOM 错误回报与清理错误同时发生时丢失逐行报告的问题;清理失败只保留诊断,不提供可应用的部分预览。
- 保留原生 CLI 优先采集,减少不必要的 CAD 回退;原生会话重试耗尽时停止整批。
## 验证与范围
两个全新工程副本完成 BOM 导入、实际替换、保存重开、独立完整属性/连接回读和资源释放。两版本保存结果一致;各工程 57 个组件、103 个引脚连接,56 个非目标组件属性和连接不变,源工程及中心库 `.lmc` 哈希不变。
相同样例单次对照:BOM 加执行总耗时 170.860 → 151.859 秒,私有桌面创建 6 → 2 次,CAD 启动 2 → 1 次。该结果不代表固定加速比例;BOM 单次测量 2.250 → 3.187 秒。
工单完整库存覆盖的 31 行回放为 20 通过、11 失败,未强行放行不匹配项;其余 406 条替换行、96 个工程未获对应完整库存,不能宣称全部工程已实机通过。用户机器的外部桌面引用残留根因尚未在本机复现。
详见[批次运行时验收](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-batch-private-runtime.md)、[实机对照](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-batch-private-runtime-live.md)和[释放日志验证](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-resource-release-logging.md)。
@@ -0,0 +1,22 @@
# 批次私有桌面实施计划
用户已在会话中明确批准完整设计及实施;自动重试策略为仅修改前重试一次。基线 e15ee25,不发布安装包。
## 实施任务
- [x] T1:批次运行时。显式传递资源,惰性共用桌面/CAD;项目关闭检查点独立于批末桌面释放;持久化生命周期,资源失败阻止新批次,支持重新清理。正常退出优先,无法确认旧资源释放不重建。
- [x] T2:写前持久化。每次尝试有新的身份与代次,首个可能修改调用前 fsync 写入标记;失败禁止写入。仅当前尝试明确尚未写入且身份与只读指纹安全时允许一次重建重试。
- [x] T3:入口及界面。BOM/任务队列分别拥有一次运行时;单独操作兼容;跨任务复用 CAD,同工程多步骤继续复用。清理状态独立于已核验任务结果,提供清理入口;原模板和队列可读。
- [x] T4:验证与审查。故障注入与取消竞争、旧结果隔离、共享数量和缓存全命中;工单回放及两份真实工程副本替换/保存回读/图连接校验;独立审查并修复,分阶段 Git 提交后推送。
## 接口约定
`BatchCadRuntime(root, kind, stop_event=None)` 为上下文管理器,提供 `native_scope(evidence)`、`run_worker(command, timeout, abort_event=None, cooperative_abort=False)`;`root` 是应用数据根。原生 scope 提供桌面式 start_process/close 接口,关闭只回收该次 CLI 子进程。CAD run_worker 仅在需要时启动并保存项目关闭检查点。
监督端不保存 COM 对象,运行时通过可选 `runtime` 参数显式传递;工作进程中继续核验桌面、CAD PID/GUID 和执行标识。可调用 `guard_pending_resources(root)` 与 `cleanup_pending_resources(root)`,后者仅清理资源,不执行替换。
写前协议模块独立于 Qt/COM。每个进程尝试写入 `DXPART_ATTEMPT_FILE`,由监督端先落盘准备记录,原生写入前验证并置为 mutation_possible。标记和身份缺失不能证明安全。
## 验收约束
所有真实 CAD 写入仅对原仓库 build 下的隔离副本进行,不触碰源工程与中心库。不删除原 feedback 或历史证据,不放宽保护锁。性能按同一副本和相同输入比较,不承诺实机残留根因已治愈。
@@ -0,0 +1,73 @@
# 批次私有桌面实机验收,2026-10-21
本机单组对照通过:两个工程的 BOM 导入及一次顺序替换合计从 170.860 秒降至 151.859 秒,减少 19.001 秒,约 11.12%。业务流程的私有桌面数从 6 个降至 2 个,CAD 启动数从 2 次降至 1 次。该时间不包含独立核验器额外启动 CAD 的开销,属于本次对照的实测值。
## 输入与隔离范围
- 基线:原仓库 `e15ee25`;候选:`.worktrees/batch-runtime` 的批次运行时实现。
- 原始 fixture:原仓库 `build/cbb-bom-import-live-20261015/projects/Audio_D2128` 与 `Audio_T605A`。
- 两版本分别复制到全新目录,使用相同源文件及输入;源工程全文件 SHA-256 在结束后保持一致。
- 中心库:`F:/Sync Tool Skyworth/Work Maps/Skyworth_CBB_Lib/Mentor_lib_EE7.9/mentor_lib.lmc`,只读使用,`.lmc` SHA-256 不变。
- 两工程均只替换 `RV05`,由 `4100-CA0000-T7` 替换为 `4100-CA0000-T0`。`sync_nc=True`,`copy_part_number_to_value=False`,队列策略为 `preserve`。
- 证据根:工作树 `build/batch-runtime-live-20261021-144210`,实际基线结果在 `baseline2`,候选在 `candidate`。`comparison.json` 汇总跨版本对照。
`preserve` 保持原生目标库器件的 Value,而非恢复旧器件文字。此目标的原生 Value 属性缺失,最终期望为 Part Number=`4100-CA0000-T0`、Part Label=`0R`、Value 属性不存在。该语义来自 `docs/superpowers/specs/2026-10-14-v1.1.0-project-tasks-design.md`,与 `QPRestoreComponent`、`QPVerifyValuePolicy` 的存在性及内容核验一致。两版本的最终实际状态均符合该期望。
## 测量结果
| 指标 | 基线 | 候选 |
|---|---:|---:|
| BOM 导入两工程 | 2.250 s | 3.187 s |
| 两工程顺序替换、保存、回读和流程收尾 | 168.610 s | 148.672 s |
| 上述业务流程合计 | 170.860 s | 151.859 s |
| BOM 阶段私有桌面 | 2 | 1 |
| 执行阶段私有桌面 | 4 | 1 |
| 执行阶段 CAD 启动 | 2 | 1 |
基线执行阶段含两个只读 native CLI 桌面和两个 CAD 桌面;候选在该阶段共用一个私有桌面和同一个 CAD PID `9192`,generation 为 `1`。独立核验器使用的额外 CAD 会话单独记录在 `desktop_starts`、`cad_starts` 的 `independent_before`、`independent_after` 阶段,不计入表中的业务资源数量。
## 保存结果与释放核验
每份工程均通过独立 Python COM 读取器,从空 CAD 开始重新打开磁盘工程,直接读取图页组件、全部属性和引脚网络。读取器不调用生产恢复/替换代码。
- 两版本替换前的完整独立读取 JSON 完全相等;替换后的完整 JSON 也完全相等,包含目标器件的所有属性。
- 每份工程保持 57 个组件身份、103 个引脚连接;56 个非目标器件的全部属性、位置、变换和引脚状态完全不变。
- `RV05` 的 UID、Refdes、位置、方向、比例和网络连接保持一致,目标引脚编号非空;Part Number/Part Label/Value 符合上述策略。
- 候选两份写前记录具有相同 batch session/generation、不同 attempt ID,并均为 `mutation_possible`。两个逻辑工程分别记录标记。
- 候选 BOM 和任务队列资源记录均为 `released`,无自动重试、恢复或已成功步骤的重放。
- 观察器确认所有拥有的 CAD 退出、私有桌面消失,输入桌面未切换,拥有的 CAD 未出现在用户桌面或成为前台窗口。
- 候选源代码/VBS 清单的 SHA-256 在实机执行期间保持一致。两个工程源副本及中心库 `.lmc` 的 SHA-256 均不变。
关键证据:
- `comparison.json`
- `candidate/report.json`
- `candidate/desktop-observation.json`
- `candidate/source-under-test-manifest.json`
- `candidate/data/batch-resources/*/session.json`
- `candidate/data/tasks/*/quick-run/batch-attempts/*/*.json`
- `baseline2/read-only-verification/saved-verification.json`
- `baseline2/{before,after}-Audio_D2128.json`、`baseline2/{before,after}-Audio_T605A.json` 及 candidate 对应文件
初次探针在 `baseline` 目录因属性 COM 对象未强制转为 `IVdAttr` 而停止,尚未执行替换。修复读取器后使用全新 `baseline2` 副本执行。`baseline2` 两次替换均已完成,随后旧核验断言误把默认关闭复制选项视为 Part Label 复制;保留原错误报告,依据既有 Value 策略修正独立预期,再仅补充缺少的磁盘读取。该补充过程没有调用队列、恢复或替换。原始错误记录和历史输出均保留。
## 复现
`tools/probe_batch_runtime_live.py` 使用服务层 BOM 导入和任务队列入口,自动创建全新工程副本,无恢复/重放选项。`--repo` 指向被测版本,工具本身保持相同。
```powershell
$batchProbeRepo = 'F:/Sync Tool Skyworth/Work Maps/dxdesigner-part-replacement'
$batchProbeWorktree = "$batchProbeRepo/.worktrees/batch-runtime"
$batchProbeSources = "$batchProbeRepo/build/cbb-bom-import-live-20261015/projects"
$batchProbeLibrary = 'F:/Sync Tool Skyworth/Work Maps/Skyworth_CBB_Lib/Mentor_lib_EE7.9/mentor_lib.lmc'
$batchProbeOutput = "$batchProbeWorktree/build/batch-runtime-live-NEW-TIMESTAMP"
$batchProbeBaseline = "$batchProbeWorktree/build/baseline-e15ee25"
git -C $batchProbeRepo worktree add --detach $batchProbeBaseline e15ee25
python "$batchProbeWorktree/tools/probe_batch_runtime_live.py" --repo $batchProbeBaseline --source-root $batchProbeSources --library $batchProbeLibrary --output "$batchProbeOutput/baseline" --run-native
python "$batchProbeWorktree/tools/probe_batch_runtime_live.py" --repo $batchProbeWorktree --source-root $batchProbeSources --library $batchProbeLibrary --output "$batchProbeOutput/candidate" --run-native
```
基线须固定到 `e15ee25` 的独立目录;合入后原仓库已经是新实现,不能继续把原仓库当前 HEAD 当作旧基线。上述输出及基线目录均应使用尚不存在的新路径。
仅在两个任务均已完成而独立核验需补读时,可使用 `--verify-existing <已有输出目录>`。该分支只读已保存工程,不执行任务队列;核验结果写入独立的 `read-only-verification` 目录,且拒绝覆盖已有核验目录。
@@ -0,0 +1,84 @@
# 批次私有桌面复用与安全重建验收
本次仅更新源码和 Git,不修改 V1.1.3 发布包。实施基线为 `e15ee25`。
## 已实施的边界
一次 BOM 导入、一次顺序执行分别拥有一个 `BatchCadRuntime`。库存缓存全部命中时不创建 Windows 资源;原生 CLI 共用批次桌面,各自进程及 Job 及时回收。需要 CAD 时才启动,同批工程串行复用该实例。工程之间由独立子进程核对桌面、CAD GUID/PID 和活动工程路径为空;每个工程使用新的 VBS 工作进程,组件、引脚及库查询缓存不跨工程传递。原有保存、关闭、重开回读继续执行。
批次标识、代次、工程/任务/作业和尝试标识保存在独立 JSON 旁文件中,旧 BOM、TSV 和已保存任务格式保持兼容。启动下一次工作前先作废该证据目录的旧关闭证明。监控消息有自己的随机标识,旧代次的资源租约不能启动新进程或把迟到关闭结果登记为新代次。
原生写入、恢复属性及保存之前,工作进程先验证完整尝试身份并持久化 `mutation_possible`。落盘失败禁止调用写入。修改前自动重试仅一次,还要求整个工程此前所有作业都处于 `prepared`、身份匹配、状态文件未改变、工程完整指纹一致且旧资源已确认释放。当前未把“数据库文件单独变化”视为自动重试许可;缺少独立身份等价证据时保守停止。明确确认的恢复操作不自动重试。
原生 CLI 仅操作临时数据库副本。启动失败、超时或特定崩溃的重试还要核对当前租约已经关闭、当前批次/代次/尝试一致,以及源 PRJ/数据库 SHA256 未变化。普通表格、匹配、解析和封装错误不触发桌面重建。
原生会话重试耗尽或预算拒绝时停止整批,不能再回退 CAD 绕过异常。该分支在实机正常路径验收后新增四项异常注入并修复,普通非零退出与格式不支持的兼容回退保留。
身份冲突单独分类:bootstrap、空工程检查点的绑定损坏/不一致,以及旧 VBS 明确报告的私有 CAD GUID 冲突,都不会被归入普通启动未就绪而自动重建。子进程用专用退出码保留该类别,异常包裹也保留不可重试的原因。
每个成功释放的代次另外保存在 `generation_releases`,含桌面、CAD PID/GUID、清理证明与释放时间。重建不会覆盖旧代次的释放证据;发布失败后重新检查也不会重复追加成功记录。
## 资源清理与 Windows 契约
- [CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop):关闭自有句柄与确认桌面对象消失分开检查。仍有外部引用时不创建更多桌面绕过问题。
- [Job Objects](https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects):工作进程和后代位于自有 Job;正常退出优先,必要终止限定于自有进程及 Job。
- [TerminateProcess](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminateprocess):终止请求后等待实际退出,随后回收句柄。
- 监督进程不连接 CAD COM,不调用 `SwitchDesktop`。工作进程通过 `STARTUPINFO.lpDesktop` 启动;不改变用户输入桌面。
`batch-resources/<批次>/session.json` 保存生命周期、进程创建时间、工程关闭证明、重建原因和耗时。批末清理失败独立报告“资源释放待处理”,保留已核验任务状态,阻止后续 CAD 入口。界面提供“重新清理资源”,仅清理/重新检查资源,不重放替换。BOM 错误仍保留逐行诊断,但移除可应用的预览。
应用重启后只能重新确认已记录进程和桌面消失;不会根据旧 PID 终止新进程、接管 COM 或自动解除历史工程保护锁。损坏或未知格式的所有权记录保持阻塞,不能把缺失进程列表解释成没有资源。清理过程中即使日志写入失败也继续回收自有资源;释放记录发布失败可再次检查并落盘。取消请求不能覆盖清理失败。
残留时保留窗口 PID/TID/类和查询错误,进一步线程诊断见[Windows 桌面研究](2026-10-21-windows-desktop-lifecycle.md)。没有查询到窗口或线程,不等于不存在外部句柄持有者。
新增真实 Windows 对象寿命回归:测试自己的独立进程连接批次桌面,批次清理确实进入 blocked 并保留 completed_result;该外部进程仍存活、用户输入桌面不变,也没有生成工程 uncertain.flag。测试关闭自己拥有的独立进程后,资源入口确认 released,generation 仍为 1,没有创建新桌面或重放任务。
## 回归范围
针对性注入包含:惰性零启动、多工程一个 CAD、关闭证明作废、一次重建和第二次失败、修改后禁止重放、写前记录失败、任务边界暂停竞争、资源清理失败优先、释放日志发布失败后重新检查、损坏记录、跨重启检查、旧代次租约及检查点拒绝。独立审查发现的关闭证明过期、重复退出首次启动失败实例、独立服务入口绕过门禁、BOM 清理失败丢报告、损坏资源记录误解除等问题均已修复。
全量检查发现一个基线已有的取消测试:假执行器仅声明清理完成,却没有写出必要的释放证据。已在原基线隔离复现,并修正测试的释放证据,生产校验条件没有放宽。
扩展旧版本回归还发现两处既有边界:旧队列冻结 Value 策略发生冲突时,原先会先访问工程进行备份,导致真正的冲突被路径错误覆盖。现在以只读检查先拒绝冲突,断言拒绝前不备份、不启动 CAD、证据不变;备份仍先于 CAD 采集。千步骤模拟测试的固定 30 秒限制在基线也失败(41.83 秒,峰值 2.35 MiB),改为直接约束命令/状态全量解析各不超过两次,保留正确性、128 MiB 内存上限及实际耗时记录,没有调高秒数来掩盖回归。
最终统一执行 `python -m pytest tests/v1_1 tests/trial_task_center -q --basetemp=build/pytest-batch-final-20261021`:**1104 passed、1 failed**,327.33 秒。唯一失败是旧测试要求工作进程路径与长路径字符串逐字相等;本次把证据目录放到较长的 build 路径后,合法的 Windows 短路径触发该断言。生产传参指向正确文件,未改生产代码;将测试改为核对 canonical 文件/目录身份并继续严格核对其余参数,整个 `test_quick_run.py` 在同样较长路径下复测 **34 passed**。两次执行合计覆盖全部 1105 个不同用例,没有未修复的失败;未为测试断言修改重复执行已成功的真实替换。
最终千步骤模拟指标:20 个工程 × 5 张图页 × 10 条规则,91 保存、909 未命中,全部作业唯一;32.147 秒、峰值 2.05 MiB,命令及状态全量解析各 **1 次**。这是注入工作进程的控制层压力测试,不能作为实际 CAD 替换吞吐量。
## 实机与工单范围
实机验收仅对两个全新工程副本进行;源工程、中心库和原始工单附件不修改。分别核对 BOM 导入、替换、保存重开、独立回读、全部非目标元件属性与引脚连接、最终资源释放及原输入桌面。性能按相同样例对照单次运行,不代表大规模实机稳定性或固定加速比例。
工单只有六个工程有完整已捕获原生库存,回放与真实 CAD 验收分别报告,不能据此宣称全部 102 个工程均已实机通过。用户机器的桌面残留根因尚未在本机复现,本次减少重复创建/释放并补齐恢复边界,没有宣称根因已根治。
### 两工程实机对照
完整过程与复现命令见[实机对照报告](2026-10-21-batch-private-runtime-live.md)。计时只包含 BOM 导入和顺序执行,不包含独立检查器额外打开 CAD 的时间。
| 指标 | 基线 | 候选 |
| --- | ---: | ---: |
| BOM 导入 | 2.250 秒 | 3.187 秒 |
| 替换、保存回读、执行收尾 | 168.610 秒 | 148.672 秒 |
| 上述总耗时 | 170.860 秒 | 151.859 秒 |
| 私有桌面创建 | 6 | 2 |
| CAD 启动 | 2 | 1 |
本次样例节省 19.001 秒(11.12%),BOM 阶段单次测量略慢。两个工程各保持 57 个组件、103 个引脚连接,56 个非目标组件全部属性及连接不变;两版本独立回读的完整结果一致。只有 RV05 更换到目标料号,RV11 保持旧料号。批次均记录 released,未自动重试,输入桌面未切换,源工程和中心库 `.lmc` 哈希不变。
正常替换验收之后,最终异常分类及释放历史补丁 `a8358d9` 又做了一次空 CAD 实机冒烟:probe、空工程 checkpoint、quit 全部通过,退出码均为 0,批次 released、pending 为空,CAD/桌面消失,输入桌面仍为 Default。该次实测 59.281 秒,没有打开任何工程,也没有重复替换。证据在工作树 `build/batch-runtime-idle-smoke-20261021-final`。
### 工单只读回放
使用[回放工具](../../tools/replay_issue_bom_report.py)复制工单任务库到新的 build 目录,再通过最新 `inspect_batch` 回放完整库存覆盖的原 Excel 行,保留原行号与工程归属。
| 项目 | 结果 |
| --- | --- |
| 工单原始范围 | 428 个任务;763 行,其中 437 条替换行、102 个工程 |
| 已捕获完整库存 | 8 份报告、6 个工程,覆盖 31 条替换行 |
| 最新逐行结果 | 20 通过、11 失败,工程/Excel 行号/状态与原报告一致 |
| 失败行 | Excel 3、4、6、7、11、12、13、16–19 行 |
| 回放副作用 | 0 CAD/桌面/进程创建,0 目标计划写入,复制数据库未变化 |
| 原证据 | ZIP、BOM、原生库存哈希不变 |
| 未回放 | 406 条替换行、96 个工程,没有相应完整已捕获库存 |
完整本机回放证据位于原仓库 `build/issue-1-batch-runtime-replay-final-20261021`。这些是报告回放,不能代替对应用户工程的实际替换验收。
@@ -0,0 +1,47 @@
# 工单 #1:大量工程 BOM 导入中断
2026-10-21,基于 V1.1.3 修复,未更新发布包。
工单:https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/1
## 附件事实
工单包含应用数据和原模板格式的 BOM。任务库有 428 个草稿任务;BOM 共 763 条数据行,其中 437 条填写了替换内容,涉及 102 个工程,其余 326 条为空白占位。未发现因工程数量触发的固定上限。
日志显示四次导入均因 CAD 采集后的私有桌面释放确认失败而整体中断;四次已完成的采集均有 `uncertain.flag`。另一次采集在打开工程阶段收到取消,附件没有最终收尾记录,不能据此判断是否成功释放。
原生清单回退 CAD 的一个原因是:`1PEF7071_PHY` 的一个非本次替换器件,展开报告中的显示位号和 `Ref Designator` 属性不同。类似情况也出现在 `2G5PHY_GPY211` 的非目标测试点。旧代码因此拒绝整个工程的原生库存。另一次回退源于数据库拒绝读取,该权限问题没有被认定为已修复。
## 修复行为
- 原生报告仍检查全部 UID、配置、层次路径、器件范围和属性唯一性;只有本次请求料号的位号差异才要求 CAD 核验。没有猜测或改写存在差异的目标位号。
- CAD 读取失败,且有明确的会话释放证据时,记录该工程失败并继续检查后续工程。许可由专用异常承载,不能仅凭错误码或保护标记不存在来继续。
- 未确认释放、未知操作异常等仍中止后续工程,但附带完整逐行报告,区分通过、失败和未检查;不生成部分预览、不写入部分计划。
- 状态机在失败后保留报告,重试时清空旧报告。取消不能覆盖未经确认的清理失败。图页证据解析和缓存错误仍保守中止。
- 桌面释放失败时额外记录残留窗口的 PID、线程和窗口类,查询失败则记录诊断错误。不会切换输入桌面或终止外部进程;释放判定和保护锁条件不变。
## 验证
| 项目 | 结果 |
| --- | --- |
| 工单 8 份完整原生报告回放 | 全部可解析,包括原先回退的 4 份报告 |
| `1PEF7071_PHY` 目标核对 | 35 个目标与附件三份 CAD 清单的原理图、图页、UID、位号、料号和 Value 完全一致 |
| 原生目标位号存在差异 | 仍拒绝原生库存,要求 CAD 核验 |
| 本地既有工程副本的新鲜原生读取 | 两个工程分别 1.418 秒、1.596 秒,原生资源释放通过 |
| 本机空 CAD 会话和完整 CAD 采集尝试 | 桌面均正常释放;完整采集因打开图页失败而拒绝,没有执行替换 |
| 实际异步界面 | 失败详情保留 → 重试旧结果清空 → 成功预览;计划未提前写入 |
| Windows 外部桌面引用 | 无窗口和有窗口均保持阻断;窗口场景识别到实际外部 PID,未终止其进程 |
| BOM、原生库存、工程采集、状态机、桌面回归 | 114 passed,34.30 秒 |
| 取消、隔离 CAD、采集缓存回归 | 63 passed,12.32 秒 |
独立代码审查发现并修复了“清理失败且保护标记写入失败时错误继续”和“同时取消掩盖未确认释放”两处边界,最终复核通过。审查者另行运行 38 项针对性检查通过。
工单原生报告中只有 6 个不同工程的完整库存。按原 BOM 回放其 31 条替换行,20 条通过,11 条无法匹配对应旧料号和位号:Excel 第 3、4、6、7、11、12、13、16、17、18、19 行。这些行没有被强行导入;其中第 16–19 行对应库存没有请求的旧料号。
## 限制与后续实机复核
本地相应源工程只有 10 个,版本与用户机器不完全相同;尚未对用户全部 102 个工程完成真实库存核验。本地新鲜读取耗时不能作为用户全部工程的加速比例。
用户机器的 CAD 桌面残留尚未在本机复现,不能宣称泄漏根因已解决。此修复减少已证实不必要的 CAD 回退,并补齐失败报告和诊断;后续需在实机重新导入,核对是否还产生 `inventory_cleanup` 及新的桌面诊断。保留原有锁、日志和附件,不自动解除历史保护锁。
原始工单附件仅在 `build/issue-1-20261021` 保留用于本地诊断,不纳入 Git;`feedback/` 未修改。工程打开尝试仅使用 `build/` 副本,没有执行 Replace Part 或发布新安装包。
@@ -0,0 +1,21 @@
# 资源释放日志
在批次资源复用实现上补齐可读历史,不改变资源释放条件、修改前重试规则或任务结果。
## 查看方式
- 界面:同批各任务的“替换结果 / 日志 → 完整日志”。
- 独立文件:`data/batch-resources/<批次ID>/resource-release.log`。
- 汇总诊断:`data/logs/application.log`;原始所有权证明仍为旁边的 `session.json`。
每次追加开始和最终结果,涵盖批次结束、安全重建、手动重新清理和重启后重新检查。记录批次ID、代次、桌面、CAD PID/GUID、退出码、耗时、句柄关闭、桌面消失、原证明路径及失败原因。不存在 CAD 绑定的历史记录不会被猜成“已正常退出”。缓存全部命中且未获取资源时,不虚构释放成功。
所有同批任务在启动前绑定日志目标;任务ID也写入批次记录,因此应用重启后的重新检查仍能关联原任务。重复点击已经成功的清理不会重复执行或追加伪造的新释放。读日志和重新清理均不执行工程替换。
日志写入错误由诊断层隔离:独立文件、各任务事件和应用日志分别尝试,单处失败仍继续其他输出;能落盘时记录 `release_log_errors`。即使日志文件被锁定或整个诊断器抛异常,也不能跳过进程等待/桌面回收、把未释放资源误判成功,或把已核验任务降为失败。所有输出位置均不可写时无法保证留下文本日志,原有资源释放证明及保护规则仍优先执行。
## 验证
新增七项验证先复现缺少日志功能,再实现通过,覆盖成功、外部持有失败后重试、多任务显示、日志写入失败、重启关联、CAD退出码及缓存零资源。既有真实 Windows 对象寿命测试增加日志断言:独立进程仍连接桌面时记录句柄关闭但桌面未消失的失败;该进程退出后重新清理记录成功。未切换输入桌面、未终止外部持有者、没有重放任务。
日志、Windows资源寿命、BOM、顺序任务、服务保护和界面入口针对性检查:63 passed,13.74秒。原有日志、服务、恢复及批次生命周期回归:72 passed,12.76秒。本次没有执行真实器件替换,没有打包。
@@ -0,0 +1,67 @@
# Windows 桌面生命周期:工单 #1 的继续排查
2026-10-21。此次没有修改生产替换流程、保护锁或发布包。
## 工单能证明什么
工单四次整批 BOM 导入的直接中断原因是 `inventory_cleanup`:受控 CAD、已知子进程退出,自有句柄关闭,但五秒后仍枚举到该私有桌面。它不能证明 Windows 因工程数量多而拒绝导入,也没有足够信息证明残留持有者是谁。BOM 行匹配失败是另外一类结果,详见 [首轮诊断](2026-10-21-issue-1-bom-import.md)。
## 对照微软官方文档
1. [CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop) 关闭一个打开的桌面句柄。成功不等于所有引用消失。调用进程的线程正在使用这个句柄,或它是初始桌面句柄时,关闭会失败。
2. [GetThreadDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getthreaddesktop) 返回指定线程关联的桌面句柄;文档明确不要对该返回值调用 `CloseDesktop`。它与应用通过 `OpenDesktop` 自己打开的句柄归属不同。
3. [Thread Connection to a Desktop](https://learn.microsoft.com/en-us/windows/win32/winstation/thread-connection-to-a-desktop) 说明线程的桌面连接、`STARTUPINFO.lpDesktop` 和继承句柄的关系。线程连接不要求存在可见窗口;不能仅凭窗口列表为空判断没有持有者。
4. [Terminating a Process](https://learn.microsoft.com/en-us/windows/win32/procthread/terminating-a-process) 明确进程终止不会自动终止其创建的全部子进程。[TerminateProcess](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminateprocess) 是异步请求,必须等待退出。现有实现已有 Job、等待与句柄关闭逻辑;本轮未直接把它认定为错误或移除检查。
5. [Traversing the Thread List](https://learn.microsoft.com/en-us/windows/win32/toolhelp/traversing-the-thread-list) 给出 Toolhelp 线程快照遍历方法。[CreateToolhelp32Snapshot](https://learn.microsoft.com/en-us/windows/win32/api/tlhelp32/nf-tlhelp32-createtoolhelp32snapshot) 要求关闭快照句柄。
6. [CreateDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createdesktopw) 说明桌面堆限制。工单已有日志没有桌面创建失败或内存不足的证据,因此本轮没有改注册表的 `SharedSection`、全局资源配额或超时常量。
这些接口没有提供“按桌面名称强制删除全部引用”的安全操作。不能把关闭自己的句柄解释成强行删除其他进程持有的桌面。
## 原生受控实验
实验仅使用临时 Python 进程和 `dx-doc-*` 桌面,不打开真实工程。
| 实验 | 结果 |
| --- | --- |
| 无额外引用,GUI 进程正常退出 | 进程清理后桌面立即消失 |
| 额外持有已退出进程的内核句柄 | 桌面正常释放,不能将进程句柄残留直接当作本案根因 |
| 额外持有已退出线程的内核句柄 | 桌面正常释放 |
| 子进程线程仍连接私有桌面,但不创建窗口 | 复现相同保护现象;窗口清单为空,线程扫描能找到实际 PID/TID/程序路径 |
| 该外部线程的进程退出后 | 再次核验桌面正常释放 |
| 对当前线程正在使用的桌面调用 CloseDesktop | 真实返回 WinError 170;恢复原线程桌面后重试成功,pywin32 没有吞掉失败或丢失重试句柄 |
实际受控残留实验还发现该会话的 `conhost.exe` 线程。此结果只证明诊断能看到无窗口持有者,不能证明用户现场也是 `conhost.exe` 导致。
实验记录在 `build/desktop-doc-research-20261021`:`retained-kernel-handles.json`、`close-api-contract.txt`、`retained-thread-session.json`、`retained-thread-diagnostic.json`、`released-after-owner-exit.json`。原始机器日志未修改。
## 新增只读诊断工具
源码工具:[diagnose_private_desktop.py](../../tools/diagnose_private_desktop.py)。需要在**发生问题的机器、桌面仍残留时**运行,事后复制到另一台电脑的旧日志不能还原已消失的 Windows 资源。
在仓库根目录使用已安装项目依赖的 Python:
```powershell
python tools/diagnose_private_desktop.py --record '实际失败会话/private-desktop.json' --output 'build/desktop-diagnostic.json'
```
也可以根据日志中的真实名称传入 `--desktop dx-实际标识`。同路径或硬链接形式的输出覆盖输入记录均被拒绝。
工具输出桌面是否仍被枚举、残留窗口、连接该桌面的线程 PID/TID/可执行程序路径,以及无法查询的线程错误统计。目标桌面查询句柄只在短生命周期子进程中打开;扫描子进程退出后再检查桌面存在性。借用的 `GetThreadDesktop` 句柄不关闭,自有进程查询句柄和快照关闭。不调用 `SetThreadDesktop`、`SwitchDesktop`、`TerminateProcess`,不关闭外部句柄。
实测 Windows 对另一进程的 `GetThreadDesktop` 查询,在仅打开 `DESKTOP_READOBJECTS`、`READOBJECTS|ENUMERATE`、`READOBJECTS|WRITEOBJECTS` 时仍返回访问拒绝,而打开拥有者 `GENERIC_ALL` 句柄后可以查询。因此仅在扫描子进程中请求该权限,仍只执行查询;权限不足会报错,不能当作“没有持有者”。
**工具的边界:**进程若只保留一个 `OpenDesktop` 句柄,却没有线程连接或窗口,线程/窗口扫描可能找不到它。空扫描不是释放证明;`desktop_still_listed` 仍独立报告。
这种情况下可使用微软官方 [Sysinternals Handle](https://learn.microsoft.com/en-us/sysinternals/downloads/handle),按其文档在管理员终端查询所有类型的句柄:
```powershell
handle64.exe -a 'dx-实际标识'
```
这里只查询,不使用 `-c` 关闭外部句柄。该工具由现场人员按微软文档运行,本轮未下载执行、接受许可或更改系统配置。拿到真实持有 PID 后,才能区分自有程序的生命周期问题、外部进程引用和资源释放延迟,并针对已确认的所有者修复。
## 验证与当前结论
原生测试验证了无窗口线程识别、外部进程不受影响、输入桌面不切换、输入记录不修改、诊断不遗留句柄,以及无匹配线程时仍报告桌面存在。四项工具测试通过;已有 Windows 桌面测试另行通过。
用户现场的持有者尚未查实,不能宣称私有桌面残留已经修好。此次交付是微软文档核对、排除实验和能在现场定位到进程/线程的独立诊断工具;没有删除保护锁、强杀不明进程或放宽准确性校验。
Binary file not shown.

After

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 261 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 296 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 163 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 109 KiB

+5 -5
View File
@@ -1,8 +1,8 @@
# DxPartReplace V1.1.2 中文使用说明
# DxPartReplace V1.1.3 中文使用说明
V1.1.0 沿用原有任务视图和料号规则编辑,增加按工程拆分的任务、按位号选择的替换计划及可选 Excel 操作。
保留原有任务视图和料号规则编辑,支持按工程拆分任务、按位号选择替换目标及可选 Excel 操作。
[窗口 GIF 操作教程](media/v1.1.2-window-bom-tutorial.gif):创建任务、导出/导入 BOM、错误行提示、保存、顺序执行、查看结果和开始新一轮。仅含工具窗口及其弹窗,实际 CAD 等待已压缩。
第一次使用请先看 [README 逐步图文教程](../../README.md#图文教程),按截图完成创建任务、BOM、替换和结果查看。以下内容用于补充查询各项规则。
## 日常流程
@@ -31,9 +31,9 @@ V1.1.0 沿用原有任务视图和料号规则编辑,增加按工程拆分的
导入先核对所有工程归属,再按工程匹配位号;缺少清单时,按该工程 BOM 中的旧料号只读采集。不会执行替换。切换任务可查看、编辑各自预览,点击“保存本批位号计划”才一次写入所有受影响任务。保存失败全部回滚,预览期间不改已保存计划。工程、图页、UID、位号、旧料号及原 Value 用于身份校验,未知工程、重复器件或跨工程目标都会拒绝;不接受公式。布尔选项可使用 TRUE/FALSE 或 是/否。
开发版新增原生采集(尚未打包发布):本机有兼容的 `icdb2ascii.exe` 时,BOM 校验和位号采集优先在临时数据库副本中一次导出真实 UID、位号及物料属性,不必逐条启动 CAD 查找。工具不可用或报告不受支持且已安全清理时,自动使用原 CAD 采集。源工程变化或后台清理不确定时停止并保留诊断;完整图页查询和实际替换仍沿用原执行链。运行前请保存并关闭目标工程。详情见 [原生采集接入记录](../research/2026-10-16-native-cli-integration.md)。
V1.1.3 已包含原生采集:本机有兼容的 `icdb2ascii.exe` 时,BOM 校验和位号采集优先在临时数据库副本中一次导出真实 UID、位号及物料属性,不必逐条启动 CAD 查找。工具不可用或报告不受支持且已安全清理时,自动使用原 CAD 采集。源工程变化或后台清理不确定时停止并保留诊断;完整图页查询和实际替换仍沿用原执行链。运行前请保存并关闭目标工程。详情见 [原生采集接入记录](../research/2026-10-16-native-cli-integration.md)。
BOM 首次采集或新增的位号默认“不替换”;已有计划中未列出的行保留原勾选和编辑状态,不会因后续 BOM 省略而自动停用。如果本次仅执行 BOM 列出的目标,请在预览中检查并取消其他启用行,或在 BOM 中显式填写对应的 `enabled=FALSE` 行。每个填写的位号须在当前工程清单中唯一匹配;同名位号跨图页时应填写图页或 UID。导入失败不会改变已保存计划。
首次采集中未被 BOM 列为替换目标的位号默认“不替换”;已有计划中未列出的行保留原勾选和编辑状态,不会因后续 BOM 省略而自动停用。如果本次仅执行 BOM 列出的目标,请在预览中检查并取消其他启用行,或在 BOM 中显式填写对应的 `enabled=FALSE` 行。每个填写的位号须在当前工程清单中唯一匹配;同名位号跨图页时应填写图页或 UID。导入失败不会改变已保存计划。
“同步 NC”默认勾选,可以按行关闭;仅按原规则识别独立 `/NC` 标记。Value 复制选项和每行自定义 Value 延续原工具的处理方式。替换计划不允许旧、新料号相同,也不允许同批链式或循环替换。
@@ -930,6 +930,7 @@ Sub QPFinishBatchItem()
Next
End If
If FinalState = "restored_ok" Then
QPMarkBatchMutation Command
QPCloseBatchProject TargetPath, True
QP_BatchSaveCommitted = True
ElseIf Not QP_SessionMode Then
@@ -975,6 +976,42 @@ Sub QPFinishBatchItem()
QPProgress "finished", 1, 1
End Sub
Sub QPMarkBatchMutation(ByRef Command)
Dim Shell, Entry, Required, CommandLine, Key, Value, ExitCode
Set Shell = CreateObject("WScript.Shell")
Required = False
For Each Entry In Shell.Environment("Process")
If InStr(1, CStr(Entry), "DXPART_ATTEMPT_FILE=", vbTextCompare) = 1 Or _
InStr(1, CStr(Entry), "DXPART_BATCH_SESSION_ID=", vbTextCompare) = 1 Or _
InStr(1, CStr(Entry), "DXPART_BATCH_GENERATION=", vbTextCompare) = 1 Or _
InStr(1, CStr(Entry), "DXPART_ATTEMPT_ID=", vbTextCompare) = 1 Then Required = True
Next
If Not Required Then Exit Sub
If Len(QP_HelperExecutable) = 0 Or Not QP_Fso.FileExists(QP_HelperExecutable) Then _
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "mutation proof helper is unavailable"
CommandLine = Chr(34) & QP_HelperExecutable & Chr(34)
If Len(QP_HelperScript) > 0 Then
If Not QP_Fso.FileExists(QP_HelperScript) Then _
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "mutation proof script is unavailable"
CommandLine = CommandLine & " " & Chr(34) & QP_HelperScript & Chr(34)
End If
CommandLine = CommandLine & " --mark-batch-attempt"
For Each Key In Array("job_id", "task_id", "project_path")
Value = CStr(Command(Key))
If Len(Value) = 0 Or InStr(Value, Chr(34)) > 0 Or InStr(Value, vbCr) > 0 Or InStr(Value, vbLf) > 0 Then _
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "invalid mutation proof identity"
Select Case Key
Case "job_id": CommandLine = CommandLine & " --attempt-job "
Case "task_id": CommandLine = CommandLine & " --attempt-task "
Case "project_path": CommandLine = CommandLine & " --attempt-project "
End Select
CommandLine = CommandLine & Chr(34) & Value & Chr(34)
Next
ExitCode = Shell.Run(CommandLine, 0, True)
If ExitCode <> 0 Then Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", _
"durable mutation proof failed; no CAD write allowed"
End Sub
Sub QPAutomaticReplaceAndRestore(ByRef Command, ByVal SnapshotPath)
Dim ErrorNumber, ErrorSource, ErrorDescription
QPRequireCommandSnapshot Command, SnapshotPath
@@ -1590,6 +1627,7 @@ Sub QPRestoreAfterOperatorReplace(ByRef Command, ByVal SnapshotPath)
QPProgress "restore_preflight", Index + 1, Records.Item("components").Count
Next
QPProgress "before_restore_transaction", 0, Plans.Count
QPMarkBatchMutation Command
QP_RestoreTransactionOpen = True
QP_App.ActiveView.Block.StartUndoTransaction
On Error Resume Next
+5 -5
View File
@@ -1,10 +1,10 @@
# UTF-8
VSVersionInfo(
ffi=FixedFileInfo(
filevers=(1, 1, 3, 0),
prodvers=(1, 1, 3, 0),
filevers=(1, 1, 4, 1),
prodvers=(1, 1, 4, 1),
mask=0x3f,
flags=0x0,
flags=0x2,
OS=0x40004,
fileType=0x1,
subtype=0x0,
@@ -15,11 +15,11 @@ VSVersionInfo(
StringTable('040904B0', [
StringStruct('CompanyName', 'DxProductivity'),
StringStruct('FileDescription', 'DxDesigner Part Replacement'),
StringStruct('FileVersion', '1.1.3.0'),
StringStruct('FileVersion', '1.1.4.1'),
StringStruct('InternalName', 'DxPartReplace'),
StringStruct('OriginalFilename', 'DxPartReplace.exe'),
StringStruct('ProductName', 'DxDesigner Part Replacement'),
StringStruct('ProductVersion', '1.1.3')
StringStruct('ProductVersion', '1.1.4-beta.1')
])
]),
VarFileInfo([VarStruct('Translation', [1033, 1200])])
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "dx-productivity"
version = "1.1.3"
version = "1.1.4b1"
requires-python = ">=3.12,<3.13"
dependencies = ["PySide6>=6.7,<7", "pywin32>=306", "pywinauto>=0.6.9,<1", "openpyxl>=3.1,<4"]
@@ -51,9 +51,13 @@ def _parse_arguments(argv: Sequence[str] | None) -> argparse.Namespace:
parser.add_argument("--data-root", type=Path, help=argparse.SUPPRESS)
parser.add_argument("--native-helper", action="store_true", help=argparse.SUPPRESS)
parser.add_argument("--snapshot", type=Path, help=argparse.SUPPRESS)
parser.add_argument('--mark-batch-attempt', action='store_true', help=argparse.SUPPRESS)
parser.add_argument('--attempt-job', help=argparse.SUPPRESS)
parser.add_argument('--attempt-task', help=argparse.SUPPRESS)
parser.add_argument('--attempt-project', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--cell-catalog-worker', type=Path, help=argparse.SUPPRESS)
parser.add_argument("--sceneos-cli", action="store_true", help=argparse.SUPPRESS)
parser.add_argument('--private-cad-action', choices=('probe', 'quit'), help=argparse.SUPPRESS)
parser.add_argument('--private-cad-action', choices=('probe', 'quit', 'checkpoint'), help=argparse.SUPPRESS)
parser.add_argument('--private-cad-evidence', type=Path, help=argparse.SUPPRESS)
parser.add_argument('--private-cad-pid', type=int, help=argparse.SUPPRESS)
return parser.parse_args(list(argv) if argv is not None else None)
@@ -116,6 +120,10 @@ def _flush(context: LoggingContext) -> None:
def main(argv: Sequence[str] | None = None) -> int:
"""Run the Qt shell, returning a conventional process status code."""
arguments = _parse_arguments(argv)
if arguments.mark_batch_attempt:
from dx_productivity.trial_task_center.native_helper import run_attempt_marker
return run_attempt_marker(job_id=arguments.attempt_job, task_id=arguments.attempt_task,
project_path=arguments.attempt_project)
if arguments.cell_catalog_worker is not None:
from dx_productivity.trial_task_center.cell_catalog_worker import run
return run(arguments.cell_catalog_worker)
@@ -41,6 +41,7 @@ def execute_batch(
stop_event: Event,
continue_after_backup_failure: bool = False,
reuse_project_session: bool = True,
runtime=None,
) -> BatchRunResult:
preparer = service._quick_run_preparer
retention = maintain_quick_run_storage(preparer.data_root)
@@ -71,6 +72,7 @@ def execute_batch(
projects,
rules,
copy_part_number_to_value=task.copy_part_number_to_value,
**({'runtime': runtime} if runtime is not None else {}),
)
metric['completed_units'] = len(session.commands)
initial_progress = progress
@@ -108,7 +110,8 @@ def execute_batch(
progress.total_items,
)
progress = session.progress
with backups, preparer.project_session_scope(enabled=reuse_project_session) as project_session:
with backups, preparer.project_session_scope(enabled=reuse_project_session,
**({'runtime': runtime} if runtime is not None else {})) as project_session:
while not progress.queue_complete:
if stop_event.is_set():
backups.commit_prefetched(progress)
@@ -137,7 +140,8 @@ def execute_batch(
try:
last_result = preparer.execute_step(task_id, projects, rules, copy_part_number_to_value=task.copy_part_number_to_value,
expected_item=expected, batch_mode=True, batch_session=session,
**({'project_session': project_session} if project_session.enabled else {}))
**({'project_session': project_session} if project_session.enabled else {}),
**({'runtime': runtime} if runtime is not None else {}))
metric['state'] = last_result.state
metric['completed_units'] = int(last_result.state in {'saved_ok', 'restored_ok', 'no_match'})
finally:
@@ -0,0 +1,133 @@
"""Durable write-ahead proof for one logical job in a private CAD attempt.
A prepared record is evidence of no write only while its complete identity still
matches. Helpers never create records and must fail closed if publication fails.
"""
import json
import os
from pathlib import Path
import re
import tempfile
import time
from uuid import uuid4
_IDENTITY = ('session_id', 'generation', 'attempt_id', 'task_id', 'job_id', 'project_path')
def _project(value):
return os.path.normcase(os.path.abspath(value))
def _validate(record):
if not isinstance(record, dict) or record.get('schema_version') != 1:
raise ValueError('invalid batch attempt schema')
for key in ('session_id', 'task_id', 'job_id', 'project_path'):
value = record.get(key)
if not isinstance(value, str) or not value.strip() or any(ord(c) < 32 for c in value):
raise ValueError(f'invalid batch attempt {key}')
generation = record.get('generation')
if type(generation) is not int or generation < 1:
raise ValueError('invalid batch attempt generation')
if not isinstance(record.get('attempt_id'), str) or not re.fullmatch(r'[0-9a-f]{32}', record['attempt_id']):
raise ValueError('invalid batch attempt ID')
if record.get('state') not in {'prepared', 'mutation_possible'}:
raise ValueError('invalid batch attempt state')
return record
def _publish(path, record):
path = Path(path)
descriptor, temporary = tempfile.mkstemp(prefix=f'.{path.name}.', suffix='.tmp', dir=path.parent)
try:
with os.fdopen(descriptor, 'w', encoding='utf-8', newline='') as handle:
json.dump(record, handle, ensure_ascii=False, indent=2)
handle.flush()
os.fsync(handle.fileno())
deadline = time.monotonic() + 1
while True:
try:
os.replace(temporary, path)
break
except PermissionError:
if time.monotonic() >= deadline:
raise
time.sleep(.02)
# POSIX needs a directory sync for the atomic rename; Windows does not
# expose fsync on directories through the CRT.
if os.name != 'nt':
directory = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory)
finally:
os.close(directory)
finally:
Path(temporary).unlink(missing_ok=True)
def prepare_attempt(path, *, session_id, generation, task_id, job_id, project_path, attempt_id=None):
record = _validate(dict(schema_version=1, session_id=str(session_id), generation=generation,
task_id=str(task_id), job_id=str(job_id), project_path=_project(project_path),
attempt_id=uuid4().hex if attempt_id is None else attempt_id, state='prepared'))
Path(path).parent.mkdir(parents=True, exist_ok=True)
_publish(path, record)
return record
def read_attempt(path):
return _validate(json.loads(Path(path).read_text(encoding='utf-8')))
def _bound_attempt(path=None, *, job_id=None, task_id=None, project_path=None):
if path is None and 'DXPART_ATTEMPT_FILE' not in os.environ:
if any(key in os.environ for key in ('DXPART_BATCH_SESSION_ID', 'DXPART_BATCH_GENERATION', 'DXPART_ATTEMPT_ID')):
raise ValueError('missing batch attempt file')
return None
raw_path = os.environ.get('DXPART_ATTEMPT_FILE') if path is None else os.fspath(path)
if not raw_path or not Path(raw_path).is_absolute():
raise ValueError('invalid batch attempt file')
path = Path(raw_path)
job_id = str(job_id) if job_id is not None else os.environ.get('DXPART_JOB_ID')
task_id = str(task_id) if task_id is not None else os.environ.get('DXPART_TASK_ID')
if path.is_dir():
if not job_id or not re.fullmatch(r'[A-Za-z0-9_-]+', job_id):
raise ValueError('missing or unsafe batch attempt job ID')
path = path / f'{job_id}.json'
record = read_attempt(path)
expected = {'session_id': os.environ.get('DXPART_BATCH_SESSION_ID'),
'attempt_id': os.environ.get('DXPART_ATTEMPT_ID'),
'generation': os.environ.get('DXPART_BATCH_GENERATION')}
if not all(expected.values()) or str(record['generation']) != expected.pop('generation'):
raise ValueError('missing or stale batch generation')
if any(record[key] != value for key, value in expected.items()):
raise ValueError('stale batch attempt identity')
if not job_id or not task_id or record['job_id'] != job_id or record['task_id'] != task_id:
raise ValueError('batch attempt task/job mismatch')
# Optional environment bindings remain authoritative even when the caller
# supplies explicit command fields.
for key, value in (('DXPART_JOB_ID', job_id), ('DXPART_TASK_ID', task_id)):
if key in os.environ and os.environ[key] != value:
raise ValueError('batch process task/job mismatch')
if project_path is not None and record['project_path'] != _project(project_path):
raise ValueError('batch attempt project mismatch')
return path, record
def mark_mutation_possible(path=None, *, job_id=None, task_id=None, project_path=None):
bound = _bound_attempt(path, job_id=job_id, task_id=task_id, project_path=project_path)
if bound is None:
return None
path, record = bound
if record['state'] != 'mutation_possible':
record = dict(record, state='mutation_possible')
_publish(path, record)
return record
def can_retry_attempt(path, expected):
try:
_validate(expected)
actual = read_attempt(path)
return actual['state'] == 'prepared' and all(actual[key] == expected[key] for key in _IDENTITY)
except (OSError, ValueError, TypeError, KeyError):
return False
@@ -0,0 +1,86 @@
"""Readable release history, separate from authoritative resource proofs."""
from datetime import datetime, timezone
import json
import os
from pathlib import Path
from .logging_setup import LoggingContext, append_task_event, configure_logging
def _append_line(path, line):
path.parent.mkdir(parents=True, exist_ok=True)
with path.open('a', encoding='utf-8', newline='\n') as stream:
stream.write(line + '\n')
stream.flush()
os.fsync(stream.fileno())
def write_release_event(root, path, record, phase, *, context=None, reason='batch_end',
elapsed=0.0, error=''):
"""Try each diagnostic destination; failures never interrupt native cleanup."""
cleanup = record.get('cleanup') if isinstance(record.get('cleanup'), dict) else {}
confirmation = cleanup.get('desktop_confirmation')
confirmation = confirmation if isinstance(confirmation, dict) else {}
cad = record.get('cad_shutdown') if isinstance(record.get('cad_shutdown'), dict) else {}
disappeared = confirmation.get('released')
if phase == 'succeeded' and record.get('reconciled'):
disappeared = True
details = {'batch_id': record.get('session_id'), 'batch_kind': record.get('kind'),
'generation': record.get('generation'), 'desktop': record.get('desktop'),
'cad_pid': record.get('cad_pid'), 'cad_guid': record.get('cad_guid'),
'reason': reason, 'result': 'releasing' if phase == 'started' else (
'released' if phase == 'succeeded' else 'blocked'),
'elapsed_seconds': elapsed, 'record_path': str(path),
'desktop_handle_closed': cleanup.get('desktop_handle_closed'),
'desktop_disappeared': disappeared, 'cad_exit_code': cad.get('cad_exit_code'),
'quit_helper_exit_code': cad.get('quit_helper_exit_code'),
'cad_shutdown': cad, 'shutdown_warning': record.get('shutdown_warning'),
'retries': record.get('retries', {}), 'cleanup': cleanup, 'error': str(error)}
title = {'started': '开始释放后台资源', 'succeeded': '后台资源释放成功',
'failed': '后台资源释放失败,资源释放待处理'}[phase]
purpose = {'batch_end': '批次结束', 'rebuild': '安全重建', 'manual_retry': '重新清理',
'restart_recheck': '重启后重新检查'}.get(reason, reason)
cad_text = f"CAD PID={details['cad_pid']}" if details['cad_pid'] else (
'CAD 未启动' if record.get('cad_start_requested') is False else
'CAD 启动未完成绑定' if record.get('cad_start_requested') else 'CAD 启动状态未知')
message = (f"{title} · {purpose} · 批次 {details['batch_id']} · 代次 {details['generation']}"
f" · 桌面 {details['desktop']} · {cad_text} · 耗时 {elapsed:.3f} 秒")
if phase != 'started':
label = lambda value: '已确认' if value is True else '未确认'
message += (f" · 桌面句柄关闭:{label(details['desktop_handle_closed'])}"
f" · 桌面消失:{label(disappeared)}")
if cad:
message += f" · CAD 退出码={details['cad_exit_code']},退出助手码={details['quit_helper_exit_code']}"
if error:
message += ' · 原因:' + str(error)
message = message.replace('\r', '\\r').replace('\n', '\\n')
level = 'ERROR' if phase == 'failed' else 'INFO'
timestamp = datetime.now(timezone.utc).isoformat(timespec='milliseconds')
errors = []
try:
_append_line(Path(path).with_name('resource-release.log'),
f'{timestamp} {level} {message} | ' + json.dumps(details, ensure_ascii=False))
except Exception as failure:
errors.append('resource-release.log: ' + str(failure))
try:
context = context if isinstance(context, LoggingContext) else configure_logging(root)
except Exception as failure:
errors.append('application log: ' + str(failure))
return errors
for task_id in record.get('log_task_ids', ()):
try:
append_task_event(context, level=level, task_id=task_id,
operation='resource_release_' + phase, message=message, details=details)
except Exception as failure:
errors.append('task ' + str(task_id) + ': ' + str(failure))
try:
context.logger.log(40 if phase == 'failed' else 20, '%s | %s', message,
json.dumps(details, ensure_ascii=False))
if errors:
context.logger.error('resource_release_log_errors batch=%s errors=%s',
record.get('session_id'), errors)
for handler in context.logger.handlers:
handler.flush()
except Exception as failure:
errors.append('application log: ' + str(failure))
return errors
@@ -0,0 +1,193 @@
"""Persistent resource ownership, independent of project replacement results."""
from __future__ import annotations
import json
import os
from pathlib import Path
import re
import time
from uuid import uuid4
from .errors import TaskCenterError
# Recovery retains owned native handles after failed cleanup. This registry is
# never an implicit execution context and never contains COM application objects.
LIVE_RESOURCES = {}
class BatchResourceCleanupError(TaskCenterError):
def __init__(self, message, *, completed_result=None):
super().__init__('batch_resources_pending', message)
self.completed_result = completed_result
def write_record(path, record):
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + '.' + uuid4().hex + '.tmp')
try:
with temporary.open('w', encoding='utf-8', newline='\n') as stream:
json.dump(record, stream, ensure_ascii=False, indent=2)
stream.flush()
os.fsync(stream.fileno())
deadline = time.monotonic() + 1.0
while True:
try:
os.replace(temporary, path)
break
except PermissionError:
if time.monotonic() >= deadline:
raise
time.sleep(.02)
finally:
temporary.unlink(missing_ok=True)
def _valid_record(record, path):
if not isinstance(record, dict):
return False
session = record.get('session_id')
processes = record.get('processes')
return (record.get('version') == 1 and isinstance(session, str)
and re.fullmatch('[0-9a-f]{32}', session) is not None and path.parent.name == session
and isinstance(record.get('kind'), str) and bool(record['kind'])
and type(record.get('generation')) is int and record['generation'] >= 1
and isinstance(record.get('desktop'), str)
and re.fullmatch('dx-[0-9a-f]{32}', record['desktop']) is not None
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released'}
and isinstance(processes, list) and all(isinstance(item, dict)
and type(item.get('pid')) is int and item['pid'] > 0
and 'creation_time' in item and (item['creation_time'] is None
or isinstance(item['creation_time'], str)) for item in processes)
and 'cad_pid' in record and (record['cad_pid'] is None
or type(record['cad_pid']) is int and record['cad_pid'] > 0)
and 'cad_guid' in record and (record['cad_guid'] is None
or isinstance(record['cad_guid'], str) and bool(record['cad_guid']))
and all(isinstance(record.get(key), dict) for key in ('checkpoints', 'retries'))
and isinstance(record.get('timings'), list))
def pending_resources(root):
pending = []
for path in sorted((Path(root).resolve() / 'batch-resources').glob('*/session.json')):
try:
record = json.loads(path.read_text(encoding='utf-8'))
if not _valid_record(record, path):
raise ValueError('invalid resource ownership record')
if record.get('state') != 'released':
pending.append(dict(record, record_path=str(path)))
except (OSError, ValueError, TypeError) as error:
pending.append({'state': 'blocked', 'record_path': str(path), 'error': str(error)})
return tuple(pending)
def guard_pending_resources(root):
if pending_resources(root):
raise BatchResourceCleanupError('上一批资源尚未确认释放;请先重试释放资源,再开始新批次。')
def process_identity(process):
result = {'pid': process.pid, 'creation_time': None}
handle = getattr(process, 'handle', None)
if handle is not None:
import win32process
result['creation_time'] = str(win32process.GetProcessTimes(handle)['CreationTime'])
return result
def _process_gone(identity):
import win32api
import win32event
import win32process
try:
handle = win32api.OpenProcess(0x101000, False, identity['pid'])
except Exception as error:
code = getattr(error, 'winerror', error.args[0] if error.args else None)
if code == 87:
return True
raise
try:
if win32event.WaitForSingleObject(handle, 0) == 0:
return True
created = identity.get('creation_time')
# A reused PID is an unrelated process and must never be terminated.
return created is not None and str(win32process.GetProcessTimes(handle)['CreationTime']) != created
finally:
handle.Close()
def cleanup_pending_resources(root, *, logging_context=None):
from .isolated_desktop import _Win32Api
from .batch_release_log import write_release_event
for record in pending_resources(root):
path = Path(record['record_path'])
live = LIVE_RESOURCES.get(str(path))
if live is not None:
try:
live.close(reason='manual_retry')
except Exception:
pass # The live owner records the failure and retains ownership.
continue
started = time.monotonic()
def log(phase, error=''):
try:
failures = write_release_event(root, path, record, phase, context=logging_context,
reason='restart_recheck', elapsed=time.monotonic() - started, error=error)
except Exception as failure:
failures = [str(failure)]
if failures:
record.setdefault('release_log_errors', []).extend(failures)
try:
diagnostic = dict(record)
diagnostic.pop('record_path', None)
write_record(path, diagnostic)
except Exception:
pass
try:
if not record.get('session_id'):
continue # Corrupt evidence cannot authorize resource actions.
log('started')
desktop = record.get('desktop')
if desktop:
_Win32Api().confirm_desktop_released(desktop)
if not all(_process_gone(item) for item in record.get('processes', ())):
log('failed', '记录中的进程仍未确认退出')
continue
# After restart only confirm disappearance. Never attach to COM,
# close a project, or terminate a PID whose handles we no longer own.
record.pop('record_path', None)
record = dict(record, state='released', reconciled=True, released_at=time.time())
write_record(path, record)
log('succeeded')
except Exception as error:
if record.get('session_id'):
record.pop('record_path', None)
log('failed', str(error))
write_record(path, dict(record, state='blocked', error=str(error)))
return pending_resources(root)
def project_checkpoint_released(root):
root = Path(root).resolve()
try:
checkpoint = json.loads((root / 'project-checkpoint.json').read_text(encoding='utf-8'))
path = Path(checkpoint['resource_record']).resolve()
record = json.loads(path.read_text(encoding='utf-8'))
key = str(root).casefold()
return (_valid_record(record, path) and checkpoint.get('version') == 1
and type(checkpoint.get('generation')) is int
and isinstance(checkpoint.get('attempt_id'), str)
and re.fullmatch('[0-9a-f]{32}', checkpoint['attempt_id']) is not None
and type(checkpoint.get('cad_pid')) is int and checkpoint['cad_pid'] > 0
and isinstance(checkpoint.get('cad_guid'), str) and bool(checkpoint['cad_guid'])
and isinstance(checkpoint.get('project_path'), str)
and Path(checkpoint['project_path']).is_absolute()
and checkpoint['project_closed'] is True and checkpoint['worker_closed'] is True
and checkpoint['evidence_root'] == str(root)
and record['state'] in {'ready', 'closing', 'released'}
and (checkpoint['session_id'], checkpoint['generation'], checkpoint['cad_pid'], checkpoint['cad_guid'])
== (record['session_id'], record['generation'], record['cad_pid'], record['cad_guid'])
and record['checkpoints'][key] == checkpoint['attempt_id'])
except (OSError, ValueError, TypeError, KeyError):
return False
@@ -0,0 +1,388 @@
"""One lazy private desktop/CAD owner per import or execution batch."""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import subprocess
import time
from uuid import uuid4
from .batch_resources import (BatchResourceCleanupError, LIVE_RESOURCES, cleanup_pending_resources,
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record)
from .errors import TaskCenterError
from .isolated_desktop import PrivateDesktop
from .operation_control import check_cancelled
from .worker_monitor import WorkerMonitorError
class _DesktopLease:
"""Close only this operation's processes; the batch owns the desktop itself."""
def __init__(self, owner, evidence, *, native=False):
self.owner, self.evidence = owner, Path(evidence)
self.native, self.attempt_id = native, uuid4().hex
self.generation, self.desktop = None, None
self.processes = []
self.cleanup_record = None
@property
def name(self):
return (self.desktop if self.desktop is not None else self.owner._ensure_desktop()).name
def __enter__(self):
self.desktop = self.owner._ensure_desktop()
self.generation = self.owner.generation
if self.native:
self.owner.record['native_operations'][str(self.evidence.resolve())] = {
'binding': self.identity, 'workers_closed': False}
self.owner._save()
return self
@property
def identity(self):
return {'session_id': self.owner.session_id, 'generation': self.generation,
'attempt_id': self.attempt_id}
def __exit__(self, *_exc):
self.close()
def start_process(self, command, **options):
available = self.owner.record['state'] == 'ready' or (
not self.native and self.owner.record['state'] == 'closing')
if self.desktop is None or self.generation != self.owner.generation or not available:
raise WorkerMonitorError('stale or unavailable batch resource lease')
process = self.desktop.start_process(command, **options)
self.processes.append(process)
self.owner._track(process)
return process
def close(self):
errors, remaining, records = [], [], []
for process in reversed(self.processes):
try:
self.owner._release_process(process, desktop=self.desktop, generation=self.generation)
records.append(getattr(process, 'cleanup_record', {}))
except Exception as error:
errors.append(str(error))
remaining.append(process)
self.processes = list(reversed(remaining))
self.cleanup_record = {'scope': 'operation', 'processes': records,
'workers_closed': not errors, 'desktop_retained_by_batch': True, 'errors': errors}
if errors:
self.owner._save(state='blocked', error='; '.join(errors))
raise BatchResourceCleanupError('工作进程未能确认退出;批次已停止。' + '; '.join(errors))
if self.native and self.generation == self.owner.generation:
self.owner.record['native_operations'][str(self.evidence.resolve())] = {
'binding': self.identity, 'workers_closed': True}
self.owner._save()
class BatchCadRuntime:
def __init__(self, root, kind, stop_event=None, *, desktop_factory=None, cad_factory=None):
self.root = Path(root).resolve()
self.kind, self.stop_event = kind, stop_event
self.session_id = uuid4().hex
self.path = self.root / 'batch-resources' / self.session_id / 'session.json'
self.desktop_factory = desktop_factory or PrivateDesktop
self.cad_factory = cad_factory
self.desktop = self.cad = None
self.generation = 0
self.result = None
self.record = {'version': 1, 'session_id': self.session_id, 'kind': kind, 'state': 'new',
'generation': 0, 'desktop': None, 'cad_pid': None, 'cad_guid': None,
'processes': [], 'checkpoints': {}, 'retries': {}, 'timings': [], 'native_operations': {}}
self._retry_counts = {}
self._project_attempts = {}
self._logging = None
def bind_logs(self, context, task_ids):
"""Keep every participating task's final release outcome visible."""
tasks = tuple(task_ids)
if any(not isinstance(task, str) or not task or Path(task).name != task
or task in {'.', '..'} for task in tasks):
raise ValueError('resource logs require safe task identifiers')
self._logging = context
self.record['log_task_ids'] = list(dict.fromkeys((*self.record.get('log_task_ids', ()), *tasks)))
if self.generation:
self._save()
def __enter__(self):
guard_pending_resources(self.root)
return self
def __exit__(self, error_type, error, traceback):
try:
self.close()
except Exception as cleanup:
raise BatchResourceCleanupError('批次资源释放待处理;已核验任务结果保留。' + str(cleanup),
completed_result=self.result) from cleanup
def _save(self, **values):
record = dict(self.record, **values)
write_record(self.path, record)
self.record = record
def _timing(self, phase, started, **details):
self.record['timings'].append({'phase': phase, 'seconds': time.monotonic() - started,
'generation': self.generation, **details})
def _ensure_desktop(self, abort_event=None):
if self.record['state'] == 'blocked':
raise BatchResourceCleanupError('当前批次资源已阻断,不能启动新的工作进程。')
if self.desktop is None:
check_cancelled(self.stop_event if abort_event is None else abort_event)
self.generation += 1
name = 'dx-' + uuid4().hex
self._save(state='starting', generation=self.generation, desktop=name,
processes=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
cad_shutdown=None, cad_start_requested=False, cleanup=None)
LIVE_RESOURCES[str(self.path)] = self
self.desktop = self.desktop_factory(name, enabled=True)
started = time.monotonic()
try:
self.desktop.__enter__()
except Exception:
self._save(state='blocked', error='private desktop startup failed')
raise
self._timing('desktop_start', started)
self._save(state='ready')
return self.desktop
def _track(self, process):
identity = process_identity(process)
if identity not in self.record['processes']:
self.record['processes'].append(identity)
self._save()
def _release_process(self, process, *, desktop=None, generation=None):
process.close()
release = getattr(desktop if desktop is not None else self.desktop, 'forget_closed_process', None)
if release:
release(process)
if generation is None or generation == self.generation:
self.record['processes'] = [item for item in self.record['processes'] if item['pid'] != process.pid]
def native_scope(self, evidence):
return _DesktopLease(self, evidence, native=True)
def _ensure_cad(self, abort_event=None):
if self.cad is None:
from .isolated_cad import IsolatedCadSession
factory = self.cad_factory or IsolatedCadSession
lease = _DesktopLease(self, self.path.parent)
started = time.monotonic()
self._save(cad_start_requested=True)
cad = factory(self.path.parent / f'generation-{self.generation + (self.desktop is None)}',
abort_event=abort_event, desktop=lease)
self.cad = cad # Retain cleanup ownership even if startup fails.
try:
cad.__enter__()
except Exception as error:
from .isolated_cad import PrivateCadStartupError
if isinstance(error, PrivateCadStartupError):
# __enter__ certified its local processes/handles closed.
# A never-ready instance must not receive a second Quit.
self.cad = None
raise
self._timing('cad_start', started)
self._save(state='ready', cad_pid=cad.binding.pid, cad_guid=cad.binding.guid)
return self.cad
def _prepare_attempts(self, root, command, attempt_id):
from .batch_attempt import prepare_attempt
from .quick_run import _read_commands
rows = _read_commands(root / 'command.tsv')
selected_rows = [row for row in rows if row[1] == command[-1]]
if len(selected_rows) != 1 or len({row[1] for row in rows}) != len(rows):
raise TaskCenterError('target_identity', '批次工作命令身份缺失或重复,已停止。')
selected = selected_rows[0]
project = Path(selected[2]).resolve()
self._save(active_project=str(project), active_task=selected[0], active_job=selected[1])
directory = root / 'batch-attempts' / (self.session_id + f'-g{self.generation}-' + attempt_id)
expected = []
for row in rows:
if Path(row[2]).resolve() == project:
path = directory / (row[1] + '.json')
record = prepare_attempt(path, session_id=self.session_id, generation=self.generation,
task_id=row[0], job_id=row[1], project_path=str(project), attempt_id=attempt_id)
expected.append((path, record))
self._project_attempts.setdefault(str(project).casefold(), []).extend(expected)
return project, directory, expected
def _can_retry(self, project, expected):
from .batch_attempt import can_retry_attempt
key = str(project).casefold()
return (not self._retry_counts.get(key) and expected
and all(can_retry_attempt(path, record) for path, record in self._project_attempts.get(key, ())))
def _checkpoint(self, root, project, attempt_id):
self.cad.checkpoint(root / ('cad-closed-' + attempt_id + '.json'))
key = str(root).casefold()
self.record['checkpoints'][key] = attempt_id
self._save(state='ready')
write_record(root / 'project-checkpoint.json', {'version': 1, 'resource_record': str(self.path),
'session_id': self.session_id, 'generation': self.generation, 'attempt_id': attempt_id,
'evidence_root': str(root), 'project_path': str(project), 'project_closed': True,
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']})
def run_worker(self, command, timeout, *, abort_event=None, cooperative_abort=False):
from .discovery_cache import project_fingerprint
from .isolated_cad import PrivateCadStartupError, is_identity_conflict
from .worker_monitor import run_monitored_process
root = Path(command[3]).resolve()
self._ensure_desktop(abort_event)
while True:
check_cancelled(abort_event or self.stop_event)
attempt_id = uuid4().hex
# No previous empty-project certificate may authorize continuation
# once another operation could attach or open a project.
self.record['checkpoints'].pop(str(root).casefold(), None)
self._save(active_attempt=attempt_id)
project, directory, expected = self._prepare_attempts(root, command, attempt_id)
before = project_fingerprint(project, self.root)
status = root / 'status.tsv'
status_before = status.read_bytes() if status.exists() else None
started = time.monotonic()
failure = None
try:
cad = self._ensure_cad(abort_event)
cad.checkpoint(root / ('cad-ready-' + attempt_id + '.json'))
environment = {'DXPART_ATTEMPT_FILE': str(directory), 'DXPART_ATTEMPT_ID': attempt_id,
'DXPART_BATCH_SESSION_ID': self.session_id, 'DXPART_BATCH_GENERATION': str(self.generation)}
def start_worker(launch, **options):
return cad.start_worker(launch, extra_environment=environment, **options)
result = run_monitored_process(command, timeout, abort_event=abort_event,
cooperative_abort=cooperative_abort, process_factory=start_worker)
self._checkpoint(root, project, attempt_id)
if result.returncode not in (2, 3):
return result
failure = WorkerMonitorError(result.stderr or 'pre-mutation worker startup failed')
except (PrivateCadStartupError, WorkerMonitorError, subprocess.TimeoutExpired, OSError) as error:
failure = error
finally:
self._timing('project_worker', started, project=str(project), attempt_id=attempt_id,
task_id=self.record['active_task'], job_id=self.record['active_job'])
# Missing proof, changed inputs/status, cancellation or a previous
# write anywhere in this project can never authorize replay.
after = project_fingerprint(project, self.root)
status_after = status.read_bytes() if status.exists() else None
# Runtime database changes require separate identity evidence. Until
# available for a failed read, use the stricter full fingerprint.
safe = (not is_identity_conflict(failure) and before is not None and before == after
and status_before == status_after and self._can_retry(project, expected)
and not (root / 'recovery.request').exists()
and not (abort_event is not None and abort_event.is_set())
and not (self.stop_event is not None and self.stop_event.is_set()))
if not safe:
raise failure
key = str(project).casefold()
self._retry_counts[key] = 1
self.record['retries'][key] = {'count': 1, 'reason': str(failure), 'attempt_id': attempt_id}
self._save(state='recycling')
self.close(rebuilding=True) # Prove disappearance before creating another.
self.desktop = self.cad = None
self._ensure_desktop(abort_event)
def recover_native_failure(self, project, evidence, reason):
"""One read-only recovery, based on durable source and worker proof."""
from .batch_attempt import can_retry_attempt
project, evidence = Path(project).resolve(), Path(evidence).resolve()
key = str(project).casefold()
if (self._retry_counts.get(key) or self.record['state'] == 'blocked'
or self.stop_event is not None and self.stop_event.is_set()):
return False
try:
proof = json.loads((evidence / 'native-retry-proof.json').read_text(encoding='utf-8'))
sources = proof['source_hashes']
operation = self.record['native_operations'][str(evidence)]
if (proof['version'] != 1 or proof['native_read_only'] is not True
or operation['workers_closed'] is not True or proof['binding'] != operation['binding']
or proof['binding']['session_id'] != self.session_id
or proof['binding']['generation'] != self.generation
or proof['workers_closed'] is not True or proof['project_path'] != str(project)
or proof['evidence_root'] != str(evidence) or str(project) not in sources
or len(sources) != 2
or not all(Path(name).resolve().is_relative_to(project.parent) for name in sources)
or not all(hashlib.sha256(Path(name).read_bytes()).hexdigest() == digest
for name, digest in sources.items())
or not all(can_retry_attempt(path, record)
for path, record in self._project_attempts.get(key, ()))):
return False
except (OSError, ValueError, KeyError, TypeError):
return False
self._retry_counts[key] = 1
self.record['retries'][key] = {'count': 1, 'reason': str(reason), 'backend': 'native_cli'}
self._save(state='recycling')
self.close(rebuilding=True)
self.desktop = self.cad = None
self._ensure_desktop()
return True
def _release_log(self, phase, *, reason, elapsed=0.0, error=''):
from .batch_release_log import write_release_event
try:
errors = write_release_event(self.root, self.path, self.record, phase,
context=self._logging, reason=reason, elapsed=elapsed, error=error)
except Exception as failure:
errors = [str(failure)]
if errors:
self.record.setdefault('release_log_errors', []).extend(errors)
try:
self._save()
except Exception:
pass # A diagnostic write cannot bypass process/desktop cleanup.
def close(self, *, rebuilding=False, reason=None):
if self.desktop is None and not self.generation:
return # A cache-only batch never acquired resources.
if self.record['state'] == 'released':
return
reason = reason or ('rebuild' if rebuilding else 'batch_end')
started = time.monotonic()
self._release_log('started', reason=reason)
try:
self._close_resources(rebuilding=rebuilding)
except Exception as error:
self._release_log('failed', reason=reason, elapsed=time.monotonic() - started, error=error)
raise
self._release_log('succeeded', reason=reason, elapsed=time.monotonic() - started)
def _close_resources(self, *, rebuilding=False):
started = time.monotonic()
errors = []
try:
self._save(state='closing')
except Exception as error:
errors.append('resource journal: ' + str(error))
if self.cad is not None:
cad, self.cad = self.cad, None
try:
cad.__exit__(None, None, None)
except Exception as error:
if rebuilding:
self.record['shutdown_warning'] = str(error)
else:
errors.append(str(error))
finally:
cad_record = getattr(cad, 'record', {})
cad_record = cad_record if isinstance(cad_record, dict) else {}
self.record['cad_shutdown'] = {key: cad_record[key] for key in (
'state', 'clean', 'cad_exit_code', 'quit_helper_exit_code', 'cleanup_errors',
'shutdown_seconds') if key in cad_record}
if self.desktop is not None:
try:
self.desktop.close()
except Exception as error:
errors.append(str(error))
self._timing('batch_release', started)
self.record['cleanup'] = getattr(self.desktop, 'cleanup_record', None)
if errors:
self._save(state='blocked', error='; '.join(errors))
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
released_at = time.time()
release = {'generation': self.generation, 'desktop': self.record['desktop'],
'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid'],
'cleanup': self.record['cleanup'], 'released_at': released_at}
releases = [*self.record.get('generation_releases', ()), release]
self._save(state='released', released_at=released_at, processes=[], generation_releases=releases)
LIVE_RESOURCES.pop(str(self.path), None)
@@ -55,11 +55,11 @@ def _assert_editable(service, task_id):
raise TaskCenterError('target_plan_locked', f'工程任务当前不能修改 BOM 计划:{context.task.name}')
def preview_batch(service, task_id, path, *, stop_event=None, on_progress=None):
def preview_batch(service, task_id, path, *, stop_event=None, on_progress=None, runtime=None):
"""Strict adapter to the same validation engine used by the import report."""
from .bom_import_report import inspect_batch
report = inspect_batch(service, task_id, path, stop_event=stop_event, on_progress=on_progress)
report = inspect_batch(service, task_id, path, stop_event=stop_event, on_progress=on_progress, runtime=runtime)
if report.preview is not None:
return report.preview
failed = next((row for row in report.rows if row.status == 'failed'), None)
@@ -1,4 +1,6 @@
"""Read-only, exhaustive BOM validation with an all-or-nothing preview."""
import inspect
from dataclasses import dataclass, replace
from pathlib import Path
from zipfile import BadZipFile
@@ -8,6 +10,7 @@ from openpyxl import load_workbook
from .bom_batch import BomBatchPreview, BomProjectPreview, _assert_editable, _identity, _scope
from .bom_plan import BomMatcher, _normal_headers, _text, parse_bom_row
from .errors import TaskCenterError
from .project_discovery import ReleasedProjectReadError
from .bom_import_state import BomImportPhase
from .store_target_plan import merge_target_inventory
from .target_plan import ProjectIdentityResolver, compile_target_rules
@@ -70,6 +73,15 @@ class BomImportReport:
return sum(project.status == 'unchecked' for project in self.projects)
class BomInspectionError(TaskCenterError):
"""Stop unsafe collection while retaining the complete, read-only row report."""
def __init__(self, cause: TaskCenterError, report: BomImportReport):
super().__init__(cause.code, cause.message, error_id=cause.error_id,
traceback_detail=cause.traceback_detail)
self.bom_report = report
def _report(rows, preview=None, errors=()):
grouped = {}
for row in rows:
@@ -111,12 +123,30 @@ def _check_record_conflicts(rows, records, indices):
rows[index] = replace(rows[index], status='failed', reason=reason)
def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None, on_phase=None):
def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None, on_phase=None, runtime=None):
"""Validate all independent rows; never persist inventory or return partial plans.
Operational failures (including cancellation and native cleanup) propagate so
callers cannot accidentally continue opening CAD after an unsafe failure.
"""
if runtime is None:
from .batch_runtime import BatchCadRuntime, BatchResourceCleanupError
report = None
try:
with BatchCadRuntime(service._logging.root, 'bom_import', stop_event) as owned_runtime:
try:
report = inspect_batch(service, task_id, path, stop_event=stop_event,
on_progress=on_progress, on_phase=on_phase, runtime=owned_runtime)
except BomInspectionError as error:
report = error.bom_report
raise
owned_runtime.result = report
return report
except BatchResourceCleanupError as error:
error.bom_report = replace(report, preview=None) if report is not None else None
if isinstance(error.completed_result, BomImportReport):
error.completed_result = replace(error.completed_result, preview=None)
raise
def check_cancelled():
if stop_event is not None and stop_event.is_set():
raise TaskCenterError('operation_cancelled', 'BOM 导入已取消')
@@ -130,6 +160,7 @@ def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None,
check_cancelled()
notify('正在读取并校验 BOM Excel 表格…', BomImportPhase.READING)
scope = _scope(service._store, task_id)
runtime.bind_logs(service._logging, tuple(task.task_id for task, _ in scope))
routing = ProjectIdentityResolver()
owners = {routing.project(project.path): (task, project) for task, project in scope}
if len(owners) != len(scope):
@@ -233,13 +264,34 @@ def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None,
inventory = tuple(row.target for row in baseline)
else:
progress(f'「{task.name}」正在获取位号库存,仅采集本表涉及的旧料号及已有计划料号')
controls = {}
collector = service.for_project_task(task.task_id)._quick_run_preparer.collect_inventory
controls = ({'runtime': runtime} if any(
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
for parameter in inspect.signature(collector).parameters.values()) else {})
if stop_event is not None:
controls['stop_event'] = stop_event
if on_progress is not None:
controls['on_progress'] = progress
inventory = tuple(service.for_project_task(task.task_id)._quick_run_preparer.collect_inventory(
project.path, tuple(dict.fromkeys((*known, *requested))), **controls))
try:
inventory = tuple(collector(
project.path, tuple(dict.fromkeys((*known, *requested))), **controls))
except TaskCenterError as error:
if error.code == 'operation_cancelled':
raise
reason = f'工程采集失败:{error.message}(错误编号:{error.error_id})'
for index in indices:
rows[index] = replace(rows[index], status='failed', reason=reason)
# Continue only with positive session-release evidence, not
# merely a shared error code or an absent uncertainty marker.
if isinstance(error, ReleasedProjectReadError):
progress(f'「{task.name}」采集失败,已记录;继续核验其他工程')
continue
for index, row in enumerate(rows):
if row.status == 'unchecked':
rows[index] = replace(row, reason='前序工程采集异常,已停止后续核验')
report = _report(rows, errors=(
f'已停止批量采集:{task.name};{error.message}(错误编号:{error.error_id})',))
raise BomInspectionError(error, report) from error
check_cancelled()
progress(f'「{task.name}」正在匹配 {len(indices)} 行 BOM 与工程位号…', BomImportPhase.MATCHING)
merged = merge_target_inventory(baseline, inventory)
@@ -90,8 +90,12 @@ class BomImportMachine:
return False
cancelled_error = isinstance(error, TaskCenterError) and error.code == 'operation_cancelled'
if error is not None and not cancelled_error:
from .bom_import_report import BomImportReport
reason = error.message if isinstance(error, TaskCenterError) else str(error)
self.state = replace(self.state, phase=BomImportPhase.FAILED, report=None,
report = getattr(error, 'bom_report', None)
if not isinstance(report, BomImportReport):
report = None
self.state = replace(self.state, phase=BomImportPhase.FAILED, report=report,
message=f'本次 BOM 导入未完成:{reason[:240]};原有预览和编辑保留。')
elif cancelled_error or self.state.phase is BomImportPhase.CANCELLING:
self.state = replace(self.state, phase=BomImportPhase.CANCELLED, report=None,
@@ -2,6 +2,7 @@
from __future__ import annotations
import inspect
from pathlib import Path
from .errors import TaskCenterError
@@ -58,11 +59,15 @@ def read_component_inventory(path: Path, project_path: Path,
def collect_target_inventory(preparer, project_path: Path,
old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None) -> tuple[ComponentTarget, ...]:
old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None, runtime=None) -> tuple[ComponentTarget, ...]:
if not old_part_numbers:
raise TaskCenterError("target_inventory", "at least one old Part Number is required")
from .operation_control import check_cancelled, control_options
check_cancelled(stop_event)
rows = preparer.collect_inventory(project_path, old_part_numbers, **control_options(stop_event, on_progress))
runtime_options = ({'runtime': runtime} if runtime is not None and any(
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
for parameter in inspect.signature(preparer.collect_inventory).parameters.values()) else {})
rows = preparer.collect_inventory(project_path, old_part_numbers, **runtime_options,
**control_options(stop_event, on_progress))
check_cancelled(stop_event)
return rows
@@ -18,6 +18,34 @@ class PrivateCadStartupError(WorkerMonitorError):
"""The owned desktop was released before any project worker was launched."""
class CadIdentityConflict(WorkerMonitorError, ValueError):
"""An invalid CAD ownership certificate can never authorize automatic replay."""
class PrivateCadIdentityError(PrivateCadStartupError, CadIdentityConflict):
"""Startup ownership failed, and its local resources were released."""
PRIVATE_CAD_IDENTITY_EXIT_CODE = 4
def is_identity_conflict(error):
"""Keep the no-retry decision when cleanup or evidence I/O wraps the cause."""
pending, seen = [error], set()
while pending:
item = pending.pop()
if item is None or id(item) in seen:
continue
seen.add(id(item))
# The legacy VBS launcher transports these ownership failures as exit
# 2 plus text. Treat only its explicit identity diagnostics as conflicts.
if (isinstance(item, CadIdentityConflict) or any(message in str(item).casefold()
for message in ('incomplete private cad identity', 'private cad guid mismatch'))):
return True
pending.extend((item.__cause__, item.__context__))
return False
def _helper_command(action, evidence, pid):
if getattr(sys, 'frozen', False):
return (sys.executable, '--private-cad-action', action, '--private-cad-evidence', str(evidence),
@@ -65,10 +93,11 @@ def _designer_executable():
class IsolatedCadSession:
"""The supervisor never attaches to COM on the user's input desktop."""
def __init__(self, root, *, abort_event=None):
def __init__(self, root, *, abort_event=None, desktop=None):
self.root = Path(root).resolve()
self.token = uuid4().hex
self.desktop = PrivateDesktop('dx-' + self.token, enabled=True)
self.borrowed_desktop = desktop is not None
self.desktop = desktop if desktop is not None else PrivateDesktop('dx-' + self.token, enabled=True)
self.cad = None
self.binding = None
self.started = time.monotonic()
@@ -110,13 +139,19 @@ class IsolatedCadSession:
time.sleep(0.05)
code = probe.poll()
if code:
raise WorkerMonitorError('private CAD startup identity failed: ' + probe.stderr.read().decode('utf-8', errors='replace'))
failure_type = CadIdentityConflict if code == PRIVATE_CAD_IDENTITY_EXIT_CODE else WorkerMonitorError
raise failure_type('private CAD startup failed: ' + probe.stderr.read().decode('utf-8', errors='replace'))
finally:
probe.close()
result = json.loads(bootstrap.read_text(encoding='utf-8'))
self.binding = CadBinding(**result)
if (self.binding.pid, self.binding.desktop, self.binding.token) != (self.cad.pid, self.desktop.name, self.token):
raise WorkerMonitorError('private CAD bootstrap returned a foreign identity')
try:
result = json.loads(bootstrap.read_text(encoding='utf-8'))
binding = CadBinding(**result)
if (type(binding.pid) is not int or not isinstance(binding.guid, str) or not binding.guid
or (binding.pid, binding.desktop, binding.token) != (self.cad.pid, self.desktop.name, self.token)):
raise ValueError('private CAD bootstrap returned a foreign identity')
except (OSError, ValueError, TypeError) as failure:
raise CadIdentityConflict('private CAD bootstrap identity evidence is invalid: ' + str(failure)) from failure
self.binding = binding
self._record(state='ready', guid=self.binding.guid, startup_seconds=time.monotonic() - self.started)
return self
except Exception as error:
@@ -126,16 +161,51 @@ class IsolatedCadSession:
self._uncertain('startup cleanup failed: ' + str(failure))
raise WorkerMonitorError('private CAD startup cleanup requires reconciliation: ' + str(failure)) from error
self._record(state='released', clean=False, startup_error=str(error))
if is_identity_conflict(error):
raise PrivateCadIdentityError(str(error)) from error
raise PrivateCadStartupError(str(error)) from error
def _check_abort(self):
if self.abort_event is not None and self.abort_event.is_set():
raise WorkerMonitorError('private CAD startup was cancelled before execution')
def start_worker(self, command, **options):
def start_worker(self, command, *, extra_environment=None, **options):
if self.binding is None or self.cad.poll() is not None:
raise WorkerMonitorError('owned private CAD is unavailable')
return self.desktop.start_process(command, env=_child_environment(self.binding.environment()), **options)
environment = self.binding.environment()
environment.update(extra_environment or {})
# The verified CAD binding cannot be overridden by attempt metadata.
binding = self.binding.environment()
for key in ('DXPART_DESKTOP', 'DXPART_CAD_PID', 'DXPART_CAD_GUID', 'DXPART_SESSION_TOKEN'):
environment[key] = binding[key]
return self.desktop.start_process(command, env=_child_environment(environment), **options)
def checkpoint(self, evidence):
"""Certify an idle, identity-verified CAD before crossing project boundaries."""
if self.binding is None or self.cad.poll() is not None:
raise WorkerMonitorError('owned CAD unavailable at project checkpoint')
evidence = Path(evidence)
evidence.parent.mkdir(parents=True, exist_ok=True)
process = self.start_worker(_helper_command('checkpoint', evidence, self.cad.pid),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
code = process.wait(timeout=20)
if code:
failure_type = CadIdentityConflict if code == PRIVATE_CAD_IDENTITY_EXIT_CODE else WorkerMonitorError
raise failure_type('project close checkpoint failed: ' +
process.stderr.read().decode('utf-8', errors='replace'))
try:
result = json.loads(evidence.read_text(encoding='utf-8'))
if (not isinstance(result, dict) or result.get('binding') != asdict(self.binding)
or result.get('project_path') != '' or result.get('project_closed') is not True):
raise ValueError('project close checkpoint identity mismatch')
except (OSError, ValueError, TypeError) as failure:
raise CadIdentityConflict('project close checkpoint identity evidence is invalid: ' + str(failure)) from failure
if self.cad.poll() is not None:
raise WorkerMonitorError('owned CAD exited during project checkpoint')
finally:
process.close()
return result
def __exit__(self, error_type, error, traceback):
closing_started = time.monotonic()
@@ -174,7 +244,9 @@ class IsolatedCadSession:
finally:
try:
self.desktop.close()
self._record(state='released', clean=error is None and cleanup_error is None,
self._record(state='session_closed' if self.borrowed_desktop else 'released',
desktop_retained_by_batch=self.borrowed_desktop,
clean=error is None and cleanup_error is None,
shutdown_seconds=time.monotonic() - closing_started,
desktop_cleanup=getattr(self.desktop, 'cleanup_record', None), **exit_codes)
except Exception as failure:
@@ -190,12 +262,17 @@ class IsolatedCadSession:
self._record(state='uncertain', error=message, **details)
def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abort=False):
def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abort=False, runtime=None):
from .worker_monitor import run_monitored_process
if runtime is not None:
return runtime.run_worker(command, timeout, abort_event=abort_event,
cooperative_abort=cooperative_abort)
try:
with IsolatedCadSession(Path(command[3]), abort_event=abort_event) as session:
return run_monitored_process(command, timeout, abort_event=abort_event, process_factory=session.start_worker,
**({'cooperative_abort': True} if cooperative_abort else {}))
except CadIdentityConflict:
raise
except PrivateCadStartupError as error:
# The existing handoff recognizes exit=2 with unchanged status as a
# retryable pre-mutation startup failure, without an uncertainty lock.
@@ -203,6 +280,14 @@ def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abor
def native_action(action, evidence, expected_pid):
try:
return _run_native_action(action, evidence, expected_pid)
except CadIdentityConflict as error:
print(str(error), file=sys.stderr)
return PRIVATE_CAD_IDENTITY_EXIT_CODE
def _run_native_action(action, evidence, expected_pid):
"""Runs in a child created with lpDesktop, never in a Python thread switch."""
if action != 'probe':
return _native_action(action, evidence, expected_pid)
@@ -228,10 +313,10 @@ def _native_action(action, evidence, expected_pid):
desktop = os.environ.get('DXPART_DESKTOP', '')
token = os.environ.get('DXPART_SESSION_TOKEN', '')
if not desktop.startswith('dx-') or current_desktop_name() != desktop:
raise ValueError('CAD bootstrap is not on the requested private desktop')
raise CadIdentityConflict('CAD bootstrap is not on the requested private desktop')
expected_pid = int(expected_pid)
if expected_pid <= 0 or str(expected_pid) != os.environ.get('DXPART_CAD_PID'):
raise ValueError('invalid CAD bootstrap PID')
raise CadIdentityConflict('invalid CAD bootstrap PID')
deadline = time.monotonic() + 60
while True:
try:
@@ -250,16 +335,29 @@ def _native_action(action, evidence, expected_pid):
# ROT is scoped to this desktop. Certify the freshly launched instance
# here; every subsequent VBS/helper attachment checks its GUID.
binding = CadBinding(desktop, expected_pid, str(app.Guid()), token)
verify_application(app, binding, require_view=False)
try:
verify_application(app, binding, require_view=False)
except ValueError as error:
raise CadIdentityConflict(str(error)) from error
Path(evidence).write_text(json.dumps(asdict(binding)), encoding='utf-8')
elif action == 'quit':
binding = CadBinding.from_environment()
elif action in {'quit', 'checkpoint'}:
try:
binding = CadBinding.from_environment()
if binding is None:
raise ValueError('missing private CAD identity')
verify_application(app, binding, require_view=False)
opened_project = project_file_path(app)
except ValueError as error:
raise CadIdentityConflict(str(error)) from error
# ActiveViewHandle can retain a destroyed/reused HWND after closing
# the last project. The private ROT/GUID binds this empty instance.
verify_application(app, binding, require_view=False)
if project_file_path(app):
raise ValueError('owned CAD still has a project open; no implicit save or discard')
app.Quit()
if opened_project:
raise CadIdentityConflict('owned CAD still has a project open; no implicit save or discard')
if action == 'quit':
app.Quit()
else:
from .batch_resources import write_record
write_record(evidence, {'binding': asdict(binding), 'project_path': '', 'project_closed': True})
else:
raise ValueError('unknown private CAD action')
except BaseException as failure:
@@ -74,9 +74,40 @@ class _Win32Api:
self.desktop_release_check['caller_desktop'] = current_desktop_name()
except Exception as error:
self.desktop_release_check['caller_desktop_error'] = str(error)
try:
self.desktop_release_check['remaining_windows'] = self._desktop_windows(name)
except Exception as error:
self.desktop_release_check['window_query_error'] = str(error)
raise TaskCenterError('private_desktop_cleanup', 'owned private desktop still exists after cleanup')
time.sleep(min(0.05, max(0.0, deadline - time.monotonic())))
def _desktop_windows(self, name):
"""Diagnose residue without switching desktops or terminating foreign owners."""
import win32gui
desktop = self.service.OpenDesktop(name, 0, False, 1) # DESKTOP_READOBJECTS
windows = []
try:
user = ctypes.WinDLL('user32', use_last_error=True)
callback_type = ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
enumerate_windows = user.EnumDesktopWindows
enumerate_windows.argtypes = [wintypes.HANDLE, callback_type, wintypes.LPARAM]
enumerate_windows.restype = wintypes.BOOL
handles = []
callback = callback_type(lambda handle, _: handles.append(handle) or True)
# An empty desktop can return FALSE with no last error. Our
# callback always continues, so only a nonzero error is a failure.
ctypes.set_last_error(0)
if not enumerate_windows(int(desktop), callback, 0) and ctypes.get_last_error():
raise ctypes.WinError(ctypes.get_last_error())
for handle in handles:
thread, pid = self.process.GetWindowThreadProcessId(handle)
windows.append({'hwnd': int(handle), 'pid': pid, 'thread_id': thread,
'class': win32gui.GetClassName(handle)})
finally:
desktop.CloseDesktop()
return windows
def create_process(self, command: Sequence[str], desktop: str, cwd: Path | None, env: Mapping[str, str] | None,
*, stdout=None, stderr=None):
startup = self.process.STARTUPINFO()
@@ -442,6 +473,16 @@ class PrivateDesktop:
self._processes.append(process)
return process
def forget_closed_process(self, process):
"""Drop a worker only after its owning wrapper confirmed all cleanup."""
record = getattr(process, 'cleanup_record', {})
if (record.get('process_handle_closed') is not True
or record.get('job_handle_closed') is not True
or record.get('unconfirmed_descendant_handles', 0)
or record.get('errors') or process.poll() is None):
raise OSError('cannot forget a worker with unconfirmed owned resources')
self._processes = [item for item in self._processes if item is not process]
def close(self) -> None:
errors, remaining = [], []
self.cleanup_record = {'desktop': self.name, 'processes': []}
@@ -2,6 +2,7 @@
import argparse
import faulthandler
import ctypes
import csv
import hashlib
import json
import os
@@ -13,6 +14,48 @@ import time
import traceback
def snapshot_attempt(snapshot):
"""Bind batch evidence to the actual snapshot and its confirmed queue row."""
from .batch_attempt import _bound_attempt
if 'DXPART_ATTEMPT_FILE' not in os.environ:
return _bound_attempt()
snapshot = Path(snapshot)
metadata, _ = read_snapshot(snapshot)
if not snapshot.name.startswith('snapshot-') or snapshot.suffix != '.tsv':
raise ValueError('batch snapshot has no job identity')
job_id = snapshot.stem[len('snapshot-'):]
with (snapshot.parent / 'command.tsv').open(encoding='utf-8-sig', newline='') as handle:
commands = [row for row in csv.DictReader(handle, delimiter='\t') if row.get('job_id') == job_id]
if len(commands) != 1:
raise ValueError('batch snapshot must match one confirmed command')
command = commands[0]
if os.path.normcase(os.path.abspath(command['project_path'])) != os.path.normcase(os.path.abspath(metadata[1])):
raise ValueError('batch command and snapshot projects differ')
return _bound_attempt(job_id=job_id, task_id=command['task_id'], project_path=metadata[1])
def mark_snapshot_mutation(snapshot):
from .batch_attempt import mark_mutation_possible
bound = snapshot_attempt(snapshot)
if bound is None:
return None
path, record = bound
return mark_mutation_possible(path, job_id=record['job_id'], task_id=record['task_id'],
project_path=record['project_path'])
def run_attempt_marker(*, job_id, task_id, project_path):
from .batch_attempt import mark_mutation_possible
if not job_id or not task_id or not project_path:
return 2
try:
mark_mutation_possible(job_id=job_id, task_id=task_id, project_path=project_path)
return 0
except (OSError, ValueError, KeyError, TypeError) as error:
print(f'Batch mutation proof failed: {error}', file=sys.stderr)
return 1
def decode(value):
characters = []
index = 0
@@ -542,6 +585,7 @@ def run_native(snapshot, report):
slot_count = int(binding.get("slot_count", report.get("slot_count", 1)))
if slot_count < 1:
raise ValueError("Recorded replacement has no valid slots")
mark_snapshot_mutation(snapshot)
report["promoted_pins"] = promote_pin_numbers(app, api, report["replaced_uids"], slot_count,
lambda phase: _promotion_phase(snapshot, report, phase))
report["already_replaced"] = True
@@ -706,6 +750,7 @@ def run_native(snapshot, report):
cell_catalog=cell_proof)
report["stage"] = "native_replace_started"
save_report(snapshot, report)
mark_snapshot_mutation(snapshot)
click(dlg_item(dialog, 1695))
replacement_poll_count = 0
@@ -820,6 +865,15 @@ def run(snapshot):
watchdog.daemon = True
watchdog.start()
try:
bound = snapshot_attempt(snapshot)
if bound is not None:
_, current = bound
identity = {key: current[key] for key in
('session_id', 'generation', 'attempt_id', 'task_id', 'job_id', 'project_path')}
if report.get('batch_attempt') != identity:
report.clear()
report.update(snapshot=str(snapshot), ok=False, stage='preflight')
report['batch_attempt'] = identity
run_native(snapshot, report)
report["ok"] = True
return 0
@@ -17,6 +17,15 @@ from .native_inventory_protocol import NativeInventoryUnsupported, read_ascii_di
from .operation_control import check_cancelled, report_progress
_RETRY_NATIVE_READ = object()
_NATIVE_CRASH_CODES = frozenset({0xC0000005, 0xC000001D, 0xC0000094,
0xC00000FD, 0xC0000374, 0xC0000409})
class _NativeInventoryProcessError(NativeInventoryUnsupported):
"""A launch, watchdog or explicit crash failure, separate from invalid input."""
def _hash(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
@@ -71,7 +80,7 @@ class NativeInventoryReader:
raise NativeInventoryUnsupported('本机未安装完整的原生器件清单工具')
return tools
def read(self, project, old_part_numbers, *, stop_event=None, on_progress=None):
def read(self, project, old_part_numbers, *, stop_event=None, on_progress=None, runtime=None):
check_cancelled(stop_event)
try:
project = Path(project).resolve(strict=True)
@@ -80,10 +89,15 @@ class NativeInventoryReader:
except (NativeInventoryUnsupported, OSError, ImportError, UnicodeError, TaskCenterError) as error:
report_progress(on_progress, f'使用 CAD 核验位号:{str(error)[:160]}')
return None
return self._read(project, text, block, snapshot, database, tools, old_part_numbers,
stop_event=stop_event, on_progress=on_progress)
for attempt in range(2):
result = self._read(project, text, block, snapshot, database, tools, old_part_numbers,
stop_event=stop_event, on_progress=on_progress, runtime=runtime,
allow_retry=runtime is not None and attempt == 0)
if result is not _RETRY_NATIVE_READ:
return result
return None
def _read(self, project, text, block, snapshot, database, tools, old_part_numbers, *, stop_event, on_progress):
def _read(self, project, text, block, snapshot, database, tools, old_part_numbers, *, stop_event, on_progress, runtime=None, allow_retry=False):
token = uuid4().hex
evidence = self.data_root / 'quick-run-history' / ('native-inventory-' + token)
evidence.mkdir(parents=True, exist_ok=False)
@@ -91,7 +105,8 @@ class NativeInventoryReader:
/ 'DxProductivity' / 'native-inventory').resolve()
base.mkdir(parents=True, exist_ok=True)
workspace = Path(tempfile.mkdtemp(prefix='dx-cli-', dir=base)).resolve()
desktop = self.desktop_factory('dx-cli-' + token, enabled=True)
desktop = (runtime.native_scope(evidence) if runtime is not None
else self.desktop_factory('dx-cli-' + token, enabled=True))
record = {'backend': 'native_cli', 'project': str(project), 'state': 'reading',
'workspace': str(workspace), 'tools': []}
started = time.monotonic()
@@ -151,9 +166,40 @@ class NativeInventoryReader:
if record['state'] == 'uncertain':
raise TaskCenterError('inventory_cleanup', f'原生采集清理状态不确定:{evidence}') from error
_verify_sources(source_hashes)
record.update(state='fallback', reason=str(error))
report_progress(on_progress, f'原生清单未能完整核对,改用 CAD 采集:{str(error)[:160]}')
record.update(source_hashes_unchanged=True,
source_hashes={str(path): digest for path, digest in source_hashes.items()})
retry = False
cleanup = getattr(desktop, 'cleanup_record', None)
session_failure = runtime is not None and isinstance(error, _NativeInventoryProcessError)
if (session_failure
and clean and isinstance(cleanup, dict) and cleanup.get('workers_closed') is True):
check_cancelled(stop_event)
from .batch_resources import write_record
write_record(evidence / 'native-retry-proof.json', {
'version': 1, 'native_read_only': True, 'project_path': str(project),
'source_hashes': {str(path): digest for path, digest in source_hashes.items()},
'workers_closed': True, 'evidence_root': str(evidence.resolve()),
'binding': dict(desktop.identity),
})
_verify_sources(source_hashes)
if allow_retry:
retry = runtime.recover_native_failure(project, evidence, str(error))
if session_failure and not retry:
record.update(state='session_failed', reason=str(error))
raise TaskCenterError('inventory_session',
f'原生读取会话异常,未能完成安全重试,批次已停止。请保留诊断记录后排查:{evidence};{error}') from error
record.update(state='retrying' if retry else 'fallback', reason=str(error))
if retry:
result = _RETRY_NATIVE_READ
report_progress(on_progress, '原生读取会话已安全释放,正在重新核验一次…')
else:
report_progress(on_progress, f'原生清单未能完整核对,改用 CAD 采集:{str(error)[:160]}')
except TaskCenterError as error:
if record['state'] == 'uncertain':
try:
(evidence / 'uncertain.flag').write_text(error.message, encoding='utf-8')
except OSError as marker_error:
record['uncertain_marker_error'] = str(marker_error)
if record['state'] != 'uncertain':
record.update(state='cancelled' if error.code == 'operation_cancelled' else 'failed', reason=error.message)
raise
@@ -181,19 +227,32 @@ class NativeInventoryReader:
started = time.monotonic()
metric = {'name': name, 'report': report_name, 'exit_code': None}
record['tools'].append(metric)
process = None
try:
with (output / (log_name + '.stdout.txt')).open('wb') as stdout, (
output / (log_name + '.stderr.txt')).open('wb') as stderr:
process = desktop.start_process(command, cwd=workspace, env=environment,
stdout=stdout, stderr=stderr)
try:
process = desktop.start_process(command, cwd=workspace, env=environment,
stdout=stdout, stderr=stderr)
except OSError as error:
raise _NativeInventoryProcessError(f'{name} 启动失败:{error}') from error
deadline = started + self.timeout
while process.poll() is None:
check_cancelled(stop_event)
if time.monotonic() >= deadline:
raise NativeInventoryUnsupported(f'{name} 原生读取超时')
raise _NativeInventoryProcessError(f'{name} 原生读取超时')
time.sleep(0.05)
metric['exit_code'] = process.poll()
if metric['exit_code'] & 0xFFFFFFFF in _NATIVE_CRASH_CODES:
raise _NativeInventoryProcessError(f'{name} 原生读取进程崩溃:{metric["exit_code"]}')
if metric['exit_code'] != 0:
raise NativeInventoryUnsupported(f'{name} 返回退出码 {metric["exit_code"]}')
finally:
metric['seconds'] = time.monotonic() - started
if process is not None and callable(getattr(process, 'close', None)):
try:
process.close()
except Exception as error:
record.update(state='uncertain', cleanup_error=str(error))
raise TaskCenterError('inventory_cleanup',
f'原生读取进程及其子进程未能确认退出:{error}') from error
@@ -41,6 +41,7 @@ def _effective_properties(text):
if len(re.findall(r'^Config .+', text, re.MULTILINE)) != 1:
raise NativeInventoryUnsupported('展开报告必须具有唯一配置')
result, attributes, paths = {}, {}, []
inconsistent_refs = set()
symbol = None
def finish():
@@ -54,7 +55,7 @@ def _effective_properties(text):
if uid in result or paths != ['']:
raise NativeInventoryUnsupported('展开报告含重复或嵌套实例,需由 CAD 核验')
if ref != attributes.get('Ref Designator', ''):
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
inconsistent_refs.add(uid)
result[uid] = dict(attributes)
if len(result) > 100000:
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
@@ -76,10 +77,10 @@ def _effective_properties(text):
raise NativeInventoryUnsupported('展开报告的器件属性不唯一')
attributes[key] = value or ''
finish()
return result
return result, inconsistent_refs
def _parse_scope(text, requested, effective):
def _parse_scope(text, requested, effective, inconsistent_refs):
blocks, sheets, targets = set(), set(), set()
seen_uids = set()
block, symbol, attributes = '', None, {}
@@ -100,6 +101,11 @@ def _parse_scope(text, requested, effective):
if 'Value' in attributes and 'Value' not in values:
raise NativeInventoryUnsupported('展开报告缺少 Value 生效属性')
if values.get('Part Number', '').casefold() in requested:
# A library reference override on an unrelated part cannot affect
# the requested identities. Still validate every UID/config/path,
# and require CAD whenever the ambiguous component is requested.
if uid in inconsistent_refs:
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
ref, part = values.get('Ref Designator', ''), values['Part Number']
if not ref or 'Ref Designator' not in attributes:
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
@@ -146,9 +152,9 @@ def read_ascii_discovery(project, report_path, old_part_numbers, *, expected_roo
_require_report(text, 'HIER BLOCKS')
if effective_report_path is None:
raise NativeInventoryUnsupported('缺少实例生效属性报告,需由 CAD 核验')
effective = _effective_properties(read_native_text(effective_report_path))
effective, inconsistent_refs = _effective_properties(read_native_text(effective_report_path))
requested = {part.casefold() for part in old_part_numbers}
blocks, available, targets = _parse_scope(text, requested, effective)
blocks, available, targets = _parse_scope(text, requested, effective, inconsistent_refs)
if expected_root is not None and expected_root not in blocks:
raise NativeInventoryUnsupported('原生全工程报告与工程根块身份不一致')
components = tuple(ComponentTarget(Path(project), *row) for row in sorted(targets))
@@ -3,6 +3,7 @@ from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
import json
from pathlib import Path
import shutil
import subprocess
@@ -13,6 +14,30 @@ from .quick_run_protocol import ProjectDiscovery, _encode_vbs_field
from .operation_control import check_cancelled, report_progress
class ReleasedProjectReadError(TaskCenterError):
"""Project read failed, with positive evidence that its CAD session released."""
def _read_failure(root, message):
error_type = TaskCenterError
if (root / 'project-checkpoint.json').exists():
from .batch_runtime import project_checkpoint_released
if project_checkpoint_released(root):
error_type = ReleasedProjectReadError
return error_type('quick_run_sheet_discovery', message)
try:
record = json.loads((root / 'private-desktop.json').read_text(encoding='utf-8'))
cleanup = record['desktop_cleanup']
if (record['state'] == 'released' and record['clean'] is True
and record['cad_exit_code'] == 0 and record['quit_helper_exit_code'] == 0
and cleanup['desktop_handle_closed'] is True and cleanup['errors'] == []
and cleanup['desktop_confirmation']['released'] is True):
error_type = ReleasedProjectReadError
except (OSError, ValueError, KeyError, TypeError):
pass # Absence of an uncertainty marker is never release evidence.
return error_type('quick_run_sheet_discovery', message)
@dataclass(frozen=True, slots=True)
class DiscoveryRuntime:
"""Worker and evidence adapters shared with the execution entry point."""
@@ -83,18 +108,23 @@ def discover_project(
completed = runtime.process_runner(command, runtime.timeout)
except (OSError, subprocess.SubprocessError) as error:
_require_released_reader(root)
check_cancelled(stop_event)
raise TaskCenterError(
"quick_run_sheet_discovery",
failure = _read_failure(
root,
f"枚举工程图页时执行器未能安全结束:{project};{error}",
) from error
)
if isinstance(failure, ReleasedProjectReadError):
check_cancelled(stop_event)
raise failure from error
_require_released_reader(root)
check_cancelled(stop_event)
report_progress(on_progress, f'读取会话已结束,正在核对位号清单:{project.name}')
(root / "worker-output.txt").write_text(completed.stdout + "\n" + completed.stderr, encoding="utf-8")
if completed.returncode != 0:
detail = completed.stderr.strip() or completed.stdout.strip() or "sheet discovery failed"
raise TaskCenterError("quick_run_sheet_discovery", runtime.failure_message(detail))
failure = _read_failure(root, runtime.failure_message(detail))
if isinstance(failure, ReleasedProjectReadError):
check_cancelled(stop_event)
raise failure
check_cancelled(stop_event)
report_progress(on_progress, f'读取会话已结束,正在核对位号清单:{project.name}')
sheets = runtime.read_sheets(root / "sheets.tsv", project)
matches = runtime.read_matches(root / "inventory.tsv", project, sheets)
if collect_components:
@@ -107,6 +137,10 @@ def discover_project(
def _require_released_reader(root):
if (root / 'project-checkpoint.json').exists():
from .batch_runtime import project_checkpoint_released
if not project_checkpoint_released(root):
raise TaskCenterError('inventory_cleanup', f'工程读取关闭检查点无效,请保留诊断记录:{root}')
if (root / 'uncertain.flag').exists():
raise TaskCenterError('inventory_cleanup', f'采集会话未能正常收尾,请检查诊断记录后重试:{root}')
@@ -33,7 +33,7 @@ class ProjectSessionWorker:
The VBS worker owns CAD. This driver never attaches to COM or terminates CAD.
A separate monitor thread retains the existing idle watchdog and worker job.
"""
def __init__(self, project: Path, root: Path, *, private_desktop=False):
def __init__(self, project: Path, root: Path, *, private_desktop=False, runtime=None):
self.project, self.root = Path(project), Path(root).resolve()
self.token = uuid4().hex
self._done, self._abort = Event(), Event()
@@ -46,12 +46,13 @@ class ProjectSessionWorker:
self._timeout = 150
self.starts = 0
self.private_desktop = private_desktop
self.runtime = runtime
def _monitor(self, command, timeout):
try:
if self.private_desktop:
if self.private_desktop or self.runtime is not None:
from .isolated_cad import run_isolated_process
self._result = run_isolated_process(command, timeout, abort_event=self._abort)
self._result = run_isolated_process(command, timeout, abort_event=self._abort, **({'runtime': self.runtime} if self.runtime is not None else {}))
else:
self._result = run_monitored_process(command, timeout, abort_event=self._abort)
except Exception as error:
@@ -130,10 +131,11 @@ class ProjectSessionWorker:
class ProjectSessionScope:
"""One serial task can visit projects, but can hold only one worker at a time."""
def __init__(self, fallback, *, enabled, private_desktop=False):
def __init__(self, fallback, *, enabled, private_desktop=False, runtime=None):
self.fallback, self.enabled = fallback, enabled
self.worker = None
self.private_desktop = private_desktop
self.runtime = runtime
def __enter__(self):
return self
@@ -143,7 +145,7 @@ class ProjectSessionScope:
return self.fallback(command, timeout)
self.prepare(project)
if self.worker is None:
self.worker = ProjectSessionWorker(Path(project), Path(command[3]), private_desktop=self.private_desktop)
self.worker = ProjectSessionWorker(Path(project), Path(command[3]), private_desktop=self.private_desktop, runtime=self.runtime)
return self.worker.run_step(command, timeout)
def prepare(self, project):
@@ -7,6 +7,7 @@ from contextlib import contextmanager
from dataclasses import dataclass
from datetime import datetime, timezone
from functools import lru_cache
import inspect
import json
import os
from pathlib import Path
@@ -171,6 +172,7 @@ class QuickRunPreparer:
*,
copy_part_number_to_value: bool = False,
before_sheet_discovery: Callable[[Path], None] | None = None,
runtime=None,
) -> QuickRunResult:
"""Create a page-addressed queue after read-only sheet discovery."""
all_selected_projects = _selected_projects(projects)
@@ -201,7 +203,8 @@ class QuickRunPreparer:
before_sheet_discovery(project.path)
if self._sheet_discoverer is None:
discovery = self.discover_project(project.path, old_part_numbers,
**({'collect_components': True} if targeted else {}))
**({'collect_components': True} if targeted else {}),
**({'runtime': runtime} if runtime is not None else {}))
if targeted:
from .inventory import validate_target_inventory
validate_target_inventory(project.path, enabled_rules, discovery.components)
@@ -244,6 +247,7 @@ class QuickRunPreparer:
*,
copy_part_number_to_value: bool = False,
before_sheet_discovery: Callable[[Path], None] | None = None,
runtime=None,
) -> QuickRunBatchSession:
enabled_rules = tuple(rule for rule in rules if rule.enabled)
handoff = self.ensure_batch_handoff(
@@ -252,6 +256,7 @@ class QuickRunPreparer:
rules,
copy_part_number_to_value=copy_part_number_to_value,
before_sheet_discovery=before_sheet_discovery,
**({'runtime': runtime} if runtime is not None else {}),
)
source = self._required_script_source()
if source.resolve() != handoff.script_path.resolve():
@@ -301,13 +306,13 @@ class QuickRunPreparer:
)
@contextmanager
def project_session_scope(self, *, enabled=True):
def project_session_scope(self, *, enabled=True, runtime=None):
"""Retain only the built-in supervised worker; injected workers stay compatible."""
from .project_session import ProjectSessionScope
from .worker_monitor import WorkerMonitorError
try:
with ProjectSessionScope(self._process_runner,
enabled=enabled and self._process_runner is _run_process, private_desktop=True) as scope:
enabled=enabled and self._process_runner is _run_process, private_desktop=True, runtime=runtime) as scope:
yield scope
except (WorkerMonitorError, subprocess.TimeoutExpired) as error:
_write_uncertain_flag(self._data_root / 'quick-run/uncertain.flag', 'project_session_cleanup')
@@ -325,6 +330,7 @@ class QuickRunPreparer:
batch_mode: bool = False,
batch_session: QuickRunBatchSession | None = None,
project_session=None,
runtime=None,
) -> QuickRunStepResult:
"""Prepare or resume a queue, then execute one replacement rule end to end."""
uncertain_path = self._data_root / "quick-run" / "uncertain.flag"
@@ -332,7 +338,8 @@ class QuickRunPreparer:
raise TaskCenterError("quick_run_uncertain", "a previous quick-run worker timed out or returned an uncorrelated status; reconcile the project before retrying")
if self._process_runner is _run_process and not batch_mode:
from .single_step import prepare_private_single_step
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value)
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value,
**({'runtime': runtime} if runtime is not None else {}))
batch_mode = True
current_root = self._data_root / "quick-run"
if batch_session is None and self.evidence_path(task_id) == current_root and (current_root / "status.tsv").is_file():
@@ -367,7 +374,8 @@ class QuickRunPreparer:
if handoff is None:
if any(getattr(rule, "scope", "all") == "targets" for rule in enabled_rules):
handoff = self.ensure_batch_handoff(task_id, projects, rules,
copy_part_number_to_value=copy_part_number_to_value)
copy_part_number_to_value=copy_part_number_to_value,
**({'runtime': runtime} if runtime is not None else {}))
else:
handoff = self.prepare(task_id, projects, rules,
copy_part_number_to_value=copy_part_number_to_value)
@@ -407,6 +415,8 @@ class QuickRunPreparer:
try:
if project_session is not None:
completed = project_session.run_step(Path(selected_command[2]), command, IDLE_PROGRESS_TIMEOUT_SECONDS)
elif runtime is not None and self._process_runner is _run_process:
completed = _run_process(command, IDLE_PROGRESS_TIMEOUT_SECONDS, runtime=runtime)
else:
completed = self._process_runner(command, IDLE_PROGRESS_TIMEOUT_SECONDS)
except subprocess.TimeoutExpired as error:
@@ -681,10 +691,14 @@ class QuickRunPreparer:
self._discovery_cache.invalidate(project_path)
def discover_project(self, project_path: Path, old_part_numbers: tuple[str, ...], *,
collect_components: bool = False, stop_event=None, on_progress=None) -> ProjectDiscovery:
collect_components: bool = False, stop_event=None, on_progress=None, runtime=None) -> ProjectDiscovery:
from .operation_control import check_cancelled, control_options, report_progress
check_cancelled(stop_event)
controls = control_options(stop_event, on_progress)
if runtime is not None and any(
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
for parameter in inspect.signature(self._discover_project_sheets).parameters.values()):
controls['runtime'] = runtime
attempt = 0
def load():
nonlocal attempt
@@ -700,14 +714,15 @@ class QuickRunPreparer:
Path(project_path), old_part_numbers, collect_components, load,
**({'stop_event': stop_event} if stop_event is not None else {}))
def collect_inventory(self, project_path: Path, old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None) -> tuple:
def collect_inventory(self, project_path: Path, old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None, runtime=None) -> tuple:
"""Collect validated component targets, retaining reusable sheet evidence."""
if not old_part_numbers:
raise TaskCenterError("target_inventory", "at least one old Part Number is required")
requested = {part.casefold() for part in old_part_numbers}
from .operation_control import control_options
return tuple(component for component in self.discover_project(
project_path, old_part_numbers, collect_components=True, **control_options(stop_event, on_progress)).components
project_path, old_part_numbers, collect_components=True, **control_options(stop_event, on_progress),
**({'runtime': runtime} if runtime is not None else {})).components
if component.old_part_number.casefold() in requested)
def _discover_project_sheets(
@@ -718,6 +733,7 @@ class QuickRunPreparer:
collect_components: bool = False,
stop_event=None,
on_progress=None,
runtime=None,
) -> _ProjectDiscovery:
"""Enumerate sheets and requested source parts while the owned project is open."""
from .project_discovery import DiscoveryRuntime, discover_project
@@ -726,14 +742,16 @@ class QuickRunPreparer:
if collect_components and self._process_runner is _run_process:
from .native_inventory import NativeInventoryReader
native = NativeInventoryReader(self._data_root).read(
project_path, old_part_numbers, stop_event=stop_event, on_progress=on_progress)
project_path, old_part_numbers, stop_event=stop_event, on_progress=on_progress,
**({'runtime': runtime} if runtime is not None else {}))
if native is not None:
return native
process_runner = self._process_runner
if stop_event is not None and process_runner is _run_process:
if (stop_event is not None or runtime is not None) and process_runner is _run_process:
def process_runner(command, timeout):
return _run_process(command, timeout, abort_event=stop_event)
return _run_process(command, timeout, abort_event=stop_event,
**({'runtime': runtime} if runtime is not None else {}))
return discover_project(project_path, old_part_numbers,
data_root=self._data_root, script_source=self._required_script_source(),
@@ -1751,12 +1769,13 @@ def _cached_native_helper_launch_arguments(frozen: bool, executable: str) -> tup
return (resolved_executable, str(Path(__file__).with_name("application.py")))
def _run_process(command: tuple[str, ...], timeout: int, *, abort_event=None) -> subprocess.CompletedProcess[str]:
def _run_process(command: tuple[str, ...], timeout: int, *, abort_event=None, runtime=None) -> subprocess.CompletedProcess[str]:
from .isolated_cad import run_isolated_process
from .legacy_paths import validate_project_access
validate_project_access(Path(command[3]))
result = run_isolated_process(command, timeout,
**({'runtime': runtime} if runtime is not None else {}),
**({'abort_event': abort_event, 'cooperative_abort': True} if abort_event is not None else {}))
validate_project_access(Path(command[3]))
return result
@@ -105,7 +105,7 @@ def require_current_cell_binding(snapshot, binding, project, metadata):
return current
def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=False, confirmed=False):
def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=False, confirmed=False, runtime=None):
if not confirmed:
raise TaskCenterError('quick_run_recovery_confirmation', '核验恢复需要明确确认,不能作为自动重试执行。')
maintain_quick_run_storage(preparer._data_root)
@@ -227,7 +227,11 @@ def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=Fal
launch = (str(preparer._cscript_resolver()), '//nologo', script_argument, root_argument,
*_native_helper_launch_arguments(), job_id)
try:
result = preparer._process_runner(launch, IDLE_PROGRESS_TIMEOUT_SECONDS)
from .quick_run import _run_process
if runtime is not None and preparer._process_runner is _run_process:
result = _run_process(launch, IDLE_PROGRESS_TIMEOUT_SECONDS, runtime=runtime)
else:
result = preparer._process_runner(launch, IDLE_PROGRESS_TIMEOUT_SECONDS)
except (OSError, subprocess.SubprocessError) as error:
raise TaskCenterError('quick_run_recovery', '恢复执行未能确认结束,原保护锁仍在;不要重新替换或删除证据。') from error
(audit / 'worker-output.txt').write_text(result.stdout + '\n' + result.stderr, encoding='utf-8')
@@ -142,10 +142,11 @@ class TaskCenterService:
return self._operation("export_bom_template", task_id,
lambda: export_template(self._store, task_id, path))
def preview_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None):
def preview_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, runtime=None):
from .bom_batch import preview_batch
return self._idle_quick_run_operation('preview_bom_batch', task_id,
lambda: preview_batch(self, task_id, path, **control_options(stop_event, on_progress)))
lambda: preview_batch(self, task_id, path, **control_options(stop_event, on_progress),
**({'runtime': runtime} if runtime is not None else {})))
def save_bom_batch(self, preview, rows_by_task):
from .bom_batch import save_batch
@@ -388,10 +389,15 @@ class TaskCenterService:
return self._operation("list_projects", task_id, lambda: self._store.list_projects(task_id))
def collect_target_inventory(self, task_id: str, old_part_numbers: tuple[str, ...] | None = None,
*, stop_event=None, on_progress=None):
*, stop_event=None, on_progress=None, runtime=None):
if runtime is None:
return self._owned_cad_operation('collect_inventory', lambda owned:
self.collect_target_inventory(task_id, old_part_numbers,
stop_event=stop_event, on_progress=on_progress, runtime=owned), stop_event, task_id=task_id)
delegated = self._project_task_service(task_id)
if delegated is not self:
return delegated.collect_target_inventory(task_id, old_part_numbers, **control_options(stop_event, on_progress))
return delegated.collect_target_inventory(task_id, old_part_numbers, runtime=runtime,
**control_options(stop_event, on_progress))
def collect():
from .bom_batch import _assert_editable
_assert_editable(self, task_id)
@@ -409,7 +415,7 @@ class TaskCenterService:
project_path=str(project[0].path)) as metric:
report_progress(on_progress, f'正在采集工程:{project[0].display_name}')
rows = collect_target_inventory(self._quick_run_preparer, project[0].path, parts,
**control_options(stop_event, on_progress))
runtime=runtime, **control_options(stop_event, on_progress))
check_cancelled(stop_event)
metric['completed_units'] = len(rows)
with self._store.transaction():
@@ -555,11 +561,14 @@ class TaskCenterService:
return self._operation("expected_probe_path", task_id, expected_path)
def prepare_quick_run(self, task_id: str) -> QuickRunResult:
def prepare_quick_run(self, task_id: str, *, runtime=None) -> QuickRunResult:
"""Create the legacy VBS handoff without starting a task-center trial."""
if runtime is None:
return self._owned_cad_operation('prepare_quick_run', lambda owned:
self.prepare_quick_run(task_id, runtime=owned), task_id=task_id)
delegated = self._project_task_service(task_id)
if delegated is not self:
return delegated.prepare_quick_run(task_id)
return delegated.prepare_quick_run(task_id, runtime=runtime)
def prepare() -> QuickRunResult:
task = self._store.get_task(task_id)
preparer = (self._quick_run_preparer.ensure_batch_handoff
@@ -569,6 +578,7 @@ class TaskCenterService:
self._store.list_projects(task_id),
self._execution_rules(task_id),
copy_part_number_to_value=task.copy_part_number_to_value,
**({'runtime': runtime} if task.target_plan_mode else {}),
)
append_task_event(
self._logging,
@@ -624,11 +634,14 @@ class TaskCenterService:
with _quick_run_lease(self._logging.root):
return self._operation("set_value_policy", task_id, update)
def execute_quick_run(self, task_id: str, *, expected_item: tuple[str, str, str, str] | None = None) -> QuickRunStepResult:
def execute_quick_run(self, task_id: str, *, expected_item: tuple[str, str, str, str] | None = None, runtime=None) -> QuickRunStepResult:
"""Execute one replacement rule through restoration and verification."""
if runtime is None:
return self._owned_cad_operation('quick_run_step', lambda owned:
self.execute_quick_run(task_id, expected_item=expected_item, runtime=owned), task_id=task_id)
delegated = self._project_task_service(task_id)
if delegated is not self:
return delegated.execute_quick_run(task_id, expected_item=expected_item)
return delegated.execute_quick_run(task_id, expected_item=expected_item, runtime=runtime)
def execute() -> QuickRunStepResult:
task = self._store.get_task(task_id)
result = self._quick_run_preparer.execute_step(
@@ -637,6 +650,7 @@ class TaskCenterService:
self._execution_rules(task_id),
copy_part_number_to_value=task.copy_part_number_to_value,
**({"expected_item": expected_item} if expected_item is not None else {}),
runtime=runtime,
)
self._synchronize_quick_run_progress(task_id)
append_task_event(
@@ -663,12 +677,22 @@ class TaskCenterService:
stop_event: threading.Event,
continue_after_backup_failure: bool = False,
reuse_project_session: bool = True,
runtime=None,
):
if runtime is None:
from .batch_runtime import BatchCadRuntime
with BatchCadRuntime(self._logging.root, 'quick_run_queue', stop_event) as owned_runtime:
owned_runtime.bind_logs(self._logging, (task_id,))
result = self.execute_quick_run_queue(task_id, expected_item=expected_item,
stop_event=stop_event, continue_after_backup_failure=continue_after_backup_failure,
reuse_project_session=reuse_project_session, runtime=owned_runtime)
owned_runtime.result = result
return result
delegated = self._project_task_service(task_id)
if delegated is not self:
return delegated.execute_quick_run_queue(task_id, expected_item=expected_item,
stop_event=stop_event, continue_after_backup_failure=continue_after_backup_failure,
reuse_project_session=reuse_project_session)
reuse_project_session=reuse_project_session, runtime=runtime)
from .batch import execute_batch
from .task_queue import assert_task_queue_idle
@@ -690,6 +714,7 @@ class TaskCenterService:
stop_event=stop_event,
continue_after_backup_failure=continue_after_backup_failure,
reuse_project_session=reuse_project_session,
runtime=runtime,
),
)
metric['state'] = 'paused' if result.paused else 'failed' if result.progress.failed_items else 'completed'
@@ -700,17 +725,22 @@ class TaskCenterService:
if result.paused:
self._pause_quick_run_queue(task_id, 'queue paused at item boundary')
return result
except Exception:
except Exception as error:
if getattr(error, 'code', None) == 'batch_resources_pending':
raise
self._pause_quick_run_queue(task_id, 'queue stopped after execution error')
raise
finally:
self._batch_feedback = None
def prepare_failed_quick_run(self, task_id: str) -> QuickRunProgress:
def prepare_failed_quick_run(self, task_id: str, *, runtime=None) -> QuickRunProgress:
"""Reset only failed projects and discard the finished queue handoff."""
if runtime is None:
return self._owned_cad_operation('prepare_failed_quick_run', lambda owned:
self.prepare_failed_quick_run(task_id, runtime=owned), task_id=task_id)
delegated = self._project_task_service(task_id)
if delegated is not self:
return delegated.prepare_failed_quick_run(task_id)
return delegated.prepare_failed_quick_run(task_id, runtime=runtime)
def prepare_retry() -> QuickRunProgress:
task = self._store.get_task(task_id)
if self._quick_run_preparer.has_interrupted(task_id):
@@ -756,6 +786,7 @@ class TaskCenterService:
self._store.list_projects(task_id),
self._execution_rules(task_id),
copy_part_number_to_value=task.copy_part_number_to_value,
runtime=runtime,
)
return self._quick_run_preparer.progress(
task_id,
@@ -775,10 +806,31 @@ class TaskCenterService:
return self._idle_quick_run_operation('start_new_round', task_id,
lambda: start_new_round(self, task_id))
def inspect_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, on_phase=None):
def _owned_cad_operation(self, kind, action, stop_event=None, *, task_id=None):
from .batch_runtime import BatchCadRuntime
with BatchCadRuntime(self._logging.root, kind, stop_event) as runtime:
runtime.bind_logs(self._logging, (task_id,) if task_id is not None else ())
result = action(runtime)
runtime.result = result
return result
def pending_batch_resources(self):
from .batch_runtime import pending_resources
return pending_resources(self._logging.root)
def cleanup_batch_resources(self):
"""Retry owned-resource release without opening or changing any project."""
from .batch_runtime import cleanup_pending_resources
from .task_queue import assert_task_queue_idle
assert_task_queue_idle(self._logging.root)
with _quick_run_lease(self._logging.root):
return self._operation('cleanup_batch_resources', None,
lambda: cleanup_pending_resources(self._logging.root, logging_context=self._logging))
def inspect_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, on_phase=None, runtime=None):
from .bom_import_report import inspect_batch
return self._idle_quick_run_operation('inspect_bom_batch', task_id,
lambda: inspect_batch(self, task_id, path, on_phase=on_phase,
lambda: inspect_batch(self, task_id, path, on_phase=on_phase, runtime=runtime,
**control_options(stop_event, on_progress)))
def _pause_quick_run_queue(self, task_id: str, message: str) -> None:
@@ -835,10 +887,13 @@ class TaskCenterService:
return delegated.has_interrupted_quick_run(task_id)
return self._quick_run_preparer.has_interrupted(task_id)
def recover_quick_run(self, task_id: str, *, confirmed: bool = False) -> QuickRunStepResult:
def recover_quick_run(self, task_id: str, *, confirmed: bool = False, runtime=None) -> QuickRunStepResult:
if runtime is None:
return self._owned_cad_operation('recover_quick_run', lambda owned:
self.recover_quick_run(task_id, confirmed=confirmed, runtime=owned), task_id=task_id)
delegated = self._project_task_service(task_id)
if delegated is not self:
return delegated.recover_quick_run(task_id, confirmed=confirmed)
return delegated.recover_quick_run(task_id, confirmed=confirmed, runtime=runtime)
from .recovery import recover
def reconcile() -> QuickRunStepResult:
@@ -852,7 +907,7 @@ class TaskCenterService:
raise TaskCenterError("quick_run_recovery", "工程包含手动执行阶段或失败记录,不能混用自动队列恢复。")
result = recover(self._quick_run_preparer, task_id, projects,
self._execution_rules(task_id), copy_part_number_to_value=task.copy_part_number_to_value,
confirmed=confirmed)
confirmed=confirmed, runtime=runtime)
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED}:
self._store.set_task_status(task_id, TaskStatus.READY, summary="interrupted quick-run reconciled")
self._synchronize_quick_run_progress(task_id)
@@ -5,7 +5,19 @@ from .errors import TaskCenterError
from .schematic_backup import create_schematic_backup
def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_number_to_value):
def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_number_to_value, *, runtime=None):
from .quick_run import (_queue_value_mode, _value_mode, _read_status_details,
_queue_has_mutation_evidence)
root = preparer.data_root / 'quick-run'
command, status = root / 'command.tsv', root / 'status.tsv'
# Reject a frozen-policy conflict before even opening the project tree for
# backup. This preflight only reads evidence and cannot archive/rewrite it.
if (command.is_file() and status.is_file()
and _queue_value_mode(command) != _value_mode(copy_part_number_to_value)
and (_read_status_details(status) or any(root.glob('snapshot-*'))
or _queue_has_mutation_evidence(root))):
raise TaskCenterError('quick_run_pending',
'recorded Value policy differs; reconcile the old queue before creating a new task')
progress = preparer.progress(task_id, projects, rules,
copy_part_number_to_value=copy_part_number_to_value)
if progress.queue_complete or progress.next_project_path is None:
@@ -15,4 +27,5 @@ def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_nu
create_schematic_backup(preparer.data_root, progress.next_project_path,
preparer.data_root / 'quick-run-history' / ('batch-' + uuid4().hex))
preparer.ensure_batch_handoff(task_id, projects, rules,
copy_part_number_to_value=copy_part_number_to_value)
copy_part_number_to_value=copy_part_number_to_value,
**({'runtime': runtime} if runtime is not None else {}))
@@ -3,6 +3,7 @@
from contextlib import contextmanager
from dataclasses import asdict, dataclass, is_dataclass
import hashlib
import inspect
import json
import os
from pathlib import Path
@@ -119,8 +120,15 @@ def _save(path: Path, state: dict) -> None:
def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
queue_id: str | None = None) -> TaskQueueResult:
queue_id: str | None = None, runtime=None) -> TaskQueueResult:
"""Execute the frozen order; uncertain state stops before the next project."""
if runtime is None:
from .batch_runtime import BatchCadRuntime
with BatchCadRuntime(service._logging.root, 'task_queue', stop_event) as owned_runtime:
result = execute_task_queue(service, task_ids, stop_event=stop_event,
queue_id=queue_id, runtime=owned_runtime)
owned_runtime.result = result
return result
ids = tuple(task_ids)
if not ids or len(set(ids)) != len(ids):
raise TaskCenterError('task_queue_selection', '请选择不同的单工程任务。')
@@ -136,6 +144,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
project_paths.append(str(projects[0].path.resolve(strict=False)).casefold())
if len(set(project_paths)) != len(ids):
raise TaskCenterError('task_queue_selection', '不能在同一队列重复处理同一个工程。')
runtime.bind_logs(service._logging, ids)
hashes = {task_id: _plan_hash(child, task_id) for task_id, child in children.items()}
token = queue_id or uuid4().hex
if len(token) != 32 or any(char not in '0123456789abcdef' for char in token):
@@ -183,8 +192,13 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
expected = (str(progress.next_project_path), progress.next_old_part_number,
progress.next_new_part_number,
'part_label' if child.get_task(task_id).copy_part_number_to_value else 'preserve')
result = child.execute_quick_run_queue(task_id, expected_item=expected,
stop_event=stop_event)
execute = child.execute_quick_run_queue
parameters = inspect.signature(execute).parameters.values()
runtime_options = ({'runtime': runtime} if any(
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
for parameter in parameters) else {})
result = execute(task_id, expected_item=expected, stop_event=stop_event,
**runtime_options)
if child.has_interrupted_quick_run(task_id) or (
result.last_result is not None and result.last_result.state == 'batch_saved_unverified'
):
@@ -88,6 +88,7 @@ class TrialTaskCenterWindow(QMainWindow):
self.run_panel.stop_queue_button.clicked.connect(self._pause_queue)
self.run_panel.retry_button.clicked.connect(self._retry_failed)
self.run_panel.recover_button.clicked.connect(self._recover_quick_run)
self.run_panel.cleanup_resources_button.clicked.connect(self._cleanup_batch_resources)
self.run_panel.new_round_button.clicked.connect(self._start_new_round)
self.run_panel.history_button.clicked.connect(self._open_round_history)
self.run_panel.copy_value_check.toggled.connect(self._value_option_changed)
@@ -285,15 +286,18 @@ class TrialTaskCenterWindow(QMainWindow):
def _update_gates(self):
busy = bool(self._workers) or self._loading
resources_pending = bool(self.service.pending_batch_resources())
self.run_panel.cleanup_resources_button.setVisible(resources_pending)
self.run_panel.cleanup_resources_button.setEnabled(resources_pending and not busy)
state = execution_state(self._context, busy=busy, dirty=self.plan_panel.table.dirty)
group_unsaved = self._group_has_unsaved_plans() if state.editable or state.new_round else False
context = self._context
self.run_panel.quick_run_button.setText(state.execute_text)
self.run_panel.quick_run_button.setEnabled(state.execute)
self.run_panel.quick_run_button.setEnabled(state.execute and not resources_pending)
self.run_panel.retry_button.setVisible(bool(context and context.phase == 'retry'))
self.run_panel.retry_button.setEnabled(state.retry)
self.run_panel.retry_button.setEnabled(state.retry and not resources_pending)
self.run_panel.recover_button.setVisible(bool(context and context.interrupted))
self.run_panel.recover_button.setEnabled(state.recover)
self.run_panel.recover_button.setEnabled(state.recover and not resources_pending)
self.run_panel.new_round_button.setVisible(bool(context and context.task.status is TaskStatus.COMPLETED))
self.run_panel.new_round_button.setEnabled(state.new_round and not group_unsaved)
self.run_panel.history_button.setVisible(bool(context and
@@ -305,6 +309,9 @@ class TrialTaskCenterWindow(QMainWindow):
rule_editable = state.editable and not group_unsaved
self.mapping_table.set_rules_editable(rule_editable)
self.plan_panel.set_plan_editable(state.editable, busy=busy)
if resources_pending:
self.plan_panel.collect_button.setEnabled(False)
self.plan_panel.import_bom_button.setEnabled(False)
self.add_mapping_button.setEnabled(rule_editable)
selected = self.mapping_table.current_rule_id() is not None
for button in (self.edit_mapping_button, self.delete_mapping_button,
@@ -331,7 +338,7 @@ class TrialTaskCenterWindow(QMainWindow):
and t.status is not TaskStatus.DELETED}
group_dirty = self.plan_panel.has_unsaved_plan(group_ids)
group_active = any(t.status in {TaskStatus.RUNNING, TaskStatus.PAUSED} for t in self._tasks if t.task_id in group_ids)
self.task_list.queue_button.setEnabled(grouped and not busy and not group_dirty and not bool(context.interrupted))
self.task_list.queue_button.setEnabled(grouped and not busy and not group_dirty and not bool(context.interrupted) and not resources_pending)
for button in (self.task_list.up_button, self.task_list.down_button):
button.setEnabled(grouped and not busy and not group_active
and self.task_list.sort.currentData() == 'execution'
@@ -342,7 +349,7 @@ class TrialTaskCenterWindow(QMainWindow):
if self._loading and not self._message:
self.run_panel.operator_band.setText('正在读取任务,可继续选择其他任务。')
if not busy:
self.run_panel.execution_status.setText('任务读取失败' if read_failed else
self.run_panel.execution_status.setText('资源待清理' if resources_pending else '任务读取失败' if read_failed else
'结果待核验' if context and context.interrupted else '空闲')
def _group_has_unsaved_plans(self):
@@ -573,6 +580,17 @@ class TrialTaskCenterWindow(QMainWindow):
self._set_operator_message(f"任务「{name}」{'已暂停' if result.paused else '已结束'}:已保存 {p.saved_items},失败 {p.failed_items},未命中 {p.no_match_items}。",
'warning' if result.paused or p.failed_items else 'success')
def _cleanup_batch_resources(self):
if not self.run_panel.cleanup_resources_button.isEnabled():
return
def done(pending):
self._set_operator_message(
'后台资源仍待清理,请保留诊断记录后重试。' if pending else '后台资源已释放,可以继续。',
'warning' if pending else 'success')
self._update_gates()
self._run_async(self.service.cleanup_batch_resources, done,
self.run_panel.cleanup_resources_button)
def _retry_failed(self):
if self.run_panel.retry_button.isEnabled():
task_id = self._task_id
@@ -4,6 +4,8 @@ import json
import re
ERROR_TEXT = {
"batch_resources_pending": "已核验的替换结果已保留。后台资源尚未释放,请使用“重新清理资源”后再开始下一批。",
"inventory_session": "原生读取会话异常,安全重试未完成,已停止本批处理。请查看诊断记录并排除原因后重试。",
"task_required": "请先选择任务。",
"directory_required": "请先选择工程目录。",
"discovery_root_locked": "旧目录已有执行记录,不能在同一任务中更换目录;请新建任务。",
@@ -25,6 +25,8 @@ class RunPanel(QWidget):
self.stop_queue_button = QPushButton('完成当前项后暂停', objectName='stopQueueButton')
self.retry_button = QPushButton('重试失败项', objectName='retryTaskItems')
self.recover_button = QPushButton('核验并恢复', objectName='recoverQuickRunButton')
self.cleanup_resources_button = QPushButton('重新清理资源', objectName='cleanupBatchResources')
self.cleanup_resources_button.setToolTip('只释放本程序拥有的后台资源,不执行工程替换。')
self.new_round_button = QPushButton('开始新一轮', objectName='startNewRoundButton')
self.history_button = QPushButton('查看轮次证据', objectName='roundHistoryButton')
self.copy_value_check = QCheckBox('Value 策略:替换后将 Part Label 写入 Value(逐行自定义优先)', objectName='copyPartNumberToValue')
@@ -37,7 +39,7 @@ class RunPanel(QWidget):
self.log_dialog = None
self.toggle_log_button.toggled.connect(self._toggle_log)
buttons = QHBoxLayout()
for button in (self.quick_run_button, self.retry_button, self.recover_button,
for button in (self.quick_run_button, self.retry_button, self.recover_button, self.cleanup_resources_button,
self.new_round_button, self.history_button, self.stop_queue_button):
buttons.addWidget(button)
buttons.addStretch()
@@ -1 +1 @@
VERSION = "1.1.3"
VERSION = "1.1.4-beta.1"
+15 -1
View File
@@ -355,6 +355,7 @@ def test_batch_clears_worker_current_state_after_saved_completion(tmp_path):
def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp_path, monkeypatch):
import dx_productivity.trial_task_center.batch as batch_module
import dx_productivity.trial_task_center.quick_run as quick_run_module
project_count = 20
sheet_count = 5
@@ -396,6 +397,15 @@ def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp
initial = service.quick_run_progress(task.task_id)
expected = (str(initial.next_project_path), 'OLD-00', 'NEW-00', 'preserve')
reads = {'_read_commands': 0, '_read_status_details': 0}
def observe(name):
original = getattr(quick_run_module, name)
def counted(*args, **kwargs):
reads[name] += 1
return original(*args, **kwargs)
monkeypatch.setattr(quick_run_module, name, counted)
for name in reads:
observe(name)
tracemalloc.start()
started = time.perf_counter()
result = service.execute_quick_run_queue(task.task_id, expected_item=expected, stop_event=Event())
@@ -410,13 +420,17 @@ def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp
'sheets_per_project': sheet_count, 'rules_per_sheet': rule_count,
'steps': total, 'saved_steps': saved, 'elapsed_seconds': elapsed,
'peak_bytes': peak, 'steps_per_second': total / elapsed,
'full_queue_parses': reads,
}, indent=2), encoding='utf-8')
assert result.progress.queue_complete
assert result.progress.total_items == total
assert result.progress.saved_items == saved
assert result.progress.no_match_items == total - saved
assert len(calls) == total and len(set(calls)) == total
assert elapsed < 30
# Wall-clock budgets depend on the machine and tracemalloc overhead. Guard
# the repeated full-file work directly; keep timings as benchmark evidence.
assert reads['_read_commands'] <= 2
assert reads['_read_status_details'] <= 2
assert peak < 128 * 1024 * 1024
+9 -10
View File
@@ -360,16 +360,15 @@ def test_execute_quick_step_launches_automatic_runtime_and_reports_completion(tm
result = preparer.execute_step("task", (project,), (rule,))
assert calls == [
((
"C:\\Windows\\SysWOW64\\cscript.exe",
"//nologo",
str(result.script_path),
str(tmp_path / "quick-run"),
*_native_helper_launch_arguments(),
_command_field(result.command_path, "job_id"),
), 150)
]
assert len(calls) == 1
command, timeout = calls[0]
assert timeout == 150
assert command[:2] == ("C:\\Windows\\SysWOW64\\cscript.exe", "//nologo")
# Long evidence paths legitimately use 8.3 names or a managed alias.
assert Path(command[2]).resolve() == result.script_path.resolve()
assert Path(command[3]).resolve() == (tmp_path / 'quick-run').resolve()
assert command[4:] == (*_native_helper_launch_arguments(),
_command_field(result.command_path, 'job_id'))
assert result.state == "restored_ok"
assert result.message == "Replaced and verified 2 current-sheet matches."
+12 -2
View File
@@ -78,14 +78,19 @@ def test_result_for_other_queued_project_is_rejected(tmp_path):
assert (tmp_path / 'quick-run/uncertain.flag').exists()
def test_policy_change_cannot_reinterpret_partial_queue(tmp_path):
def test_policy_change_cannot_reinterpret_partial_queue(tmp_path, monkeypatch):
projects, rules = inputs()
preparer = QuickRunPreparer(tmp_path, script_source=SOURCE)
result = preparer.prepare('task', projects, rules)
job = result.command_path.read_text(encoding='utf-8').splitlines()[1].split('\t')[1]
result.status_path.write_text(f'job_id\tstate\tcode\tmessage\n{job}\tawaiting_native_replace\t\tcaptured\n', encoding='utf-8')
before = {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
from dx_productivity.trial_task_center import single_step
monkeypatch.setattr(single_step, 'create_schematic_backup',
lambda *a, **k: pytest.fail('frozen policy must be rejected before backup or CAD'))
with pytest.raises(TaskCenterError, match='quick_run_pending'):
preparer.execute_step('task', projects, rules, copy_part_number_to_value=True)
assert before == {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
def test_service_freezes_setting_after_queue_is_prepared(tmp_path):
@@ -189,7 +194,7 @@ def test_serial_three_project_two_rule_queue_pins_each_job(tmp_path):
assert preparer.progress('task', projects, rules).queue_complete
def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_path):
def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_path, monkeypatch):
projects, rules = inputs()
preparer = QuickRunPreparer(tmp_path, script_source=SOURCE)
result = preparer.prepare('task', projects, rules)
@@ -200,8 +205,13 @@ def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_pa
encoding='utf-8',
)
(result.command_path.parent / 'snapshot-old.tsv').write_text('retained evidence', encoding='utf-8')
before = {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
from dx_productivity.trial_task_center import single_step
monkeypatch.setattr(single_step, 'create_schematic_backup',
lambda *a, **k: pytest.fail('legacy snapshot must be checked before backup or CAD'))
with pytest.raises(TaskCenterError, match='quick_run_pending'):
preparer.execute_step('task', projects, rules)
assert before == {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
def test_frozen_launcher_reserves_helper_script_slot(monkeypatch, tmp_path):
+163
View File
@@ -0,0 +1,163 @@
import json
import os
import pytest
from dx_productivity.trial_task_center import batch_attempt as attempts
@pytest.fixture
def prepared(tmp_path, monkeypatch):
path = tmp_path / 'job.json'
record = attempts.prepare_attempt(path, session_id='session-a', generation=1,
task_id='task-a', job_id='job-a', project_path=tmp_path / 'board.prj')
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(path))
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
monkeypatch.setenv('DXPART_ATTEMPT_ID', record['attempt_id'])
monkeypatch.setenv('DXPART_TASK_ID', 'task-a')
monkeypatch.setenv('DXPART_JOB_ID', 'job-a')
return path, record
def test_only_current_prepared_attempt_can_retry(prepared):
path, expected = prepared
assert attempts.can_retry_attempt(path, expected)
attempts.mark_mutation_possible(job_id='job-a')
assert attempts.read_attempt(path)['state'] == 'mutation_possible'
assert not attempts.can_retry_attempt(path, expected)
@pytest.mark.parametrize('field,value', [('session_id', 'old'), ('generation', 2),
('attempt_id', '0' * 32), ('task_id', 'old'), ('job_id', 'old'),
('project_path', 'other.prj')])
def test_retry_rejects_stale_identity(prepared, field, value):
path, expected = prepared
assert not attempts.can_retry_attempt(path, dict(expected, **{field: value}))
@pytest.mark.parametrize('contents', [None, '{truncated', '{}', '[]'])
def test_unreadable_or_incomplete_proof_never_allows_retry(prepared, contents):
path, expected = prepared
if contents is None:
path.unlink()
else:
path.write_text(contents)
assert not attempts.can_retry_attempt(path, expected)
with pytest.raises((OSError, ValueError)):
attempts.mark_mutation_possible(job_id='job-a')
@pytest.mark.parametrize('name,value', [('DXPART_BATCH_SESSION_ID', 'old'),
('DXPART_BATCH_GENERATION', '2'), ('DXPART_ATTEMPT_ID', '0' * 32),
('DXPART_TASK_ID', 'old'), ('DXPART_JOB_ID', 'old')])
def test_marker_blocks_stale_process_identity(prepared, monkeypatch, name, value):
path, _ = prepared
monkeypatch.setenv(name, value)
with pytest.raises(ValueError):
attempts.mark_mutation_possible(job_id='job-a')
assert attempts.read_attempt(path)['state'] == 'prepared'
def test_marker_fsync_failure_blocks_caller_without_publishing_mutation(prepared, monkeypatch):
path, _ = prepared
def fail(_):
raise OSError('durability failed')
monkeypatch.setattr(os, 'fsync', fail)
with pytest.raises(OSError, match='durability failed'):
attempts.mark_mutation_possible(job_id='job-a')
assert json.loads(path.read_text())['state'] == 'prepared'
def test_legacy_no_environment_remains_supported(monkeypatch):
for name in tuple(os.environ):
if name.startswith('DXPART_') and ('ATTEMPT' in name or 'BATCH_' in name):
monkeypatch.delenv(name)
assert attempts.mark_mutation_possible() is None
def test_blank_batch_path_is_not_legacy(monkeypatch):
monkeypatch.setenv('DXPART_ATTEMPT_FILE', '')
with pytest.raises(ValueError):
attempts.mark_mutation_possible()
def test_preparation_is_durable_before_return(tmp_path, monkeypatch):
fsynced = []
real_fsync = os.fsync
def fsync(fd):
fsynced.append(True)
return real_fsync(fd)
monkeypatch.setattr(os, 'fsync', fsync)
path = tmp_path / 'attempt.json'
first = attempts.prepare_attempt(path, session_id='s', generation=1,
task_id='t', job_id='j', project_path=tmp_path / 'board.prj')
assert fsynced and attempts.read_attempt(path)['state'] == 'prepared'
second = attempts.prepare_attempt(tmp_path / 'second.json', session_id='s', generation=1,
task_id='t', job_id='j', project_path=tmp_path / 'board.prj')
assert first['attempt_id'] != second['attempt_id']
def test_reused_worker_marks_each_logical_job(tmp_path, monkeypatch):
directory = tmp_path / 'attempt'
for job in ('job-a', 'job-b'):
attempts.prepare_attempt(directory / f'{job}.json', session_id='session-a', generation=1,
task_id='task-a', job_id=job, project_path=tmp_path / 'board.prj', attempt_id='a' * 32)
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(directory))
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
monkeypatch.setenv('DXPART_ATTEMPT_ID', 'a' * 32)
for job in ('job-a', 'job-b'):
attempts.mark_mutation_possible(job_id=job, task_id='task-a', project_path=tmp_path / 'board.prj')
assert attempts.read_attempt(directory / f'{job}.json')['state'] == 'mutation_possible'
def test_native_guard_binds_snapshot_to_command_and_current_attempt(prepared, tmp_path):
from dx_productivity.trial_task_center.native_helper import mark_snapshot_mutation
path, _ = prepared
snapshot = tmp_path / 'snapshot-job-a.tsv'
snapshot.write_text(f'M\t{tmp_path / "board.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
assert mark_snapshot_mutation(snapshot)['state'] == 'mutation_possible'
assert attempts.read_attempt(path)['state'] == 'mutation_possible'
def test_native_guard_rejects_other_project_before_marking(prepared, tmp_path):
from dx_productivity.trial_task_center.native_helper import mark_snapshot_mutation
path, _ = prepared
snapshot = tmp_path / 'snapshot-job-a.tsv'
snapshot.write_text(f'M\t{tmp_path / "other.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
with pytest.raises(ValueError):
mark_snapshot_mutation(snapshot)
assert attempts.read_attempt(path)['state'] == 'prepared'
def test_marker_cli_fails_closed_on_stale_job(prepared):
from dx_productivity.trial_task_center.application import main
path, record = prepared
assert main(['--mark-batch-attempt', '--attempt-job', 'other', '--attempt-task', 'task-a',
'--attempt-project', record['project_path']]) == 1
assert attempts.read_attempt(path)['state'] == 'prepared'
def test_native_failure_does_not_reuse_previous_attempt_transaction(prepared, tmp_path, monkeypatch):
from dx_productivity.trial_task_center import native_helper as native
path, _ = prepared
snapshot = tmp_path / 'snapshot-job-a.tsv'
snapshot.write_text(f'M\t{tmp_path / "board.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
report_path = tmp_path / 'snapshot-job-a.tsv.native.json'
report_path.write_text(json.dumps({'stage': 'native_replace_started',
'transaction': {'old': 'unrelated'}, 'batch_attempt': {'attempt_id': 'old'}}))
def unavailable_cad(snapshot, report):
raise RuntimeError('CAD unavailable before mutation')
monkeypatch.setattr(native, 'run_native', unavailable_cad)
assert native.run(snapshot) == 1
report = json.loads(report_path.read_text())
assert report['stage'] == 'preflight' and 'transaction' not in report
assert report['batch_attempt']['job_id'] == 'job-a'
assert attempts.read_attempt(path)['state'] == 'prepared'
+60
View File
@@ -0,0 +1,60 @@
import json
from pathlib import Path
import sys
import pytest
from test_accuracy_vbs_guards import run_vbs
from dx_productivity.trial_task_center.batch_attempt import prepare_attempt
@pytest.mark.parametrize('fault', ['', 'missing', 'stale', 'wrong-project', 'missing-path'])
def test_vbs_guard_prevents_property_write_without_durable_current_proof(tmp_path, monkeypatch, fault):
root = Path(__file__).resolve().parents[2]
monkeypatch.setenv('PYTHONPATH', str(root / 'src'))
directory = tmp_path / 'attempt'
record_path = directory / 'job-a.json'
record = prepare_attempt(record_path, session_id='session-a', generation=1,
task_id='task-a', job_id='job-a', project_path=tmp_path / 'board.prj')
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(directory))
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
monkeypatch.setenv('DXPART_ATTEMPT_ID', '0' * 32 if fault == 'stale' else record['attempt_id'])
if fault == 'missing':
record_path.unlink()
if fault == 'missing-path':
monkeypatch.delenv('DXPART_ATTEMPT_FILE')
monkeypatch.delenv('DXPART_BATCH_GENERATION')
monkeypatch.delenv('DXPART_ATTEMPT_ID')
project = tmp_path / ('other.prj' if fault == 'wrong-project' else 'board.prj')
output = run_vbs(tmp_path, ['QPMarkBatchMutation'], '''
Dim Command, Attr, Failure
Set Command = CreateObject("Scripting.Dictionary")
Command.Add "job_id", "job-a": Command.Add "task_id", "task-a"
Command.Add "project_path", "''' + str(project) + '''"
Set Attr = New PropertySink
On Error Resume Next
GuardedWrite Command, Attr
Failure = Err.Number
On Error GoTo 0
WScript.Echo CStr(Failure <> 0)
WScript.Echo Attr.Value
''', '''
Dim QP_HelperExecutable, QP_HelperScript, QP_Fso
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
QP_HelperExecutable = "''' + sys.executable + '''"
QP_HelperScript = "''' + str(root / 'src/dx_productivity/trial_task_center/application.py') + '''"
Class PropertySink
Public Value
Private Sub Class_Initialize
Value = "unchanged"
End Sub
End Class
Sub GuardedWrite(ByRef Command, ByRef Attr)
QPMarkBatchMutation Command
Attr.Value = "changed"
End Sub
''')
assert output.splitlines() == (['False', 'changed'] if not fault else ['True', 'unchanged'])
if not fault:
assert json.loads(record_path.read_text())['state'] == 'mutation_possible'
+153
View File
@@ -0,0 +1,153 @@
"""Release diagnostics remain visible without changing cleanup decisions."""
import json
import pytest
from dx_productivity.trial_task_center import batch_resources, batch_runtime
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.ui.messages import _format_task_log_lines
from test_batch_runtime import Desktop
@pytest.fixture(autouse=True)
def reset_desktop():
Desktop.created = []
Desktop.fail_release = False
def events(root, task='one'):
return [json.loads(line) for line in (root / 'tasks' / task / 'events.jsonl')
.read_text(encoding='utf-8').splitlines()]
def runtime(root):
result = batch_runtime.BatchCadRuntime(root, 'task_queue', desktop_factory=Desktop)
result.bind_logs(configure_logging(root), ('one', 'two'))
return result
def test_batch_release_is_readable_and_visible_for_each_participating_task(tmp_path):
with runtime(tmp_path) as owner:
with owner.native_scope(tmp_path / 'native'):
pass
log = owner.path.with_name('resource-release.log').read_text(encoding='utf-8')
assert '开始释放' in log and '资源释放成功' in log
assert owner.session_id in log and owner.record['desktop'] in log
assert 'CAD 未启动' in log
for task in ('one', 'two'):
rows = events(tmp_path, task)
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_succeeded']
details = rows[-1]['details']
assert details['batch_id'] == owner.session_id and details['generation'] == 1
assert details['desktop_handle_closed'] is True and details['desktop_disappeared'] is True
assert details['elapsed_seconds'] >= 0 and details['record_path'] == str(owner.path)
assert '资源释放成功' in '\n'.join(_format_task_log_lines(tuple(json.dumps(row) for row in rows)))
assert '资源释放成功' in (tmp_path / 'logs/application.log').read_text(encoding='utf-8')
def test_failed_release_and_manual_retry_keep_both_results_without_replay(tmp_path):
Desktop.fail_release = True
owner = runtime(tmp_path)
with pytest.raises(batch_resources.BatchResourceCleanupError):
with owner:
with owner.native_scope(tmp_path / 'native'):
pass
owner.result = 'saved and verified'
failed = events(tmp_path)[-1]
assert failed['operation'] == 'resource_release_failed'
assert 'outside owner retains desktop' in failed['message']
assert failed['details']['desktop_disappeared'] is False
assert failed['details']['result'] == 'blocked'
Desktop.fail_release = False
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
rows = events(tmp_path)
assert [row['sequence'] for row in rows] == [1, 2, 3, 4]
assert rows[-1]['operation'] == 'resource_release_succeeded'
assert rows[-1]['details']['reason'] == 'manual_retry'
assert owner.result == 'saved and verified' and len(Desktop.created) == 1
def test_release_log_failure_does_not_skip_cleanup_or_falsify_its_result(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import batch_release_log
def denied(*_args):
raise OSError('release log locked')
monkeypatch.setattr(batch_release_log, '_append_line', denied)
with runtime(tmp_path) as owner:
with owner.native_scope(tmp_path / 'native'):
pass
assert Desktop.created[0].released
assert owner.record['state'] == 'released' and not batch_resources.pending_resources(tmp_path)
persisted = json.loads(owner.path.read_text(encoding='utf-8'))
assert any('release log locked' in entry for entry in persisted['release_log_errors'])
assert events(tmp_path)[-1]['operation'] == 'resource_release_succeeded'
def test_restart_recheck_logs_original_batch_and_tasks(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import isolated_desktop
Desktop.fail_release = True
owner = runtime(tmp_path)
with pytest.raises(batch_resources.BatchResourceCleanupError):
with owner:
with owner.native_scope(tmp_path / 'native'):
pass
batch_resources.LIVE_RESOURCES.pop(str(owner.path))
class Api:
def confirm_desktop_released(self, name):
assert name == owner.record['desktop']
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
assert batch_resources.cleanup_pending_resources(tmp_path, logging_context=configure_logging(tmp_path)) == ()
row = events(tmp_path)[-1]
assert row['operation'] == 'resource_release_succeeded'
assert row['details']['reason'] == 'restart_recheck'
assert row['details']['batch_id'] == owner.session_id
assert row['details']['desktop_disappeared'] is True
assert len(Desktop.created) == 1
def test_cached_batch_has_no_misleading_release_success(tmp_path):
with runtime(tmp_path) as owner:
pass
assert not owner.path.exists()
assert not owner.path.with_name('resource-release.log').exists()
assert not (tmp_path / 'tasks/one/events.jsonl').exists()
def test_cad_exit_and_cleanup_evidence_are_logged(tmp_path):
with runtime(tmp_path) as owner:
with owner.native_scope(tmp_path / 'native'):
pass
class Cad:
record = {'state': 'session_closed', 'clean': True,
'cad_exit_code': 0, 'quit_helper_exit_code': 0, 'shutdown_seconds': .01}
def __exit__(self, *_):
pass
owner.cad = Cad()
owner._save(cad_pid=123, cad_guid='owned')
event = events(tmp_path)[-1]
assert event['details']['cad_pid'] == 123
assert event['details']['cad_exit_code'] == event['details']['quit_helper_exit_code'] == 0
assert 'CAD 退出码=0' in event['message']
def test_release_logger_crash_cannot_hide_final_cleanup_failure(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import batch_release_log
def crash(*_args, **_options):
raise OSError('diagnostic unavailable')
monkeypatch.setattr(batch_release_log, 'write_release_event', crash)
Desktop.fail_release = True
owner = runtime(tmp_path)
with pytest.raises(batch_resources.BatchResourceCleanupError):
with owner:
with owner.native_scope(tmp_path / 'native'):
pass
assert batch_resources.pending_resources(tmp_path)
Desktop.fail_release = False
batch_resources.LIVE_RESOURCES.pop(str(owner.path))
from dx_productivity.trial_task_center import isolated_desktop
class Api:
def confirm_desktop_released(self, _name):
pass
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
record = json.loads(owner.path.read_text())
assert record['state'] == 'released' and record['release_log_errors']
+500
View File
@@ -0,0 +1,500 @@
"""Batch resources are lazy, shared, and independently blocked after failed release."""
import json
from pathlib import Path
from types import SimpleNamespace
import pytest
from dx_productivity.trial_task_center.errors import TaskCenterError
class Desktop:
created = []
fail_release = False
def __init__(self, name, **_options):
self.name = name
self.processes = []
self.released = False
self.cleanup_record = None
def __enter__(self):
self.created.append(self)
return self
def start_process(self, command, **_options):
process = SimpleNamespace(pid=100 + len(self.processes), closed=False,
cleanup_record={}, returncode=0)
def close():
process.closed = True
process.cleanup_record = {'pid': process.pid, 'process_handle_closed': True,
'job_handle_closed': True, 'errors': []}
process.close = close
process.poll = lambda: process.returncode
self.processes.append(process)
return process
def close(self):
for process in self.processes:
process.close()
self.cleanup_record = {'desktop_handle_closed': True,
'desktop_confirmation': {'released': not self.fail_release}, 'errors': []}
if self.fail_release:
raise OSError('outside owner retains desktop')
self.released = True
@pytest.fixture(autouse=True)
def reset_desktop():
Desktop.created = []
Desktop.fail_release = False
def test_cache_only_batch_does_not_create_desktop_or_cad(tmp_path):
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop):
pass
assert not Desktop.created
def test_native_operations_share_one_desktop_but_release_each_worker(tmp_path):
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
names = []
for name in ('first', 'second'):
with runtime.native_scope(tmp_path / name) as scope:
names.append(scope.name)
process = scope.start_process(('inventory.exe',))
assert not process.closed
assert process.closed and not Desktop.created[0].released
assert names[0] == names[1]
assert len(Desktop.created) == 1 and Desktop.created[0].released
def test_failed_final_release_blocks_next_batch_and_resource_cleanup_never_replays(tmp_path):
from dx_productivity.trial_task_center.batch_runtime import (
BatchCadRuntime, cleanup_pending_resources, guard_pending_resources, pending_resources)
Desktop.fail_release = True
with pytest.raises(TaskCenterError) as caught:
with BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop) as runtime:
with runtime.native_scope(tmp_path / 'operation'):
pass
runtime.result = 'verified completed task'
assert caught.value.code == 'batch_resources_pending'
assert caught.value.completed_result == 'verified completed task'
assert pending_resources(tmp_path)
with pytest.raises(TaskCenterError):
guard_pending_resources(tmp_path)
assert not list(tmp_path.rglob('uncertain.flag'))
Desktop.fail_release = False
assert cleanup_pending_resources(tmp_path) == ()
guard_pending_resources(tmp_path)
assert len(Desktop.created) == 1
def test_cleanup_failure_takes_priority_over_cancel(tmp_path):
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
Desktop.fail_release = True
with pytest.raises(TaskCenterError) as caught:
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
with runtime.native_scope(tmp_path / 'operation'):
pass
raise TaskCenterError('operation_cancelled', 'cancel requested')
assert caught.value.code == 'batch_resources_pending'
Desktop.fail_release = False
from dx_productivity.trial_task_center.batch_runtime import cleanup_pending_resources
cleanup_pending_resources(tmp_path)
class Cad:
created = []
def __init__(self, root, *, abort_event, desktop):
self.root, self.desktop, self.abort_event = root, desktop, abort_event
self.binding = SimpleNamespace(pid=500, guid='owned')
self.closed = False
def __enter__(self):
self.desktop.__enter__()
self.created.append(self)
return self
def checkpoint(self, evidence):
assert not self.closed
def start_worker(self, *args, **kwargs):
raise AssertionError('monitor is injected')
def __exit__(self, *_exc):
self.closed = True
self.desktop.close()
def command_for(tmp_path, name='one'):
root = tmp_path / 'data' / name
root.mkdir(parents=True)
project = tmp_path / ('project-' + name) / 'test.prj'
project.parent.mkdir()
project.write_text('project identity')
(root / 'command.tsv').write_text('task_id\tjob_id\tproject_path\told_part\tnew_part\n'
f'task\tjob\t{project}\told\tnew\n')
return ('cscript', '//nologo', 'worker.vbs', str(root), 'python', 'job'), project
def worker_runtime(tmp_path, monkeypatch, monitor):
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
from dx_productivity.trial_task_center import worker_monitor
monkeypatch.setattr(worker_monitor, 'run_monitored_process', monitor)
Cad.created = []
return BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=Desktop, cad_factory=Cad)
def test_two_projects_reuse_cad_and_produce_distinct_close_checkpoints(tmp_path, monkeypatch):
import subprocess
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
one, _ = command_for(tmp_path)
two, _ = command_for(tmp_path, 'two')
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
subprocess.CompletedProcess(command, 0, '', ''))
with runtime:
runtime.run_worker(one, 5)
assert project_checkpoint_released(Path(one[3]))
runtime.run_worker(two, 5)
assert project_checkpoint_released(Path(two[3]))
assert len(Cad.created) == len(Desktop.created) == 1
assert Cad.created[0].closed and Desktop.created[0].released
@pytest.mark.parametrize('second_fails', [False, True])
def test_prewrite_failure_releases_old_desktop_and_retries_only_once(tmp_path, monkeypatch, second_fails):
import subprocess
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
command, _ = command_for(tmp_path)
calls = []
def monitor(command, *args, **kwargs):
calls.append(len(calls))
if len(calls) == 2:
assert Desktop.created[0].released
return subprocess.CompletedProcess(command, 2 if len(calls) == 1 or second_fails else 0, '', 'startup')
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
with runtime:
if second_fails:
with pytest.raises(WorkerMonitorError):
runtime.run_worker(command, 5)
else:
runtime.run_worker(command, 5)
assert len(calls) == len(Desktop.created) == len(Cad.created) == 2
assert runtime.generation == 2
releases = json.loads(runtime.path.read_text())['generation_releases']
assert [release['generation'] for release in releases] == [1, 2]
assert [release['desktop'] for release in releases] == [desktop.name for desktop in Desktop.created]
assert all(release['cad_pid'] == 500 and release['cad_guid'] == 'owned'
and release['cleanup']['desktop_confirmation']['released'] is True
and release['released_at'] > 0 for release in releases)
def test_possible_mutation_prevents_automatic_replay(tmp_path, monkeypatch):
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
command, _ = command_for(tmp_path)
calls = []
def monitor(*args, **kwargs):
calls.append(1)
path = next((Path(command[3]) / 'batch-attempts').rglob('job.json'))
record = json.loads(path.read_text())
record['state'] = 'mutation_possible'
path.write_text(json.dumps(record))
raise WorkerMonitorError('lost connection after possible write')
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
with runtime, pytest.raises(WorkerMonitorError):
runtime.run_worker(command, 5)
assert len(calls) == len(Desktop.created) == 1
def test_previous_checkpoint_invalidated_before_new_worker(tmp_path, monkeypatch):
import subprocess
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
command, _ = command_for(tmp_path)
calls = []
def monitor(*args, **kwargs):
assert not project_checkpoint_released(Path(command[3]))
calls.append(1)
return subprocess.CompletedProcess(command, 0, '', '')
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
with runtime:
runtime.run_worker(command, 5)
assert project_checkpoint_released(Path(command[3]))
def fail_checkpoint(*args):
raise WorkerMonitorError('cannot certify empty project')
runtime.cad.checkpoint = fail_checkpoint
# No retry after evidence has been marked as possibly modified.
path = next((Path(command[3]) / 'batch-attempts').rglob('job.json'))
record = json.loads(path.read_text())
record['state'] = 'mutation_possible'
path.write_text(json.dumps(record))
with pytest.raises(WorkerMonitorError):
runtime.run_worker(command, 5)
assert not project_checkpoint_released(Path(command[3]))
def test_attempt_publication_failure_never_launches_worker(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import batch_attempt
command, _ = command_for(tmp_path)
runtime = worker_runtime(tmp_path, monkeypatch, lambda *a, **k: pytest.fail('must not launch'))
def fail(*args, **kwargs):
raise OSError('disk unavailable')
monkeypatch.setattr(batch_attempt, 'prepare_attempt', fail)
with runtime, pytest.raises(OSError):
runtime.run_worker(command, 5)
assert not Cad.created
def test_boundary_pause_does_not_abort_current_cad_startup(tmp_path, monkeypatch):
import subprocess
from threading import Event
command, _ = command_for(tmp_path)
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
subprocess.CompletedProcess(command, 0, '', ''))
pause, worker_abort = Event(), Event()
runtime.stop_event = pause
original_enter = Cad.__enter__
def enter(cad):
pause.set() # User requests pause after the current task has begun.
assert cad.abort_event is worker_abort
assert not cad.abort_event.is_set()
return original_enter(cad)
monkeypatch.setattr(Cad, '__enter__', enter)
with runtime:
runtime.run_worker(command, 5, abort_event=worker_abort)
assert not list(tmp_path.rglob('uncertain.flag'))
def test_cleanup_still_runs_when_resource_journal_write_fails(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import batch_runtime
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
with pytest.raises(TaskCenterError) as caught:
with runtime:
with runtime.native_scope(tmp_path / 'read'):
pass
real_write = batch_runtime.write_record
def failing_write(path, record):
if record['state'] == 'closing':
raise OSError('journal unavailable')
return real_write(path, record)
monkeypatch.setattr(batch_runtime, 'write_record', failing_write)
assert Desktop.created[0].released
assert caught.value.code == 'batch_resources_pending'
monkeypatch.setattr(batch_runtime, 'write_record', real_write)
runtime.close()
def test_real_startup_failure_with_released_lease_can_retry(tmp_path, monkeypatch):
import io
import subprocess
from dx_productivity.trial_task_center import isolated_cad, worker_monitor
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
command, _ = command_for(tmp_path)
starts = []
def executable():
starts.append(1)
if len(starts) == 1:
raise worker_monitor.WorkerMonitorError('registry temporarily unavailable')
return 'cad.exe'
class BootstrapDesktop(Desktop):
def start_process(self, command, **options):
process = super().start_process(command, **options)
process.stderr = io.BytesIO()
process.wait = lambda timeout: process.returncode
if command[0] == 'cad.exe':
self.cad = process
process.returncode = None
else:
action, evidence, pid = command
environment = options['env']
binding = {'desktop': self.name, 'pid': pid, 'guid': 'owned',
'token': environment['DXPART_SESSION_TOKEN']}
if action == 'probe':
Path(evidence).write_text(json.dumps(binding))
elif action == 'checkpoint':
Path(evidence).write_text(json.dumps({'binding': binding, 'project_path': '', 'project_closed': True}))
elif action == 'quit':
self.cad.returncode = 0
return process
monkeypatch.setattr(isolated_cad, '_designer_executable', executable)
monkeypatch.setattr(isolated_cad, '_helper_command', lambda *args: args)
monkeypatch.setattr(worker_monitor, 'run_monitored_process', lambda *a, **k:
subprocess.CompletedProcess(command, 0, '', ''))
with BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=BootstrapDesktop) as runtime:
runtime.run_worker(command, 5)
assert len(starts) == len(Desktop.created) == 2
assert all(desktop.released for desktop in Desktop.created)
def test_native_rebuild_requires_strict_source_and_worker_proof(tmp_path):
import hashlib
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
command, project = command_for(tmp_path)
database = project.parent / 'database.dat'
database.write_bytes(b'database')
evidence = Path(command[3]) / 'native'
evidence.mkdir()
proof = {'version': 1, 'native_read_only': True, 'workers_closed': True,
'project_path': str(project.resolve()), 'evidence_root': str(evidence.resolve()),
'source_hashes': {str(path.resolve()): hashlib.sha256(path.read_bytes()).hexdigest()
for path in (project, database)}}
path = evidence / 'native-retry-proof.json'
path.write_text(json.dumps(proof))
with BatchCadRuntime(tmp_path / 'data', 'bom_import', desktop_factory=Desktop) as runtime:
with runtime.native_scope(evidence) as scope:
pass
proof['binding'] = scope.identity
path.write_text(json.dumps(proof))
database.write_bytes(b'changed externally')
assert not runtime.recover_native_failure(project, evidence, 'timeout')
database.write_bytes(b'database')
assert runtime.recover_native_failure(project, evidence, 'timeout')
assert Desktop.created[0].released
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
with pytest.raises(WorkerMonitorError, match='stale'):
scope.start_process(('late-old-worker.exe',))
assert scope.identity['generation'] == 1 and runtime.generation == 2
scope.close() # A late close cannot grant the old operation a new identity.
assert str(evidence.resolve()) not in runtime.record['native_operations']
assert not runtime.recover_native_failure(project, evidence, 'another timeout')
assert len(Desktop.created) == 2
def test_release_publication_failure_can_be_reconciled_without_replaying(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import batch_runtime
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
real_write = batch_runtime.write_record
def failing_write(path, record):
if record['state'] == 'released':
raise OSError('cannot publish release')
return real_write(path, record)
with pytest.raises(TaskCenterError):
with runtime:
with runtime.native_scope(tmp_path / 'read'):
pass
monkeypatch.setattr(batch_runtime, 'write_record', failing_write)
assert Desktop.created[0].released and batch_runtime.pending_resources(tmp_path)
monkeypatch.setattr(batch_runtime, 'write_record', real_write)
assert batch_runtime.cleanup_pending_resources(tmp_path) == ()
assert len(Desktop.created) == 1
releases = json.loads(runtime.path.read_text())['generation_releases']
assert len(releases) == 1 and releases[0]['generation'] == 1
@pytest.mark.parametrize('record', [
{'version': 1, 'session_id': 'old', 'state': 'blocked'},
{'version': 1, 'session_id': 'old', 'state': 'released'},
[],
])
def test_unknown_resource_evidence_never_authorizes_cleanup(tmp_path, record):
from dx_productivity.trial_task_center.batch_resources import cleanup_pending_resources, pending_resources
path = tmp_path / 'batch-resources' / 'old' / 'session.json'
path.parent.mkdir(parents=True)
path.write_text(json.dumps(record))
assert pending_resources(tmp_path)
assert cleanup_pending_resources(tmp_path)
assert json.loads(path.read_text()) == record
def test_stale_generation_and_unknown_checkpoint_schema_are_rejected(tmp_path, monkeypatch):
import subprocess
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
command, _ = command_for(tmp_path)
runtime = worker_runtime(tmp_path, monkeypatch, lambda *a, **k:
subprocess.CompletedProcess(command, 0, '', ''))
with runtime:
runtime.run_worker(command, 5)
path = Path(command[3]) / 'project-checkpoint.json'
checkpoint = json.loads(path.read_text())
assert project_checkpoint_released(Path(command[3]))
for changes in ({'version': 999}, {'generation': 999}, {'attempt_id': 'old'}, {'cad_guid': 'foreign'}):
path.write_text(json.dumps(dict(checkpoint, **changes)))
assert not project_checkpoint_released(Path(command[3]))
def test_restart_reconciles_only_confirmed_disappearance_without_replacement(tmp_path, monkeypatch):
from dx_productivity.trial_task_center import batch_resources, isolated_desktop
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
Desktop.fail_release = True
runtime = BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
with pytest.raises(TaskCenterError):
with runtime:
with runtime.native_scope(tmp_path / 'read'):
pass
# Simulate restarting the UI: there are no retained owned handles to act on.
batch_resources.LIVE_RESOURCES.pop(str(runtime.path))
present = [True]
class Api:
def confirm_desktop_released(self, name):
assert name == runtime.record['desktop']
if present[0]:
raise OSError('desktop still held by external thread')
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
assert batch_resources.cleanup_pending_resources(tmp_path)
present[0] = False
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
assert len(Desktop.created) == 1
@pytest.mark.parametrize('conflict', [
'bootstrap_binding', 'bootstrap_json', 'bootstrap_schema', 'helper_identity',
'checkpoint_binding', 'checkpoint_json', 'worker_identity',
])
def test_identity_conflict_never_rebuilds_or_launches_another_worker(tmp_path, monkeypatch, conflict):
import io
import subprocess
from dx_productivity.trial_task_center import isolated_cad, worker_monitor
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
command, _ = command_for(tmp_path)
workers = []
class IdentityDesktop(Desktop):
def start_process(self, launch, **options):
process = super().start_process(launch, **options)
process.stderr = io.BytesIO(b'CAD COM GUID does not match its owned session')
process.wait = lambda timeout: process.returncode
if launch[0] == 'cad.exe':
self.cad = process
process.returncode = None
return process
action, evidence, pid = launch
binding = {'desktop': self.name, 'pid': pid, 'guid': 'owned',
'token': options['env']['DXPART_SESSION_TOKEN']}
if action == 'probe':
if conflict == 'helper_identity':
process.returncode = 4 # Private helper identity-conflict exit.
return process
if conflict == 'bootstrap_binding':
binding['pid'] = pid + 1
if conflict == 'bootstrap_schema':
binding.pop('token')
Path(evidence).write_text('{' if conflict == 'bootstrap_json' else json.dumps(binding))
elif action == 'checkpoint':
closing = Path(evidence).name.startswith('cad-closed-')
if closing and conflict == 'checkpoint_binding':
binding['guid'] = 'foreign'
record = {'binding': binding, 'project_path': '', 'project_closed': True}
Path(evidence).write_text('{' if closing and conflict == 'checkpoint_json' else json.dumps(record))
elif action == 'quit':
self.cad.returncode = 0
return process
monkeypatch.setattr(isolated_cad, '_designer_executable', lambda: 'cad.exe')
monkeypatch.setattr(isolated_cad, '_helper_command', lambda *args: args)
def monitor(*args, **kwargs):
workers.append(1)
if conflict == 'worker_identity':
return subprocess.CompletedProcess(command, 2, '', 'private CAD GUID mismatch; no project was changed')
return subprocess.CompletedProcess(command, 0, '', '')
monkeypatch.setattr(worker_monitor, 'run_monitored_process', monitor)
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=IdentityDesktop)
with runtime, pytest.raises(worker_monitor.WorkerMonitorError):
runtime.run_worker(command, 5)
assert len(Desktop.created) == 1
assert len(workers) == (1 if conflict.startswith('checkpoint') or conflict == 'worker_identity' else 0)
assert runtime.generation == 1 and not runtime.record['retries']
assert Desktop.created[0].released
@@ -0,0 +1,122 @@
"""Batch entry points own resource lifetime independently of task outcomes."""
import sys
from types import SimpleNamespace, ModuleType
from threading import Event
import pytest
from dx_productivity.trial_task_center import task_queue
from dx_productivity.trial_task_center.errors import TaskCenterError
from test_task_queue import Service
from test_bom_batch import batch, records_for
@pytest.fixture
def runtime_module(monkeypatch):
module = ModuleType('dx_productivity.trial_task_center.batch_runtime')
scopes = []
class CleanupError(TaskCenterError):
def __init__(self):
super().__init__('batch_resources_pending', 'resource cleanup blocked')
class Runtime:
result = None
fail_cleanup = False
def __init__(self, root, kind, stop_event=None):
self.root, self.kind = root, kind
self.closed = False
scopes.append(self)
def __enter__(self):
return self
def bind_logs(self, context, task_ids):
self.log_task_ids = tuple(task_ids)
def __exit__(self, *args):
self.closed = True
if self.fail_cleanup:
error = CleanupError()
error.completed_result = self.result
raise error
module.BatchCadRuntime = Runtime
module.BatchResourceCleanupError = CleanupError
monkeypatch.setitem(sys.modules, module.__name__, module)
return Runtime, scopes
def test_task_queue_owns_one_runtime_and_passes_it_to_every_child(tmp_path, runtime_module):
Runtime, scopes = runtime_module
service = Service(tmp_path)
original = service.for_project_task
seen = []
def child(task_id):
result = original(task_id)
execute = result.execute_quick_run_queue
def run(task_id, *, runtime, **options):
assert not runtime.closed
seen.append(runtime)
return execute(task_id, **options)
result.execute_quick_run_queue = run
return result
service.for_project_task = child
result = task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event())
assert [entry.state for entry in result.entries] == ['completed', 'completed']
assert len(scopes) == 1 and scopes[0].closed
assert seen == [scopes[0], scopes[0]]
assert scopes[0].log_task_ids == ('A', 'B')
def test_final_cleanup_failure_preserves_completed_tasks_and_resume_does_not_repeat(tmp_path, runtime_module):
Runtime, scopes = runtime_module
Runtime.fail_cleanup = True
service = Service(tmp_path)
with pytest.raises(TaskCenterError) as caught:
task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event())
assert caught.value.code == 'batch_resources_pending'
result = caught.value.completed_result
assert [entry.state for entry in result.entries] == ['completed', 'completed']
Runtime.fail_cleanup = False
task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event(), queue_id=result.queue_id)
assert service.calls == [('start', 'A'), ('end', 'A'), ('start', 'B'), ('end', 'B')]
@pytest.mark.parametrize('current', [True, False])
def test_reader_requires_current_batch_project_checkpoint(tmp_path, runtime_module, current):
from dx_productivity.trial_task_center.project_discovery import (
_read_failure, _require_released_reader, ReleasedProjectReadError)
module = sys.modules['dx_productivity.trial_task_center.batch_runtime']
module.project_checkpoint_released = lambda root: current
(tmp_path / 'project-checkpoint.json').write_text('{}')
if current:
_require_released_reader(tmp_path)
assert isinstance(_read_failure(tmp_path, 'read failed'), ReleasedProjectReadError)
else:
with pytest.raises(TaskCenterError) as caught:
_require_released_reader(tmp_path)
assert caught.value.code == 'inventory_cleanup'
assert not isinstance(_read_failure(tmp_path, 'read failed'), ReleasedProjectReadError)
@pytest.mark.parametrize('read_error', [False, True])
def test_bom_cleanup_failure_preserves_completed_or_partial_report(batch, tmp_path, runtime_module, read_error):
from dx_productivity.trial_task_center.bom_import_report import inspect_batch
from dx_productivity.trial_task_center.bom_plan import write_bom_template
Runtime, scopes = runtime_module
Runtime.fail_cleanup = True
service, store, tasks, targets, calls = batch
if read_error:
def fail(*args, **kwargs):
raise TaskCenterError('inventory_cleanup', 'reader did not close')
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = fail
with pytest.raises(TaskCenterError) as caught:
inspect_batch(service, tasks[0].task_id,
write_bom_template(tmp_path / 'cleanup.xlsx', records_for(tasks, targets)))
assert caught.value.code == 'batch_resources_pending'
report = caught.value.bom_report
assert report is not None and report.total_rows == 2
assert report.failed_rows == (1 if read_error else 0)
assert report.passed_rows == (0 if read_error else 2)
assert report.preview is None
original = caught.value.__context__.bom_report if read_error else scopes[0].result
assert report is not original
assert (original.preview is not None) == (not read_error)
if caught.value.completed_result is not None:
assert caught.value.completed_result.preview is None
assert len(scopes) == 1 and scopes[0].closed
+58
View File
@@ -0,0 +1,58 @@
"""An external Windows reference blocks a batch without replaying its results."""
import json
import os
from pathlib import Path
import subprocess
import sys
import pytest
from dx_productivity.trial_task_center.batch_runtime import (
BatchCadRuntime, BatchResourceCleanupError, cleanup_pending_resources, guard_pending_resources)
from dx_productivity.trial_task_center.cad_identity import current_desktop_name
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api, build_startup_info
from dx_productivity.trial_task_center.logging_setup import configure_logging
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop object lifetime')
def test_external_reference_preserves_completed_result_until_resource_only_cleanup(tmp_path):
before = current_desktop_name()
runtime = BatchCadRuntime(tmp_path, 'execution')
runtime.bind_logs(configure_logging(tmp_path), ('windows-resource-test',))
external = None
try:
with pytest.raises(BatchResourceCleanupError) as caught:
with runtime:
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
# The test owns this independent actor, outside runtime's
# process/job list. The runtime must leave it alive.
external = _Win32Api().create_process((sys.executable, '-u', '-c',
'import win32service,time,sys; '
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000); '
'd.SetThreadDesktop();print("ready",flush=True);time.sleep(30)', scope.name),
build_startup_info(scope.name).lpDesktop, None, None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
assert external.stdout.readline() == b'ready\r\n'
runtime.result = 'already saved and independently verified'
assert caught.value.completed_result == runtime.result
assert external.poll() is None
assert json.loads(runtime.path.read_text())['state'] == 'blocked'
log = runtime.path.with_name('resource-release.log').read_text(encoding='utf-8')
assert '资源释放失败' in log and '桌面消失:未确认' in log
with pytest.raises(BatchResourceCleanupError):
guard_pending_resources(tmp_path)
assert not list(Path(tmp_path).rglob('uncertain.flag'))
assert current_desktop_name() == before
finally:
if external is not None:
external.close() # Only the test closes its own independent actor.
assert cleanup_pending_resources(tmp_path) == ()
assert json.loads(runtime.path.read_text())['state'] == 'released'
assert runtime.result == 'already saved and independently verified'
assert runtime.generation == 1
rows = [json.loads(line) for line in (tmp_path / 'tasks/windows-resource-test/events.jsonl')
.read_text(encoding='utf-8').splitlines()]
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_failed',
'resource_release_started', 'resource_release_succeeded']
assert rows[-1]['details']['desktop_disappeared'] is True
guard_pending_resources(tmp_path)
+94
View File
@@ -35,6 +35,26 @@ def test_reports_all_matching_errors_and_success_is_read_only(batch, tmp_path):
assert all(not store.list_target_plan(task.task_id) for task in tasks)
def test_bom_batch_passes_one_runtime_to_all_project_reads(batch, tmp_path, monkeypatch):
from types import SimpleNamespace
service, store, tasks, targets, calls = batch
bound = []
runtime = SimpleNamespace(bind_logs=lambda context, task_ids: bound.append((context, task_ids)))
seen = []
for task in tasks:
preparer = service.for_project_task(task.task_id)._quick_run_preparer
original = preparer.collect_inventory
def read(project, parts, *, runtime=None, _read=original, **options):
seen.append(runtime)
return _read(project, parts, **options)
monkeypatch.setattr(preparer, 'collect_inventory', read)
report = inspect_batch(service, tasks[0].task_id,
write_bom_template(tmp_path / 'shared.xlsx', records_for(tasks, targets)), runtime=runtime)
assert report.passed_projects == 2
assert seen == [runtime, runtime]
assert bound == [(service._logging, tuple(task.task_id for task in tasks))]
def test_duplicate_marks_both_rows_and_cross_row_chain_rejected(batch, tmp_path):
service, store, tasks, targets, calls = batch
record = records_for(tasks, targets)[0]
@@ -111,6 +131,45 @@ def test_cancel_and_native_cleanup_errors_propagate_without_next_project(batch,
assert not calls
def test_released_project_read_failure_reports_rows_and_checks_next_project(batch, tmp_path):
from dx_productivity.trial_task_center.project_discovery import ReleasedProjectReadError
service, store, tasks, targets, calls = batch
def failed_reader(*args, **kwargs):
raise ReleasedProjectReadError('quick_run_sheet_discovery', 'project read failed', error_id='READ-1')
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = failed_reader
report = inspect_batch(service, tasks[0].task_id,
write_bom_template(tmp_path / 'read-failure.xlsx', records_for(tasks, targets)))
assert (report.failed_rows, report.passed_rows, report.unchecked_rows) == (1, 1, 0)
assert 'READ-1' in report.rows[0].reason and 'project read failed' in report.rows[0].reason
assert report.preview is None
assert all(not store.list_target_plan(task.task_id) for task in tasks)
@pytest.mark.parametrize('cancel_requested', [False, True])
@pytest.mark.parametrize('failure_first', [False, True])
@pytest.mark.parametrize('code', ['inventory_cleanup', 'quick_run_sheet_discovery'])
def test_unsafe_reader_failure_carries_full_report_without_opening_later_project(batch, tmp_path, cancel_requested, failure_first, code):
service, store, tasks, targets, calls = batch
records = list(records_for(tasks, targets))
records = ([records[0], dict(records[0], refdes='R2'), records[1]] if failure_first else
[records[1], records[0], dict(records[0], refdes='R2')])
stop = Event()
def failed_reader(*args, **kwargs):
if cancel_requested:
stop.set()
raise TaskCenterError(code, 'desktop retained', error_id='CLEANUP-1')
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = failed_reader
with pytest.raises(TaskCenterError) as caught:
inspect_batch(service, tasks[0].task_id,
write_bom_template(tmp_path / 'unsafe.xlsx', records), stop_event=stop)
assert caught.value.code == code and caught.value.error_id == 'CLEANUP-1'
report = caught.value.bom_report
assert (report.passed_rows, report.failed_rows, report.unchecked_rows) == ((0, 2, 1) if failure_first else (1, 2, 0))
assert len(calls) == (0 if failure_first else 1)
assert report.preview is None and 'CLEANUP-1' in report.rows[1].reason
assert all(not store.list_target_plan(task.task_id) for task in tasks)
def test_cancellation_after_reader_prevents_opening_next_project(batch, tmp_path):
service, store, tasks, targets, calls = batch
@@ -202,3 +261,38 @@ def test_ui_report_is_persistent_and_failure_does_not_stage(batch, tmp_path, qtb
assert '失败 1 行' in panel.report_text
assert all(not store.list_target_plan(task.task_id) for task in tasks)
window.close()
def test_ui_cleanup_failure_keeps_details_and_retry_clears_previous_result(batch, tmp_path, qtbot, monkeypatch):
from PySide6.QtWidgets import QFileDialog
from dx_productivity.trial_task_center.ui.main_window import TrialTaskCenterWindow
from dx_productivity.trial_task_center.bom_import_state import BomImportPhase
service, store, tasks, targets, calls = batch
preparer = service.for_project_task(tasks[0].task_id)._quick_run_preparer
original_reader = preparer.collect_inventory
def failed_reader(*args, **kwargs):
raise TaskCenterError('inventory_cleanup', 'desktop retained', error_id='UI-CLEANUP-1')
preparer.collect_inventory = failed_reader
path = write_bom_template(tmp_path / 'cleanup-ui.xlsx', records_for(tasks, targets))
window = TrialTaskCenterWindow(service)
qtbot.addWidget(window)
window.open_task(tasks[0].task_id)
qtbot.waitUntil(lambda: not window._loading and not window._context_query.busy)
monkeypatch.setattr(QFileDialog, 'getOpenFileName', lambda *args: (str(path), 'Excel (*.xlsx)'))
panel = window.plan_panel
panel.import_bom_button.click()
qtbot.waitUntil(lambda: not window._workers)
assert panel.bom_import_state.phase is BomImportPhase.FAILED
assert panel.last_bom_report.failed_rows == 1 and panel.last_bom_report.unchecked_rows == 1
assert 'UI-CLEANUP-1' in panel.report_text and panel.report_toggle.isVisibleTo(panel)
assert not panel._bom_batches and not calls
qtbot.waitUntil(lambda: not window._loading and not window._context_query.busy)
preparer.collect_inventory = original_reader
panel.import_bom_button.click()
assert panel.last_bom_report is None and not panel.report_toggle.isVisibleTo(panel)
qtbot.waitUntil(lambda: not window._workers)
assert panel.bom_import_state.phase is BomImportPhase.PREVIEW_READY
assert panel.last_bom_report.passed_rows == 2 and 'UI-CLEANUP-1' not in panel.report_text
assert all(not store.list_target_plan(task.task_id) for task in tasks)
panel.discard_all()
window.close()
+25
View File
@@ -64,3 +64,28 @@ def test_ready_preview_is_separate_from_saved_or_discarded_plan():
machine.release_preview(saved=True)
assert machine.state.phase is Phase.PREVIEW_READY
assert machine.state.report.preview is None and '已保存' in machine.state.notice
def test_failed_import_preserves_attached_row_report_and_retry_resets_it():
from dx_productivity.trial_task_center.bom_import_report import BomInspectionError
machine = BomImportMachine()
attempt = machine.start()
report = BomImportReport((), (), errors=('resources uncertain',))
error = BomInspectionError(TaskCenterError('inventory_cleanup', 'resources uncertain'), report)
machine.finish(attempt, error=error)
assert machine.state.phase is Phase.FAILED and machine.state.report is report
machine.start()
assert machine.state.report is None and machine.state.phase is Phase.READING
def test_resource_cleanup_failure_preserves_bom_row_report_after_cancellation():
from dx_productivity.trial_task_center.batch_runtime import BatchResourceCleanupError
machine = BomImportMachine()
attempt = machine.start()
machine.cancel(attempt)
report = BomImportReport((), (), errors=('project read failed',))
error = BatchResourceCleanupError('desktop cleanup failed')
error.bom_report = report
machine.finish(attempt, error=error)
assert machine.state.phase is Phase.FAILED and machine.state.report is report
assert 'desktop cleanup failed' in machine.state.message
+74
View File
@@ -0,0 +1,74 @@
"""Desktop diagnosis must find threads without windows and leave their owner alone."""
import json
import os
from pathlib import Path
import subprocess
import sys
from uuid import uuid4
import pytest
from dx_productivity.trial_task_center.isolated_desktop import PrivateDesktop, _Win32Api, build_startup_info
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop diagnosis')
def test_diagnostic_finds_windowless_desktop_thread_without_changing_owner(tmp_path):
from dx_productivity.trial_task_center.cad_identity import current_desktop_name
before = current_desktop_name()
desktop = PrivateDesktop('dx-diagnostic-' + uuid4().hex, enabled=True)
desktop.__enter__()
# Explicitly connect a thread without creating any windows.
child = _Win32Api().create_process((sys.executable, '-u', '-c',
'import win32service,time,sys; '
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000);d.SetThreadDesktop(); '
'print("ready",flush=True);time.sleep(30)', desktop.name), build_startup_info(desktop.name).lpDesktop,
None, None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
assert child.stdout.readline() == b'ready\r\n'
record = tmp_path / 'private-desktop.json'
record.write_text(json.dumps({'desktop': desktop.name}))
original = record.read_bytes()
output = tmp_path / 'diagnostic.json'
script = Path(__file__).resolve().parents[2] / 'tools/diagnose_private_desktop.py'
result = subprocess.run((sys.executable, str(script), '--record', str(record), '--output', str(output)),
capture_output=True, timeout=20)
assert result.returncode == 0, result.stderr.decode(errors='replace')
report = json.loads(output.read_text(encoding='utf-8'))
assert any(row['pid'] == child.pid for row in report['thread_scan']['threads'])
assert report['windows'] == [] and report['desktop_still_listed']
assert child.poll() is None and current_desktop_name() == before
assert record.read_bytes() == original
finally:
child.close()
desktop.close()
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop diagnosis')
def test_no_thread_owner_is_not_mistaken_for_released_desktop(tmp_path):
with PrivateDesktop('dx-diagnostic-' + uuid4().hex, enabled=True) as desktop:
output = tmp_path / 'diagnostic.json'
script = Path(__file__).resolve().parents[2] / 'tools/diagnose_private_desktop.py'
result = subprocess.run((sys.executable, str(script), '--desktop', desktop.name, '--output', str(output)),
capture_output=True, timeout=20)
assert result.returncode == 0, result.stderr.decode(errors='replace')
report = json.loads(output.read_text(encoding='utf-8'))
assert report['thread_scan']['threads'] == []
assert report['desktop_still_listed'] and report['windows'] == []
# A leaked query handle would make this context's strict close fail.
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop diagnosis')
@pytest.mark.parametrize('hardlink', [False, True])
def test_output_cannot_overwrite_input_record_even_through_hardlink(tmp_path, hardlink):
record = tmp_path / 'private-desktop.json'
original = json.dumps({'desktop': 'dx-no-such-' + uuid4().hex}).encode()
record.write_bytes(original)
output = record
if hardlink:
output = tmp_path / 'diagnostic.json'
os.link(record, output)
script = Path(__file__).resolve().parents[2] / 'tools/diagnose_private_desktop.py'
result = subprocess.run((sys.executable, str(script), '--record', str(record), '--output', str(output)),
capture_output=True, timeout=20)
assert result.returncode != 0
assert record.read_bytes() == original
+8
View File
@@ -1,4 +1,5 @@
from dataclasses import replace
import json
from pathlib import Path
from subprocess import CompletedProcess
from threading import Event
@@ -93,6 +94,13 @@ def test_default_discovery_forwards_abort_to_private_runner_and_waits_cleanup(tm
assert cooperative_abort
stop.set()
cleanup.append('released')
# Match the private runner's durable proof after its owned cleanup.
(Path(command[3]) / 'private-desktop.json').write_text(json.dumps({
'state': 'released', 'clean': True, 'cad_exit_code': 0,
'quit_helper_exit_code': 0,
'desktop_cleanup': {'desktop_handle_closed': True, 'errors': [],
'desktop_confirmation': {'released': True}},
}), encoding='utf-8')
return CompletedProcess(command, 2, '', 'cancelled')
monkeypatch.setattr(isolated_cad, 'run_isolated_process', run)
monkeypatch.setattr(quick_run, '_native_helper_launch_arguments', lambda: ())
+13
View File
@@ -314,6 +314,19 @@ def test_released_startup_failure_returns_retryable_pre_mutation_code(monkeypatc
assert not (tmp_path / 'uncertain.flag').exists()
def test_released_startup_identity_conflict_is_not_a_retryable_exit(monkeypatch, tmp_path):
class FailedIdentity:
def __init__(self, *args, **kwargs):
pass
def __enter__(self):
raise runtime.PrivateCadIdentityError('bootstrap returned a foreign identity')
def __exit__(self, *args):
pytest.fail('failed identity must not enter execution')
monkeypatch.setattr(runtime, 'IsolatedCadSession', FailedIdentity)
with pytest.raises(runtime.CadIdentityConflict):
runtime.run_isolated_process(('cscript.exe', '//nologo', 'helper.vbs', str(tmp_path)), 1)
@pytest.mark.parametrize('failure', ['quit', 'cad_exit', 'unexpected_exit'])
def test_uncertain_cleanup_is_durable_before_forced_owned_release(monkeypatch, tmp_path, failure):
events, cad = fake_desktop(monkeypatch, tmp_path,
+13 -5
View File
@@ -33,12 +33,17 @@ def test_release_waits_for_a_temporary_external_reference_without_stopping_its_o
@pytest.mark.skipif(os.name != 'nt', reason='native Windows desktop lifetime')
def test_release_does_not_report_success_while_another_handle_keeps_desktop_alive():
@pytest.mark.parametrize('with_window', [False, True])
def test_release_does_not_report_success_while_another_handle_keeps_desktop_alive(with_window):
desktop = PrivateDesktop('dx-test-' + uuid4().hex, enabled=True)
desktop.__enter__()
retained = subprocess.Popen((sys.executable, '-u', '-c',
'import win32service,sys,time; h=win32service.OpenDesktop(sys.argv[1],0,False,1); print("ready",flush=True); time.sleep(30)',
desktop.name), stdout=subprocess.PIPE)
script = ('import win32service,sys,time; h=win32service.OpenDesktop(sys.argv[1],0,False,1); ' +
('import win32gui; w=win32gui.CreateWindowEx(0,"STATIC","residue",0,0,0,1,1,0,0,0,None); '
if with_window else '') + 'print("ready",flush=True); time.sleep(30)')
# subprocess.STARTUPINFO ignores lpDesktop. Use the native launch boundary
# so the diagnostic window really belongs to the requested desktop.
retained = _Win32Api().create_process((sys.executable, '-u', '-c', script, desktop.name),
build_startup_info(desktop.name).lpDesktop, None, None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
assert retained.stdout.readline() == b'ready\r\n'
with pytest.raises(TaskCenterError, match='still exists'):
@@ -46,11 +51,14 @@ def test_release_does_not_report_success_while_another_handle_keeps_desktop_aliv
record = desktop.cleanup_record
assert record['desktop_handle_closed']
assert not record['desktop_confirmation']['released']
windows = record['desktop_confirmation']['remaining_windows']
assert {row['pid'] for row in windows} == ({retained.pid} if with_window else set())
assert retained.poll() is None # Diagnostics must not terminate a foreign owner.
assert record['desktop_confirmation']['elapsed_seconds'] >= 2
finally:
retained.terminate()
retained.wait(timeout=5)
retained.stdout.close()
retained.close()
desktop.close()
assert desktop.cleanup_record['desktop_confirmation']['released']
+12
View File
@@ -138,3 +138,15 @@ def test_component_cache_cannot_substitute_for_complete_page_discovery(tmp_path)
def test_inventory_requires_effective_values_even_when_no_requested_base_part_matches(tmp_path):
with pytest.raises(protocol.NativeInventoryUnsupported):
protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('NEW',))
def test_unrequested_reference_override_does_not_block_certified_target_inventory(tmp_path):
flat = effective_report(tmp_path)
flat.write_text(flat.read_text().replace('Prop Ref Designator R2', 'Prop Ref Designator LIB_R2'))
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('NEW',),
effective_report_path=flat)
assert [(row.uid, row.refdes, row.old_part_number) for row in result.components] == [('$27I100', 'R1', 'NEW')]
# The ambiguous component is still refused when its part is requested.
with pytest.raises(protocol.NativeInventoryUnsupported, match='位号与属性不一致'):
protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('OLD',),
effective_report_path=flat)
+136
View File
@@ -73,6 +73,142 @@ def test_reader_uses_a_disposable_database_and_returns_certified_native_identity
assert messages
def test_native_reader_borrows_batch_desktop_and_closes_only_native_scope(tmp_path):
project, reader, stop, calls, released = reader_fixture(tmp_path)
borrowed = reader.desktop_factory('shared')
evidence = []
class Runtime:
def native_scope(self, root):
evidence.append(root)
return borrowed
def forbidden(*args, **kwargs):
raise AssertionError('batch native read must not create another desktop')
reader.desktop_factory = forbidden
result = reader.read(project, ('OLD',), runtime=Runtime())
assert [row.refdes for row in result.components] == ['R1']
assert len(calls) == 2 and released == [True]
assert len(evidence) == 1 and evidence[0].parent.name == 'quick-run-history'
@pytest.mark.parametrize('fail_close', [False, True])
def test_native_jobs_release_descendants_before_next_command_or_fallback(tmp_path, fail_close):
project, reader, stop, calls, released = reader_fixture(tmp_path)
factory = reader.desktop_factory
events = []
def desktop(*args, **options):
value = factory(*args, **options)
original = value.start_process
def start(command, **kwargs):
events.append('start')
process = original(command, **kwargs)
def close():
events.append('close')
if fail_close:
raise OSError('native descendant remained alive')
process.close = close
return process
value.start_process = start
return value
reader.desktop_factory = desktop
if fail_close:
with pytest.raises(TaskCenterError) as caught:
reader.read(project, ('OLD',))
assert caught.value.code == 'inventory_cleanup'
assert events == ['start', 'close']
assert list((tmp_path / 'evidence').rglob('uncertain.flag'))
else:
assert reader.read(project, ('OLD',)).components
assert events == ['start', 'close', 'start', 'close']
@pytest.mark.parametrize('failure', ['launch', 'timeout', 'crash', 'input_error', 'parser'])
def test_only_transient_native_process_failure_requests_one_proven_rebuild(tmp_path, failure):
project, reader, stop, calls, released = reader_fixture(tmp_path, corrupt=failure == 'parser')
reader.timeout = 0
scopes, recoveries = [], []
class Runtime:
def native_scope(self, evidence):
value = reader.desktop_factory('shared')
value.cleanup_record = {'workers_closed': True}
value.identity = {'session_id': 'current-batch', 'generation': 1,
'attempt_id': f'native-{len(scopes) + 1}'}
original = value.start_process
first = not scopes
scopes.append(evidence)
def start(command, **options):
if first and failure == 'launch':
raise OSError('temporary process launch failed')
process = original(command, **options)
if first and failure == 'timeout':
process.poll = lambda: None
elif first and failure in {'crash', 'input_error'}:
process.poll = lambda: 0xC0000005 if failure == 'crash' else 2
process.close = lambda: None
return process
value.start_process = start
return value
def recover_native_failure(self, path, evidence, reason):
proof = json.loads((evidence / 'native-retry-proof.json').read_text())
assert path == project and reason
assert proof['native_read_only'] and proof['workers_closed']
assert proof['binding'] == {'session_id': 'current-batch', 'generation': 1,
'attempt_id': 'native-1'}
assert proof['project_path'] == str(project.resolve())
assert set(proof['source_hashes']) == {str(project), str(project.parent / 'database/icdb.dat')}
assert released == [True]
recoveries.append(evidence)
return True
result = reader.read(project, ('OLD',), runtime=Runtime())
if failure in {'launch', 'timeout', 'crash'}:
assert [row.refdes for row in result.components] == ['R1']
assert len(scopes) == 2 and len(recoveries) == 1
else:
assert result is None and len(scopes) == 1 and not recoveries
@pytest.mark.parametrize('failure', ['timeout', 'crash'])
@pytest.mark.parametrize('recovery_allowed', [True, False])
def test_native_session_failure_stops_batch_after_retry_or_denied_budget_without_cad_fallback(
tmp_path, monkeypatch, failure, recovery_allowed):
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer
project, reader, stop, calls, released = reader_fixture(tmp_path)
reader.timeout = 0
scopes, recoveries, cad_calls = [], [], []
class BatchRuntime:
def native_scope(self, evidence):
scope = reader.desktop_factory('shared')
scope.cleanup_record = {'workers_closed': True}
scope.identity = {'session_id': 'batch', 'generation': 1,
'attempt_id': f'native-{len(scopes) + 1}'}
original = scope.start_process
def start(command, **options):
process = original(command, **options)
process.poll = lambda: None if failure == 'timeout' else 0xC0000005
process.close = lambda: None
return process
scope.start_process = start
scopes.append(evidence)
return scope
def recover_native_failure(self, project, evidence, reason):
recoveries.append(evidence)
return recovery_allowed
def run_worker(self, *args, **kwargs):
cad_calls.append(args)
raise AssertionError('native session failure must stop before CAD fallback')
monkeypatch.setattr(runtime, 'NativeInventoryReader', lambda *args, **kwargs: reader)
preparer = QuickRunPreparer(tmp_path / 'cad-data')
with pytest.raises(TaskCenterError) as caught:
preparer._discover_project_sheets(project, ('OLD',), collect_components=True, runtime=BatchRuntime())
assert caught.value.code == 'inventory_session'
assert len(scopes) == len(released) == (2 if recovery_allowed else 1)
assert len(recoveries) == 1 and not cad_calls
record = json.loads((scopes[-1] / 'native-inventory.json').read_text(encoding='utf-8'))
assert record['state'] == 'session_failed' and record['source_hashes_unchanged']
assert record['cleanup']['workers_closed']
proof = json.loads((scopes[-1] / 'native-retry-proof.json').read_text(encoding='utf-8'))
assert proof['workers_closed'] and proof['binding']['attempt_id'] == f'native-{len(scopes)}'
@pytest.mark.parametrize('failure', ['malformed', 'timeout'])
def test_safe_native_failure_allows_com_fallback_only_after_desktop_release(tmp_path, failure):
project, reader, stop, calls, released = reader_fixture(
+43
View File
@@ -1,6 +1,8 @@
"""Discovery orchestration uses injected workers and validates their evidence."""
from pathlib import Path
import json
import subprocess
from threading import Event
import pytest
@@ -73,6 +75,20 @@ def test_discovery_without_components_never_requests_or_requires_component_file(
assert result.components == ()
def test_discovery_reuses_explicit_runtime_on_built_in_process_path(discovery, monkeypatch):
project, roots, prepare = discovery
shared = object()
seen = []
def process(command, timeout, *, runtime=None, **options):
seen.append(runtime)
write_evidence(roots[-1], project)
return subprocess.CompletedProcess(command, 0, '', '')
monkeypatch.setattr(quick_run, '_run_process', process)
result = prepare(process).discover_project(project, ('OLD',), runtime=shared)
assert result.sheets == (('MAIN', '1'),)
assert seen == [shared]
@pytest.mark.parametrize('failure', [OSError('launch failed'), subprocess.TimeoutExpired('worker', 150)])
def test_discovery_worker_failures_keep_stable_error_code(discovery, failure):
project, roots, prepare = discovery
@@ -103,3 +119,30 @@ def test_discovery_rejects_foreign_sheet_evidence(discovery):
return subprocess.CompletedProcess(command, 0, '', '')
with pytest.raises(TaskCenterError, match='another project'):
prepare(worker)._discover_project_sheets(project, ('OLD',))
@pytest.mark.parametrize('release', ['confirmed', 'missing', 'uncertain', 'malformed'])
@pytest.mark.parametrize('cancelled', [False, True])
def test_only_positive_release_evidence_certifies_failed_reader(discovery, release, cancelled):
from dx_productivity.trial_task_center.project_discovery import ReleasedProjectReadError
project, roots, prepare = discovery
stop = Event()
def worker(command, timeout):
if release != 'missing':
record = {'state': 'released' if release == 'confirmed' else 'uncertain',
'clean': True, 'cad_exit_code': 0, 'quit_helper_exit_code': 0,
'desktop_cleanup': {'desktop_handle_closed': True, 'errors': [],
'desktop_confirmation': {'released': True}}}
(roots[-1] / 'private-desktop.json').write_text(
'{' if release == 'malformed' else json.dumps(record))
# No uncertain.flag exists, including when its write would have failed.
if cancelled:
stop.set()
raise PermissionError('uncertain.flag cannot be written')
with pytest.raises(TaskCenterError) as caught:
prepare(worker)._discover_project_sheets(project, ('OLD',), stop_event=stop)
if cancelled and release == 'confirmed':
assert caught.value.code == 'operation_cancelled'
else:
assert caught.value.code == 'quick_run_sheet_discovery'
assert isinstance(caught.value, ReleasedProjectReadError) == (release == 'confirmed')
+16
View File
@@ -51,6 +51,22 @@ def test_same_project_steps_retain_one_supervised_worker_and_close(tmp_path):
assert not (tmp_path / 'project-session.request').exists()
def test_batch_runtime_retains_same_project_worker_without_owning_cad(tmp_path):
from dx_productivity.trial_task_center.worker_monitor import run_monitored_process
calls = []
class Runtime:
def run_worker(self, command, timeout, **options):
calls.append(command)
return run_monitored_process(command, timeout, **options)
runtime = Runtime()
command = toy_worker(tmp_path)
with ProjectSessionScope(None, enabled=True, private_desktop=True, runtime=runtime) as scope:
assert scope.run_step(tmp_path / 'board.prj', command, 3).returncode == 0
assert scope.run_step(tmp_path / 'board.prj', (*command[:-1], 'job2'), 3).returncode == 0
assert len(calls) == 1
assert (tmp_path / 'closed.txt').exists()
def test_session_progress_keeps_monitor_identity_and_resets_each_step():
watch = ProgressWatch('token', 'first', 10, 0)
first = b'DX_PROGRESS\ttoken\tfirst\t1\topen_project\t0\t1\t1\tfirst\n'
+30
View File
@@ -1,4 +1,5 @@
from dataclasses import replace
import json
from threading import Event
import pytest
@@ -73,3 +74,32 @@ def test_cancelled_service_inventory_does_not_save_any_targets(group):
service.collect_target_inventory(task_id, ('OLD',), stop_event=stop)
assert error.value.code == 'operation_cancelled'
assert not store.list_target_plan(task_id) and not store.get_task(task_id).target_plan_mode
@pytest.mark.parametrize('entry', ['collect', 'prepare', 'execute', 'retry', 'recover', 'preview', 'inspect'])
def test_pending_resources_block_every_standalone_cad_service_before_target_or_worker_changes(group, tmp_path, entry):
service, store, tasks, targets = group
task_id = tasks[0].task_id
calls = []
def collect(*args, **kwargs):
calls.append('read')
return (targets[task_id],)
service.for_project_task(task_id)._quick_run_preparer.collect_inventory = collect
record = service._logging.root / 'batch-resources/previous/session.json'
record.parent.mkdir(parents=True)
record.write_text(json.dumps({'version': 1, 'session_id': 'previous', 'state': 'blocked'}))
path = write_bom_template(tmp_path / 'scope.xlsx', ({'project_path': str(targets[task_id].project_path),
'old_part_number': 'OLD', 'new_part_number': 'NEW', 'refdes': 'R1'},))
invoke = {
'collect': lambda: service.collect_target_inventory(task_id, ('OLD',)),
'prepare': lambda: service.prepare_quick_run(task_id),
'execute': lambda: service.execute_quick_run(task_id),
'retry': lambda: service.prepare_failed_quick_run(task_id),
'recover': lambda: service.recover_quick_run(task_id, confirmed=True),
'preview': lambda: service.preview_bom_batch(task_id, path),
'inspect': lambda: service.inspect_bom_batch(task_id, path),
}[entry]
with pytest.raises(TaskCenterError) as caught:
invoke()
assert caught.value.code == 'batch_resources_pending'
assert not calls and not store.list_target_plan(task_id)
+22
View File
@@ -10,6 +10,28 @@ from test_ui_batch_retry import open_project
from test_ui_remediation import window
def test_pending_batch_resources_block_new_work_and_cleanup_does_not_replace(window, qtbot, tmp_path, monkeypatch):
open_project(window, qtbot, tmp_path)
pending = [{'state': 'cleanup_pending'}]
monkeypatch.setattr(window.service, 'pending_batch_resources', lambda: tuple(pending))
calls = []
def cleanup():
calls.append('cleanup')
pending.clear()
return ()
monkeypatch.setattr(window.service, 'cleanup_batch_resources', cleanup)
window._update_gates()
assert not window.run_panel.quick_run_button.isEnabled()
assert not window.plan_panel.collect_button.isEnabled()
assert not window.plan_panel.import_bom_button.isEnabled()
assert window.run_panel.cleanup_resources_button.isEnabled()
window.run_panel.cleanup_resources_button.click()
qtbot.waitUntil(lambda: not window._workers)
assert calls == ['cleanup']
assert not window.run_panel.cleanup_resources_button.isEnabled()
assert '已释放' in window.run_panel.operator_band.text()
@pytest.mark.parametrize('cleanup_failure', [False, True])
def test_collection_cancel_waits_for_cleanup_then_shows_terminal_state(window, qtbot, tmp_path, monkeypatch, cleanup_failure):
open_project(window, qtbot, tmp_path)
+157
View File
@@ -0,0 +1,157 @@
"""Read-only desktop thread diagnosis, with borrowed handles confined to a child.
Windows contracts: GetThreadDesktop handles must NOT be closed; Toolhelp
snapshots must be closed. A missing thread/window owner does not prove that
no other process holds an OpenDesktop handle. See the accompanying research.
"""
import argparse
from collections import Counter
import ctypes
from ctypes import wintypes
import json
import os
from pathlib import Path
import re
import subprocess
import sys
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'src'))
def scan_threads(desktop_name):
"""Run only in the short-lived scan child, never in the application UI."""
import win32service
class ThreadEntry(ctypes.Structure):
_fields_ = [('dwSize', wintypes.DWORD), ('cntUsage', wintypes.DWORD),
('th32ThreadID', wintypes.DWORD), ('th32OwnerProcessID', wintypes.DWORD),
('tpBasePri', wintypes.LONG), ('tpDeltaPri', wintypes.LONG),
('dwFlags', wintypes.DWORD)]
kernel = ctypes.WinDLL('kernel32', use_last_error=True)
user = ctypes.WinDLL('user32', use_last_error=True)
snapshot = kernel.CreateToolhelp32Snapshot
snapshot.argtypes, snapshot.restype = [wintypes.DWORD, wintypes.DWORD], wintypes.HANDLE
first, next_thread = kernel.Thread32First, kernel.Thread32Next
for function in (first, next_thread):
function.argtypes, function.restype = [wintypes.HANDLE, ctypes.POINTER(ThreadEntry)], wintypes.BOOL
kernel.CloseHandle.argtypes, kernel.CloseHandle.restype = [wintypes.HANDLE], wintypes.BOOL
user.GetThreadDesktop.argtypes, user.GetThreadDesktop.restype = [wintypes.DWORD], wintypes.HANDLE
user.GetUserObjectInformationW.argtypes = [wintypes.HANDLE, ctypes.c_int, ctypes.c_void_p,
wintypes.DWORD, ctypes.POINTER(wintypes.DWORD)]
user.GetUserObjectInformationW.restype = wintypes.BOOL
kernel.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD]
kernel.OpenProcess.restype = wintypes.HANDLE
kernel.QueryFullProcessImageNameW.argtypes = [wintypes.HANDLE, wintypes.DWORD, wintypes.LPWSTR,
ctypes.POINTER(wintypes.DWORD)]
kernel.QueryFullProcessImageNameW.restype = wintypes.BOOL
images = {}
def image(pid):
if pid not in images:
process = kernel.OpenProcess(0x1000, False, pid) # QUERY_LIMITED_INFORMATION
if not process:
images[pid] = {'image_error': ctypes.get_last_error()}
else:
try:
value, length = ctypes.create_unicode_buffer(32768), wintypes.DWORD(32768)
images[pid] = ({'image': value.value} if kernel.QueryFullProcessImageNameW(
process, 0, value, ctypes.byref(length)) else {'image_error': ctypes.get_last_error()})
finally:
kernel.CloseHandle(process)
return images[pid]
handle = snapshot(0x4, 0) # TH32CS_SNAPTHREAD, read-only system snapshot
if handle == ctypes.c_void_p(-1).value:
raise ctypes.WinError(ctypes.get_last_error())
threads, unreadable, total = [], Counter(), 0
target = None
try:
# On Windows a READOBJECTS-only handle still made GetThreadDesktop
# return ACCESS_DENIED for another process on this desktop. Open with
# the owner's full access, but perform queries only. Never switch to it.
try:
target = win32service.OpenDesktop(desktop_name, 0, False, 0x10000000)
except Exception as error:
code = getattr(error, 'winerror', error.args[0] if error.args else None)
if code == 2:
return {'threads': [], 'snapshot_threads': 0, 'unreadable_by_winerror': {}, 'target_gone': True}
raise
entry = ThreadEntry()
entry.dwSize = ctypes.sizeof(entry)
valid = first(handle, ctypes.byref(entry))
while valid:
total += 1
desktop = user.GetThreadDesktop(entry.th32ThreadID)
if not desktop:
unreadable[str(ctypes.get_last_error())] += 1
else:
name, required = ctypes.create_unicode_buffer(256), wintypes.DWORD()
if not user.GetUserObjectInformationW(desktop, 2, name, ctypes.sizeof(name), ctypes.byref(required)):
unreadable[str(ctypes.get_last_error())] += 1
elif name.value.casefold() == desktop_name.casefold():
pid = entry.th32OwnerProcessID
threads.append({'pid': pid, 'thread_id': entry.th32ThreadID, **image(pid)})
# GetThreadDesktop is borrowed. CloseDesktop is forbidden here.
valid = next_thread(handle, ctypes.byref(entry))
if ctypes.get_last_error() != 18: # ERROR_NO_MORE_FILES
raise ctypes.WinError(ctypes.get_last_error())
finally:
if target is not None:
target.CloseDesktop()
if not kernel.CloseHandle(handle):
raise ctypes.WinError(ctypes.get_last_error())
return {'threads': threads, 'snapshot_threads': total, 'unreadable_by_winerror': dict(unreadable)}
def diagnose(desktop_name):
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api
# The child exits before we recheck desktop lifetime. GetThreadDesktop can
# never make the long-lived supervisor responsible for borrowed handles.
completed = subprocess.run((sys.executable, str(Path(__file__).resolve()), '--scan-worker', desktop_name),
capture_output=True, text=True, encoding='utf-8', timeout=15, creationflags=subprocess.CREATE_NO_WINDOW)
completed.check_returncode()
report = {'desktop': desktop_name, 'thread_scan': json.loads(completed.stdout)}
api = _Win32Api()
listed = {name.casefold() for name in api.service.GetProcessWindowStation().EnumDesktops()}
report['desktop_still_listed'] = desktop_name.casefold() in listed
if report['desktop_still_listed']:
try:
report['windows'] = api._desktop_windows(desktop_name)
except Exception as error:
report['window_query_error'] = str(error)
else:
report['windows'] = []
report['scope'] = 'Thread/window scan only; external handle-only owners may require Microsoft Sysinternals Handle.'
return report
def main():
parser = argparse.ArgumentParser(description=__doc__)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument('--record', type=Path, help='Existing private-desktop.json (read only)')
source.add_argument('--desktop', help='Owned dx- desktop name')
source.add_argument('--scan-worker', help=argparse.SUPPRESS)
parser.add_argument('--output', type=Path)
args = parser.parse_args()
if os.name != 'nt':
parser.error('Windows is required')
desktop = args.scan_worker or args.desktop
if args.record:
desktop = json.loads(args.record.read_text(encoding='utf-8-sig'))['desktop']
if args.output and (args.output.resolve() == args.record.resolve()
or args.output.exists() and args.output.samefile(args.record)):
parser.error('Diagnostic output must not overwrite its input record')
if not isinstance(desktop, str) or not re.fullmatch(r'dx-[A-Za-z0-9-]{1,100}', desktop):
parser.error('Only a valid owned dx- desktop name is accepted')
report = scan_threads(desktop) if args.scan_worker else diagnose(desktop)
text = json.dumps(report, ensure_ascii=True, indent=2)
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text, encoding='utf-8')
print(text)
if __name__ == '__main__':
main()
+344
View File
@@ -0,0 +1,344 @@
"""One-shot, two-project BOM/replace comparison on newly copied projects only.
The same tool imports either checkout via --repo. A failed run is retained and
cannot be resumed: completed writes are never automatically repeated.
"""
from __future__ import annotations
import argparse
from dataclasses import asdict
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
from threading import Event
import time
import traceback
OLD, NEW = '4100-CA0000-T7', '4100-CA0000-T0'
NAMES = ('Audio_D2128', 'Audio_T605A')
def save(path, value):
path.write_text(json.dumps(value, ensure_ascii=False, indent=2, default=str), encoding='utf-8')
def hashes(root):
return {str(path.relative_to(root)): hashlib.sha256(path.read_bytes()).hexdigest()
for path in root.rglob('*') if path.is_file()}
def inspect_native(spec_path):
"""Read the saved sheet directly, independently of the replacement snapshot."""
import pythoncom
import win32com.client
from dx_productivity.trial_task_center.cad_identity import project_file_path, verify_application
from dx_productivity.trial_task_center.native_helper import attach, attribute_value, components_on_sheet
spec = json.loads(spec_path.read_text(encoding='utf-8'))
assert Path(spec['project']).resolve().is_relative_to(Path(__file__).resolve().parents[1] / 'build')
pythoncom.CoInitialize()
app = document = None
try:
app = win32com.client.GetActiveObject('Viewdraw.Application')
verify_application(app, require_view=False)
assert not project_file_path(app), 'reader must start with empty CAD'
assert app.OpenProject(spec['project'])
rows = {}
for schematic, sheet in spec['sheets']:
document = app.SchematicSheetDocuments().Open(schematic, sheet)
document.Activate()
app, api = attach()
assert Path(project_file_path(app)).resolve() == Path(spec['project']).resolve()
pins = {}
for item in app.ActiveView.Query(4096, 0):
pin = api.IVdCmpPin(item._oleobj_)
point = pin.GetLocation()
connection = pin.Connection
net = None if connection is None else connection.Net
assert connection is None or net is not None
pins.setdefault(str(pin.Component.UID), []).append(dict(number=str(pin.Number),
x=float(point.X), y=float(point.Y), net_uid=None if net is None else str(net.UID),
net_name=None if net is None else str(net.LogicalNetName)))
for component in components_on_sheet(app, api):
uid = str(component.UID)
point = component.GetLocation()
attributes = {}
for item in component.Attributes:
attr = api.IVdAttr(item._oleobj_)
location = attr.GetLocation()
attributes[str(attr.Name)] = dict(value=str(attr.Value or ''),
instance_value=str(attr.InstanceValue or ''), x=float(location.X), y=float(location.Y),
color=int(attr.Color), font=int(attr.Font), size=float(attr.Size),
orientation=int(attr.Orientation), visible=bool(attr.Visible))
rows[f'{schematic}:{sheet}:{uid}'] = dict(uid=uid, refdes=str(component.Refdes),
schematic=schematic, sheet=sheet, x=float(point.X), y=float(point.Y),
orientation=int(component.Orientation), scale=float(component.Scale), attributes=attributes,
part_number=attribute_value(component, 'Part Number'), value=attribute_value(component, 'Value'),
part_label=attribute_value(component, 'Part Label'),
pins=sorted(pins.get(uid, []), key=lambda pin: (pin['x'], pin['y'], pin['number'])))
document.Close(False, '')
document = None
app.CloseProject()
assert not project_file_path(app)
save(Path(spec['destination']), rows)
finally:
if document is not None:
document.Close(False, '')
if app is not None and project_file_path(app):
assert Path(project_file_path(app)).resolve() == Path(spec['project']).resolve()
app.CloseProject()
app = None
pythoncom.CoUninitialize()
def compare_saved(before, after):
assert set(before) == set(after), 'component identity set changed'
selected = 0
for key, old in before.items():
new = after[key]
if old['refdes'] != 'RV05':
assert old == new, f'non-target changed: {key}'
continue
selected += 1
assert old['part_number'] == OLD and new['part_number'] == NEW
assert new['part_label'] == '0R'
# This fixture uses the unchecked copy option. Its native target has no
# Value attribute; preserve means retaining that absence, not restoring
# OLD's Value. See the 2026-10-14 project-tasks design and the production
# QPVerifyValuePolicy existence/content check after native replacement.
assert new['value'] == '' and 'Value' not in new['attributes']
for field in ('uid', 'refdes', 'schematic', 'sheet', 'x', 'y', 'orientation', 'scale'):
assert old[field] == new[field], (key, field)
graph = lambda row: sorted((pin['x'], pin['y'], pin['net_uid'], pin['net_name']) for pin in row['pins'])
assert graph(old) == graph(new) and new['pins'], f'target connectivity changed: {key}'
assert all(pin['number'] for pin in new['pins']), f'unpromoted target pin: {key}'
assert selected == 1
return {'components': len(after), 'targets': selected, 'unchanged_components': len(after) - selected,
'pin_connections': sum(len(row['pins']) for row in after.values()),
'target_state': [{key: row[key] for key in ('uid', 'refdes', 'part_number', 'part_label', 'value')}
| {'value_attribute_exists': 'Value' in row['attributes']}
for row in after.values() if row['refdes'] == 'RV05']}
def verify_existing(repo, root):
"""Finish readback only; this branch has no task queue or replacement call."""
import sqlite3
import win32api
from dx_productivity.trial_task_center.isolated_cad import IsolatedCadSession
from private_desktop_observer import PrivateDesktopObserver
root = root.resolve(strict=True)
assert root.is_relative_to(Path(__file__).resolve().parents[1] / 'build')
original = json.loads((root / 'report.json').read_text(encoding='utf-8'))
with sqlite3.connect(root / 'tasks.db') as database:
assert database.execute('select count(*) from tasks where status="completed" and copy_part_number_to_value=0').fetchone()[0] == 2
verification = root / 'read-only-verification'
verification.mkdir(exist_ok=False)
report = {'mode': 'read_only_saved_verification', 'copy_part_number_to_value': False,
'value_policy': 'preserve native target Value; this target has no Value attribute', 'projects': []}
try:
with PrivateDesktopObserver(verification) as observer:
for raw_project in original['projects']:
project = Path(raw_project).resolve(strict=True)
assert project.is_relative_to(root / 'projects')
before = json.loads((root / f'before-{project.stem}.json').read_text(encoding='utf-8'))
destination = root / f'after-{project.stem}.json'
if not destination.exists():
evidence = verification / project.stem
evidence.mkdir()
spec = evidence / 'spec.json'
save(spec, {'project': str(project), 'sheets': sorted({(row['schematic'], row['sheet']) for row in before.values()}),
'destination': str(destination)})
with IsolatedCadSession(evidence) as cad:
observer.records[cad.desktop.name] = dict(cad.record)
observer.handles[cad.cad.pid] = win32api.OpenProcess(0x100000, False, cad.cad.pid)
process = cad.start_worker((sys.executable, str(Path(__file__).resolve()), '--repo', str(repo),
'--inspect-file', str(spec), '--run-native'), stdout=-1, stderr=-1)
try:
code = process.wait(timeout=180)
(evidence / 'worker-output.txt').write_bytes(process.stdout.read() + b'\n' + process.stderr.read())
assert code == 0, (evidence / 'worker-output.txt').read_text(errors='replace')
finally:
process.close()
after = json.loads(destination.read_text(encoding='utf-8'))
report['projects'].append({'project': str(project), **compare_saved(before, after)})
observation = json.loads((verification / 'desktop-observation.json').read_text(encoding='utf-8'))
assert not observation['observer_errors']
assert observation['input_desktop_before'] == observation['input_desktop_after']
assert not observation['owned_cad_became_foreground'] and not observation['owned_cad_on_input_desktop']
assert all(observation['cad_exit_confirmed'].values()) and all(observation['desktop_absent'].values())
report['passed'] = True
finally:
save(verification / 'saved-verification.json', report)
print(json.dumps(report, ensure_ascii=False), flush=True)
return 0 if report.get('passed') else 1
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--repo', type=Path, required=True)
parser.add_argument('--source-root', type=Path)
parser.add_argument('--library', type=Path)
parser.add_argument('--output', type=Path)
parser.add_argument('--run-native', action='store_true', required=True)
parser.add_argument('--inspect-file', type=Path)
parser.add_argument('--verify-existing', type=Path, help='Read saved completed run only; never execute replacement')
args = parser.parse_args()
repo = args.repo.resolve(strict=True)
sys.path.insert(0, str(repo / 'src'))
sys.path.insert(0, str(repo / 'tools'))
if args.inspect_file:
inspect_native(args.inspect_file)
return 0
if args.verify_existing:
return verify_existing(repo, args.verify_existing)
assert args.output and args.source_root and args.library
output = args.output.resolve()
assert output.is_relative_to(Path(__file__).resolve().parents[1] / 'build')
output.mkdir(parents=True, exist_ok=False)
source_manifest = {'git_head': subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=repo, text=True).strip(),
'files': {str(path.relative_to(repo)): hashlib.sha256(path.read_bytes()).hexdigest()
for folder, pattern in (('src', '*.py'), ('integration', '*.vbs'))
for path in (repo / folder).rglob(pattern)}}
save(output / 'source-under-test-manifest.json', source_manifest)
library = args.library.resolve(strict=True)
library_hash = hashlib.sha256(library.read_bytes()).hexdigest()
source_root = args.source_root.resolve(strict=True)
source_hashes, projects = {}, []
for name in NAMES:
source = source_root / name
assert (source / (name + '.prj')).is_file()
source_hashes[name] = hashes(source)
folder = output / 'projects' / name
shutil.copytree(source, folder)
project = folder / (name + '.prj')
text, count = re.subn(r'(?m)^KEY CentralLibrary .*$',
lambda _: f'KEY CentralLibrary "{library.as_posix()}"', project.read_text(encoding='utf-8-sig'))
assert count == 1
project.write_text(text, encoding='utf-8')
projects.append(project)
from dx_productivity.trial_task_center.bom_plan import write_bom_template
from dx_productivity.trial_task_center.isolated_cad import IsolatedCadSession
from dx_productivity.trial_task_center.isolated_desktop import PrivateDesktop
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.store import TaskStore
from dx_productivity.trial_task_center.task_queue import execute_task_queue
from private_desktop_observer import PrivateDesktopObserver
report = {'repo': str(repo), 'library': str(library), 'source_root': str(source_root),
'projects': list(map(str, projects)), 'input': {'old': OLD, 'new': NEW, 'refdes': 'RV05',
'copy_part_number_to_value': False, 'value_mode': 'preserve'},
'steps': [], 'desktop_starts': [], 'cad_starts': [], 'source_hashes': source_hashes}
save(output / 'report.json', report)
phase = 'setup'
observer = PrivateDesktopObserver(output)
original_desktop_enter, original_cad_enter = PrivateDesktop.__enter__, IsolatedCadSession.__enter__
def enter_desktop(desktop):
started = time.monotonic()
try:
return original_desktop_enter(desktop)
finally:
report['desktop_starts'].append({'phase': phase, 'desktop': desktop.name,
'seconds': time.monotonic() - started})
def enter_cad(cad):
started = time.monotonic()
try:
return original_cad_enter(cad)
finally:
entry = {'phase': phase, 'desktop': cad.desktop.name, 'seconds': time.monotonic() - started,
'pid': None if cad.cad is None else cad.cad.pid}
report['cad_starts'].append(entry)
if entry['pid']:
import win32api
observer.records[entry['desktop']] = dict(cad.record)
try:
observer.handles[entry['pid']] = win32api.OpenProcess(0x100000, False, entry['pid'])
except Exception:
pass
PrivateDesktop.__enter__, IsolatedCadSession.__enter__ = enter_desktop, enter_cad
store = TaskStore(output / 'tasks.db')
service = TaskCenterService(store, configure_logging(output / 'data'))
def record(stage, started, **details):
entry = {'stage': stage, 'seconds': time.monotonic() - started, **details}
report['steps'].append(entry)
save(output / 'report.json', report)
print(json.dumps(entry, ensure_ascii=False, default=str), flush=True)
def independent(project, sheets, label):
name = project.stem
root = output / 'independent-read' / 'quick-run-history' / f'sheet-discovery-{label}-{name}'
root.mkdir(parents=True)
destination, spec = output / f'{label}-{name}.json', root / 'spec.json'
save(spec, {'project': str(project), 'sheets': sorted(sheets), 'destination': str(destination)})
with IsolatedCadSession(root) as cad:
process = cad.start_worker((sys.executable, str(Path(__file__).resolve()), '--repo', str(repo),
'--inspect-file', str(spec), '--run-native'), stdout=-1, stderr=-1)
try:
code = process.wait(timeout=180)
(root / 'worker-output.txt').write_bytes(process.stdout.read() + b'\n' + process.stderr.read())
assert code == 0, (root / 'worker-output.txt').read_text(errors='replace')
finally:
process.close()
return json.loads(destination.read_text(encoding='utf-8'))
started_all = time.monotonic()
try:
with observer:
tasks = service.create_project_tasks(output / 'projects', selected_paths=tuple(projects))
assert len(tasks) == 2
bom = write_bom_template(output / 'input.xlsx', ({'project_path': str(project),
'old_part_number': OLD, 'new_part_number': NEW, 'refdes': 'RV05', 'enabled': True,
'sync_nc': True} for project in projects))
phase = 'bom_import'
started = time.monotonic()
imported = service.inspect_bom_batch(tasks[0].task_id, bom, on_progress=lambda message: print(message, flush=True))
assert imported.passed_rows == 2 and imported.preview is not None and not imported.failed_rows
service.save_bom_batch(imported.preview, {item.task_id: item.rows for item in imported.preview.projects})
record(phase, started, passed_rows=imported.passed_rows)
phase = 'independent_before'
before = {}
for project in projects:
rows = next(item.rows for item in imported.preview.projects if item.project_path.resolve() == project.resolve())
sheets = {(row.target.schematic_name, row.target.sheet_number) for row in rows}
before[project.stem] = independent(project, sheets, 'before')
phase = 'queue_execution'
started = time.monotonic()
result = execute_task_queue(service, tuple(task.task_id for task in tasks), stop_event=Event())
assert all(entry.state == 'completed' for entry in result.entries), result
record(phase, started, result=asdict(result))
phase = 'independent_after'
for project in projects:
sheets = {(row['schematic'], row['sheet']) for row in before[project.stem].values()}
started = time.monotonic()
after = independent(project, sheets, 'after')
record(f'independent-{project.stem}', started, **compare_saved(before[project.stem], after))
observation = json.loads((output / 'desktop-observation.json').read_text(encoding='utf-8'))
assert not observation['observer_errors']
assert observation['input_desktop_before'] == observation['input_desktop_after']
assert not observation['owned_cad_became_foreground'] and not observation['owned_cad_on_input_desktop']
assert all(observation['cad_exit_confirmed'].values()) and all(observation['desktop_absent'].values())
report['verified'] = True
except BaseException as error:
report['error'] = f'{type(error).__name__}: {error}'
report['traceback'] = traceback.format_exc()
traceback.print_exc()
finally:
report['total_seconds_including_independent_reads'] = time.monotonic() - started_all
report['workflow_seconds'] = sum(step['seconds'] for step in report['steps'] if step['stage'] in {'bom_import', 'queue_execution'})
report['source_unchanged'] = all(hashes(source_root / name) == source_hashes[name] for name in NAMES)
report['library_lmc_unchanged'] = hashlib.sha256(library.read_bytes()).hexdigest() == library_hash
PrivateDesktop.__enter__, IsolatedCadSession.__enter__ = original_desktop_enter, original_cad_enter
store.close()
from dx_productivity.trial_task_center.legacy_paths import cleanup_worker_aliases
report['alias_cleanup_errors'] = list(map(str, cleanup_worker_aliases()))
save(output / 'report.json', report)
print(json.dumps({key: report.get(key) for key in ('verified', 'error', 'workflow_seconds', 'source_unchanged',
'library_lmc_unchanged', 'desktop_starts', 'cad_starts')}, ensure_ascii=False), flush=True)
return 0 if (report.get('verified') and report['source_unchanged'] and report['library_lmc_unchanged']
and not report['alias_cleanup_errors']) else 1
if __name__ == '__main__':
raise SystemExit(main())
+201
View File
@@ -0,0 +1,201 @@
"""Replay an issue BOM through inspect_batch using only captured native reports.
All fixtures and the copied task database are created under --output. This tool
never opens CAD and refuses to create a desktop or persist a target plan.
"""
from __future__ import annotations
import argparse
from collections import defaultdict
from dataclasses import asdict
import hashlib
import json
from pathlib import Path
import sqlite3
import subprocess
import sys
from unittest.mock import patch
import zipfile
def digest(path):
return hashlib.sha256(Path(path).read_bytes()).hexdigest()
def save(path, value):
Path(path).write_text(json.dumps(value, ensure_ascii=False, indent=2, default=str), encoding='utf-8')
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--issue-root', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--repo', type=Path, default=Path(__file__).resolve().parents[1])
args = parser.parse_args()
issue, output, repo = args.issue_root.resolve(), args.output.resolve(), args.repo.resolve()
assert not output.is_relative_to(issue), 'Original evidence is read-only'
assert 'build' in output.parts, 'Replay output must stay inside a build directory'
output.mkdir(parents=True, exist_ok=False)
sys.path.insert(0, str(repo / 'src'))
from openpyxl import load_workbook
from dx_productivity.trial_task_center import isolated_cad, isolated_desktop, native_inventory
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
from dx_productivity.trial_task_center.bom_import_report import inspect_batch
from dx_productivity.trial_task_center.logging_setup import configure_logging
from dx_productivity.trial_task_center.native_inventory_protocol import read_ascii_discovery
from dx_productivity.trial_task_center.service import TaskCenterService
from dx_productivity.trial_task_center.store import TaskStore
from dx_productivity.trial_task_center.target_plan import ProjectIdentityResolver
identities = ProjectIdentityResolver()
expected = json.loads((issue / 'reported-row-replay.json').read_text(encoding='utf-8'))
replayed_rows = {row['excel_row'] for row in expected}
replayed_projects = {row['project'] for row in expected}
baseline_native = json.loads((issue / 'native-replay-report.json').read_text(encoding='utf-8'))
native_names = {row['evidence'] for row in baseline_native if row['status'] == 'ready'}
captured = defaultdict(list)
original_files = [issue / name for name in (
'reported-app.zip', 'reported-bom.xlsx', 'reported-row-replay.json', 'native-replay-report.json')]
for evidence in (issue / 'evidence/tasks').glob('*/quick-run-history/native-inventory-*'):
if evidence.name not in native_names:
continue
metadata = json.loads((evidence / 'native-inventory.json').read_text(encoding='utf-8'))
project = Path(metadata['project'])
if project.name not in replayed_projects:
continue
scope, effective = evidence / 'output/scope.txt', evidence / 'output/effective.txt'
assert scope.is_file() and effective.is_file()
captured[identities.project(project)].append(evidence)
original_files.extend((evidence / 'native-inventory.json', scope, effective))
assert len(captured) == len(replayed_projects) == 6
before_sources = {str(path): digest(path) for path in original_files}
database = output / 'app-copy/task-center.db'
database.parent.mkdir()
with zipfile.ZipFile(issue / 'reported-app.zip') as archive:
names = [name for name in archive.namelist() if name.endswith('/data/task-center.db')]
assert len(names) == 1
database.write_bytes(archive.read(names[0]))
connection = sqlite3.connect(database)
connection.row_factory = sqlite3.Row
source_counts = {'tasks': connection.execute('SELECT count(*) FROM tasks').fetchone()[0],
'target_plan': connection.execute('SELECT count(*) FROM target_plan').fetchone()[0]}
assert source_counts == {'tasks': 428, 'target_plan': 0}
project_rows = list(connection.execute('SELECT project_id,task_id,path FROM projects'))
remap, source_by_local, task_by_source = {}, {}, {}
for project_row in project_rows:
source = Path(project_row['path'])
local = output / 'projects' / project_row['project_id'] / source.name
local.parent.mkdir(parents=True)
local.write_text('KEY FrontEndSnapshot "DxD"\nKEY iCDBDir "database"\n', encoding='utf-8')
(local.parent / 'database').mkdir()
(local.parent / 'database/icdb.dat').write_bytes(b'captured read-only replay fixture')
source_key, local_key = identities.project(source), identities.project(local)
remap[source_key] = local
source_by_local[local_key] = source_key
task_by_source[source_key] = project_row['task_id']
connection.execute('UPDATE projects SET path=?,path_identity=?,discovery_root=? WHERE project_id=?',
(str(local), local_key, str(output / 'projects'), project_row['project_id']))
connection.execute('UPDATE tasks SET discovery_root=?', (str(output / 'projects'),))
connection.commit()
connection.close()
workbook = load_workbook(issue / 'reported-bom.xlsx', data_only=False)
sheet = workbook.active
assert len(workbook.worksheets) == 1
assert tuple(cell.value for cell in sheet[1]) == (
'project_name', 'project_path', 'old_part_number', 'new_part_number', 'refdes',
'schematic_name', 'sheet_number', 'uid', 'enabled', 'custom_value', 'sync_nc')
original_rows, replacement_rows, replacement_projects = sheet.max_row - 1, [], set()
for excel_row in range(2, sheet.max_row + 1):
source_path = sheet.cell(excel_row, 2).value
key = identities.project(source_path)
assert key in remap
sheet.cell(excel_row, 2, str(remap[key]))
has_replacement = any(sheet.cell(excel_row, column).value not in (None, '') for column in (3, 4, 5))
if has_replacement:
replacement_rows.append(excel_row)
replacement_projects.add(key)
if excel_row not in replayed_rows:
# Preserve row positions, routing columns, header and original cell
# styles. Uncaptured replacements are explicitly outside this replay.
for column in (3, 4, 5, 6, 7, 8, 10):
sheet.cell(excel_row, column).value = None
else:
assert key in captured and has_replacement
assert (original_rows, len(replacement_rows), len(replacement_projects)) == (763, 437, 102)
replay_file = output / 'captured-scope-original-format.xlsx'
workbook.save(replay_file)
workbook.close()
store = TaskStore(database)
service = TaskCenterService(store, configure_logging(output / 'data'))
loads, phases, forbidden_calls = [], [], []
def forbidden(name):
def fail(*args, **kwargs):
forbidden_calls.append(name)
raise AssertionError(f'Read-only replay attempted {name}')
return fail
for source_key, evidence_paths in captured.items():
child = service.for_project_task(task_by_source[source_key])
def captured_loader(project, parts, *, collect_components=False, _key=source_key,
_evidence=tuple(evidence_paths), **controls):
assert source_by_local[identities.project(project)] == _key
assert collect_components and controls.get('runtime') is not None
reads = [read_ascii_discovery(project, evidence / 'output/scope.txt', parts,
effective_report_path=evidence / 'output/effective.txt') for evidence in _evidence]
assert all(read == reads[0] for read in reads), 'Repeated native captures disagree'
loads.append({'project': str(project), 'original_project': _key, 'requested_parts': parts,
'components': len(reads[0].components), 'evidence': list(_evidence)})
return reads[0]
child._quick_run_preparer._discover_project_sheets = captured_loader
initial_dump = tuple(store._connection.iterdump())
with (patch.object(BatchCadRuntime, '_ensure_desktop', forbidden('batch desktop')),
patch.object(isolated_desktop.PrivateDesktop, '__enter__', forbidden('private desktop')),
patch.object(isolated_cad.IsolatedCadSession, '__enter__', forbidden('CAD')),
patch.object(native_inventory.NativeInventoryReader, 'read', forbidden('native process')),
patch.object(subprocess, 'Popen', forbidden('process launch')),
patch.object(store, 'record_target_inventory', forbidden('inventory persistence')),
patch.object(store, 'replace_target_plan', forbidden('target-plan persistence'))):
report = inspect_batch(service, task_by_source[next(iter(captured))], replay_file,
on_phase=lambda phase, message: phases.append({'phase': phase.value, 'message': message}))
assert tuple(store._connection.iterdump()) == initial_dump, 'Replay changed the copied task database'
assert not forbidden_calls
assert report.preview is None
assert len(loads) == len(captured) == 6
current = [{'excel_row': row.excel_row, 'project': Path(row.project_path).name,
'status': row.status, 'reason': row.reason} for row in report.rows]
expected_outcomes = [(row['excel_row'], row['project'], row['status']) for row in expected]
current_outcomes = [(row['excel_row'], row['project'], row['status']) for row in current]
assert current_outcomes == expected_outcomes
assert (report.total_rows, report.passed_rows, report.failed_rows, report.unchecked_rows) == (31, 20, 11, 0)
assert [row.excel_row for row in report.rows if row.status == 'failed'] == [3, 4, 6, 7, 11, 12, 13, 16, 17, 18, 19]
assert {str(path): digest(path) for path in original_files} == before_sources
save(output / 'report.json', asdict(report))
save(output / 'row-comparison.json', [{'original': old, 'current': new,
'row_project_status_equal': True, 'reason_text_equal': old['reason'] == new['reason']}
for old, new in zip(expected, current)])
save(output / 'captured-loads.json', loads)
save(output / 'source-hashes.json', before_sources)
save(output / 'phases.json', phases)
summary = {'source_repo': str(repo), 'source_commit': subprocess.check_output(
['git', 'rev-parse', 'HEAD'], cwd=repo, text=True).strip(),
'application_tasks': 428, 'original_bom_rows': original_rows,
'original_replacement_rows': len(replacement_rows), 'original_replacement_projects': 102,
'captured_projects': 6, 'captured_report_files': sum(map(len, captured.values())),
'replayed_rows': report.total_rows, 'passed_rows': report.passed_rows,
'failed_rows': report.failed_rows, 'unchecked_rows': report.unchecked_rows,
'unreplayed_replacement_rows': len(replacement_rows) - report.total_rows,
'unreplayed_projects': 96, 'row_project_status_equal': True,
'cad_desktop_or_process_creations': 0, 'target_plan_writes': 0,
'copied_database_unchanged_during_inspection': True, 'original_evidence_hashes_unchanged': True,
'limitations': 'Only the 31 rows covered by six captured project inventories were replayed. '
'Original and current reason wording can differ; Excel row/project/status match exactly.'}
save(output / 'summary.json', summary)
store.close()
print(json.dumps(summary, ensure_ascii=False, indent=2))
if __name__ == '__main__':
main()