Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ce42acc654 | ||
|
|
1ee519e5d2 | ||
|
|
bc91b98385 | ||
|
|
9c4ccc5bf5 | ||
|
|
46bcafd7a3 | ||
|
|
e46982af6c | ||
|
|
2a9398ab50 | ||
|
|
faad50df80 | ||
|
|
74c1cb752c | ||
|
|
a8358d90d7 | ||
|
|
486ea05ad5 | ||
|
|
4b78ca0f27 | ||
|
|
0151552f55 | ||
|
|
2c88be99ce | ||
|
|
a011e8eab6 | ||
|
|
9a3dff2f14 | ||
|
|
bdb096fb88 | ||
|
|
89c7c603f6 | ||
|
|
681403a4d2 | ||
|
|
a94276494a | ||
|
|
00861a59d6 | ||
|
|
832ab5d563 | ||
|
|
e15ee259f8 | ||
|
|
e4ef6a533e | ||
|
|
3b3b648580 |
No files matched your search
@@ -4,72 +4,162 @@
|
||||
|
||||
面向 **Windows + DxDesigner / Expedition Enterprise 7.9.5** 的器件批量替换工具。一个工程对应一个任务,可以只替换同料号下的部分位号;支持直接编辑和 Excel BOM 导入,同批工程按顺序执行。
|
||||
|
||||
**最新版本:V1.1.3** · [下载便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.3/DxPartReplace-1.1.3.zip) · [版本说明与附件](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/tag/v1.1.3) · [完整使用说明](docs/user/v1.1.0-project-tasks.md)
|
||||
**最新版本:V1.1.4 Beta 2(预发布)** · [下载便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.4-beta.2/DxPartReplace-1.1.4-beta.2.zip) · [版本说明与附件](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/tag/v1.1.4-beta.2) · [完整使用说明](docs/user/v1.1.0-project-tasks.md)
|
||||
|
||||
所有发布均标记为 **Beta 预发布**。V1.1.4 Beta 1 因 BOM 导入缺少 `win32timezone` 已撤回,请使用 Beta 2;旧 Git 标签保留供回溯。升级前备份 `data`,完整更新 EXE 与 `_internal`,不要只替换 EXE。
|
||||
|
||||
本版提供[目标封装选择](docs/user/target-cell-selection.md):目标封装可留空,此时工具不主动覆写封装,可后续与中心库同步;需要指定时,在料号规则中读取并选择合法封装。BOM 保持原 11 列。
|
||||
|
||||
## 窗口操作教程
|
||||
## 图文教程
|
||||
|
||||
下面的 GIF 录制于 V1.1.2,操作步骤仍适用于 V1.1.3,图标与版本号以新版为准。包含创建任务、BOM 导入、错误行提示、保存、顺序执行、查看结果和开始新一轮。共 20 个步骤、47.5 秒;仅记录工具及其弹窗,CAD 等待已压缩,Excel 填表示例在录制之外完成。
|
||||
按下面 **13 步**,完成“创建任务 → 填 BOM → 导入校验 → 保存计划 → 替换 → 查看结果”。本例有两个工程 `Audio_D2128`、`Audio_T605A`,各自只替换 `RV05`,保留同料号下的 `RV11`。
|
||||
|
||||

|
||||
截图来自真实窗口:起始页为 V1.1.3,后续操作图取自已验收的 V1.1.2 业务流程,操作位置相同。点击图片可以放大查看;BOM 填写图为示意。
|
||||
|
||||
如果图片没有自动播放,可以[单独打开 GIF](docs/user/media/v1.1.2-window-bom-tutorial.gif)。
|
||||
### 第 1 步:解压并打开应用
|
||||
|
||||
## 开始使用
|
||||
[下载 V1.1.4 Beta 2 便携包](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/releases/download/v1.1.4-beta.2/DxPartReplace-1.1.4-beta.2.zip),**完整解压**,双击 `DxPartReplace.exe`。保留它与 `_internal` 的相对位置,无需安装 Python;本机需要可用的 DxDesigner、许可证和中央库。
|
||||
|
||||
1. 下载并完整解压便携包,保留 `DxPartReplace.exe` 和 `_internal` 的相对位置。**便携包无需安装 Python**,本机仍需安装可用的 DxDesigner、许可证和中央库。
|
||||
2. 双击 `DxPartReplace.exe`。首次运行会自动创建同目录的 `data`,保存任务、日志和执行证据。
|
||||
3. 点击左侧 **新建**,在弹窗填写工程根目录,点击 **读取工程**,勾选需要处理的 `.prj`,再点击 **创建任务**。每个工程自动创建一个任务。
|
||||
打开后看到下图主窗口。首次使用请准备完整的工程副本,执行前保存并关闭待处理工程。
|
||||
|
||||
首次使用请在完整工程副本上验证映射与库配置。执行前保存并关闭待处理工程;工具会启动自己的后台 CAD,主桌面仍可查看其他文件。
|
||||

|
||||
|
||||
## 方式一:通过 BOM 批量配置
|
||||
### 第 2 步:新建任务,读取工程目录
|
||||
|
||||
### 1. 导出本批模板
|
||||
点击左下方 **新建**。在弹窗中填写工程根目录,或点击 **选择目录**,再点击 **读取工程**。
|
||||
|
||||
选择任务,打开顶部 **本批位号计划**,点击 **BOM 模板**。只要已导入工程,就能导出同批所有工程的一张空白表,**不需要先采集位号或填写料号规则**。
|
||||
**完成标志:** 弹窗列表显示该目录下找到的 `.prj` 工程。
|
||||
|
||||
工程名称、工程路径自动填写。工程路径用于唯一分配任务,请保持原样;同一工程有多条替换时,可以复制该工程行。
|
||||

|
||||
|
||||
### 2. 在 Excel 填写替换项
|
||||
### 第 3 步:勾选工程,创建任务
|
||||
|
||||
保留原模板的 11 列表头,按需要填写:
|
||||
勾选这次要处理的工程,点击右下角 **创建任务**。本例勾选两个工程。
|
||||
|
||||
| 列名 | 填写内容 |
|
||||
**完成标志:** 左侧出现两个任务,**一个工程对应一个任务**。
|
||||
|
||||

|
||||
|
||||
### 第 4 步:导出本批 BOM 模板
|
||||
|
||||
选中左侧任意一个任务,切换到顶部 **本批位号计划**,点击 **BOM 模板**,选择文件保存位置。
|
||||
|
||||
**完成标志:** 导出的 Excel 包含本批所有工程,工程名称和完整路径已经填写好。此时不需要先采集位号,也不需要先添加料号规则。
|
||||
|
||||

|
||||
|
||||
### 第 5 步:在 Excel 中填写替换项
|
||||
|
||||
打开刚导出的模板,**保留全部 11 列表头以及原工程路径**。同一工程需要填写多条替换时,复制该工程行后再填写。
|
||||
|
||||
本例在两个工程行中分别填写相同的旧、新料号和 `RV05`:
|
||||
|
||||

|
||||
|
||||
| 要填写的列 | 填法 |
|
||||
| --- | --- |
|
||||
| `old_part_number` | 工程当前的旧料号 |
|
||||
| `new_part_number` | 中央库中的目标新料号 |
|
||||
| `refdes` | 要处理的位号,例如 `RV05`;一行可以写多个位号,以逗号、分号、空格或顿号分隔 |
|
||||
| `enabled` | `TRUE` / `是` 表示替换,`FALSE` / `否` 表示保留 |
|
||||
| `sync_nc` | 是否同步 NC,默认开启,可逐行关闭 |
|
||||
| `custom_value` | 可选的自定义 Value;填写后优先写入,留空沿用任务的 Value 策略 |
|
||||
| `old_part_number` | 工程当前的旧料号,本例为 `4100-CA0000-T7` |
|
||||
| `new_part_number` | 中央库中的目标新料号,本例为 `4100-CA0000-T0` |
|
||||
| `refdes` | 要替换的位号,本例为 `RV05`;多个位号可用逗号、分号、空格或顿号分隔 |
|
||||
| `enabled` | `TRUE` / `是` 表示替换,`FALSE` / `否` 表示保留;留空默认替换,建议显式填写 |
|
||||
| `sync_nc` | 是否同步 NC,默认开启;按需要填 `TRUE` 或 `FALSE` |
|
||||
| `custom_value` | 可选的自定义 Value;留空时使用任务的 Value 策略 |
|
||||
|
||||
例如,首次配置的旧料号同时对应 `RV05`、`RV11`,只想替换 `RV05`,可以只填写 `RV05` 的启用项;首次采集的其余位号默认不替换。
|
||||
料号按**文本**填写,不要使用公式。相同位号出现在多个图页时,补填 `schematic_name`、`sheet_number` 或 `uid` 消除歧义。旧新相同、链式或循环替换会被拒绝。
|
||||
|
||||
**再次导入时,BOM 中未列出的既有计划行会保留原勾选和编辑状态。** 如果这轮只想执行 BOM 列出的目标,请在预览中检查其他启用行,并明确取消勾选,或在 BOM 中填写对应的 `enabled=FALSE` 行。
|
||||
**本例首次导入只列出 RV05,采集到的 RV11 默认保留。再次导入时,未列出的既有计划行会保留原勾选;需要停用时,请取消勾选或明确填写 `enabled=FALSE`。**
|
||||
|
||||
同名位号分布在不同图页时,补填 `schematic_name`、`sheet_number` 或 `uid` 消除歧义。`enabled` 留空默认替换,`sync_nc` 留空默认开启;建议显式填写。料号按文本填写;导入不接受公式,也不接受旧新相同、链式或循环替换。
|
||||
填完后保存 Excel 文件。
|
||||
|
||||
### 3. 导入并检查报告
|
||||
### 第 6 步:导入填写好的 BOM
|
||||
|
||||
点击 **导入 BOM**,选择填好的文件。工具先校验表格格式、工程归属,以及能从表中确定的重复声明和料号冲突;通过后,只核验本表涉及的工程。已有位号库存覆盖旧料号时直接匹配;缺少库存时优先通过原生 CLI 采集,不可用时才回退到后台 CAD 只读采集。
|
||||
返回应用,在 **本批位号计划** 点击 **导入 BOM**,选择刚保存的 Excel,等待校验结果。
|
||||
|
||||
新一轮导入开始后,旧报告会清除,界面显示读表、获取库存、匹配位号等进度;取消选择文件则保留原报告。导入失败或取消会保留已有预览和手动编辑。保存和执行前仍会核验工程。
|
||||
导入会检查表格和工程归属,并匹配相关工程的位号。缺少库存时优先调用原生 CLI 采集,不可用时回退到后台 CAD 只读采集。**导入不会执行替换,也不会直接保存计划。**
|
||||
|
||||
- 校验报告显示通过、失败、未检查的行数,并标明 Excel 行号及原因。
|
||||
- 有错误时,整批不会生成新的可保存预览,也不会部分写入计划。修正后重新导入。
|
||||
- 全部通过后,先得到预览;可以切换任务,检查或修改各工程的位号计划。
|
||||

|
||||
|
||||
### 4. 保存并执行
|
||||
### 第 7 步:检查导入报告,有错误先修正
|
||||
|
||||
确认预览后点击 **保存本批位号计划**;普通单工程编辑时按钮显示 **保存位号计划**。
|
||||
点击校验状态条中的 **详情**。报告会显示通过、失败、未检查的行数,以及 Excel 行号和错误原因。
|
||||
|
||||
点击 **执行当前任务** 处理当前工程,或点击左侧 **顺序执行本批任务** 处理同批工程。确认范围后,工具逐项执行原生 Replace Part、保存、关闭并重新打开核验。
|
||||
下图是故意填入不存在位号的示例:**通过 2 行、失败 1 行**,错误定位到 Excel 第 4 行。按报告修正 Excel、保存文件,再次点击 **导入 BOM**。
|
||||
|
||||
**左侧搜索和显示排序只影响浏览,不改变本批执行范围。** 调整执行顺序请使用上移、下移任务按钮。
|
||||
**有错误时不会部分保存。全部通过后再继续下一步。** 新一次导入开始后旧报告会清除;导入失败仍保留已有预览和手动编辑,取消文件选择则保留原报告。
|
||||
|
||||
## 方式二:直接在界面编辑
|
||||

|
||||
|
||||
### 第 8 步:核对每个工程的预览,保存计划
|
||||
|
||||
报告全部通过后,关闭详情,依次选中左侧各任务,核对:
|
||||
|
||||
- 需要替换的 `RV05` 已勾选,新料号正确。
|
||||
- 需要保留的 `RV11` 未勾选。
|
||||
- 自定义 Value、同步 NC 等逐行设置符合预期。
|
||||
|
||||
确认后点击 **保存本批位号计划**;单工程编辑或保存完成后,按钮显示 **保存位号计划**。
|
||||
|
||||
**完成标志:** 底部提示计划保存完成。下图为保存后的状态,只有 RV05 启用。
|
||||
|
||||

|
||||
|
||||
### 第 9 步:按任务确认 Value 与封装设置
|
||||
|
||||
选中一个任务,按需设置顶部 **Value 策略**:勾选后,将替换后的 **Part Label 写入 Value**;逐行自定义 Value 优先。处理多个工程时,分别确认各任务的开关。
|
||||
|
||||
本例开启该选项。**复制的是 Part Label,不是把新料号直接写进 Value。**
|
||||
|
||||

|
||||
|
||||
目标封装可留空,工具不主动覆写 Cell Name,可后续与中心库同步。需要本轮指定时,在 **本批料号规则** 中编辑对应旧、新料号规则,读取并选择合法封装,核对计划中的选择后保存。详见[封装操作说明](docs/user/target-cell-selection.md)。
|
||||
|
||||
### 第 10 步:选择执行范围,确认后开始
|
||||
|
||||
只处理选中工程,点击底部 **执行当前任务**;处理同批工程,点击左侧 **顺序执行本批任务**。本例使用后者。
|
||||
|
||||
在确认弹窗中核对工程数量和执行范围,确认无误再点击 **Yes / 是**。
|
||||
|
||||

|
||||
|
||||
**搜索和显示排序只影响浏览,不改变本批执行范围。** 调整实际执行顺序,请使用左下方 **上移任务 / 下移任务**。
|
||||
|
||||
### 第 11 步:等待后台执行和保存重开核验
|
||||
|
||||
工具按工程顺序执行替换,保存、关闭并重新打开核验。左侧显示各任务状态,底部显示当前阶段和进度。
|
||||
|
||||

|
||||
|
||||
运行期间可以在主桌面查看其他文件,请勿打开或修改本次正在处理的同一工程。需要暂停时,点击 **完成当前项后暂停**,等待当前步骤安全收尾。
|
||||
|
||||
**完成标志:** 任务显示已完成,底部提示本批执行结束。还要打开结果核对,不能只看进度条。
|
||||
|
||||
### 第 12 步:查看实际修改结果与日志
|
||||
|
||||
选中要检查的任务,点击右下角 **替换结果 / 日志**,查看 **位号替换结果**。
|
||||
|
||||
本例应看到:`RV05` 的旧料号为 `4100-CA0000-T7`,回读料号为 `4100-CA0000-T0`,核验状态为 **已保存并核验**。
|
||||
|
||||

|
||||
|
||||
需要排查原因时,切到 **完整日志**,查看具体阶段和耗时;可用 **上一页日志 / 最新日志** 翻页。失败、未命中或未核验不会被当作已保存成功。
|
||||
|
||||

|
||||
|
||||
### 第 13 步:需要继续替换时,开始新一轮
|
||||
|
||||
已完成任务可以点击底部 **开始新一轮**:
|
||||
|
||||

|
||||
|
||||
核对弹窗中的任务范围,再确认。
|
||||
|
||||

|
||||
|
||||
**完成标志:** 已完成任务恢复为待配置,旧计划、命令、快照和日志归档保留。接下来按工程**当前料号**重新填 BOM 或修改规则,重新采集、保存后执行;旧轮次通过 **查看轮次证据** 打开。
|
||||
|
||||
出现状态不确定或恢复提示时,先使用 **核验并恢复**,按[恢复说明](docs/user/interrupted-recovery.md)处理。
|
||||
|
||||
## 不使用 Excel:直接在界面编辑
|
||||
|
||||
1. 在 **本批料号规则** 添加旧、新料号,按需配置 Value 与同步 NC;规则提供采集条件和默认值。
|
||||
2. 切到 **本批位号计划**,点击 **采集所有匹配位号**。
|
||||
@@ -78,7 +168,7 @@
|
||||
|
||||
两种方式可以配合使用,不必为了编辑而先导出 Excel。切换任务会保留编辑草稿;关闭时有未保存内容会提示保存全部、放弃或取消。
|
||||
|
||||
## Value、暂停与执行结果
|
||||
## Value、NC 与日志选项
|
||||
|
||||
- 顶部 **Value 策略** 开关控制当前任务是否将替换后的 Part Label 写入 Value;逐行自定义 Value 优先。处理多个任务时,分别确认各任务的开关。
|
||||
- **同步 NC** 是逐行选项,默认开启,可以关闭。
|
||||
@@ -86,12 +176,6 @@
|
||||
- 点击 **替换结果 / 日志** 查看图页、位号、旧料号、实际回读料号和核验状态。失败、未命中、未核验不会显示为已保存成功;完整日志支持翻页。
|
||||
- 日志和 BOM 详情按需打开,关闭后释放窗口;数据和证据仍保留。
|
||||
|
||||
## 完成后再执行一轮
|
||||
|
||||
已完成任务可以点击 **开始新一轮**。工具归档旧计划、命令、快照与执行证据,清空过期位号基线,将同批已完成任务恢复为待配置。
|
||||
|
||||
接着按工程**当前料号**重新填 BOM 或修改规则,重新采集、保存并执行。旧轮次可通过 **查看轮次证据** 打开。出现状态不确定或恢复提示时,先使用 **核验并恢复**,不要直接重复执行。
|
||||
|
||||
## 更新程序与保留数据
|
||||
|
||||
先关闭旧程序并备份 `data`。完整解压新包,用新 EXE 和 `_internal` 配合原 `data` 使用;不要混用不同版本的依赖目录,也不要删除原数据。
|
||||
@@ -110,6 +194,10 @@ DxPartReplace/
|
||||
|
||||
## 验证范围与已知限制
|
||||
|
||||
V1.1.4 Beta 2 新增批次后台资源管理:一次 BOM 导入或顺序执行共用一个私有桌面,CAD 按需启动并在同批工程间复用,批次结束统一退出。若提示 **资源释放待处理**,已保存核验的结果保留,请点击 **重新清理资源**;该操作只清理后台资源,不会再次替换。旧的工程保护锁仍按“核验并恢复”处理。设计、异常边界与实机结果见[批次运行时验收记录](docs/testing/2026-10-21-batch-private-runtime.md)。
|
||||
|
||||
资源释放会追加日志:在 **替换结果 / 日志 → 完整日志** 查看本批开始释放、释放成功/失败及重新清理的结果。独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`,记录批次、桌面、CAD PID、退出码、释放耗时、句柄关闭和桌面消失的分别确认,以及失败原因;同时保留 `session.json` 的原始释放证明。重启后重新检查也会追加记录,不覆盖原失败日志。
|
||||
|
||||
V1.1.3 的业务代码沿用本机已验收候选:两工程实际替换和独立回读、362 字符工程路径、BOM 失败/成功导入、Part Label 写入 Value、可选封装、旧版任务数据副本、新轮次和自有后台资源释放。新增图标及版本元数据另做冻结包检查。详见[验收报告](docs/testing/2026-10-19-release-preparation.md)。
|
||||
|
||||
当前按 **evaluation-only(评估试用)** 交付。不同机器、中央库和 CAD 版本仍需用工程副本确认兼容性;替换回读通过不代表已通过 Packager、ERC 或网表检查。
|
||||
@@ -128,6 +216,7 @@ py -3.12 -m venv .venv
|
||||
|
||||
源码默认数据目录为 `%LOCALAPPDATA%\DxProductivity\trial-task-center`,可用 `DX_TASK_CENTER_HOME` 指定独立目录。运行回归:`python -m pytest -q`;构建使用 `packaging/Build-DxPartReplace.ps1`。
|
||||
|
||||
- [历史窗口 GIF(补充演示,录制于 V1.1.2)](docs/user/media/v1.1.2-window-bom-tutorial.gif)
|
||||
- [完整桌面使用说明](docs/user/v1.1.0-project-tasks.md)
|
||||
- [中断核验与恢复](docs/user/interrupted-recovery.md)
|
||||
- [Value 与属性继承](docs/user/value-copy-and-layout.md)
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"version": "1.1.4-beta.1",
|
||||
"release_tag": "v1.1.4-beta.1",
|
||||
"classification": "beta",
|
||||
"prerelease": true,
|
||||
"code_commit": "e46982af6c3ea145cfa04486a54227ca02ecdd89",
|
||||
"application_entry_matches_source": true,
|
||||
"vbs_match_source": {
|
||||
"QuickPartReplace.vbs": true,
|
||||
"NativeSnapshotReader.vbs": true
|
||||
},
|
||||
"data_directory_excluded": true,
|
||||
"gif_recorded_version": "1.1.2",
|
||||
"acceptance_report": "docs/testing/2026-10-21-batch-private-runtime.md",
|
||||
"build_log": "build/release-1.1.4-beta.1-build.log",
|
||||
"zip_sha256": "acf9f13617b5c25ddfe4c9e4df90481f6ca7838329bca1c201e7c1fdfe27d4a1",
|
||||
"zip_bytes": 60266953,
|
||||
"file_count": 396,
|
||||
"extracted_zip_smoke": true,
|
||||
"exe_sha256": "c0a43c302568206c9bbfc02203daded1df8e19df1d2083c2f0961eddd213a900",
|
||||
"build_verification": {
|
||||
"passed": 822,
|
||||
"warnings": 1,
|
||||
"seconds": 211.19
|
||||
},
|
||||
"frozen_helper_verification": {
|
||||
"passed": true,
|
||||
"version": "1.1.4-beta.1",
|
||||
"windows_numeric_version": "1.1.4.1",
|
||||
"windows_prerelease_flag": true,
|
||||
"frozen_marker_written": true,
|
||||
"stale_marker_rejected": true,
|
||||
"commands": [
|
||||
"probe",
|
||||
"checkpoint",
|
||||
"quit"
|
||||
],
|
||||
"opened_projects": [],
|
||||
"resources_released": true,
|
||||
"observer_errors": []
|
||||
},
|
||||
"business_module_count": 84,
|
||||
"business_modules_all_match_source": true
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
# V1.1.4 Beta 1 构建与发布记录
|
||||
|
||||
源提交:`e46982af6c3ea145cfa04486a54227ca02ecdd89`。本次后续提交只补充构建记录,程序代码不变。版本显示为 `1.1.4-beta.1`,Python 包元数据为等价的 `1.1.4b1`;Windows 数字版本为 `1.1.4.1`,设置 `VS_FF_PRERELEASE`。
|
||||
|
||||
## 本次检查
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 标准打包检查 | 822 passed,211.19 秒;1 条已知 pywinauto COM 模式提示 |
|
||||
| 冻结代码对应源码 | 84 个业务模块及应用入口编译码全部一致,两份 VBS SHA256 一致 |
|
||||
| ZIP 解压 | 完整性检查、逐文件校验和解压后的 EXE 启动/干净退出通过 |
|
||||
| 版本信息 | 数字版本 1.1.4.1 与 Windows 预发布标志通过 |
|
||||
| 冻结保护标记 CLI | 当前尝试写入成功;旧代次被拒绝且原记录不变 |
|
||||
| 冻结 CAD 助手 | 实际调用 ZIP 解压 EXE 的 probe → checkpoint → quit,通过 |
|
||||
| CAD 与桌面释放 | CAD/退出助手码均为 0,批次 released,pending 为空,释放日志保留 |
|
||||
| 主桌面 | Default 保持不变,无自有 CAD 前台/主桌面污染,无观察错误 |
|
||||
| 用户数据 | 包内不含运行数据目录、任务数据库、现场日志或工单附件 |
|
||||
|
||||
本轮冻结助手验证没有打开任何工程,也没有重复替换;真实两工程替换范围和限制沿用[已记录的源码实机验收](../testing/2026-10-21-batch-private-runtime-live.md)。
|
||||
|
||||
初次便携包暂存被隐私目录检查阻止:仓库公开的 `docs/testing/data` 内有三份已提交的验收 JSON,被宽泛的 `data` 名称检查误判。保留失败暂存,调整为明确禁止包根运行数据和所有数据库/日志文件,在新暂存目录重新生成并通过;未放行用户数据。
|
||||
|
||||
## 附件
|
||||
|
||||
- `DxPartReplace-1.1.4-beta.1.zip`:60,266,953 字节;包含程序、完整离线文档、GIF 和空 BOM 模板。
|
||||
- `BOM-template.xlsx`、`window-tutorial-1.1.4-beta.1.gif`、`release-manifest-1.1.4-beta.1.json`、`SHA256SUMS-1.1.4-beta.1.txt`。
|
||||
- ZIP SHA256:`acf9f13617b5c25ddfe4c9e4df90481f6ca7838329bca1c201e7c1fdfe27d4a1`。
|
||||
- [紧凑构建记录](v1.1.4-beta.1-build.json);完整逐文件清单在发布附件的 manifest。
|
||||
|
||||
新发布与四个旧发布都设为 Beta 预发布。旧发布只增加 Beta 标题并调整 prerelease 状态,逐一确认原标签、正文和附件 ID/名称/大小不变。新附件先上传到草稿,逐项回下载核对 SHA256,再转为公开 Beta。
|
||||
|
||||
本机原始证据位于 `.worktrees/batch-runtime/build/release-1.1.4-beta.1`,构建日志为同工作树 `build/release-1.1.4-beta.1-build.log`。
|
||||
@@ -0,0 +1,31 @@
|
||||
# DxPartReplace V1.1.4 Beta 1
|
||||
|
||||
发布日期:2026-10-21。**Beta 预发布版**。此前发布也统一标记为 Beta,原标签、附件及下载地址保留。
|
||||
|
||||
## 下载与升级
|
||||
|
||||
完整解压 `DxPartReplace-1.1.4-beta.1.zip`,运行 `DxPartReplace.exe`。无需安装 Python;本机需要可用的 DxDesigner / Expedition Enterprise 7.9.5、许可证和中央库。
|
||||
|
||||
升级前关闭旧程序并备份 `data`,保留原数据,配套更新 EXE 与 `_internal`。不要让不同版本同时打开同一数据目录;历史保护锁不会自动解除。
|
||||
|
||||
[中文图文教程](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/README.md)。包内包含离线图文教程、GIF、默认 11 列 BOM 模板、构建清单和 SHA256 校验。教程图片及 GIF 沿用已验收的旧版录制,新增的“重新清理资源”与释放日志以本版界面为准。
|
||||
|
||||
## 本版更新
|
||||
|
||||
- 每次 BOM 导入和顺序执行分别作为一个批次,共用一个私有桌面;CAD 按需启动,在批内跨工程复用,批末统一退出。
|
||||
- 工程之间确认上一工程已关闭,重新核验路径、中心库、图页及目标身份;保留保存、关闭、重开回读流程。
|
||||
- 首次可能写入前持久化保护标记。只有明确证明尚未修改且旧资源已释放时,允许重建重试一次;身份冲突、已修改或结果不确定时不自动重放。
|
||||
- 释放失败保留已保存并核验的任务结果,阻止新批次,提供“重新清理资源”;清理操作不重复替换,重启后仍检查未收尾批次。
|
||||
- 释放日志显示开始、成功、失败和重新检查,包含耗时、CAD退出码、句柄关闭与桌面消失的分别确认、失败原因。同批各任务“完整日志”可见,独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`。
|
||||
- 修复 BOM 错误回报与清理错误同时发生时丢失逐行报告的问题;清理失败只保留诊断,不提供可应用的部分预览。
|
||||
- 保留原生 CLI 优先采集,减少不必要的 CAD 回退;原生会话重试耗尽时停止整批。
|
||||
|
||||
## 验证与范围
|
||||
|
||||
两个全新工程副本完成 BOM 导入、实际替换、保存重开、独立完整属性/连接回读和资源释放。两版本保存结果一致;各工程 57 个组件、103 个引脚连接,56 个非目标组件属性和连接不变,源工程及中心库 `.lmc` 哈希不变。
|
||||
|
||||
相同样例单次对照:BOM 加执行总耗时 170.860 → 151.859 秒,私有桌面创建 6 → 2 次,CAD 启动 2 → 1 次。该结果不代表固定加速比例;BOM 单次测量 2.250 → 3.187 秒。
|
||||
|
||||
工单完整库存覆盖的 31 行回放为 20 通过、11 失败,未强行放行不匹配项;其余 406 条替换行、96 个工程未获对应完整库存,不能宣称全部工程已实机通过。用户机器的外部桌面引用残留根因尚未在本机复现。
|
||||
|
||||
详见[批次运行时验收](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-batch-private-runtime.md)、[实机对照](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-batch-private-runtime-live.md)和[释放日志验证](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.1/docs/testing/2026-10-21-resource-release-logging.md)。
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"version": "1.1.4-beta.2",
|
||||
"release_tag": "v1.1.4-beta.2",
|
||||
"classification": "beta",
|
||||
"prerelease": true,
|
||||
"code_commit": "1ee519e5d25e4052f20b7b32ad18dcd837eb948d",
|
||||
"application_entry_matches_source": true,
|
||||
"vbs_match_source": {
|
||||
"QuickPartReplace.vbs": true,
|
||||
"NativeSnapshotReader.vbs": true
|
||||
},
|
||||
"data_directory_excluded": true,
|
||||
"gif_recorded_version": "1.1.2",
|
||||
"build_log": "build/release-1.1.4-beta.2-build.log",
|
||||
"zip_sha256": "21d242d1c4bd015f3b140d01f08e6779be0b6408f4d0deef09b457ec22a00325",
|
||||
"zip_bytes": 60682629,
|
||||
"file_count": 402,
|
||||
"extracted_zip_smoke": true,
|
||||
"exe_sha256": "0247347e33389025e9dab61065040a1589ed14876eaa9db5cf8fabd24604c360",
|
||||
"build_verification": {
|
||||
"passed": 824,
|
||||
"warnings": 1,
|
||||
"seconds": 177.39
|
||||
},
|
||||
"packaging_checks_passed": 2,
|
||||
"acceptance_guard_checks_passed": 15,
|
||||
"business_module_count": 84,
|
||||
"business_modules_all_match_source": true,
|
||||
"frozen_main": {
|
||||
"passed": true,
|
||||
"win32timezone_archived": true,
|
||||
"process_creation_time_read": true,
|
||||
"resources_released": true,
|
||||
"windows_numeric_version": "1.1.4.2",
|
||||
"windows_prerelease_flag": true
|
||||
},
|
||||
"ui_acceptance": {
|
||||
"full_exe_all_program_files_match_zip": true,
|
||||
"template_columns": 11,
|
||||
"imports": [
|
||||
{
|
||||
"file": "BOM-含错误位号.xlsx",
|
||||
"seconds": 9.889999999999418,
|
||||
"title": "本批 BOM 校验未通过 · 通过 2 / 失败 1 / 未检查 0 行 · 详情"
|
||||
},
|
||||
{
|
||||
"file": "BOM-替换RV05.xlsx",
|
||||
"seconds": 4.734000000011292,
|
||||
"title": "本批 BOM 校验通过 · 通过 2 / 失败 0 / 未检查 0 行 · 详情"
|
||||
}
|
||||
],
|
||||
"search_preserves_tasks": true,
|
||||
"execution_seconds": 209.40600000001723,
|
||||
"completed_tasks": 2,
|
||||
"cad_starts": 1,
|
||||
"queue_desktop_generations": 1,
|
||||
"all_batches_released": true,
|
||||
"results_window_reopened": true,
|
||||
"observer_initial_path_mismatch_retained": true,
|
||||
"read_only_post_verification_passed": true,
|
||||
"replacement_replayed": false
|
||||
},
|
||||
"independent_graphs": {
|
||||
"passed": true,
|
||||
"components_each": 57,
|
||||
"pin_connections_each": 103,
|
||||
"unchanged_non_targets_each": 56,
|
||||
"source_hashes_unchanged": true,
|
||||
"library_lmc_hash_unchanged": true
|
||||
},
|
||||
"acceptance_report": "docs/testing/2026-10-23-issue-2-frozen-runtime.md",
|
||||
"old_release": {
|
||||
"tag": "v1.1.4-beta.1",
|
||||
"withdrawn_to_draft": true,
|
||||
"public_release_api_status": 404,
|
||||
"git_tag_retained": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# V1.1.4 Beta 2 构建与发布记录
|
||||
|
||||
日期:2026-10-23。源快照 `1ee519e5d25e4052f20b7b32ad18dcd837eb948d`;后续提交仅补充本次构建记录。
|
||||
|
||||
修复工单 #2 的 pywin32 动态依赖漏打包。版本 `1.1.4-beta.2`,Python 元数据 `1.1.4b2`,Windows 数字版本 `1.1.4.2`,带预发布标志。
|
||||
|
||||
| 验证 | 结果 |
|
||||
| --- | --- |
|
||||
| 发布门禁 | 824 passed,177.39 秒;1 条既有 pywinauto STA COM 警告 |
|
||||
| 额外检查 | 打包配置 2 项,验收脚本 15 项通过 |
|
||||
| 冻结主进程 | 最终 ZIP 解压 EXE 实际读取工作进程创建时间,确认进程退出和桌面释放 |
|
||||
| 源码一致性 | 84 个业务模块、应用入口编译码、两份 VBS 哈希一致 |
|
||||
| 完整 EXE 双工程 UI | 创建任务、默认模板导出、错误/正确 BOM 导入、保存计划、实际替换、保存重开、结果日志重开通过 |
|
||||
| 执行批次 | 209.406 秒;一次 CAD 启动,一个桌面代次;资源最终释放 |
|
||||
| 独立前后对照 | 每工程 57 组件、103 引脚连接,56 个非目标组件属性/连接不变,目标料号和 Value 正确 |
|
||||
| 原始数据 | 源工程目录完整哈希、中央库 .lmc 哈希不变 |
|
||||
| 最终 ZIP | 完整性、解压 smoke、逐文件与已验收 UI 包比对通过;不含运行 data、任务数据库或现场日志 |
|
||||
|
||||
旧外部观察器未扫描批次资源目录造成的验收断言保留在原始记录中。已补齐观察器并完成测试,采用只读续验确认已完成的执行和资源释放,没有再次替换。细节见[工单修复与实机记录](../testing/2026-10-23-issue-2-frozen-runtime.md)。
|
||||
|
||||
ZIP:`DxPartReplace-1.1.4-beta.2.zip`,60,682,629 字节。SHA256:`21d242d1c4bd015f3b140d01f08e6779be0b6408f4d0deef09b457ec22a00325`。
|
||||
|
||||
附件包括便携 ZIP、默认 BOM 模板、GIF、SHA256SUMS 和逐文件 manifest。GIF 沿用 1.1.2 教程,包内说明已标明。
|
||||
|
||||
Beta 1 已撤回为草稿,匿名发布 API 返回 404;其 Git 标签和原附件保留用于回溯。Beta 2 先以草稿上传,回下载核对附件哈希后再公开为 Beta。紧凑证据见[构建 JSON](v1.1.4-beta.2-build.json)。本机原始证据为 `build/release-1.1.4-beta.2`。
|
||||
@@ -0,0 +1,43 @@
|
||||
# DxPartReplace V1.1.4 Beta 2
|
||||
|
||||
发布日期:2026-10-23。**Beta 预发布版**。替代已撤回的 V1.1.4 Beta 1;旧 Git 标签保留供回溯,旧发布不再提供公开下载。
|
||||
|
||||
## 下载与升级
|
||||
|
||||
完整解压 `DxPartReplace-1.1.4-beta.2.zip`,运行 `DxPartReplace.exe`。无需安装 Python;本机需要可用的 DxDesigner / Expedition Enterprise 7.9.5、许可证和中央库。
|
||||
|
||||
升级前关闭旧程序并备份 `data`,保留原数据,配套更新 EXE 与 `_internal`。不要让不同版本同时打开同一数据目录;历史保护锁不会自动解除。
|
||||
|
||||
[中文图文教程](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/README.md)。包内包含离线图文教程、GIF、默认 11 列 BOM 模板、构建清单和 SHA256 校验。教程图片及 GIF 沿用已验收的旧版录制,新增的“重新清理资源”与释放日志以本版界面为准。
|
||||
|
||||
## Beta 2 修复
|
||||
|
||||
- 修复工单 #2(DXT-20261023-A1F7FADF):发布包缺失 `win32timezone`,导致 BOM 导入记录采集进程创建时间时失败。显式打包 pywin32 的动态依赖,无需用户安装 Python。
|
||||
- 发布门禁改为在完整 EXE 主进程中验证私有桌面、子进程启动、创建时间记录及最终释放,防止源码 Python 的完整环境掩盖打包缺失。
|
||||
- 不改变替换、保存回读、保护锁及资源恢复规则。
|
||||
|
||||
## 继承的批次功能
|
||||
|
||||
- 每次 BOM 导入和顺序执行分别作为一个批次,共用一个私有桌面;CAD 按需启动,在批内跨工程复用,批末统一退出。
|
||||
- 工程之间确认上一工程已关闭,重新核验路径、中心库、图页及目标身份;保留保存、关闭、重开回读流程。
|
||||
- 首次可能写入前持久化保护标记。只有明确证明尚未修改且旧资源已释放时,允许重建重试一次;身份冲突、已修改或结果不确定时不自动重放。
|
||||
- 释放失败保留已保存并核验的任务结果,阻止新批次,提供“重新清理资源”;清理操作不重复替换,重启后仍检查未收尾批次。
|
||||
- 释放日志显示开始、成功、失败和重新检查,包含耗时、CAD退出码、句柄关闭与桌面消失的分别确认、失败原因。同批各任务“完整日志”可见,独立文件位于 `data/batch-resources/<批次ID>/resource-release.log`。
|
||||
- 修复 BOM 错误回报与清理错误同时发生时丢失逐行报告的问题;清理失败只保留诊断,不提供可应用的部分预览。
|
||||
- 保留原生 CLI 优先采集,减少不必要的 CAD 回退;原生会话重试耗尽时停止整批。
|
||||
|
||||
## Beta 2 发布验证
|
||||
|
||||
发布门禁 824 项通过;打包配置检查 2 项、外部验收脚本检查 15 项通过。完整 EXE 在两个全新工程副本上完成创建任务、导出默认模板、错误 BOM 报告(2 通过 / 1 失败)、修正后导入(2 通过 / 0 失败)、保存计划、实际替换、保存重开回读及结果日志查看。两工程各 57 个组件、103 个引脚连接,56 个非目标组件不变;源文件与中心库 `.lmc` 哈希不变。执行批次共一次 CAD 启动、一个桌面代次,最终释放已独立确认。
|
||||
|
||||
本轮[工单修复与验收记录](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-23-issue-2-frozen-runtime.md)包含原验证漏检范围及外部观察器修正后的只读续验说明;已完成替换没有重放。
|
||||
|
||||
## 继承的历史验证与范围
|
||||
|
||||
两个全新工程副本完成 BOM 导入、实际替换、保存重开、独立完整属性/连接回读和资源释放。两版本保存结果一致;各工程 57 个组件、103 个引脚连接,56 个非目标组件属性和连接不变,源工程及中心库 `.lmc` 哈希不变。
|
||||
|
||||
相同样例单次对照:BOM 加执行总耗时 170.860 → 151.859 秒,私有桌面创建 6 → 2 次,CAD 启动 2 → 1 次。该结果不代表固定加速比例;BOM 单次测量 2.250 → 3.187 秒。
|
||||
|
||||
工单完整库存覆盖的 31 行回放为 20 通过、11 失败,未强行放行不匹配项;其余 406 条替换行、96 个工程未获对应完整库存,不能宣称全部工程已实机通过。用户机器的外部桌面引用残留根因尚未在本机复现。
|
||||
|
||||
详见[批次运行时验收](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-21-batch-private-runtime.md)、[实机对照](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-21-batch-private-runtime-live.md)和[释放日志验证](https://git.simgreek.com/simgreek/dxdesigner-part-replacement/src/tag/v1.1.4-beta.2/docs/testing/2026-10-21-resource-release-logging.md)。
|
||||
@@ -0,0 +1,22 @@
|
||||
# 批次私有桌面实施计划
|
||||
|
||||
用户已在会话中明确批准完整设计及实施;自动重试策略为仅修改前重试一次。基线 e15ee25,不发布安装包。
|
||||
|
||||
## 实施任务
|
||||
|
||||
- [x] T1:批次运行时。显式传递资源,惰性共用桌面/CAD;项目关闭检查点独立于批末桌面释放;持久化生命周期,资源失败阻止新批次,支持重新清理。正常退出优先,无法确认旧资源释放不重建。
|
||||
- [x] T2:写前持久化。每次尝试有新的身份与代次,首个可能修改调用前 fsync 写入标记;失败禁止写入。仅当前尝试明确尚未写入且身份与只读指纹安全时允许一次重建重试。
|
||||
- [x] T3:入口及界面。BOM/任务队列分别拥有一次运行时;单独操作兼容;跨任务复用 CAD,同工程多步骤继续复用。清理状态独立于已核验任务结果,提供清理入口;原模板和队列可读。
|
||||
- [x] T4:验证与审查。故障注入与取消竞争、旧结果隔离、共享数量和缓存全命中;工单回放及两份真实工程副本替换/保存回读/图连接校验;独立审查并修复,分阶段 Git 提交后推送。
|
||||
|
||||
## 接口约定
|
||||
|
||||
`BatchCadRuntime(root, kind, stop_event=None)` 为上下文管理器,提供 `native_scope(evidence)`、`run_worker(command, timeout, abort_event=None, cooperative_abort=False)`;`root` 是应用数据根。原生 scope 提供桌面式 start_process/close 接口,关闭只回收该次 CLI 子进程。CAD run_worker 仅在需要时启动并保存项目关闭检查点。
|
||||
|
||||
监督端不保存 COM 对象,运行时通过可选 `runtime` 参数显式传递;工作进程中继续核验桌面、CAD PID/GUID 和执行标识。可调用 `guard_pending_resources(root)` 与 `cleanup_pending_resources(root)`,后者仅清理资源,不执行替换。
|
||||
|
||||
写前协议模块独立于 Qt/COM。每个进程尝试写入 `DXPART_ATTEMPT_FILE`,由监督端先落盘准备记录,原生写入前验证并置为 mutation_possible。标记和身份缺失不能证明安全。
|
||||
|
||||
## 验收约束
|
||||
|
||||
所有真实 CAD 写入仅对原仓库 build 下的隔离副本进行,不触碰源工程与中心库。不删除原 feedback 或历史证据,不放宽保护锁。性能按同一副本和相同输入比较,不承诺实机残留根因已治愈。
|
||||
@@ -0,0 +1,73 @@
|
||||
# 批次私有桌面实机验收,2026-10-21
|
||||
|
||||
本机单组对照通过:两个工程的 BOM 导入及一次顺序替换合计从 170.860 秒降至 151.859 秒,减少 19.001 秒,约 11.12%。业务流程的私有桌面数从 6 个降至 2 个,CAD 启动数从 2 次降至 1 次。该时间不包含独立核验器额外启动 CAD 的开销,属于本次对照的实测值。
|
||||
|
||||
## 输入与隔离范围
|
||||
|
||||
- 基线:原仓库 `e15ee25`;候选:`.worktrees/batch-runtime` 的批次运行时实现。
|
||||
- 原始 fixture:原仓库 `build/cbb-bom-import-live-20261015/projects/Audio_D2128` 与 `Audio_T605A`。
|
||||
- 两版本分别复制到全新目录,使用相同源文件及输入;源工程全文件 SHA-256 在结束后保持一致。
|
||||
- 中心库:`F:/Sync Tool Skyworth/Work Maps/Skyworth_CBB_Lib/Mentor_lib_EE7.9/mentor_lib.lmc`,只读使用,`.lmc` SHA-256 不变。
|
||||
- 两工程均只替换 `RV05`,由 `4100-CA0000-T7` 替换为 `4100-CA0000-T0`。`sync_nc=True`,`copy_part_number_to_value=False`,队列策略为 `preserve`。
|
||||
- 证据根:工作树 `build/batch-runtime-live-20261021-144210`,实际基线结果在 `baseline2`,候选在 `candidate`。`comparison.json` 汇总跨版本对照。
|
||||
|
||||
`preserve` 保持原生目标库器件的 Value,而非恢复旧器件文字。此目标的原生 Value 属性缺失,最终期望为 Part Number=`4100-CA0000-T0`、Part Label=`0R`、Value 属性不存在。该语义来自 `docs/superpowers/specs/2026-10-14-v1.1.0-project-tasks-design.md`,与 `QPRestoreComponent`、`QPVerifyValuePolicy` 的存在性及内容核验一致。两版本的最终实际状态均符合该期望。
|
||||
|
||||
## 测量结果
|
||||
|
||||
| 指标 | 基线 | 候选 |
|
||||
|---|---:|---:|
|
||||
| BOM 导入两工程 | 2.250 s | 3.187 s |
|
||||
| 两工程顺序替换、保存、回读和流程收尾 | 168.610 s | 148.672 s |
|
||||
| 上述业务流程合计 | 170.860 s | 151.859 s |
|
||||
| BOM 阶段私有桌面 | 2 | 1 |
|
||||
| 执行阶段私有桌面 | 4 | 1 |
|
||||
| 执行阶段 CAD 启动 | 2 | 1 |
|
||||
|
||||
基线执行阶段含两个只读 native CLI 桌面和两个 CAD 桌面;候选在该阶段共用一个私有桌面和同一个 CAD PID `9192`,generation 为 `1`。独立核验器使用的额外 CAD 会话单独记录在 `desktop_starts`、`cad_starts` 的 `independent_before`、`independent_after` 阶段,不计入表中的业务资源数量。
|
||||
|
||||
## 保存结果与释放核验
|
||||
|
||||
每份工程均通过独立 Python COM 读取器,从空 CAD 开始重新打开磁盘工程,直接读取图页组件、全部属性和引脚网络。读取器不调用生产恢复/替换代码。
|
||||
|
||||
- 两版本替换前的完整独立读取 JSON 完全相等;替换后的完整 JSON 也完全相等,包含目标器件的所有属性。
|
||||
- 每份工程保持 57 个组件身份、103 个引脚连接;56 个非目标器件的全部属性、位置、变换和引脚状态完全不变。
|
||||
- `RV05` 的 UID、Refdes、位置、方向、比例和网络连接保持一致,目标引脚编号非空;Part Number/Part Label/Value 符合上述策略。
|
||||
- 候选两份写前记录具有相同 batch session/generation、不同 attempt ID,并均为 `mutation_possible`。两个逻辑工程分别记录标记。
|
||||
- 候选 BOM 和任务队列资源记录均为 `released`,无自动重试、恢复或已成功步骤的重放。
|
||||
- 观察器确认所有拥有的 CAD 退出、私有桌面消失,输入桌面未切换,拥有的 CAD 未出现在用户桌面或成为前台窗口。
|
||||
- 候选源代码/VBS 清单的 SHA-256 在实机执行期间保持一致。两个工程源副本及中心库 `.lmc` 的 SHA-256 均不变。
|
||||
|
||||
关键证据:
|
||||
|
||||
- `comparison.json`
|
||||
- `candidate/report.json`
|
||||
- `candidate/desktop-observation.json`
|
||||
- `candidate/source-under-test-manifest.json`
|
||||
- `candidate/data/batch-resources/*/session.json`
|
||||
- `candidate/data/tasks/*/quick-run/batch-attempts/*/*.json`
|
||||
- `baseline2/read-only-verification/saved-verification.json`
|
||||
- `baseline2/{before,after}-Audio_D2128.json`、`baseline2/{before,after}-Audio_T605A.json` 及 candidate 对应文件
|
||||
|
||||
初次探针在 `baseline` 目录因属性 COM 对象未强制转为 `IVdAttr` 而停止,尚未执行替换。修复读取器后使用全新 `baseline2` 副本执行。`baseline2` 两次替换均已完成,随后旧核验断言误把默认关闭复制选项视为 Part Label 复制;保留原错误报告,依据既有 Value 策略修正独立预期,再仅补充缺少的磁盘读取。该补充过程没有调用队列、恢复或替换。原始错误记录和历史输出均保留。
|
||||
|
||||
## 复现
|
||||
|
||||
`tools/probe_batch_runtime_live.py` 使用服务层 BOM 导入和任务队列入口,自动创建全新工程副本,无恢复/重放选项。`--repo` 指向被测版本,工具本身保持相同。
|
||||
|
||||
```powershell
|
||||
$batchProbeRepo = 'F:/Sync Tool Skyworth/Work Maps/dxdesigner-part-replacement'
|
||||
$batchProbeWorktree = "$batchProbeRepo/.worktrees/batch-runtime"
|
||||
$batchProbeSources = "$batchProbeRepo/build/cbb-bom-import-live-20261015/projects"
|
||||
$batchProbeLibrary = 'F:/Sync Tool Skyworth/Work Maps/Skyworth_CBB_Lib/Mentor_lib_EE7.9/mentor_lib.lmc'
|
||||
$batchProbeOutput = "$batchProbeWorktree/build/batch-runtime-live-NEW-TIMESTAMP"
|
||||
$batchProbeBaseline = "$batchProbeWorktree/build/baseline-e15ee25"
|
||||
|
||||
git -C $batchProbeRepo worktree add --detach $batchProbeBaseline e15ee25
|
||||
python "$batchProbeWorktree/tools/probe_batch_runtime_live.py" --repo $batchProbeBaseline --source-root $batchProbeSources --library $batchProbeLibrary --output "$batchProbeOutput/baseline" --run-native
|
||||
python "$batchProbeWorktree/tools/probe_batch_runtime_live.py" --repo $batchProbeWorktree --source-root $batchProbeSources --library $batchProbeLibrary --output "$batchProbeOutput/candidate" --run-native
|
||||
```
|
||||
|
||||
基线须固定到 `e15ee25` 的独立目录;合入后原仓库已经是新实现,不能继续把原仓库当前 HEAD 当作旧基线。上述输出及基线目录均应使用尚不存在的新路径。
|
||||
|
||||
仅在两个任务均已完成而独立核验需补读时,可使用 `--verify-existing <已有输出目录>`。该分支只读已保存工程,不执行任务队列;核验结果写入独立的 `read-only-verification` 目录,且拒绝覆盖已有核验目录。
|
||||
@@ -0,0 +1,84 @@
|
||||
# 批次私有桌面复用与安全重建验收
|
||||
|
||||
本次仅更新源码和 Git,不修改 V1.1.3 发布包。实施基线为 `e15ee25`。
|
||||
|
||||
## 已实施的边界
|
||||
|
||||
一次 BOM 导入、一次顺序执行分别拥有一个 `BatchCadRuntime`。库存缓存全部命中时不创建 Windows 资源;原生 CLI 共用批次桌面,各自进程及 Job 及时回收。需要 CAD 时才启动,同批工程串行复用该实例。工程之间由独立子进程核对桌面、CAD GUID/PID 和活动工程路径为空;每个工程使用新的 VBS 工作进程,组件、引脚及库查询缓存不跨工程传递。原有保存、关闭、重开回读继续执行。
|
||||
|
||||
批次标识、代次、工程/任务/作业和尝试标识保存在独立 JSON 旁文件中,旧 BOM、TSV 和已保存任务格式保持兼容。启动下一次工作前先作废该证据目录的旧关闭证明。监控消息有自己的随机标识,旧代次的资源租约不能启动新进程或把迟到关闭结果登记为新代次。
|
||||
|
||||
原生写入、恢复属性及保存之前,工作进程先验证完整尝试身份并持久化 `mutation_possible`。落盘失败禁止调用写入。修改前自动重试仅一次,还要求整个工程此前所有作业都处于 `prepared`、身份匹配、状态文件未改变、工程完整指纹一致且旧资源已确认释放。当前未把“数据库文件单独变化”视为自动重试许可;缺少独立身份等价证据时保守停止。明确确认的恢复操作不自动重试。
|
||||
|
||||
原生 CLI 仅操作临时数据库副本。启动失败、超时或特定崩溃的重试还要核对当前租约已经关闭、当前批次/代次/尝试一致,以及源 PRJ/数据库 SHA256 未变化。普通表格、匹配、解析和封装错误不触发桌面重建。
|
||||
|
||||
原生会话重试耗尽或预算拒绝时停止整批,不能再回退 CAD 绕过异常。该分支在实机正常路径验收后新增四项异常注入并修复,普通非零退出与格式不支持的兼容回退保留。
|
||||
|
||||
身份冲突单独分类:bootstrap、空工程检查点的绑定损坏/不一致,以及旧 VBS 明确报告的私有 CAD GUID 冲突,都不会被归入普通启动未就绪而自动重建。子进程用专用退出码保留该类别,异常包裹也保留不可重试的原因。
|
||||
|
||||
每个成功释放的代次另外保存在 `generation_releases`,含桌面、CAD PID/GUID、清理证明与释放时间。重建不会覆盖旧代次的释放证据;发布失败后重新检查也不会重复追加成功记录。
|
||||
|
||||
## 资源清理与 Windows 契约
|
||||
|
||||
- [CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop):关闭自有句柄与确认桌面对象消失分开检查。仍有外部引用时不创建更多桌面绕过问题。
|
||||
- [Job Objects](https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects):工作进程和后代位于自有 Job;正常退出优先,必要终止限定于自有进程及 Job。
|
||||
- [TerminateProcess](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminateprocess):终止请求后等待实际退出,随后回收句柄。
|
||||
- 监督进程不连接 CAD COM,不调用 `SwitchDesktop`。工作进程通过 `STARTUPINFO.lpDesktop` 启动;不改变用户输入桌面。
|
||||
|
||||
`batch-resources/<批次>/session.json` 保存生命周期、进程创建时间、工程关闭证明、重建原因和耗时。批末清理失败独立报告“资源释放待处理”,保留已核验任务状态,阻止后续 CAD 入口。界面提供“重新清理资源”,仅清理/重新检查资源,不重放替换。BOM 错误仍保留逐行诊断,但移除可应用的预览。
|
||||
|
||||
应用重启后只能重新确认已记录进程和桌面消失;不会根据旧 PID 终止新进程、接管 COM 或自动解除历史工程保护锁。损坏或未知格式的所有权记录保持阻塞,不能把缺失进程列表解释成没有资源。清理过程中即使日志写入失败也继续回收自有资源;释放记录发布失败可再次检查并落盘。取消请求不能覆盖清理失败。
|
||||
|
||||
残留时保留窗口 PID/TID/类和查询错误,进一步线程诊断见[Windows 桌面研究](2026-10-21-windows-desktop-lifecycle.md)。没有查询到窗口或线程,不等于不存在外部句柄持有者。
|
||||
|
||||
新增真实 Windows 对象寿命回归:测试自己的独立进程连接批次桌面,批次清理确实进入 blocked 并保留 completed_result;该外部进程仍存活、用户输入桌面不变,也没有生成工程 uncertain.flag。测试关闭自己拥有的独立进程后,资源入口确认 released,generation 仍为 1,没有创建新桌面或重放任务。
|
||||
|
||||
## 回归范围
|
||||
|
||||
针对性注入包含:惰性零启动、多工程一个 CAD、关闭证明作废、一次重建和第二次失败、修改后禁止重放、写前记录失败、任务边界暂停竞争、资源清理失败优先、释放日志发布失败后重新检查、损坏记录、跨重启检查、旧代次租约及检查点拒绝。独立审查发现的关闭证明过期、重复退出首次启动失败实例、独立服务入口绕过门禁、BOM 清理失败丢报告、损坏资源记录误解除等问题均已修复。
|
||||
|
||||
全量检查发现一个基线已有的取消测试:假执行器仅声明清理完成,却没有写出必要的释放证据。已在原基线隔离复现,并修正测试的释放证据,生产校验条件没有放宽。
|
||||
|
||||
扩展旧版本回归还发现两处既有边界:旧队列冻结 Value 策略发生冲突时,原先会先访问工程进行备份,导致真正的冲突被路径错误覆盖。现在以只读检查先拒绝冲突,断言拒绝前不备份、不启动 CAD、证据不变;备份仍先于 CAD 采集。千步骤模拟测试的固定 30 秒限制在基线也失败(41.83 秒,峰值 2.35 MiB),改为直接约束命令/状态全量解析各不超过两次,保留正确性、128 MiB 内存上限及实际耗时记录,没有调高秒数来掩盖回归。
|
||||
|
||||
最终统一执行 `python -m pytest tests/v1_1 tests/trial_task_center -q --basetemp=build/pytest-batch-final-20261021`:**1104 passed、1 failed**,327.33 秒。唯一失败是旧测试要求工作进程路径与长路径字符串逐字相等;本次把证据目录放到较长的 build 路径后,合法的 Windows 短路径触发该断言。生产传参指向正确文件,未改生产代码;将测试改为核对 canonical 文件/目录身份并继续严格核对其余参数,整个 `test_quick_run.py` 在同样较长路径下复测 **34 passed**。两次执行合计覆盖全部 1105 个不同用例,没有未修复的失败;未为测试断言修改重复执行已成功的真实替换。
|
||||
|
||||
最终千步骤模拟指标:20 个工程 × 5 张图页 × 10 条规则,91 保存、909 未命中,全部作业唯一;32.147 秒、峰值 2.05 MiB,命令及状态全量解析各 **1 次**。这是注入工作进程的控制层压力测试,不能作为实际 CAD 替换吞吐量。
|
||||
|
||||
## 实机与工单范围
|
||||
|
||||
实机验收仅对两个全新工程副本进行;源工程、中心库和原始工单附件不修改。分别核对 BOM 导入、替换、保存重开、独立回读、全部非目标元件属性与引脚连接、最终资源释放及原输入桌面。性能按相同样例对照单次运行,不代表大规模实机稳定性或固定加速比例。
|
||||
|
||||
工单只有六个工程有完整已捕获原生库存,回放与真实 CAD 验收分别报告,不能据此宣称全部 102 个工程均已实机通过。用户机器的桌面残留根因尚未在本机复现,本次减少重复创建/释放并补齐恢复边界,没有宣称根因已根治。
|
||||
|
||||
### 两工程实机对照
|
||||
|
||||
完整过程与复现命令见[实机对照报告](2026-10-21-batch-private-runtime-live.md)。计时只包含 BOM 导入和顺序执行,不包含独立检查器额外打开 CAD 的时间。
|
||||
|
||||
| 指标 | 基线 | 候选 |
|
||||
| --- | ---: | ---: |
|
||||
| BOM 导入 | 2.250 秒 | 3.187 秒 |
|
||||
| 替换、保存回读、执行收尾 | 168.610 秒 | 148.672 秒 |
|
||||
| 上述总耗时 | 170.860 秒 | 151.859 秒 |
|
||||
| 私有桌面创建 | 6 | 2 |
|
||||
| CAD 启动 | 2 | 1 |
|
||||
|
||||
本次样例节省 19.001 秒(11.12%),BOM 阶段单次测量略慢。两个工程各保持 57 个组件、103 个引脚连接,56 个非目标组件全部属性及连接不变;两版本独立回读的完整结果一致。只有 RV05 更换到目标料号,RV11 保持旧料号。批次均记录 released,未自动重试,输入桌面未切换,源工程和中心库 `.lmc` 哈希不变。
|
||||
|
||||
正常替换验收之后,最终异常分类及释放历史补丁 `a8358d9` 又做了一次空 CAD 实机冒烟:probe、空工程 checkpoint、quit 全部通过,退出码均为 0,批次 released、pending 为空,CAD/桌面消失,输入桌面仍为 Default。该次实测 59.281 秒,没有打开任何工程,也没有重复替换。证据在工作树 `build/batch-runtime-idle-smoke-20261021-final`。
|
||||
|
||||
### 工单只读回放
|
||||
|
||||
使用[回放工具](../../tools/replay_issue_bom_report.py)复制工单任务库到新的 build 目录,再通过最新 `inspect_batch` 回放完整库存覆盖的原 Excel 行,保留原行号与工程归属。
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 工单原始范围 | 428 个任务;763 行,其中 437 条替换行、102 个工程 |
|
||||
| 已捕获完整库存 | 8 份报告、6 个工程,覆盖 31 条替换行 |
|
||||
| 最新逐行结果 | 20 通过、11 失败,工程/Excel 行号/状态与原报告一致 |
|
||||
| 失败行 | Excel 3、4、6、7、11、12、13、16–19 行 |
|
||||
| 回放副作用 | 0 CAD/桌面/进程创建,0 目标计划写入,复制数据库未变化 |
|
||||
| 原证据 | ZIP、BOM、原生库存哈希不变 |
|
||||
| 未回放 | 406 条替换行、96 个工程,没有相应完整已捕获库存 |
|
||||
|
||||
完整本机回放证据位于原仓库 `build/issue-1-batch-runtime-replay-final-20261021`。这些是报告回放,不能代替对应用户工程的实际替换验收。
|
||||
@@ -0,0 +1,47 @@
|
||||
# 工单 #1:大量工程 BOM 导入中断
|
||||
|
||||
2026-10-21,基于 V1.1.3 修复,未更新发布包。
|
||||
|
||||
工单:https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/1
|
||||
|
||||
## 附件事实
|
||||
|
||||
工单包含应用数据和原模板格式的 BOM。任务库有 428 个草稿任务;BOM 共 763 条数据行,其中 437 条填写了替换内容,涉及 102 个工程,其余 326 条为空白占位。未发现因工程数量触发的固定上限。
|
||||
|
||||
日志显示四次导入均因 CAD 采集后的私有桌面释放确认失败而整体中断;四次已完成的采集均有 `uncertain.flag`。另一次采集在打开工程阶段收到取消,附件没有最终收尾记录,不能据此判断是否成功释放。
|
||||
|
||||
原生清单回退 CAD 的一个原因是:`1PEF7071_PHY` 的一个非本次替换器件,展开报告中的显示位号和 `Ref Designator` 属性不同。类似情况也出现在 `2G5PHY_GPY211` 的非目标测试点。旧代码因此拒绝整个工程的原生库存。另一次回退源于数据库拒绝读取,该权限问题没有被认定为已修复。
|
||||
|
||||
## 修复行为
|
||||
|
||||
- 原生报告仍检查全部 UID、配置、层次路径、器件范围和属性唯一性;只有本次请求料号的位号差异才要求 CAD 核验。没有猜测或改写存在差异的目标位号。
|
||||
- CAD 读取失败,且有明确的会话释放证据时,记录该工程失败并继续检查后续工程。许可由专用异常承载,不能仅凭错误码或保护标记不存在来继续。
|
||||
- 未确认释放、未知操作异常等仍中止后续工程,但附带完整逐行报告,区分通过、失败和未检查;不生成部分预览、不写入部分计划。
|
||||
- 状态机在失败后保留报告,重试时清空旧报告。取消不能覆盖未经确认的清理失败。图页证据解析和缓存错误仍保守中止。
|
||||
- 桌面释放失败时额外记录残留窗口的 PID、线程和窗口类,查询失败则记录诊断错误。不会切换输入桌面或终止外部进程;释放判定和保护锁条件不变。
|
||||
|
||||
## 验证
|
||||
|
||||
| 项目 | 结果 |
|
||||
| --- | --- |
|
||||
| 工单 8 份完整原生报告回放 | 全部可解析,包括原先回退的 4 份报告 |
|
||||
| `1PEF7071_PHY` 目标核对 | 35 个目标与附件三份 CAD 清单的原理图、图页、UID、位号、料号和 Value 完全一致 |
|
||||
| 原生目标位号存在差异 | 仍拒绝原生库存,要求 CAD 核验 |
|
||||
| 本地既有工程副本的新鲜原生读取 | 两个工程分别 1.418 秒、1.596 秒,原生资源释放通过 |
|
||||
| 本机空 CAD 会话和完整 CAD 采集尝试 | 桌面均正常释放;完整采集因打开图页失败而拒绝,没有执行替换 |
|
||||
| 实际异步界面 | 失败详情保留 → 重试旧结果清空 → 成功预览;计划未提前写入 |
|
||||
| Windows 外部桌面引用 | 无窗口和有窗口均保持阻断;窗口场景识别到实际外部 PID,未终止其进程 |
|
||||
| BOM、原生库存、工程采集、状态机、桌面回归 | 114 passed,34.30 秒 |
|
||||
| 取消、隔离 CAD、采集缓存回归 | 63 passed,12.32 秒 |
|
||||
|
||||
独立代码审查发现并修复了“清理失败且保护标记写入失败时错误继续”和“同时取消掩盖未确认释放”两处边界,最终复核通过。审查者另行运行 38 项针对性检查通过。
|
||||
|
||||
工单原生报告中只有 6 个不同工程的完整库存。按原 BOM 回放其 31 条替换行,20 条通过,11 条无法匹配对应旧料号和位号:Excel 第 3、4、6、7、11、12、13、16、17、18、19 行。这些行没有被强行导入;其中第 16–19 行对应库存没有请求的旧料号。
|
||||
|
||||
## 限制与后续实机复核
|
||||
|
||||
本地相应源工程只有 10 个,版本与用户机器不完全相同;尚未对用户全部 102 个工程完成真实库存核验。本地新鲜读取耗时不能作为用户全部工程的加速比例。
|
||||
|
||||
用户机器的 CAD 桌面残留尚未在本机复现,不能宣称泄漏根因已解决。此修复减少已证实不必要的 CAD 回退,并补齐失败报告和诊断;后续需在实机重新导入,核对是否还产生 `inventory_cleanup` 及新的桌面诊断。保留原有锁、日志和附件,不自动解除历史保护锁。
|
||||
|
||||
原始工单附件仅在 `build/issue-1-20261021` 保留用于本地诊断,不纳入 Git;`feedback/` 未修改。工程打开尝试仅使用 `build/` 副本,没有执行 Replace Part 或发布新安装包。
|
||||
@@ -0,0 +1,21 @@
|
||||
# 资源释放日志
|
||||
|
||||
在批次资源复用实现上补齐可读历史,不改变资源释放条件、修改前重试规则或任务结果。
|
||||
|
||||
## 查看方式
|
||||
|
||||
- 界面:同批各任务的“替换结果 / 日志 → 完整日志”。
|
||||
- 独立文件:`data/batch-resources/<批次ID>/resource-release.log`。
|
||||
- 汇总诊断:`data/logs/application.log`;原始所有权证明仍为旁边的 `session.json`。
|
||||
|
||||
每次追加开始和最终结果,涵盖批次结束、安全重建、手动重新清理和重启后重新检查。记录批次ID、代次、桌面、CAD PID/GUID、退出码、耗时、句柄关闭、桌面消失、原证明路径及失败原因。不存在 CAD 绑定的历史记录不会被猜成“已正常退出”。缓存全部命中且未获取资源时,不虚构释放成功。
|
||||
|
||||
所有同批任务在启动前绑定日志目标;任务ID也写入批次记录,因此应用重启后的重新检查仍能关联原任务。重复点击已经成功的清理不会重复执行或追加伪造的新释放。读日志和重新清理均不执行工程替换。
|
||||
|
||||
日志写入错误由诊断层隔离:独立文件、各任务事件和应用日志分别尝试,单处失败仍继续其他输出;能落盘时记录 `release_log_errors`。即使日志文件被锁定或整个诊断器抛异常,也不能跳过进程等待/桌面回收、把未释放资源误判成功,或把已核验任务降为失败。所有输出位置均不可写时无法保证留下文本日志,原有资源释放证明及保护规则仍优先执行。
|
||||
|
||||
## 验证
|
||||
|
||||
新增七项验证先复现缺少日志功能,再实现通过,覆盖成功、外部持有失败后重试、多任务显示、日志写入失败、重启关联、CAD退出码及缓存零资源。既有真实 Windows 对象寿命测试增加日志断言:独立进程仍连接桌面时记录句柄关闭但桌面未消失的失败;该进程退出后重新清理记录成功。未切换输入桌面、未终止外部持有者、没有重放任务。
|
||||
|
||||
日志、Windows资源寿命、BOM、顺序任务、服务保护和界面入口针对性检查:63 passed,13.74秒。原有日志、服务、恢复及批次生命周期回归:72 passed,12.76秒。本次没有执行真实器件替换,没有打包。
|
||||
@@ -0,0 +1,67 @@
|
||||
# Windows 桌面生命周期:工单 #1 的继续排查
|
||||
|
||||
2026-10-21。此次没有修改生产替换流程、保护锁或发布包。
|
||||
|
||||
## 工单能证明什么
|
||||
|
||||
工单四次整批 BOM 导入的直接中断原因是 `inventory_cleanup`:受控 CAD、已知子进程退出,自有句柄关闭,但五秒后仍枚举到该私有桌面。它不能证明 Windows 因工程数量多而拒绝导入,也没有足够信息证明残留持有者是谁。BOM 行匹配失败是另外一类结果,详见 [首轮诊断](2026-10-21-issue-1-bom-import.md)。
|
||||
|
||||
## 对照微软官方文档
|
||||
|
||||
1. [CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop) 关闭一个打开的桌面句柄。成功不等于所有引用消失。调用进程的线程正在使用这个句柄,或它是初始桌面句柄时,关闭会失败。
|
||||
2. [GetThreadDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getthreaddesktop) 返回指定线程关联的桌面句柄;文档明确不要对该返回值调用 `CloseDesktop`。它与应用通过 `OpenDesktop` 自己打开的句柄归属不同。
|
||||
3. [Thread Connection to a Desktop](https://learn.microsoft.com/en-us/windows/win32/winstation/thread-connection-to-a-desktop) 说明线程的桌面连接、`STARTUPINFO.lpDesktop` 和继承句柄的关系。线程连接不要求存在可见窗口;不能仅凭窗口列表为空判断没有持有者。
|
||||
4. [Terminating a Process](https://learn.microsoft.com/en-us/windows/win32/procthread/terminating-a-process) 明确进程终止不会自动终止其创建的全部子进程。[TerminateProcess](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminateprocess) 是异步请求,必须等待退出。现有实现已有 Job、等待与句柄关闭逻辑;本轮未直接把它认定为错误或移除检查。
|
||||
5. [Traversing the Thread List](https://learn.microsoft.com/en-us/windows/win32/toolhelp/traversing-the-thread-list) 给出 Toolhelp 线程快照遍历方法。[CreateToolhelp32Snapshot](https://learn.microsoft.com/en-us/windows/win32/api/tlhelp32/nf-tlhelp32-createtoolhelp32snapshot) 要求关闭快照句柄。
|
||||
6. [CreateDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createdesktopw) 说明桌面堆限制。工单已有日志没有桌面创建失败或内存不足的证据,因此本轮没有改注册表的 `SharedSection`、全局资源配额或超时常量。
|
||||
|
||||
这些接口没有提供“按桌面名称强制删除全部引用”的安全操作。不能把关闭自己的句柄解释成强行删除其他进程持有的桌面。
|
||||
|
||||
## 原生受控实验
|
||||
|
||||
实验仅使用临时 Python 进程和 `dx-doc-*` 桌面,不打开真实工程。
|
||||
|
||||
| 实验 | 结果 |
|
||||
| --- | --- |
|
||||
| 无额外引用,GUI 进程正常退出 | 进程清理后桌面立即消失 |
|
||||
| 额外持有已退出进程的内核句柄 | 桌面正常释放,不能将进程句柄残留直接当作本案根因 |
|
||||
| 额外持有已退出线程的内核句柄 | 桌面正常释放 |
|
||||
| 子进程线程仍连接私有桌面,但不创建窗口 | 复现相同保护现象;窗口清单为空,线程扫描能找到实际 PID/TID/程序路径 |
|
||||
| 该外部线程的进程退出后 | 再次核验桌面正常释放 |
|
||||
| 对当前线程正在使用的桌面调用 CloseDesktop | 真实返回 WinError 170;恢复原线程桌面后重试成功,pywin32 没有吞掉失败或丢失重试句柄 |
|
||||
|
||||
实际受控残留实验还发现该会话的 `conhost.exe` 线程。此结果只证明诊断能看到无窗口持有者,不能证明用户现场也是 `conhost.exe` 导致。
|
||||
|
||||
实验记录在 `build/desktop-doc-research-20261021`:`retained-kernel-handles.json`、`close-api-contract.txt`、`retained-thread-session.json`、`retained-thread-diagnostic.json`、`released-after-owner-exit.json`。原始机器日志未修改。
|
||||
|
||||
## 新增只读诊断工具
|
||||
|
||||
源码工具:[diagnose_private_desktop.py](../../tools/diagnose_private_desktop.py)。需要在**发生问题的机器、桌面仍残留时**运行,事后复制到另一台电脑的旧日志不能还原已消失的 Windows 资源。
|
||||
|
||||
在仓库根目录使用已安装项目依赖的 Python:
|
||||
|
||||
```powershell
|
||||
python tools/diagnose_private_desktop.py --record '实际失败会话/private-desktop.json' --output 'build/desktop-diagnostic.json'
|
||||
```
|
||||
|
||||
也可以根据日志中的真实名称传入 `--desktop dx-实际标识`。同路径或硬链接形式的输出覆盖输入记录均被拒绝。
|
||||
|
||||
工具输出桌面是否仍被枚举、残留窗口、连接该桌面的线程 PID/TID/可执行程序路径,以及无法查询的线程错误统计。目标桌面查询句柄只在短生命周期子进程中打开;扫描子进程退出后再检查桌面存在性。借用的 `GetThreadDesktop` 句柄不关闭,自有进程查询句柄和快照关闭。不调用 `SetThreadDesktop`、`SwitchDesktop`、`TerminateProcess`,不关闭外部句柄。
|
||||
|
||||
实测 Windows 对另一进程的 `GetThreadDesktop` 查询,在仅打开 `DESKTOP_READOBJECTS`、`READOBJECTS|ENUMERATE`、`READOBJECTS|WRITEOBJECTS` 时仍返回访问拒绝,而打开拥有者 `GENERIC_ALL` 句柄后可以查询。因此仅在扫描子进程中请求该权限,仍只执行查询;权限不足会报错,不能当作“没有持有者”。
|
||||
|
||||
**工具的边界:**进程若只保留一个 `OpenDesktop` 句柄,却没有线程连接或窗口,线程/窗口扫描可能找不到它。空扫描不是释放证明;`desktop_still_listed` 仍独立报告。
|
||||
|
||||
这种情况下可使用微软官方 [Sysinternals Handle](https://learn.microsoft.com/en-us/sysinternals/downloads/handle),按其文档在管理员终端查询所有类型的句柄:
|
||||
|
||||
```powershell
|
||||
handle64.exe -a 'dx-实际标识'
|
||||
```
|
||||
|
||||
这里只查询,不使用 `-c` 关闭外部句柄。该工具由现场人员按微软文档运行,本轮未下载执行、接受许可或更改系统配置。拿到真实持有 PID 后,才能区分自有程序的生命周期问题、外部进程引用和资源释放延迟,并针对已确认的所有者修复。
|
||||
|
||||
## 验证与当前结论
|
||||
|
||||
原生测试验证了无窗口线程识别、外部进程不受影响、输入桌面不切换、输入记录不修改、诊断不遗留句柄,以及无匹配线程时仍报告桌面存在。四项工具测试通过;已有 Windows 桌面测试另行通过。
|
||||
|
||||
用户现场的持有者尚未查实,不能宣称私有桌面残留已经修好。此次交付是微软文档核对、排除实验和能在现场定位到进程/线程的独立诊断工具;没有删除保护锁、强杀不明进程或放宽准确性校验。
|
||||
@@ -0,0 +1,40 @@
|
||||
# 工单 #2:冻结主进程依赖与实机回归
|
||||
|
||||
日期:2026-10-23。对应修正版:V1.1.4 Beta 2。
|
||||
|
||||
## 根因与复现
|
||||
|
||||
工单:https://git.simgreek.com/simgreek/dxdesigner-part-replacement/issues/2 。附件 `application.log` 的 SHA256 为 `f81f4ee602089e85cf98f6f8c64dcb4f5325885573b1b210d97f43e4ba3d98ef`。
|
||||
|
||||
Beta 1 在 `inspect_bom_batch → NativeInventoryReader → _DesktopLease.start_process → BatchCadRuntime._track → process_identity → win32process.GetProcessTimes` 抛出 `ModuleNotFoundError: No module named 'win32timezone'`。pywin32 的 C 扩展在转换进程创建时间时动态导入该模块,PyInstaller 未自动收集。已核对与公开 Beta 1 哈希一致的 EXE(`c0a43c302568206c9bbfc02203daded1df8e19df1d2083c2f0961eddd213a900`),其 PYZ 中确实没有该模块;独立进程阻断该导入后调用 GetProcessTimes 可复现。
|
||||
|
||||
附件显示尚未启动 CAD,未进入替换;清理成功,桌面句柄关闭与桌面消失分别确认,耗时 1.281 秒。现有证据不支持归因为 BOM 内容或多工程私有桌面泄漏。
|
||||
|
||||
原发布验证用源码 Python 监督打包后的 COM 子进程,GetProcessTimes 实际运行在依赖完整的源码环境,未覆盖冻结主进程,这是本次漏检的边界。
|
||||
|
||||
## 修复与门禁
|
||||
|
||||
- 在 spec 中显式加入 `win32timezone`。
|
||||
- 冻结 EXE 的 `--smoke-test` 实际创建批次私有桌面,启动自有短命进程,经生产 `_track` 读取创建时间,再释放工作进程和桌面。不打开 CAD 或工程。
|
||||
- `tools/verify_packaged_runtime.py` 清除源码 Python 环境变量,以完整 EXE 验证上述链路及持久化释放证明;构建脚本必须通过该门禁。
|
||||
- 旧版 EXE 在新增门禁中失败:`Missing win32timezone in frozen archive`。新版 EXE 通过,Windows 版本为 1.1.4.2,带预发布标志。
|
||||
- 回归覆盖模拟缺依赖时仍完成资源释放。验收脚本检查每个原生采集操作的批次归属及最终释放,支持批次保留桌面的正常语义。
|
||||
- 不改变替换规则、写入前保护标记、人工恢复边界或已有保护锁。
|
||||
|
||||
## 自动验证
|
||||
|
||||
发布构建门禁:824 passed,177.39 秒;1 条既有 pywinauto STA COM 警告。打包配置检查另有 2 项通过;外部验收脚本防误判检查 15 项通过。新版本在无 PYTHONPATH/PYTHONHOME 的冻结主进程中完成进程创建时间读取和资源释放。
|
||||
|
||||
## 实机验证范围
|
||||
|
||||
使用 Audio_D2128、Audio_T605A 的全新副本,通过 Windows 界面操作完整 EXE。原始工程和中央库不用于写入。证据目录为 `build/release-1.1.4-beta.2/ui-acceptance`。
|
||||
|
||||
工程任务创建、导出默认 11 列模板、填写并导入错误/正确 BOM、保存计划已通过。错误表为通过 2 / 失败 1 / 未检查 0;正确表为通过 2 / 失败 0 / 未检查 0。保存前数据库没有部分计划,保存后每工程仅 RV05 启用,RV11 保留。搜索过滤不改变任务范围。BOM 使用原生 CLI,无 CAD 启动;两个工程共用一个桌面,释放已确认。第二次导入缓存命中,没有新增桌面。
|
||||
|
||||
两个工程均已通过完整 EXE 的实际替换、保存关闭及重新打开核验。队列耗时 209.406 秒,同一代桌面、一次 CAD 启动;未重建、未重放。结果窗口可关闭并重新打开,显示 RV05、新料号及已保存核验状态。
|
||||
|
||||
旧外部观察器遗漏 `data/batch-resources/*/session.json`,导致任务完成后的验收断言失败。已补齐扫描路径并以回归验证;原失败记录保留。未重新执行替换,而是读取完成事件、队列/批次记录,并用 Windows API 独立确认 CAD 已退出、桌面消失。结合原观察器 943 次采样的前台 PID 和输入桌面窗口记录确认,执行 CAD 未成为前台或进入输入桌面。只读续验结果为 `completed-execution-verification.json`。
|
||||
|
||||
独立源属性/连接读取器分别在替换前后打开工程副本:每工程 57 个组件、103 个引脚连接;56 个非目标组件的身份、位置、方向、属性及连接完全相同。目标 RV05 的料号由 4100-CA0000-T7 变为 4100-CA0000-T0;启用 Part Label → Value,回读均为 0R;目标位置、方向和网络连接保持一致。源工程副本来源目录的完整文件哈希及中心库 `.lmc` 哈希不变。完整 EXE 无 ERROR 日志,窗口正常退出。
|
||||
|
||||
本次不是全部工单工程实机覆盖,也不据此承诺所有机器上的桌面残留根因均已消除。原始缺依赖故障已在完整 EXE 主进程、双工程原生采集和替换路径上覆盖。
|
||||
|
After Width: | Height: | Size: 150 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 63 KiB |
|
After Width: | Height: | Size: 261 KiB |
|
After Width: | Height: | Size: 97 KiB |
|
After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 142 KiB |
|
After Width: | Height: | Size: 296 KiB |
|
After Width: | Height: | Size: 163 KiB |
|
After Width: | Height: | Size: 73 KiB |
|
After Width: | Height: | Size: 44 KiB |
|
After Width: | Height: | Size: 109 KiB |
@@ -1,8 +1,8 @@
|
||||
# DxPartReplace V1.1.2 中文使用说明
|
||||
# DxPartReplace V1.1.3 中文使用说明
|
||||
|
||||
V1.1.0 沿用原有任务视图和料号规则编辑,增加按工程拆分的任务、按位号选择的替换计划及可选 Excel 操作。
|
||||
保留原有任务视图和料号规则编辑,支持按工程拆分任务、按位号选择替换目标及可选 Excel 操作。
|
||||
|
||||
[窗口 GIF 操作教程](media/v1.1.2-window-bom-tutorial.gif):创建任务、导出/导入 BOM、错误行提示、保存、顺序执行、查看结果和开始新一轮。仅含工具窗口及其弹窗,实际 CAD 等待已压缩。
|
||||
第一次使用请先看 [README 逐步图文教程](../../README.md#图文教程),按截图完成创建任务、BOM、替换和结果查看。以下内容用于补充查询各项规则。
|
||||
|
||||
## 日常流程
|
||||
|
||||
@@ -31,9 +31,9 @@ V1.1.0 沿用原有任务视图和料号规则编辑,增加按工程拆分的
|
||||
|
||||
导入先核对所有工程归属,再按工程匹配位号;缺少清单时,按该工程 BOM 中的旧料号只读采集。不会执行替换。切换任务可查看、编辑各自预览,点击“保存本批位号计划”才一次写入所有受影响任务。保存失败全部回滚,预览期间不改已保存计划。工程、图页、UID、位号、旧料号及原 Value 用于身份校验,未知工程、重复器件或跨工程目标都会拒绝;不接受公式。布尔选项可使用 TRUE/FALSE 或 是/否。
|
||||
|
||||
开发版新增原生采集(尚未打包发布):本机有兼容的 `icdb2ascii.exe` 时,BOM 校验和位号采集优先在临时数据库副本中一次导出真实 UID、位号及物料属性,不必逐条启动 CAD 查找。工具不可用或报告不受支持且已安全清理时,自动使用原 CAD 采集。源工程变化或后台清理不确定时停止并保留诊断;完整图页查询和实际替换仍沿用原执行链。运行前请保存并关闭目标工程。详情见 [原生采集接入记录](../research/2026-10-16-native-cli-integration.md)。
|
||||
V1.1.3 已包含原生采集:本机有兼容的 `icdb2ascii.exe` 时,BOM 校验和位号采集优先在临时数据库副本中一次导出真实 UID、位号及物料属性,不必逐条启动 CAD 查找。工具不可用或报告不受支持且已安全清理时,自动使用原 CAD 采集。源工程变化或后台清理不确定时停止并保留诊断;完整图页查询和实际替换仍沿用原执行链。运行前请保存并关闭目标工程。详情见 [原生采集接入记录](../research/2026-10-16-native-cli-integration.md)。
|
||||
|
||||
BOM 首次采集或新增的位号默认“不替换”;已有计划中未列出的行保留原勾选和编辑状态,不会因后续 BOM 省略而自动停用。如果本次仅执行 BOM 列出的目标,请在预览中检查并取消其他启用行,或在 BOM 中显式填写对应的 `enabled=FALSE` 行。每个填写的位号须在当前工程清单中唯一匹配;同名位号跨图页时应填写图页或 UID。导入失败不会改变已保存计划。
|
||||
首次采集中未被 BOM 列为替换目标的位号默认“不替换”;已有计划中未列出的行保留原勾选和编辑状态,不会因后续 BOM 省略而自动停用。如果本次仅执行 BOM 列出的目标,请在预览中检查并取消其他启用行,或在 BOM 中显式填写对应的 `enabled=FALSE` 行。每个填写的位号须在当前工程清单中唯一匹配;同名位号跨图页时应填写图页或 UID。导入失败不会改变已保存计划。
|
||||
|
||||
“同步 NC”默认勾选,可以按行关闭;仅按原规则识别独立 `/NC` 标记。Value 复制选项和每行自定义 Value 延续原工具的处理方式。替换计划不允许旧、新料号相同,也不允许同批链式或循环替换。
|
||||
|
||||
|
||||
@@ -930,6 +930,7 @@ Sub QPFinishBatchItem()
|
||||
Next
|
||||
End If
|
||||
If FinalState = "restored_ok" Then
|
||||
QPMarkBatchMutation Command
|
||||
QPCloseBatchProject TargetPath, True
|
||||
QP_BatchSaveCommitted = True
|
||||
ElseIf Not QP_SessionMode Then
|
||||
@@ -975,6 +976,42 @@ Sub QPFinishBatchItem()
|
||||
QPProgress "finished", 1, 1
|
||||
End Sub
|
||||
|
||||
Sub QPMarkBatchMutation(ByRef Command)
|
||||
Dim Shell, Entry, Required, CommandLine, Key, Value, ExitCode
|
||||
Set Shell = CreateObject("WScript.Shell")
|
||||
Required = False
|
||||
For Each Entry In Shell.Environment("Process")
|
||||
If InStr(1, CStr(Entry), "DXPART_ATTEMPT_FILE=", vbTextCompare) = 1 Or _
|
||||
InStr(1, CStr(Entry), "DXPART_BATCH_SESSION_ID=", vbTextCompare) = 1 Or _
|
||||
InStr(1, CStr(Entry), "DXPART_BATCH_GENERATION=", vbTextCompare) = 1 Or _
|
||||
InStr(1, CStr(Entry), "DXPART_ATTEMPT_ID=", vbTextCompare) = 1 Then Required = True
|
||||
Next
|
||||
If Not Required Then Exit Sub
|
||||
If Len(QP_HelperExecutable) = 0 Or Not QP_Fso.FileExists(QP_HelperExecutable) Then _
|
||||
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "mutation proof helper is unavailable"
|
||||
CommandLine = Chr(34) & QP_HelperExecutable & Chr(34)
|
||||
If Len(QP_HelperScript) > 0 Then
|
||||
If Not QP_Fso.FileExists(QP_HelperScript) Then _
|
||||
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "mutation proof script is unavailable"
|
||||
CommandLine = CommandLine & " " & Chr(34) & QP_HelperScript & Chr(34)
|
||||
End If
|
||||
CommandLine = CommandLine & " --mark-batch-attempt"
|
||||
For Each Key In Array("job_id", "task_id", "project_path")
|
||||
Value = CStr(Command(Key))
|
||||
If Len(Value) = 0 Or InStr(Value, Chr(34)) > 0 Or InStr(Value, vbCr) > 0 Or InStr(Value, vbLf) > 0 Then _
|
||||
Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", "invalid mutation proof identity"
|
||||
Select Case Key
|
||||
Case "job_id": CommandLine = CommandLine & " --attempt-job "
|
||||
Case "task_id": CommandLine = CommandLine & " --attempt-task "
|
||||
Case "project_path": CommandLine = CommandLine & " --attempt-project "
|
||||
End Select
|
||||
CommandLine = CommandLine & Chr(34) & Value & Chr(34)
|
||||
Next
|
||||
ExitCode = Shell.Run(CommandLine, 0, True)
|
||||
If ExitCode <> 0 Then Err.Raise vbObjectError + 879, "QuickPartReplace.Attempt", _
|
||||
"durable mutation proof failed; no CAD write allowed"
|
||||
End Sub
|
||||
|
||||
Sub QPAutomaticReplaceAndRestore(ByRef Command, ByVal SnapshotPath)
|
||||
Dim ErrorNumber, ErrorSource, ErrorDescription
|
||||
QPRequireCommandSnapshot Command, SnapshotPath
|
||||
@@ -1590,6 +1627,7 @@ Sub QPRestoreAfterOperatorReplace(ByRef Command, ByVal SnapshotPath)
|
||||
QPProgress "restore_preflight", Index + 1, Records.Item("components").Count
|
||||
Next
|
||||
QPProgress "before_restore_transaction", 0, Plans.Count
|
||||
QPMarkBatchMutation Command
|
||||
QP_RestoreTransactionOpen = True
|
||||
QP_App.ActiveView.Block.StartUndoTransaction
|
||||
On Error Resume Next
|
||||
|
||||
@@ -80,6 +80,9 @@ try {
|
||||
if (-not (Test-Path -LiteralPath $executable -PathType Leaf)) {
|
||||
throw "PyInstaller did not produce $executable"
|
||||
}
|
||||
$runtimeSmokeRoot = Join-Path $repositoryRoot ("build\frozen-runtime-" + [Guid]::NewGuid().ToString("N"))
|
||||
& $PythonExecutable tools\verify_packaged_runtime.py $executable $runtimeSmokeRoot
|
||||
if ($LASTEXITCODE -ne 0) { throw "Frozen supervisor verification failed; build stopped" }
|
||||
$sourceQuickPartReplace = Join-Path $repositoryRoot "integration\dxdesigner795\QuickPartReplace.vbs"
|
||||
$packagedQuickPartReplace = Join-Path $repositoryRoot "dist\DxPartReplace\_internal\legacy-runtime\dxdesigner795\QuickPartReplace.vbs"
|
||||
if (-not (Test-Path -LiteralPath $sourceQuickPartReplace -PathType Leaf)) {
|
||||
|
||||
@@ -43,6 +43,8 @@ a = Analysis(
|
||||
"win32con",
|
||||
"win32gui",
|
||||
"win32process",
|
||||
# GetProcessTimes imports this from pywin32's C extension at runtime.
|
||||
"win32timezone",
|
||||
"pywinauto",
|
||||
"pywinauto.controls.uiawrapper",
|
||||
],
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# UTF-8
|
||||
VSVersionInfo(
|
||||
ffi=FixedFileInfo(
|
||||
filevers=(1, 1, 3, 0),
|
||||
prodvers=(1, 1, 3, 0),
|
||||
filevers=(1, 1, 4, 2),
|
||||
prodvers=(1, 1, 4, 2),
|
||||
mask=0x3f,
|
||||
flags=0x0,
|
||||
flags=0x2,
|
||||
OS=0x40004,
|
||||
fileType=0x1,
|
||||
subtype=0x0,
|
||||
@@ -15,11 +15,11 @@ VSVersionInfo(
|
||||
StringTable('040904B0', [
|
||||
StringStruct('CompanyName', 'DxProductivity'),
|
||||
StringStruct('FileDescription', 'DxDesigner Part Replacement'),
|
||||
StringStruct('FileVersion', '1.1.3.0'),
|
||||
StringStruct('FileVersion', '1.1.4.2'),
|
||||
StringStruct('InternalName', 'DxPartReplace'),
|
||||
StringStruct('OriginalFilename', 'DxPartReplace.exe'),
|
||||
StringStruct('ProductName', 'DxDesigner Part Replacement'),
|
||||
StringStruct('ProductVersion', '1.1.3')
|
||||
StringStruct('ProductVersion', '1.1.4-beta.2')
|
||||
])
|
||||
]),
|
||||
VarFileInfo([VarStruct('Translation', [1033, 1200])])
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "dx-productivity"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4b2"
|
||||
requires-python = ">=3.12,<3.13"
|
||||
dependencies = ["PySide6>=6.7,<7", "pywin32>=306", "pywinauto>=0.6.9,<1", "openpyxl>=3.1,<4"]
|
||||
|
||||
|
||||
@@ -51,9 +51,13 @@ def _parse_arguments(argv: Sequence[str] | None) -> argparse.Namespace:
|
||||
parser.add_argument("--data-root", type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument("--native-helper", action="store_true", help=argparse.SUPPRESS)
|
||||
parser.add_argument("--snapshot", type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--mark-batch-attempt', action='store_true', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--attempt-job', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--attempt-task', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--attempt-project', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--cell-catalog-worker', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument("--sceneos-cli", action="store_true", help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-action', choices=('probe', 'quit'), help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-action', choices=('probe', 'quit', 'checkpoint'), help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-evidence', type=Path, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--private-cad-pid', type=int, help=argparse.SUPPRESS)
|
||||
return parser.parse_args(list(argv) if argv is not None else None)
|
||||
@@ -113,9 +117,31 @@ def _flush(context: LoggingContext) -> None:
|
||||
handler.flush()
|
||||
|
||||
|
||||
def _smoke_batch_runtime(data_root: Path, context: LoggingContext) -> None:
|
||||
"""Exercise process tracking inside the frozen supervisor before releasing."""
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
|
||||
with BatchCadRuntime(data_root, 'packaged_runtime_smoke') as runtime:
|
||||
runtime.bind_logs(context, ())
|
||||
with runtime.native_scope(data_root / 'runtime-smoke') as lease:
|
||||
command = str(Path(os.environ['SystemRoot']) / 'System32/cmd.exe')
|
||||
process = lease.start_process((command, '/d', '/c', 'exit', '0'))
|
||||
if not runtime.record['processes'][0]['creation_time']:
|
||||
raise RuntimeError('Smoke process identity has no creation time')
|
||||
if process.wait(timeout=15) != 0:
|
||||
raise RuntimeError('Smoke worker did not exit successfully')
|
||||
if runtime.record['state'] != 'released':
|
||||
raise RuntimeError('Smoke resources were not released')
|
||||
context.logger.info('packaged_runtime_smoke_passed batch_id=%s', runtime.session_id)
|
||||
|
||||
|
||||
def main(argv: Sequence[str] | None = None) -> int:
|
||||
"""Run the Qt shell, returning a conventional process status code."""
|
||||
arguments = _parse_arguments(argv)
|
||||
if arguments.mark_batch_attempt:
|
||||
from dx_productivity.trial_task_center.native_helper import run_attempt_marker
|
||||
return run_attempt_marker(job_id=arguments.attempt_job, task_id=arguments.attempt_task,
|
||||
project_path=arguments.attempt_project)
|
||||
if arguments.cell_catalog_worker is not None:
|
||||
from dx_productivity.trial_task_center.cell_catalog_worker import run
|
||||
return run(arguments.cell_catalog_worker)
|
||||
@@ -141,6 +167,8 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
try:
|
||||
database_path = data_root / "task-center.db"
|
||||
_log_startup(logging_context, data_root, database_path)
|
||||
if arguments.smoke_test and bool(getattr(sys, 'frozen', False)):
|
||||
_smoke_batch_runtime(data_root, logging_context)
|
||||
store = TaskStore(database_path)
|
||||
service = TaskCenterService(store, logging_context)
|
||||
qt_application = QApplication([sys.executable, *(argv or ())])
|
||||
|
||||
@@ -41,6 +41,7 @@ def execute_batch(
|
||||
stop_event: Event,
|
||||
continue_after_backup_failure: bool = False,
|
||||
reuse_project_session: bool = True,
|
||||
runtime=None,
|
||||
) -> BatchRunResult:
|
||||
preparer = service._quick_run_preparer
|
||||
retention = maintain_quick_run_storage(preparer.data_root)
|
||||
@@ -71,6 +72,7 @@ def execute_batch(
|
||||
projects,
|
||||
rules,
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}),
|
||||
)
|
||||
metric['completed_units'] = len(session.commands)
|
||||
initial_progress = progress
|
||||
@@ -108,7 +110,8 @@ def execute_batch(
|
||||
progress.total_items,
|
||||
)
|
||||
progress = session.progress
|
||||
with backups, preparer.project_session_scope(enabled=reuse_project_session) as project_session:
|
||||
with backups, preparer.project_session_scope(enabled=reuse_project_session,
|
||||
**({'runtime': runtime} if runtime is not None else {})) as project_session:
|
||||
while not progress.queue_complete:
|
||||
if stop_event.is_set():
|
||||
backups.commit_prefetched(progress)
|
||||
@@ -137,7 +140,8 @@ def execute_batch(
|
||||
try:
|
||||
last_result = preparer.execute_step(task_id, projects, rules, copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
expected_item=expected, batch_mode=True, batch_session=session,
|
||||
**({'project_session': project_session} if project_session.enabled else {}))
|
||||
**({'project_session': project_session} if project_session.enabled else {}),
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
metric['state'] = last_result.state
|
||||
metric['completed_units'] = int(last_result.state in {'saved_ok', 'restored_ok', 'no_match'})
|
||||
finally:
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Durable write-ahead proof for one logical job in a private CAD attempt.
|
||||
|
||||
A prepared record is evidence of no write only while its complete identity still
|
||||
matches. Helpers never create records and must fail closed if publication fails.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import tempfile
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
|
||||
_IDENTITY = ('session_id', 'generation', 'attempt_id', 'task_id', 'job_id', 'project_path')
|
||||
|
||||
|
||||
def _project(value):
|
||||
return os.path.normcase(os.path.abspath(value))
|
||||
|
||||
|
||||
def _validate(record):
|
||||
if not isinstance(record, dict) or record.get('schema_version') != 1:
|
||||
raise ValueError('invalid batch attempt schema')
|
||||
for key in ('session_id', 'task_id', 'job_id', 'project_path'):
|
||||
value = record.get(key)
|
||||
if not isinstance(value, str) or not value.strip() or any(ord(c) < 32 for c in value):
|
||||
raise ValueError(f'invalid batch attempt {key}')
|
||||
generation = record.get('generation')
|
||||
if type(generation) is not int or generation < 1:
|
||||
raise ValueError('invalid batch attempt generation')
|
||||
if not isinstance(record.get('attempt_id'), str) or not re.fullmatch(r'[0-9a-f]{32}', record['attempt_id']):
|
||||
raise ValueError('invalid batch attempt ID')
|
||||
if record.get('state') not in {'prepared', 'mutation_possible'}:
|
||||
raise ValueError('invalid batch attempt state')
|
||||
return record
|
||||
|
||||
|
||||
def _publish(path, record):
|
||||
path = Path(path)
|
||||
descriptor, temporary = tempfile.mkstemp(prefix=f'.{path.name}.', suffix='.tmp', dir=path.parent)
|
||||
try:
|
||||
with os.fdopen(descriptor, 'w', encoding='utf-8', newline='') as handle:
|
||||
json.dump(record, handle, ensure_ascii=False, indent=2)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
deadline = time.monotonic() + 1
|
||||
while True:
|
||||
try:
|
||||
os.replace(temporary, path)
|
||||
break
|
||||
except PermissionError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise
|
||||
time.sleep(.02)
|
||||
# POSIX needs a directory sync for the atomic rename; Windows does not
|
||||
# expose fsync on directories through the CRT.
|
||||
if os.name != 'nt':
|
||||
directory = os.open(path.parent, os.O_RDONLY)
|
||||
try:
|
||||
os.fsync(directory)
|
||||
finally:
|
||||
os.close(directory)
|
||||
finally:
|
||||
Path(temporary).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def prepare_attempt(path, *, session_id, generation, task_id, job_id, project_path, attempt_id=None):
|
||||
record = _validate(dict(schema_version=1, session_id=str(session_id), generation=generation,
|
||||
task_id=str(task_id), job_id=str(job_id), project_path=_project(project_path),
|
||||
attempt_id=uuid4().hex if attempt_id is None else attempt_id, state='prepared'))
|
||||
Path(path).parent.mkdir(parents=True, exist_ok=True)
|
||||
_publish(path, record)
|
||||
return record
|
||||
|
||||
|
||||
def read_attempt(path):
|
||||
return _validate(json.loads(Path(path).read_text(encoding='utf-8')))
|
||||
|
||||
|
||||
def _bound_attempt(path=None, *, job_id=None, task_id=None, project_path=None):
|
||||
if path is None and 'DXPART_ATTEMPT_FILE' not in os.environ:
|
||||
if any(key in os.environ for key in ('DXPART_BATCH_SESSION_ID', 'DXPART_BATCH_GENERATION', 'DXPART_ATTEMPT_ID')):
|
||||
raise ValueError('missing batch attempt file')
|
||||
return None
|
||||
raw_path = os.environ.get('DXPART_ATTEMPT_FILE') if path is None else os.fspath(path)
|
||||
if not raw_path or not Path(raw_path).is_absolute():
|
||||
raise ValueError('invalid batch attempt file')
|
||||
path = Path(raw_path)
|
||||
job_id = str(job_id) if job_id is not None else os.environ.get('DXPART_JOB_ID')
|
||||
task_id = str(task_id) if task_id is not None else os.environ.get('DXPART_TASK_ID')
|
||||
if path.is_dir():
|
||||
if not job_id or not re.fullmatch(r'[A-Za-z0-9_-]+', job_id):
|
||||
raise ValueError('missing or unsafe batch attempt job ID')
|
||||
path = path / f'{job_id}.json'
|
||||
record = read_attempt(path)
|
||||
expected = {'session_id': os.environ.get('DXPART_BATCH_SESSION_ID'),
|
||||
'attempt_id': os.environ.get('DXPART_ATTEMPT_ID'),
|
||||
'generation': os.environ.get('DXPART_BATCH_GENERATION')}
|
||||
if not all(expected.values()) or str(record['generation']) != expected.pop('generation'):
|
||||
raise ValueError('missing or stale batch generation')
|
||||
if any(record[key] != value for key, value in expected.items()):
|
||||
raise ValueError('stale batch attempt identity')
|
||||
if not job_id or not task_id or record['job_id'] != job_id or record['task_id'] != task_id:
|
||||
raise ValueError('batch attempt task/job mismatch')
|
||||
# Optional environment bindings remain authoritative even when the caller
|
||||
# supplies explicit command fields.
|
||||
for key, value in (('DXPART_JOB_ID', job_id), ('DXPART_TASK_ID', task_id)):
|
||||
if key in os.environ and os.environ[key] != value:
|
||||
raise ValueError('batch process task/job mismatch')
|
||||
if project_path is not None and record['project_path'] != _project(project_path):
|
||||
raise ValueError('batch attempt project mismatch')
|
||||
return path, record
|
||||
|
||||
|
||||
def mark_mutation_possible(path=None, *, job_id=None, task_id=None, project_path=None):
|
||||
bound = _bound_attempt(path, job_id=job_id, task_id=task_id, project_path=project_path)
|
||||
if bound is None:
|
||||
return None
|
||||
path, record = bound
|
||||
if record['state'] != 'mutation_possible':
|
||||
record = dict(record, state='mutation_possible')
|
||||
_publish(path, record)
|
||||
return record
|
||||
|
||||
|
||||
def can_retry_attempt(path, expected):
|
||||
try:
|
||||
_validate(expected)
|
||||
actual = read_attempt(path)
|
||||
return actual['state'] == 'prepared' and all(actual[key] == expected[key] for key in _IDENTITY)
|
||||
except (OSError, ValueError, TypeError, KeyError):
|
||||
return False
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Readable release history, separate from authoritative resource proofs."""
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from .logging_setup import LoggingContext, append_task_event, configure_logging
|
||||
|
||||
|
||||
def _append_line(path, line):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open('a', encoding='utf-8', newline='\n') as stream:
|
||||
stream.write(line + '\n')
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
|
||||
|
||||
def write_release_event(root, path, record, phase, *, context=None, reason='batch_end',
|
||||
elapsed=0.0, error=''):
|
||||
"""Try each diagnostic destination; failures never interrupt native cleanup."""
|
||||
cleanup = record.get('cleanup') if isinstance(record.get('cleanup'), dict) else {}
|
||||
confirmation = cleanup.get('desktop_confirmation')
|
||||
confirmation = confirmation if isinstance(confirmation, dict) else {}
|
||||
cad = record.get('cad_shutdown') if isinstance(record.get('cad_shutdown'), dict) else {}
|
||||
disappeared = confirmation.get('released')
|
||||
if phase == 'succeeded' and record.get('reconciled'):
|
||||
disappeared = True
|
||||
details = {'batch_id': record.get('session_id'), 'batch_kind': record.get('kind'),
|
||||
'generation': record.get('generation'), 'desktop': record.get('desktop'),
|
||||
'cad_pid': record.get('cad_pid'), 'cad_guid': record.get('cad_guid'),
|
||||
'reason': reason, 'result': 'releasing' if phase == 'started' else (
|
||||
'released' if phase == 'succeeded' else 'blocked'),
|
||||
'elapsed_seconds': elapsed, 'record_path': str(path),
|
||||
'desktop_handle_closed': cleanup.get('desktop_handle_closed'),
|
||||
'desktop_disappeared': disappeared, 'cad_exit_code': cad.get('cad_exit_code'),
|
||||
'quit_helper_exit_code': cad.get('quit_helper_exit_code'),
|
||||
'cad_shutdown': cad, 'shutdown_warning': record.get('shutdown_warning'),
|
||||
'retries': record.get('retries', {}), 'cleanup': cleanup, 'error': str(error)}
|
||||
title = {'started': '开始释放后台资源', 'succeeded': '后台资源释放成功',
|
||||
'failed': '后台资源释放失败,资源释放待处理'}[phase]
|
||||
purpose = {'batch_end': '批次结束', 'rebuild': '安全重建', 'manual_retry': '重新清理',
|
||||
'restart_recheck': '重启后重新检查'}.get(reason, reason)
|
||||
cad_text = f"CAD PID={details['cad_pid']}" if details['cad_pid'] else (
|
||||
'CAD 未启动' if record.get('cad_start_requested') is False else
|
||||
'CAD 启动未完成绑定' if record.get('cad_start_requested') else 'CAD 启动状态未知')
|
||||
message = (f"{title} · {purpose} · 批次 {details['batch_id']} · 代次 {details['generation']}"
|
||||
f" · 桌面 {details['desktop']} · {cad_text} · 耗时 {elapsed:.3f} 秒")
|
||||
if phase != 'started':
|
||||
label = lambda value: '已确认' if value is True else '未确认'
|
||||
message += (f" · 桌面句柄关闭:{label(details['desktop_handle_closed'])}"
|
||||
f" · 桌面消失:{label(disappeared)}")
|
||||
if cad:
|
||||
message += f" · CAD 退出码={details['cad_exit_code']},退出助手码={details['quit_helper_exit_code']}"
|
||||
if error:
|
||||
message += ' · 原因:' + str(error)
|
||||
message = message.replace('\r', '\\r').replace('\n', '\\n')
|
||||
level = 'ERROR' if phase == 'failed' else 'INFO'
|
||||
timestamp = datetime.now(timezone.utc).isoformat(timespec='milliseconds')
|
||||
errors = []
|
||||
try:
|
||||
_append_line(Path(path).with_name('resource-release.log'),
|
||||
f'{timestamp} {level} {message} | ' + json.dumps(details, ensure_ascii=False))
|
||||
except Exception as failure:
|
||||
errors.append('resource-release.log: ' + str(failure))
|
||||
try:
|
||||
context = context if isinstance(context, LoggingContext) else configure_logging(root)
|
||||
except Exception as failure:
|
||||
errors.append('application log: ' + str(failure))
|
||||
return errors
|
||||
for task_id in record.get('log_task_ids', ()):
|
||||
try:
|
||||
append_task_event(context, level=level, task_id=task_id,
|
||||
operation='resource_release_' + phase, message=message, details=details)
|
||||
except Exception as failure:
|
||||
errors.append('task ' + str(task_id) + ': ' + str(failure))
|
||||
try:
|
||||
context.logger.log(40 if phase == 'failed' else 20, '%s | %s', message,
|
||||
json.dumps(details, ensure_ascii=False))
|
||||
if errors:
|
||||
context.logger.error('resource_release_log_errors batch=%s errors=%s',
|
||||
record.get('session_id'), errors)
|
||||
for handler in context.logger.handlers:
|
||||
handler.flush()
|
||||
except Exception as failure:
|
||||
errors.append('application log: ' + str(failure))
|
||||
return errors
|
||||
@@ -0,0 +1,193 @@
|
||||
"""Persistent resource ownership, independent of project replacement results."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
|
||||
|
||||
# Recovery retains owned native handles after failed cleanup. This registry is
|
||||
# never an implicit execution context and never contains COM application objects.
|
||||
LIVE_RESOURCES = {}
|
||||
|
||||
|
||||
class BatchResourceCleanupError(TaskCenterError):
|
||||
def __init__(self, message, *, completed_result=None):
|
||||
super().__init__('batch_resources_pending', message)
|
||||
self.completed_result = completed_result
|
||||
|
||||
|
||||
def write_record(path, record):
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_name(path.name + '.' + uuid4().hex + '.tmp')
|
||||
try:
|
||||
with temporary.open('w', encoding='utf-8', newline='\n') as stream:
|
||||
json.dump(record, stream, ensure_ascii=False, indent=2)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
deadline = time.monotonic() + 1.0
|
||||
while True:
|
||||
try:
|
||||
os.replace(temporary, path)
|
||||
break
|
||||
except PermissionError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise
|
||||
time.sleep(.02)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _valid_record(record, path):
|
||||
if not isinstance(record, dict):
|
||||
return False
|
||||
session = record.get('session_id')
|
||||
processes = record.get('processes')
|
||||
return (record.get('version') == 1 and isinstance(session, str)
|
||||
and re.fullmatch('[0-9a-f]{32}', session) is not None and path.parent.name == session
|
||||
and isinstance(record.get('kind'), str) and bool(record['kind'])
|
||||
and type(record.get('generation')) is int and record['generation'] >= 1
|
||||
and isinstance(record.get('desktop'), str)
|
||||
and re.fullmatch('dx-[0-9a-f]{32}', record['desktop']) is not None
|
||||
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released'}
|
||||
and isinstance(processes, list) and all(isinstance(item, dict)
|
||||
and type(item.get('pid')) is int and item['pid'] > 0
|
||||
and 'creation_time' in item and (item['creation_time'] is None
|
||||
or isinstance(item['creation_time'], str)) for item in processes)
|
||||
and 'cad_pid' in record and (record['cad_pid'] is None
|
||||
or type(record['cad_pid']) is int and record['cad_pid'] > 0)
|
||||
and 'cad_guid' in record and (record['cad_guid'] is None
|
||||
or isinstance(record['cad_guid'], str) and bool(record['cad_guid']))
|
||||
and all(isinstance(record.get(key), dict) for key in ('checkpoints', 'retries'))
|
||||
and isinstance(record.get('timings'), list))
|
||||
|
||||
|
||||
def pending_resources(root):
|
||||
pending = []
|
||||
for path in sorted((Path(root).resolve() / 'batch-resources').glob('*/session.json')):
|
||||
try:
|
||||
record = json.loads(path.read_text(encoding='utf-8'))
|
||||
if not _valid_record(record, path):
|
||||
raise ValueError('invalid resource ownership record')
|
||||
if record.get('state') != 'released':
|
||||
pending.append(dict(record, record_path=str(path)))
|
||||
except (OSError, ValueError, TypeError) as error:
|
||||
pending.append({'state': 'blocked', 'record_path': str(path), 'error': str(error)})
|
||||
return tuple(pending)
|
||||
|
||||
|
||||
def guard_pending_resources(root):
|
||||
if pending_resources(root):
|
||||
raise BatchResourceCleanupError('上一批资源尚未确认释放;请先重试释放资源,再开始新批次。')
|
||||
|
||||
|
||||
def process_identity(process):
|
||||
result = {'pid': process.pid, 'creation_time': None}
|
||||
handle = getattr(process, 'handle', None)
|
||||
if handle is not None:
|
||||
import win32process
|
||||
result['creation_time'] = str(win32process.GetProcessTimes(handle)['CreationTime'])
|
||||
return result
|
||||
|
||||
|
||||
def _process_gone(identity):
|
||||
import win32api
|
||||
import win32event
|
||||
import win32process
|
||||
try:
|
||||
handle = win32api.OpenProcess(0x101000, False, identity['pid'])
|
||||
except Exception as error:
|
||||
code = getattr(error, 'winerror', error.args[0] if error.args else None)
|
||||
if code == 87:
|
||||
return True
|
||||
raise
|
||||
try:
|
||||
if win32event.WaitForSingleObject(handle, 0) == 0:
|
||||
return True
|
||||
created = identity.get('creation_time')
|
||||
# A reused PID is an unrelated process and must never be terminated.
|
||||
return created is not None and str(win32process.GetProcessTimes(handle)['CreationTime']) != created
|
||||
finally:
|
||||
handle.Close()
|
||||
|
||||
|
||||
def cleanup_pending_resources(root, *, logging_context=None):
|
||||
from .isolated_desktop import _Win32Api
|
||||
from .batch_release_log import write_release_event
|
||||
for record in pending_resources(root):
|
||||
path = Path(record['record_path'])
|
||||
live = LIVE_RESOURCES.get(str(path))
|
||||
if live is not None:
|
||||
try:
|
||||
live.close(reason='manual_retry')
|
||||
except Exception:
|
||||
pass # The live owner records the failure and retains ownership.
|
||||
continue
|
||||
started = time.monotonic()
|
||||
def log(phase, error=''):
|
||||
try:
|
||||
failures = write_release_event(root, path, record, phase, context=logging_context,
|
||||
reason='restart_recheck', elapsed=time.monotonic() - started, error=error)
|
||||
except Exception as failure:
|
||||
failures = [str(failure)]
|
||||
if failures:
|
||||
record.setdefault('release_log_errors', []).extend(failures)
|
||||
try:
|
||||
diagnostic = dict(record)
|
||||
diagnostic.pop('record_path', None)
|
||||
write_record(path, diagnostic)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if not record.get('session_id'):
|
||||
continue # Corrupt evidence cannot authorize resource actions.
|
||||
log('started')
|
||||
desktop = record.get('desktop')
|
||||
if desktop:
|
||||
_Win32Api().confirm_desktop_released(desktop)
|
||||
if not all(_process_gone(item) for item in record.get('processes', ())):
|
||||
log('failed', '记录中的进程仍未确认退出')
|
||||
continue
|
||||
# After restart only confirm disappearance. Never attach to COM,
|
||||
# close a project, or terminate a PID whose handles we no longer own.
|
||||
record.pop('record_path', None)
|
||||
record = dict(record, state='released', reconciled=True, released_at=time.time())
|
||||
write_record(path, record)
|
||||
log('succeeded')
|
||||
except Exception as error:
|
||||
if record.get('session_id'):
|
||||
record.pop('record_path', None)
|
||||
log('failed', str(error))
|
||||
write_record(path, dict(record, state='blocked', error=str(error)))
|
||||
return pending_resources(root)
|
||||
|
||||
|
||||
def project_checkpoint_released(root):
|
||||
root = Path(root).resolve()
|
||||
try:
|
||||
checkpoint = json.loads((root / 'project-checkpoint.json').read_text(encoding='utf-8'))
|
||||
path = Path(checkpoint['resource_record']).resolve()
|
||||
record = json.loads(path.read_text(encoding='utf-8'))
|
||||
key = str(root).casefold()
|
||||
return (_valid_record(record, path) and checkpoint.get('version') == 1
|
||||
and type(checkpoint.get('generation')) is int
|
||||
and isinstance(checkpoint.get('attempt_id'), str)
|
||||
and re.fullmatch('[0-9a-f]{32}', checkpoint['attempt_id']) is not None
|
||||
and type(checkpoint.get('cad_pid')) is int and checkpoint['cad_pid'] > 0
|
||||
and isinstance(checkpoint.get('cad_guid'), str) and bool(checkpoint['cad_guid'])
|
||||
and isinstance(checkpoint.get('project_path'), str)
|
||||
and Path(checkpoint['project_path']).is_absolute()
|
||||
and checkpoint['project_closed'] is True and checkpoint['worker_closed'] is True
|
||||
and checkpoint['evidence_root'] == str(root)
|
||||
and record['state'] in {'ready', 'closing', 'released'}
|
||||
and (checkpoint['session_id'], checkpoint['generation'], checkpoint['cad_pid'], checkpoint['cad_guid'])
|
||||
== (record['session_id'], record['generation'], record['cad_pid'], record['cad_guid'])
|
||||
and record['checkpoints'][key] == checkpoint['attempt_id'])
|
||||
except (OSError, ValueError, TypeError, KeyError):
|
||||
return False
|
||||
@@ -0,0 +1,388 @@
|
||||
"""One lazy private desktop/CAD owner per import or execution batch."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
from .batch_resources import (BatchResourceCleanupError, LIVE_RESOURCES, cleanup_pending_resources,
|
||||
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record)
|
||||
from .errors import TaskCenterError
|
||||
from .isolated_desktop import PrivateDesktop
|
||||
from .operation_control import check_cancelled
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
|
||||
|
||||
class _DesktopLease:
|
||||
"""Close only this operation's processes; the batch owns the desktop itself."""
|
||||
def __init__(self, owner, evidence, *, native=False):
|
||||
self.owner, self.evidence = owner, Path(evidence)
|
||||
self.native, self.attempt_id = native, uuid4().hex
|
||||
self.generation, self.desktop = None, None
|
||||
self.processes = []
|
||||
self.cleanup_record = None
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return (self.desktop if self.desktop is not None else self.owner._ensure_desktop()).name
|
||||
|
||||
def __enter__(self):
|
||||
self.desktop = self.owner._ensure_desktop()
|
||||
self.generation = self.owner.generation
|
||||
if self.native:
|
||||
self.owner.record['native_operations'][str(self.evidence.resolve())] = {
|
||||
'binding': self.identity, 'workers_closed': False}
|
||||
self.owner._save()
|
||||
return self
|
||||
|
||||
@property
|
||||
def identity(self):
|
||||
return {'session_id': self.owner.session_id, 'generation': self.generation,
|
||||
'attempt_id': self.attempt_id}
|
||||
|
||||
def __exit__(self, *_exc):
|
||||
self.close()
|
||||
|
||||
def start_process(self, command, **options):
|
||||
available = self.owner.record['state'] == 'ready' or (
|
||||
not self.native and self.owner.record['state'] == 'closing')
|
||||
if self.desktop is None or self.generation != self.owner.generation or not available:
|
||||
raise WorkerMonitorError('stale or unavailable batch resource lease')
|
||||
process = self.desktop.start_process(command, **options)
|
||||
self.processes.append(process)
|
||||
self.owner._track(process)
|
||||
return process
|
||||
|
||||
def close(self):
|
||||
errors, remaining, records = [], [], []
|
||||
for process in reversed(self.processes):
|
||||
try:
|
||||
self.owner._release_process(process, desktop=self.desktop, generation=self.generation)
|
||||
records.append(getattr(process, 'cleanup_record', {}))
|
||||
except Exception as error:
|
||||
errors.append(str(error))
|
||||
remaining.append(process)
|
||||
self.processes = list(reversed(remaining))
|
||||
self.cleanup_record = {'scope': 'operation', 'processes': records,
|
||||
'workers_closed': not errors, 'desktop_retained_by_batch': True, 'errors': errors}
|
||||
if errors:
|
||||
self.owner._save(state='blocked', error='; '.join(errors))
|
||||
raise BatchResourceCleanupError('工作进程未能确认退出;批次已停止。' + '; '.join(errors))
|
||||
if self.native and self.generation == self.owner.generation:
|
||||
self.owner.record['native_operations'][str(self.evidence.resolve())] = {
|
||||
'binding': self.identity, 'workers_closed': True}
|
||||
self.owner._save()
|
||||
|
||||
|
||||
class BatchCadRuntime:
|
||||
def __init__(self, root, kind, stop_event=None, *, desktop_factory=None, cad_factory=None):
|
||||
self.root = Path(root).resolve()
|
||||
self.kind, self.stop_event = kind, stop_event
|
||||
self.session_id = uuid4().hex
|
||||
self.path = self.root / 'batch-resources' / self.session_id / 'session.json'
|
||||
self.desktop_factory = desktop_factory or PrivateDesktop
|
||||
self.cad_factory = cad_factory
|
||||
self.desktop = self.cad = None
|
||||
self.generation = 0
|
||||
self.result = None
|
||||
self.record = {'version': 1, 'session_id': self.session_id, 'kind': kind, 'state': 'new',
|
||||
'generation': 0, 'desktop': None, 'cad_pid': None, 'cad_guid': None,
|
||||
'processes': [], 'checkpoints': {}, 'retries': {}, 'timings': [], 'native_operations': {}}
|
||||
self._retry_counts = {}
|
||||
self._project_attempts = {}
|
||||
self._logging = None
|
||||
|
||||
def bind_logs(self, context, task_ids):
|
||||
"""Keep every participating task's final release outcome visible."""
|
||||
tasks = tuple(task_ids)
|
||||
if any(not isinstance(task, str) or not task or Path(task).name != task
|
||||
or task in {'.', '..'} for task in tasks):
|
||||
raise ValueError('resource logs require safe task identifiers')
|
||||
self._logging = context
|
||||
self.record['log_task_ids'] = list(dict.fromkeys((*self.record.get('log_task_ids', ()), *tasks)))
|
||||
if self.generation:
|
||||
self._save()
|
||||
|
||||
def __enter__(self):
|
||||
guard_pending_resources(self.root)
|
||||
return self
|
||||
|
||||
def __exit__(self, error_type, error, traceback):
|
||||
try:
|
||||
self.close()
|
||||
except Exception as cleanup:
|
||||
raise BatchResourceCleanupError('批次资源释放待处理;已核验任务结果保留。' + str(cleanup),
|
||||
completed_result=self.result) from cleanup
|
||||
|
||||
def _save(self, **values):
|
||||
record = dict(self.record, **values)
|
||||
write_record(self.path, record)
|
||||
self.record = record
|
||||
|
||||
def _timing(self, phase, started, **details):
|
||||
self.record['timings'].append({'phase': phase, 'seconds': time.monotonic() - started,
|
||||
'generation': self.generation, **details})
|
||||
|
||||
def _ensure_desktop(self, abort_event=None):
|
||||
if self.record['state'] == 'blocked':
|
||||
raise BatchResourceCleanupError('当前批次资源已阻断,不能启动新的工作进程。')
|
||||
if self.desktop is None:
|
||||
check_cancelled(self.stop_event if abort_event is None else abort_event)
|
||||
self.generation += 1
|
||||
name = 'dx-' + uuid4().hex
|
||||
self._save(state='starting', generation=self.generation, desktop=name,
|
||||
processes=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
|
||||
cad_shutdown=None, cad_start_requested=False, cleanup=None)
|
||||
LIVE_RESOURCES[str(self.path)] = self
|
||||
self.desktop = self.desktop_factory(name, enabled=True)
|
||||
started = time.monotonic()
|
||||
try:
|
||||
self.desktop.__enter__()
|
||||
except Exception:
|
||||
self._save(state='blocked', error='private desktop startup failed')
|
||||
raise
|
||||
self._timing('desktop_start', started)
|
||||
self._save(state='ready')
|
||||
return self.desktop
|
||||
|
||||
def _track(self, process):
|
||||
identity = process_identity(process)
|
||||
if identity not in self.record['processes']:
|
||||
self.record['processes'].append(identity)
|
||||
self._save()
|
||||
|
||||
def _release_process(self, process, *, desktop=None, generation=None):
|
||||
process.close()
|
||||
release = getattr(desktop if desktop is not None else self.desktop, 'forget_closed_process', None)
|
||||
if release:
|
||||
release(process)
|
||||
if generation is None or generation == self.generation:
|
||||
self.record['processes'] = [item for item in self.record['processes'] if item['pid'] != process.pid]
|
||||
|
||||
def native_scope(self, evidence):
|
||||
return _DesktopLease(self, evidence, native=True)
|
||||
|
||||
def _ensure_cad(self, abort_event=None):
|
||||
if self.cad is None:
|
||||
from .isolated_cad import IsolatedCadSession
|
||||
factory = self.cad_factory or IsolatedCadSession
|
||||
lease = _DesktopLease(self, self.path.parent)
|
||||
started = time.monotonic()
|
||||
self._save(cad_start_requested=True)
|
||||
cad = factory(self.path.parent / f'generation-{self.generation + (self.desktop is None)}',
|
||||
abort_event=abort_event, desktop=lease)
|
||||
self.cad = cad # Retain cleanup ownership even if startup fails.
|
||||
try:
|
||||
cad.__enter__()
|
||||
except Exception as error:
|
||||
from .isolated_cad import PrivateCadStartupError
|
||||
if isinstance(error, PrivateCadStartupError):
|
||||
# __enter__ certified its local processes/handles closed.
|
||||
# A never-ready instance must not receive a second Quit.
|
||||
self.cad = None
|
||||
raise
|
||||
self._timing('cad_start', started)
|
||||
self._save(state='ready', cad_pid=cad.binding.pid, cad_guid=cad.binding.guid)
|
||||
return self.cad
|
||||
|
||||
def _prepare_attempts(self, root, command, attempt_id):
|
||||
from .batch_attempt import prepare_attempt
|
||||
from .quick_run import _read_commands
|
||||
rows = _read_commands(root / 'command.tsv')
|
||||
selected_rows = [row for row in rows if row[1] == command[-1]]
|
||||
if len(selected_rows) != 1 or len({row[1] for row in rows}) != len(rows):
|
||||
raise TaskCenterError('target_identity', '批次工作命令身份缺失或重复,已停止。')
|
||||
selected = selected_rows[0]
|
||||
project = Path(selected[2]).resolve()
|
||||
self._save(active_project=str(project), active_task=selected[0], active_job=selected[1])
|
||||
directory = root / 'batch-attempts' / (self.session_id + f'-g{self.generation}-' + attempt_id)
|
||||
expected = []
|
||||
for row in rows:
|
||||
if Path(row[2]).resolve() == project:
|
||||
path = directory / (row[1] + '.json')
|
||||
record = prepare_attempt(path, session_id=self.session_id, generation=self.generation,
|
||||
task_id=row[0], job_id=row[1], project_path=str(project), attempt_id=attempt_id)
|
||||
expected.append((path, record))
|
||||
self._project_attempts.setdefault(str(project).casefold(), []).extend(expected)
|
||||
return project, directory, expected
|
||||
|
||||
def _can_retry(self, project, expected):
|
||||
from .batch_attempt import can_retry_attempt
|
||||
key = str(project).casefold()
|
||||
return (not self._retry_counts.get(key) and expected
|
||||
and all(can_retry_attempt(path, record) for path, record in self._project_attempts.get(key, ())))
|
||||
|
||||
def _checkpoint(self, root, project, attempt_id):
|
||||
self.cad.checkpoint(root / ('cad-closed-' + attempt_id + '.json'))
|
||||
key = str(root).casefold()
|
||||
self.record['checkpoints'][key] = attempt_id
|
||||
self._save(state='ready')
|
||||
write_record(root / 'project-checkpoint.json', {'version': 1, 'resource_record': str(self.path),
|
||||
'session_id': self.session_id, 'generation': self.generation, 'attempt_id': attempt_id,
|
||||
'evidence_root': str(root), 'project_path': str(project), 'project_closed': True,
|
||||
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']})
|
||||
|
||||
def run_worker(self, command, timeout, *, abort_event=None, cooperative_abort=False):
|
||||
from .discovery_cache import project_fingerprint
|
||||
from .isolated_cad import PrivateCadStartupError, is_identity_conflict
|
||||
from .worker_monitor import run_monitored_process
|
||||
root = Path(command[3]).resolve()
|
||||
self._ensure_desktop(abort_event)
|
||||
while True:
|
||||
check_cancelled(abort_event or self.stop_event)
|
||||
attempt_id = uuid4().hex
|
||||
# No previous empty-project certificate may authorize continuation
|
||||
# once another operation could attach or open a project.
|
||||
self.record['checkpoints'].pop(str(root).casefold(), None)
|
||||
self._save(active_attempt=attempt_id)
|
||||
project, directory, expected = self._prepare_attempts(root, command, attempt_id)
|
||||
before = project_fingerprint(project, self.root)
|
||||
status = root / 'status.tsv'
|
||||
status_before = status.read_bytes() if status.exists() else None
|
||||
started = time.monotonic()
|
||||
failure = None
|
||||
try:
|
||||
cad = self._ensure_cad(abort_event)
|
||||
cad.checkpoint(root / ('cad-ready-' + attempt_id + '.json'))
|
||||
environment = {'DXPART_ATTEMPT_FILE': str(directory), 'DXPART_ATTEMPT_ID': attempt_id,
|
||||
'DXPART_BATCH_SESSION_ID': self.session_id, 'DXPART_BATCH_GENERATION': str(self.generation)}
|
||||
def start_worker(launch, **options):
|
||||
return cad.start_worker(launch, extra_environment=environment, **options)
|
||||
result = run_monitored_process(command, timeout, abort_event=abort_event,
|
||||
cooperative_abort=cooperative_abort, process_factory=start_worker)
|
||||
self._checkpoint(root, project, attempt_id)
|
||||
if result.returncode not in (2, 3):
|
||||
return result
|
||||
failure = WorkerMonitorError(result.stderr or 'pre-mutation worker startup failed')
|
||||
except (PrivateCadStartupError, WorkerMonitorError, subprocess.TimeoutExpired, OSError) as error:
|
||||
failure = error
|
||||
finally:
|
||||
self._timing('project_worker', started, project=str(project), attempt_id=attempt_id,
|
||||
task_id=self.record['active_task'], job_id=self.record['active_job'])
|
||||
# Missing proof, changed inputs/status, cancellation or a previous
|
||||
# write anywhere in this project can never authorize replay.
|
||||
after = project_fingerprint(project, self.root)
|
||||
status_after = status.read_bytes() if status.exists() else None
|
||||
# Runtime database changes require separate identity evidence. Until
|
||||
# available for a failed read, use the stricter full fingerprint.
|
||||
safe = (not is_identity_conflict(failure) and before is not None and before == after
|
||||
and status_before == status_after and self._can_retry(project, expected)
|
||||
and not (root / 'recovery.request').exists()
|
||||
and not (abort_event is not None and abort_event.is_set())
|
||||
and not (self.stop_event is not None and self.stop_event.is_set()))
|
||||
if not safe:
|
||||
raise failure
|
||||
key = str(project).casefold()
|
||||
self._retry_counts[key] = 1
|
||||
self.record['retries'][key] = {'count': 1, 'reason': str(failure), 'attempt_id': attempt_id}
|
||||
self._save(state='recycling')
|
||||
self.close(rebuilding=True) # Prove disappearance before creating another.
|
||||
self.desktop = self.cad = None
|
||||
self._ensure_desktop(abort_event)
|
||||
|
||||
def recover_native_failure(self, project, evidence, reason):
|
||||
"""One read-only recovery, based on durable source and worker proof."""
|
||||
from .batch_attempt import can_retry_attempt
|
||||
project, evidence = Path(project).resolve(), Path(evidence).resolve()
|
||||
key = str(project).casefold()
|
||||
if (self._retry_counts.get(key) or self.record['state'] == 'blocked'
|
||||
or self.stop_event is not None and self.stop_event.is_set()):
|
||||
return False
|
||||
try:
|
||||
proof = json.loads((evidence / 'native-retry-proof.json').read_text(encoding='utf-8'))
|
||||
sources = proof['source_hashes']
|
||||
operation = self.record['native_operations'][str(evidence)]
|
||||
if (proof['version'] != 1 or proof['native_read_only'] is not True
|
||||
or operation['workers_closed'] is not True or proof['binding'] != operation['binding']
|
||||
or proof['binding']['session_id'] != self.session_id
|
||||
or proof['binding']['generation'] != self.generation
|
||||
or proof['workers_closed'] is not True or proof['project_path'] != str(project)
|
||||
or proof['evidence_root'] != str(evidence) or str(project) not in sources
|
||||
or len(sources) != 2
|
||||
or not all(Path(name).resolve().is_relative_to(project.parent) for name in sources)
|
||||
or not all(hashlib.sha256(Path(name).read_bytes()).hexdigest() == digest
|
||||
for name, digest in sources.items())
|
||||
or not all(can_retry_attempt(path, record)
|
||||
for path, record in self._project_attempts.get(key, ()))):
|
||||
return False
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
return False
|
||||
self._retry_counts[key] = 1
|
||||
self.record['retries'][key] = {'count': 1, 'reason': str(reason), 'backend': 'native_cli'}
|
||||
self._save(state='recycling')
|
||||
self.close(rebuilding=True)
|
||||
self.desktop = self.cad = None
|
||||
self._ensure_desktop()
|
||||
return True
|
||||
|
||||
def _release_log(self, phase, *, reason, elapsed=0.0, error=''):
|
||||
from .batch_release_log import write_release_event
|
||||
try:
|
||||
errors = write_release_event(self.root, self.path, self.record, phase,
|
||||
context=self._logging, reason=reason, elapsed=elapsed, error=error)
|
||||
except Exception as failure:
|
||||
errors = [str(failure)]
|
||||
if errors:
|
||||
self.record.setdefault('release_log_errors', []).extend(errors)
|
||||
try:
|
||||
self._save()
|
||||
except Exception:
|
||||
pass # A diagnostic write cannot bypass process/desktop cleanup.
|
||||
|
||||
def close(self, *, rebuilding=False, reason=None):
|
||||
if self.desktop is None and not self.generation:
|
||||
return # A cache-only batch never acquired resources.
|
||||
if self.record['state'] == 'released':
|
||||
return
|
||||
reason = reason or ('rebuild' if rebuilding else 'batch_end')
|
||||
started = time.monotonic()
|
||||
self._release_log('started', reason=reason)
|
||||
try:
|
||||
self._close_resources(rebuilding=rebuilding)
|
||||
except Exception as error:
|
||||
self._release_log('failed', reason=reason, elapsed=time.monotonic() - started, error=error)
|
||||
raise
|
||||
self._release_log('succeeded', reason=reason, elapsed=time.monotonic() - started)
|
||||
|
||||
def _close_resources(self, *, rebuilding=False):
|
||||
started = time.monotonic()
|
||||
errors = []
|
||||
try:
|
||||
self._save(state='closing')
|
||||
except Exception as error:
|
||||
errors.append('resource journal: ' + str(error))
|
||||
if self.cad is not None:
|
||||
cad, self.cad = self.cad, None
|
||||
try:
|
||||
cad.__exit__(None, None, None)
|
||||
except Exception as error:
|
||||
if rebuilding:
|
||||
self.record['shutdown_warning'] = str(error)
|
||||
else:
|
||||
errors.append(str(error))
|
||||
finally:
|
||||
cad_record = getattr(cad, 'record', {})
|
||||
cad_record = cad_record if isinstance(cad_record, dict) else {}
|
||||
self.record['cad_shutdown'] = {key: cad_record[key] for key in (
|
||||
'state', 'clean', 'cad_exit_code', 'quit_helper_exit_code', 'cleanup_errors',
|
||||
'shutdown_seconds') if key in cad_record}
|
||||
if self.desktop is not None:
|
||||
try:
|
||||
self.desktop.close()
|
||||
except Exception as error:
|
||||
errors.append(str(error))
|
||||
self._timing('batch_release', started)
|
||||
self.record['cleanup'] = getattr(self.desktop, 'cleanup_record', None)
|
||||
if errors:
|
||||
self._save(state='blocked', error='; '.join(errors))
|
||||
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
|
||||
released_at = time.time()
|
||||
release = {'generation': self.generation, 'desktop': self.record['desktop'],
|
||||
'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid'],
|
||||
'cleanup': self.record['cleanup'], 'released_at': released_at}
|
||||
releases = [*self.record.get('generation_releases', ()), release]
|
||||
self._save(state='released', released_at=released_at, processes=[], generation_releases=releases)
|
||||
LIVE_RESOURCES.pop(str(self.path), None)
|
||||
@@ -55,11 +55,11 @@ def _assert_editable(service, task_id):
|
||||
raise TaskCenterError('target_plan_locked', f'工程任务当前不能修改 BOM 计划:{context.task.name}')
|
||||
|
||||
|
||||
def preview_batch(service, task_id, path, *, stop_event=None, on_progress=None):
|
||||
def preview_batch(service, task_id, path, *, stop_event=None, on_progress=None, runtime=None):
|
||||
"""Strict adapter to the same validation engine used by the import report."""
|
||||
from .bom_import_report import inspect_batch
|
||||
|
||||
report = inspect_batch(service, task_id, path, stop_event=stop_event, on_progress=on_progress)
|
||||
report = inspect_batch(service, task_id, path, stop_event=stop_event, on_progress=on_progress, runtime=runtime)
|
||||
if report.preview is not None:
|
||||
return report.preview
|
||||
failed = next((row for row in report.rows if row.status == 'failed'), None)
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
"""Read-only, exhaustive BOM validation with an all-or-nothing preview."""
|
||||
import inspect
|
||||
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
from zipfile import BadZipFile
|
||||
@@ -8,6 +10,7 @@ from openpyxl import load_workbook
|
||||
from .bom_batch import BomBatchPreview, BomProjectPreview, _assert_editable, _identity, _scope
|
||||
from .bom_plan import BomMatcher, _normal_headers, _text, parse_bom_row
|
||||
from .errors import TaskCenterError
|
||||
from .project_discovery import ReleasedProjectReadError
|
||||
from .bom_import_state import BomImportPhase
|
||||
from .store_target_plan import merge_target_inventory
|
||||
from .target_plan import ProjectIdentityResolver, compile_target_rules
|
||||
@@ -70,6 +73,15 @@ class BomImportReport:
|
||||
return sum(project.status == 'unchecked' for project in self.projects)
|
||||
|
||||
|
||||
class BomInspectionError(TaskCenterError):
|
||||
"""Stop unsafe collection while retaining the complete, read-only row report."""
|
||||
|
||||
def __init__(self, cause: TaskCenterError, report: BomImportReport):
|
||||
super().__init__(cause.code, cause.message, error_id=cause.error_id,
|
||||
traceback_detail=cause.traceback_detail)
|
||||
self.bom_report = report
|
||||
|
||||
|
||||
def _report(rows, preview=None, errors=()):
|
||||
grouped = {}
|
||||
for row in rows:
|
||||
@@ -111,12 +123,30 @@ def _check_record_conflicts(rows, records, indices):
|
||||
rows[index] = replace(rows[index], status='failed', reason=reason)
|
||||
|
||||
|
||||
def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None, on_phase=None):
|
||||
def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None, on_phase=None, runtime=None):
|
||||
"""Validate all independent rows; never persist inventory or return partial plans.
|
||||
|
||||
Operational failures (including cancellation and native cleanup) propagate so
|
||||
callers cannot accidentally continue opening CAD after an unsafe failure.
|
||||
"""
|
||||
if runtime is None:
|
||||
from .batch_runtime import BatchCadRuntime, BatchResourceCleanupError
|
||||
report = None
|
||||
try:
|
||||
with BatchCadRuntime(service._logging.root, 'bom_import', stop_event) as owned_runtime:
|
||||
try:
|
||||
report = inspect_batch(service, task_id, path, stop_event=stop_event,
|
||||
on_progress=on_progress, on_phase=on_phase, runtime=owned_runtime)
|
||||
except BomInspectionError as error:
|
||||
report = error.bom_report
|
||||
raise
|
||||
owned_runtime.result = report
|
||||
return report
|
||||
except BatchResourceCleanupError as error:
|
||||
error.bom_report = replace(report, preview=None) if report is not None else None
|
||||
if isinstance(error.completed_result, BomImportReport):
|
||||
error.completed_result = replace(error.completed_result, preview=None)
|
||||
raise
|
||||
def check_cancelled():
|
||||
if stop_event is not None and stop_event.is_set():
|
||||
raise TaskCenterError('operation_cancelled', 'BOM 导入已取消')
|
||||
@@ -130,6 +160,7 @@ def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None,
|
||||
check_cancelled()
|
||||
notify('正在读取并校验 BOM Excel 表格…', BomImportPhase.READING)
|
||||
scope = _scope(service._store, task_id)
|
||||
runtime.bind_logs(service._logging, tuple(task.task_id for task, _ in scope))
|
||||
routing = ProjectIdentityResolver()
|
||||
owners = {routing.project(project.path): (task, project) for task, project in scope}
|
||||
if len(owners) != len(scope):
|
||||
@@ -233,13 +264,34 @@ def inspect_batch(service, task_id, path, *, stop_event=None, on_progress=None,
|
||||
inventory = tuple(row.target for row in baseline)
|
||||
else:
|
||||
progress(f'「{task.name}」正在获取位号库存,仅采集本表涉及的旧料号及已有计划料号')
|
||||
controls = {}
|
||||
collector = service.for_project_task(task.task_id)._quick_run_preparer.collect_inventory
|
||||
controls = ({'runtime': runtime} if any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in inspect.signature(collector).parameters.values()) else {})
|
||||
if stop_event is not None:
|
||||
controls['stop_event'] = stop_event
|
||||
if on_progress is not None:
|
||||
controls['on_progress'] = progress
|
||||
inventory = tuple(service.for_project_task(task.task_id)._quick_run_preparer.collect_inventory(
|
||||
project.path, tuple(dict.fromkeys((*known, *requested))), **controls))
|
||||
try:
|
||||
inventory = tuple(collector(
|
||||
project.path, tuple(dict.fromkeys((*known, *requested))), **controls))
|
||||
except TaskCenterError as error:
|
||||
if error.code == 'operation_cancelled':
|
||||
raise
|
||||
reason = f'工程采集失败:{error.message}(错误编号:{error.error_id})'
|
||||
for index in indices:
|
||||
rows[index] = replace(rows[index], status='failed', reason=reason)
|
||||
# Continue only with positive session-release evidence, not
|
||||
# merely a shared error code or an absent uncertainty marker.
|
||||
if isinstance(error, ReleasedProjectReadError):
|
||||
progress(f'「{task.name}」采集失败,已记录;继续核验其他工程')
|
||||
continue
|
||||
for index, row in enumerate(rows):
|
||||
if row.status == 'unchecked':
|
||||
rows[index] = replace(row, reason='前序工程采集异常,已停止后续核验')
|
||||
report = _report(rows, errors=(
|
||||
f'已停止批量采集:{task.name};{error.message}(错误编号:{error.error_id})',))
|
||||
raise BomInspectionError(error, report) from error
|
||||
check_cancelled()
|
||||
progress(f'「{task.name}」正在匹配 {len(indices)} 行 BOM 与工程位号…', BomImportPhase.MATCHING)
|
||||
merged = merge_target_inventory(baseline, inventory)
|
||||
|
||||
@@ -90,8 +90,12 @@ class BomImportMachine:
|
||||
return False
|
||||
cancelled_error = isinstance(error, TaskCenterError) and error.code == 'operation_cancelled'
|
||||
if error is not None and not cancelled_error:
|
||||
from .bom_import_report import BomImportReport
|
||||
reason = error.message if isinstance(error, TaskCenterError) else str(error)
|
||||
self.state = replace(self.state, phase=BomImportPhase.FAILED, report=None,
|
||||
report = getattr(error, 'bom_report', None)
|
||||
if not isinstance(report, BomImportReport):
|
||||
report = None
|
||||
self.state = replace(self.state, phase=BomImportPhase.FAILED, report=report,
|
||||
message=f'本次 BOM 导入未完成:{reason[:240]};原有预览和编辑保留。')
|
||||
elif cancelled_error or self.state.phase is BomImportPhase.CANCELLING:
|
||||
self.state = replace(self.state, phase=BomImportPhase.CANCELLED, report=None,
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from pathlib import Path
|
||||
|
||||
from .errors import TaskCenterError
|
||||
@@ -58,11 +59,15 @@ def read_component_inventory(path: Path, project_path: Path,
|
||||
|
||||
|
||||
def collect_target_inventory(preparer, project_path: Path,
|
||||
old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None) -> tuple[ComponentTarget, ...]:
|
||||
old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None, runtime=None) -> tuple[ComponentTarget, ...]:
|
||||
if not old_part_numbers:
|
||||
raise TaskCenterError("target_inventory", "at least one old Part Number is required")
|
||||
from .operation_control import check_cancelled, control_options
|
||||
check_cancelled(stop_event)
|
||||
rows = preparer.collect_inventory(project_path, old_part_numbers, **control_options(stop_event, on_progress))
|
||||
runtime_options = ({'runtime': runtime} if runtime is not None and any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in inspect.signature(preparer.collect_inventory).parameters.values()) else {})
|
||||
rows = preparer.collect_inventory(project_path, old_part_numbers, **runtime_options,
|
||||
**control_options(stop_event, on_progress))
|
||||
check_cancelled(stop_event)
|
||||
return rows
|
||||
@@ -18,6 +18,34 @@ class PrivateCadStartupError(WorkerMonitorError):
|
||||
"""The owned desktop was released before any project worker was launched."""
|
||||
|
||||
|
||||
class CadIdentityConflict(WorkerMonitorError, ValueError):
|
||||
"""An invalid CAD ownership certificate can never authorize automatic replay."""
|
||||
|
||||
|
||||
class PrivateCadIdentityError(PrivateCadStartupError, CadIdentityConflict):
|
||||
"""Startup ownership failed, and its local resources were released."""
|
||||
|
||||
|
||||
PRIVATE_CAD_IDENTITY_EXIT_CODE = 4
|
||||
|
||||
|
||||
def is_identity_conflict(error):
|
||||
"""Keep the no-retry decision when cleanup or evidence I/O wraps the cause."""
|
||||
pending, seen = [error], set()
|
||||
while pending:
|
||||
item = pending.pop()
|
||||
if item is None or id(item) in seen:
|
||||
continue
|
||||
seen.add(id(item))
|
||||
# The legacy VBS launcher transports these ownership failures as exit
|
||||
# 2 plus text. Treat only its explicit identity diagnostics as conflicts.
|
||||
if (isinstance(item, CadIdentityConflict) or any(message in str(item).casefold()
|
||||
for message in ('incomplete private cad identity', 'private cad guid mismatch'))):
|
||||
return True
|
||||
pending.extend((item.__cause__, item.__context__))
|
||||
return False
|
||||
|
||||
|
||||
def _helper_command(action, evidence, pid):
|
||||
if getattr(sys, 'frozen', False):
|
||||
return (sys.executable, '--private-cad-action', action, '--private-cad-evidence', str(evidence),
|
||||
@@ -65,10 +93,11 @@ def _designer_executable():
|
||||
|
||||
class IsolatedCadSession:
|
||||
"""The supervisor never attaches to COM on the user's input desktop."""
|
||||
def __init__(self, root, *, abort_event=None):
|
||||
def __init__(self, root, *, abort_event=None, desktop=None):
|
||||
self.root = Path(root).resolve()
|
||||
self.token = uuid4().hex
|
||||
self.desktop = PrivateDesktop('dx-' + self.token, enabled=True)
|
||||
self.borrowed_desktop = desktop is not None
|
||||
self.desktop = desktop if desktop is not None else PrivateDesktop('dx-' + self.token, enabled=True)
|
||||
self.cad = None
|
||||
self.binding = None
|
||||
self.started = time.monotonic()
|
||||
@@ -110,13 +139,19 @@ class IsolatedCadSession:
|
||||
time.sleep(0.05)
|
||||
code = probe.poll()
|
||||
if code:
|
||||
raise WorkerMonitorError('private CAD startup identity failed: ' + probe.stderr.read().decode('utf-8', errors='replace'))
|
||||
failure_type = CadIdentityConflict if code == PRIVATE_CAD_IDENTITY_EXIT_CODE else WorkerMonitorError
|
||||
raise failure_type('private CAD startup failed: ' + probe.stderr.read().decode('utf-8', errors='replace'))
|
||||
finally:
|
||||
probe.close()
|
||||
result = json.loads(bootstrap.read_text(encoding='utf-8'))
|
||||
self.binding = CadBinding(**result)
|
||||
if (self.binding.pid, self.binding.desktop, self.binding.token) != (self.cad.pid, self.desktop.name, self.token):
|
||||
raise WorkerMonitorError('private CAD bootstrap returned a foreign identity')
|
||||
try:
|
||||
result = json.loads(bootstrap.read_text(encoding='utf-8'))
|
||||
binding = CadBinding(**result)
|
||||
if (type(binding.pid) is not int or not isinstance(binding.guid, str) or not binding.guid
|
||||
or (binding.pid, binding.desktop, binding.token) != (self.cad.pid, self.desktop.name, self.token)):
|
||||
raise ValueError('private CAD bootstrap returned a foreign identity')
|
||||
except (OSError, ValueError, TypeError) as failure:
|
||||
raise CadIdentityConflict('private CAD bootstrap identity evidence is invalid: ' + str(failure)) from failure
|
||||
self.binding = binding
|
||||
self._record(state='ready', guid=self.binding.guid, startup_seconds=time.monotonic() - self.started)
|
||||
return self
|
||||
except Exception as error:
|
||||
@@ -126,16 +161,51 @@ class IsolatedCadSession:
|
||||
self._uncertain('startup cleanup failed: ' + str(failure))
|
||||
raise WorkerMonitorError('private CAD startup cleanup requires reconciliation: ' + str(failure)) from error
|
||||
self._record(state='released', clean=False, startup_error=str(error))
|
||||
if is_identity_conflict(error):
|
||||
raise PrivateCadIdentityError(str(error)) from error
|
||||
raise PrivateCadStartupError(str(error)) from error
|
||||
|
||||
def _check_abort(self):
|
||||
if self.abort_event is not None and self.abort_event.is_set():
|
||||
raise WorkerMonitorError('private CAD startup was cancelled before execution')
|
||||
|
||||
def start_worker(self, command, **options):
|
||||
def start_worker(self, command, *, extra_environment=None, **options):
|
||||
if self.binding is None or self.cad.poll() is not None:
|
||||
raise WorkerMonitorError('owned private CAD is unavailable')
|
||||
return self.desktop.start_process(command, env=_child_environment(self.binding.environment()), **options)
|
||||
environment = self.binding.environment()
|
||||
environment.update(extra_environment or {})
|
||||
# The verified CAD binding cannot be overridden by attempt metadata.
|
||||
binding = self.binding.environment()
|
||||
for key in ('DXPART_DESKTOP', 'DXPART_CAD_PID', 'DXPART_CAD_GUID', 'DXPART_SESSION_TOKEN'):
|
||||
environment[key] = binding[key]
|
||||
return self.desktop.start_process(command, env=_child_environment(environment), **options)
|
||||
|
||||
def checkpoint(self, evidence):
|
||||
"""Certify an idle, identity-verified CAD before crossing project boundaries."""
|
||||
if self.binding is None or self.cad.poll() is not None:
|
||||
raise WorkerMonitorError('owned CAD unavailable at project checkpoint')
|
||||
evidence = Path(evidence)
|
||||
evidence.parent.mkdir(parents=True, exist_ok=True)
|
||||
process = self.start_worker(_helper_command('checkpoint', evidence, self.cad.pid),
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
try:
|
||||
code = process.wait(timeout=20)
|
||||
if code:
|
||||
failure_type = CadIdentityConflict if code == PRIVATE_CAD_IDENTITY_EXIT_CODE else WorkerMonitorError
|
||||
raise failure_type('project close checkpoint failed: ' +
|
||||
process.stderr.read().decode('utf-8', errors='replace'))
|
||||
try:
|
||||
result = json.loads(evidence.read_text(encoding='utf-8'))
|
||||
if (not isinstance(result, dict) or result.get('binding') != asdict(self.binding)
|
||||
or result.get('project_path') != '' or result.get('project_closed') is not True):
|
||||
raise ValueError('project close checkpoint identity mismatch')
|
||||
except (OSError, ValueError, TypeError) as failure:
|
||||
raise CadIdentityConflict('project close checkpoint identity evidence is invalid: ' + str(failure)) from failure
|
||||
if self.cad.poll() is not None:
|
||||
raise WorkerMonitorError('owned CAD exited during project checkpoint')
|
||||
finally:
|
||||
process.close()
|
||||
return result
|
||||
|
||||
def __exit__(self, error_type, error, traceback):
|
||||
closing_started = time.monotonic()
|
||||
@@ -174,7 +244,9 @@ class IsolatedCadSession:
|
||||
finally:
|
||||
try:
|
||||
self.desktop.close()
|
||||
self._record(state='released', clean=error is None and cleanup_error is None,
|
||||
self._record(state='session_closed' if self.borrowed_desktop else 'released',
|
||||
desktop_retained_by_batch=self.borrowed_desktop,
|
||||
clean=error is None and cleanup_error is None,
|
||||
shutdown_seconds=time.monotonic() - closing_started,
|
||||
desktop_cleanup=getattr(self.desktop, 'cleanup_record', None), **exit_codes)
|
||||
except Exception as failure:
|
||||
@@ -190,12 +262,17 @@ class IsolatedCadSession:
|
||||
self._record(state='uncertain', error=message, **details)
|
||||
|
||||
|
||||
def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abort=False):
|
||||
def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abort=False, runtime=None):
|
||||
from .worker_monitor import run_monitored_process
|
||||
if runtime is not None:
|
||||
return runtime.run_worker(command, timeout, abort_event=abort_event,
|
||||
cooperative_abort=cooperative_abort)
|
||||
try:
|
||||
with IsolatedCadSession(Path(command[3]), abort_event=abort_event) as session:
|
||||
return run_monitored_process(command, timeout, abort_event=abort_event, process_factory=session.start_worker,
|
||||
**({'cooperative_abort': True} if cooperative_abort else {}))
|
||||
except CadIdentityConflict:
|
||||
raise
|
||||
except PrivateCadStartupError as error:
|
||||
# The existing handoff recognizes exit=2 with unchanged status as a
|
||||
# retryable pre-mutation startup failure, without an uncertainty lock.
|
||||
@@ -203,6 +280,14 @@ def run_isolated_process(command, timeout, *, abort_event=None, cooperative_abor
|
||||
|
||||
|
||||
def native_action(action, evidence, expected_pid):
|
||||
try:
|
||||
return _run_native_action(action, evidence, expected_pid)
|
||||
except CadIdentityConflict as error:
|
||||
print(str(error), file=sys.stderr)
|
||||
return PRIVATE_CAD_IDENTITY_EXIT_CODE
|
||||
|
||||
|
||||
def _run_native_action(action, evidence, expected_pid):
|
||||
"""Runs in a child created with lpDesktop, never in a Python thread switch."""
|
||||
if action != 'probe':
|
||||
return _native_action(action, evidence, expected_pid)
|
||||
@@ -228,10 +313,10 @@ def _native_action(action, evidence, expected_pid):
|
||||
desktop = os.environ.get('DXPART_DESKTOP', '')
|
||||
token = os.environ.get('DXPART_SESSION_TOKEN', '')
|
||||
if not desktop.startswith('dx-') or current_desktop_name() != desktop:
|
||||
raise ValueError('CAD bootstrap is not on the requested private desktop')
|
||||
raise CadIdentityConflict('CAD bootstrap is not on the requested private desktop')
|
||||
expected_pid = int(expected_pid)
|
||||
if expected_pid <= 0 or str(expected_pid) != os.environ.get('DXPART_CAD_PID'):
|
||||
raise ValueError('invalid CAD bootstrap PID')
|
||||
raise CadIdentityConflict('invalid CAD bootstrap PID')
|
||||
deadline = time.monotonic() + 60
|
||||
while True:
|
||||
try:
|
||||
@@ -250,16 +335,29 @@ def _native_action(action, evidence, expected_pid):
|
||||
# ROT is scoped to this desktop. Certify the freshly launched instance
|
||||
# here; every subsequent VBS/helper attachment checks its GUID.
|
||||
binding = CadBinding(desktop, expected_pid, str(app.Guid()), token)
|
||||
verify_application(app, binding, require_view=False)
|
||||
try:
|
||||
verify_application(app, binding, require_view=False)
|
||||
except ValueError as error:
|
||||
raise CadIdentityConflict(str(error)) from error
|
||||
Path(evidence).write_text(json.dumps(asdict(binding)), encoding='utf-8')
|
||||
elif action == 'quit':
|
||||
binding = CadBinding.from_environment()
|
||||
elif action in {'quit', 'checkpoint'}:
|
||||
try:
|
||||
binding = CadBinding.from_environment()
|
||||
if binding is None:
|
||||
raise ValueError('missing private CAD identity')
|
||||
verify_application(app, binding, require_view=False)
|
||||
opened_project = project_file_path(app)
|
||||
except ValueError as error:
|
||||
raise CadIdentityConflict(str(error)) from error
|
||||
# ActiveViewHandle can retain a destroyed/reused HWND after closing
|
||||
# the last project. The private ROT/GUID binds this empty instance.
|
||||
verify_application(app, binding, require_view=False)
|
||||
if project_file_path(app):
|
||||
raise ValueError('owned CAD still has a project open; no implicit save or discard')
|
||||
app.Quit()
|
||||
if opened_project:
|
||||
raise CadIdentityConflict('owned CAD still has a project open; no implicit save or discard')
|
||||
if action == 'quit':
|
||||
app.Quit()
|
||||
else:
|
||||
from .batch_resources import write_record
|
||||
write_record(evidence, {'binding': asdict(binding), 'project_path': '', 'project_closed': True})
|
||||
else:
|
||||
raise ValueError('unknown private CAD action')
|
||||
except BaseException as failure:
|
||||
|
||||
@@ -74,9 +74,40 @@ class _Win32Api:
|
||||
self.desktop_release_check['caller_desktop'] = current_desktop_name()
|
||||
except Exception as error:
|
||||
self.desktop_release_check['caller_desktop_error'] = str(error)
|
||||
try:
|
||||
self.desktop_release_check['remaining_windows'] = self._desktop_windows(name)
|
||||
except Exception as error:
|
||||
self.desktop_release_check['window_query_error'] = str(error)
|
||||
raise TaskCenterError('private_desktop_cleanup', 'owned private desktop still exists after cleanup')
|
||||
time.sleep(min(0.05, max(0.0, deadline - time.monotonic())))
|
||||
|
||||
def _desktop_windows(self, name):
|
||||
"""Diagnose residue without switching desktops or terminating foreign owners."""
|
||||
import win32gui
|
||||
|
||||
desktop = self.service.OpenDesktop(name, 0, False, 1) # DESKTOP_READOBJECTS
|
||||
windows = []
|
||||
try:
|
||||
user = ctypes.WinDLL('user32', use_last_error=True)
|
||||
callback_type = ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
|
||||
enumerate_windows = user.EnumDesktopWindows
|
||||
enumerate_windows.argtypes = [wintypes.HANDLE, callback_type, wintypes.LPARAM]
|
||||
enumerate_windows.restype = wintypes.BOOL
|
||||
handles = []
|
||||
callback = callback_type(lambda handle, _: handles.append(handle) or True)
|
||||
# An empty desktop can return FALSE with no last error. Our
|
||||
# callback always continues, so only a nonzero error is a failure.
|
||||
ctypes.set_last_error(0)
|
||||
if not enumerate_windows(int(desktop), callback, 0) and ctypes.get_last_error():
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
for handle in handles:
|
||||
thread, pid = self.process.GetWindowThreadProcessId(handle)
|
||||
windows.append({'hwnd': int(handle), 'pid': pid, 'thread_id': thread,
|
||||
'class': win32gui.GetClassName(handle)})
|
||||
finally:
|
||||
desktop.CloseDesktop()
|
||||
return windows
|
||||
|
||||
def create_process(self, command: Sequence[str], desktop: str, cwd: Path | None, env: Mapping[str, str] | None,
|
||||
*, stdout=None, stderr=None):
|
||||
startup = self.process.STARTUPINFO()
|
||||
@@ -442,6 +473,16 @@ class PrivateDesktop:
|
||||
self._processes.append(process)
|
||||
return process
|
||||
|
||||
def forget_closed_process(self, process):
|
||||
"""Drop a worker only after its owning wrapper confirmed all cleanup."""
|
||||
record = getattr(process, 'cleanup_record', {})
|
||||
if (record.get('process_handle_closed') is not True
|
||||
or record.get('job_handle_closed') is not True
|
||||
or record.get('unconfirmed_descendant_handles', 0)
|
||||
or record.get('errors') or process.poll() is None):
|
||||
raise OSError('cannot forget a worker with unconfirmed owned resources')
|
||||
self._processes = [item for item in self._processes if item is not process]
|
||||
|
||||
def close(self) -> None:
|
||||
errors, remaining = [], []
|
||||
self.cleanup_record = {'desktop': self.name, 'processes': []}
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
import argparse
|
||||
import faulthandler
|
||||
import ctypes
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -13,6 +14,48 @@ import time
|
||||
import traceback
|
||||
|
||||
|
||||
def snapshot_attempt(snapshot):
|
||||
"""Bind batch evidence to the actual snapshot and its confirmed queue row."""
|
||||
from .batch_attempt import _bound_attempt
|
||||
if 'DXPART_ATTEMPT_FILE' not in os.environ:
|
||||
return _bound_attempt()
|
||||
snapshot = Path(snapshot)
|
||||
metadata, _ = read_snapshot(snapshot)
|
||||
if not snapshot.name.startswith('snapshot-') or snapshot.suffix != '.tsv':
|
||||
raise ValueError('batch snapshot has no job identity')
|
||||
job_id = snapshot.stem[len('snapshot-'):]
|
||||
with (snapshot.parent / 'command.tsv').open(encoding='utf-8-sig', newline='') as handle:
|
||||
commands = [row for row in csv.DictReader(handle, delimiter='\t') if row.get('job_id') == job_id]
|
||||
if len(commands) != 1:
|
||||
raise ValueError('batch snapshot must match one confirmed command')
|
||||
command = commands[0]
|
||||
if os.path.normcase(os.path.abspath(command['project_path'])) != os.path.normcase(os.path.abspath(metadata[1])):
|
||||
raise ValueError('batch command and snapshot projects differ')
|
||||
return _bound_attempt(job_id=job_id, task_id=command['task_id'], project_path=metadata[1])
|
||||
|
||||
|
||||
def mark_snapshot_mutation(snapshot):
|
||||
from .batch_attempt import mark_mutation_possible
|
||||
bound = snapshot_attempt(snapshot)
|
||||
if bound is None:
|
||||
return None
|
||||
path, record = bound
|
||||
return mark_mutation_possible(path, job_id=record['job_id'], task_id=record['task_id'],
|
||||
project_path=record['project_path'])
|
||||
|
||||
|
||||
def run_attempt_marker(*, job_id, task_id, project_path):
|
||||
from .batch_attempt import mark_mutation_possible
|
||||
if not job_id or not task_id or not project_path:
|
||||
return 2
|
||||
try:
|
||||
mark_mutation_possible(job_id=job_id, task_id=task_id, project_path=project_path)
|
||||
return 0
|
||||
except (OSError, ValueError, KeyError, TypeError) as error:
|
||||
print(f'Batch mutation proof failed: {error}', file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def decode(value):
|
||||
characters = []
|
||||
index = 0
|
||||
@@ -542,6 +585,7 @@ def run_native(snapshot, report):
|
||||
slot_count = int(binding.get("slot_count", report.get("slot_count", 1)))
|
||||
if slot_count < 1:
|
||||
raise ValueError("Recorded replacement has no valid slots")
|
||||
mark_snapshot_mutation(snapshot)
|
||||
report["promoted_pins"] = promote_pin_numbers(app, api, report["replaced_uids"], slot_count,
|
||||
lambda phase: _promotion_phase(snapshot, report, phase))
|
||||
report["already_replaced"] = True
|
||||
@@ -706,6 +750,7 @@ def run_native(snapshot, report):
|
||||
cell_catalog=cell_proof)
|
||||
report["stage"] = "native_replace_started"
|
||||
save_report(snapshot, report)
|
||||
mark_snapshot_mutation(snapshot)
|
||||
click(dlg_item(dialog, 1695))
|
||||
|
||||
replacement_poll_count = 0
|
||||
@@ -820,6 +865,15 @@ def run(snapshot):
|
||||
watchdog.daemon = True
|
||||
watchdog.start()
|
||||
try:
|
||||
bound = snapshot_attempt(snapshot)
|
||||
if bound is not None:
|
||||
_, current = bound
|
||||
identity = {key: current[key] for key in
|
||||
('session_id', 'generation', 'attempt_id', 'task_id', 'job_id', 'project_path')}
|
||||
if report.get('batch_attempt') != identity:
|
||||
report.clear()
|
||||
report.update(snapshot=str(snapshot), ok=False, stage='preflight')
|
||||
report['batch_attempt'] = identity
|
||||
run_native(snapshot, report)
|
||||
report["ok"] = True
|
||||
return 0
|
||||
|
||||
@@ -17,6 +17,15 @@ from .native_inventory_protocol import NativeInventoryUnsupported, read_ascii_di
|
||||
from .operation_control import check_cancelled, report_progress
|
||||
|
||||
|
||||
_RETRY_NATIVE_READ = object()
|
||||
_NATIVE_CRASH_CODES = frozenset({0xC0000005, 0xC000001D, 0xC0000094,
|
||||
0xC00000FD, 0xC0000374, 0xC0000409})
|
||||
|
||||
|
||||
class _NativeInventoryProcessError(NativeInventoryUnsupported):
|
||||
"""A launch, watchdog or explicit crash failure, separate from invalid input."""
|
||||
|
||||
|
||||
def _hash(path):
|
||||
with path.open('rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
@@ -71,7 +80,7 @@ class NativeInventoryReader:
|
||||
raise NativeInventoryUnsupported('本机未安装完整的原生器件清单工具')
|
||||
return tools
|
||||
|
||||
def read(self, project, old_part_numbers, *, stop_event=None, on_progress=None):
|
||||
def read(self, project, old_part_numbers, *, stop_event=None, on_progress=None, runtime=None):
|
||||
check_cancelled(stop_event)
|
||||
try:
|
||||
project = Path(project).resolve(strict=True)
|
||||
@@ -80,10 +89,15 @@ class NativeInventoryReader:
|
||||
except (NativeInventoryUnsupported, OSError, ImportError, UnicodeError, TaskCenterError) as error:
|
||||
report_progress(on_progress, f'使用 CAD 核验位号:{str(error)[:160]}')
|
||||
return None
|
||||
return self._read(project, text, block, snapshot, database, tools, old_part_numbers,
|
||||
stop_event=stop_event, on_progress=on_progress)
|
||||
for attempt in range(2):
|
||||
result = self._read(project, text, block, snapshot, database, tools, old_part_numbers,
|
||||
stop_event=stop_event, on_progress=on_progress, runtime=runtime,
|
||||
allow_retry=runtime is not None and attempt == 0)
|
||||
if result is not _RETRY_NATIVE_READ:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _read(self, project, text, block, snapshot, database, tools, old_part_numbers, *, stop_event, on_progress):
|
||||
def _read(self, project, text, block, snapshot, database, tools, old_part_numbers, *, stop_event, on_progress, runtime=None, allow_retry=False):
|
||||
token = uuid4().hex
|
||||
evidence = self.data_root / 'quick-run-history' / ('native-inventory-' + token)
|
||||
evidence.mkdir(parents=True, exist_ok=False)
|
||||
@@ -91,7 +105,8 @@ class NativeInventoryReader:
|
||||
/ 'DxProductivity' / 'native-inventory').resolve()
|
||||
base.mkdir(parents=True, exist_ok=True)
|
||||
workspace = Path(tempfile.mkdtemp(prefix='dx-cli-', dir=base)).resolve()
|
||||
desktop = self.desktop_factory('dx-cli-' + token, enabled=True)
|
||||
desktop = (runtime.native_scope(evidence) if runtime is not None
|
||||
else self.desktop_factory('dx-cli-' + token, enabled=True))
|
||||
record = {'backend': 'native_cli', 'project': str(project), 'state': 'reading',
|
||||
'workspace': str(workspace), 'tools': []}
|
||||
started = time.monotonic()
|
||||
@@ -151,9 +166,40 @@ class NativeInventoryReader:
|
||||
if record['state'] == 'uncertain':
|
||||
raise TaskCenterError('inventory_cleanup', f'原生采集清理状态不确定:{evidence}') from error
|
||||
_verify_sources(source_hashes)
|
||||
record.update(state='fallback', reason=str(error))
|
||||
report_progress(on_progress, f'原生清单未能完整核对,改用 CAD 采集:{str(error)[:160]}')
|
||||
record.update(source_hashes_unchanged=True,
|
||||
source_hashes={str(path): digest for path, digest in source_hashes.items()})
|
||||
retry = False
|
||||
cleanup = getattr(desktop, 'cleanup_record', None)
|
||||
session_failure = runtime is not None and isinstance(error, _NativeInventoryProcessError)
|
||||
if (session_failure
|
||||
and clean and isinstance(cleanup, dict) and cleanup.get('workers_closed') is True):
|
||||
check_cancelled(stop_event)
|
||||
from .batch_resources import write_record
|
||||
write_record(evidence / 'native-retry-proof.json', {
|
||||
'version': 1, 'native_read_only': True, 'project_path': str(project),
|
||||
'source_hashes': {str(path): digest for path, digest in source_hashes.items()},
|
||||
'workers_closed': True, 'evidence_root': str(evidence.resolve()),
|
||||
'binding': dict(desktop.identity),
|
||||
})
|
||||
_verify_sources(source_hashes)
|
||||
if allow_retry:
|
||||
retry = runtime.recover_native_failure(project, evidence, str(error))
|
||||
if session_failure and not retry:
|
||||
record.update(state='session_failed', reason=str(error))
|
||||
raise TaskCenterError('inventory_session',
|
||||
f'原生读取会话异常,未能完成安全重试,批次已停止。请保留诊断记录后排查:{evidence};{error}') from error
|
||||
record.update(state='retrying' if retry else 'fallback', reason=str(error))
|
||||
if retry:
|
||||
result = _RETRY_NATIVE_READ
|
||||
report_progress(on_progress, '原生读取会话已安全释放,正在重新核验一次…')
|
||||
else:
|
||||
report_progress(on_progress, f'原生清单未能完整核对,改用 CAD 采集:{str(error)[:160]}')
|
||||
except TaskCenterError as error:
|
||||
if record['state'] == 'uncertain':
|
||||
try:
|
||||
(evidence / 'uncertain.flag').write_text(error.message, encoding='utf-8')
|
||||
except OSError as marker_error:
|
||||
record['uncertain_marker_error'] = str(marker_error)
|
||||
if record['state'] != 'uncertain':
|
||||
record.update(state='cancelled' if error.code == 'operation_cancelled' else 'failed', reason=error.message)
|
||||
raise
|
||||
@@ -181,19 +227,32 @@ class NativeInventoryReader:
|
||||
started = time.monotonic()
|
||||
metric = {'name': name, 'report': report_name, 'exit_code': None}
|
||||
record['tools'].append(metric)
|
||||
process = None
|
||||
try:
|
||||
with (output / (log_name + '.stdout.txt')).open('wb') as stdout, (
|
||||
output / (log_name + '.stderr.txt')).open('wb') as stderr:
|
||||
process = desktop.start_process(command, cwd=workspace, env=environment,
|
||||
stdout=stdout, stderr=stderr)
|
||||
try:
|
||||
process = desktop.start_process(command, cwd=workspace, env=environment,
|
||||
stdout=stdout, stderr=stderr)
|
||||
except OSError as error:
|
||||
raise _NativeInventoryProcessError(f'{name} 启动失败:{error}') from error
|
||||
deadline = started + self.timeout
|
||||
while process.poll() is None:
|
||||
check_cancelled(stop_event)
|
||||
if time.monotonic() >= deadline:
|
||||
raise NativeInventoryUnsupported(f'{name} 原生读取超时')
|
||||
raise _NativeInventoryProcessError(f'{name} 原生读取超时')
|
||||
time.sleep(0.05)
|
||||
metric['exit_code'] = process.poll()
|
||||
if metric['exit_code'] & 0xFFFFFFFF in _NATIVE_CRASH_CODES:
|
||||
raise _NativeInventoryProcessError(f'{name} 原生读取进程崩溃:{metric["exit_code"]}')
|
||||
if metric['exit_code'] != 0:
|
||||
raise NativeInventoryUnsupported(f'{name} 返回退出码 {metric["exit_code"]}')
|
||||
finally:
|
||||
metric['seconds'] = time.monotonic() - started
|
||||
if process is not None and callable(getattr(process, 'close', None)):
|
||||
try:
|
||||
process.close()
|
||||
except Exception as error:
|
||||
record.update(state='uncertain', cleanup_error=str(error))
|
||||
raise TaskCenterError('inventory_cleanup',
|
||||
f'原生读取进程及其子进程未能确认退出:{error}') from error
|
||||
@@ -41,6 +41,7 @@ def _effective_properties(text):
|
||||
if len(re.findall(r'^Config .+', text, re.MULTILINE)) != 1:
|
||||
raise NativeInventoryUnsupported('展开报告必须具有唯一配置')
|
||||
result, attributes, paths = {}, {}, []
|
||||
inconsistent_refs = set()
|
||||
symbol = None
|
||||
|
||||
def finish():
|
||||
@@ -54,7 +55,7 @@ def _effective_properties(text):
|
||||
if uid in result or paths != ['']:
|
||||
raise NativeInventoryUnsupported('展开报告含重复或嵌套实例,需由 CAD 核验')
|
||||
if ref != attributes.get('Ref Designator', ''):
|
||||
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
|
||||
inconsistent_refs.add(uid)
|
||||
result[uid] = dict(attributes)
|
||||
if len(result) > 100000:
|
||||
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
|
||||
@@ -76,10 +77,10 @@ def _effective_properties(text):
|
||||
raise NativeInventoryUnsupported('展开报告的器件属性不唯一')
|
||||
attributes[key] = value or ''
|
||||
finish()
|
||||
return result
|
||||
return result, inconsistent_refs
|
||||
|
||||
|
||||
def _parse_scope(text, requested, effective):
|
||||
def _parse_scope(text, requested, effective, inconsistent_refs):
|
||||
blocks, sheets, targets = set(), set(), set()
|
||||
seen_uids = set()
|
||||
block, symbol, attributes = '', None, {}
|
||||
@@ -100,6 +101,11 @@ def _parse_scope(text, requested, effective):
|
||||
if 'Value' in attributes and 'Value' not in values:
|
||||
raise NativeInventoryUnsupported('展开报告缺少 Value 生效属性')
|
||||
if values.get('Part Number', '').casefold() in requested:
|
||||
# A library reference override on an unrelated part cannot affect
|
||||
# the requested identities. Still validate every UID/config/path,
|
||||
# and require CAD whenever the ambiguous component is requested.
|
||||
if uid in inconsistent_refs:
|
||||
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
|
||||
ref, part = values.get('Ref Designator', ''), values['Part Number']
|
||||
if not ref or 'Ref Designator' not in attributes:
|
||||
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
|
||||
@@ -146,9 +152,9 @@ def read_ascii_discovery(project, report_path, old_part_numbers, *, expected_roo
|
||||
_require_report(text, 'HIER BLOCKS')
|
||||
if effective_report_path is None:
|
||||
raise NativeInventoryUnsupported('缺少实例生效属性报告,需由 CAD 核验')
|
||||
effective = _effective_properties(read_native_text(effective_report_path))
|
||||
effective, inconsistent_refs = _effective_properties(read_native_text(effective_report_path))
|
||||
requested = {part.casefold() for part in old_part_numbers}
|
||||
blocks, available, targets = _parse_scope(text, requested, effective)
|
||||
blocks, available, targets = _parse_scope(text, requested, effective, inconsistent_refs)
|
||||
if expected_root is not None and expected_root not in blocks:
|
||||
raise NativeInventoryUnsupported('原生全工程报告与工程根块身份不一致')
|
||||
components = tuple(ComponentTarget(Path(project), *row) for row in sorted(targets))
|
||||
|
||||
@@ -3,6 +3,7 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
@@ -13,6 +14,30 @@ from .quick_run_protocol import ProjectDiscovery, _encode_vbs_field
|
||||
from .operation_control import check_cancelled, report_progress
|
||||
|
||||
|
||||
class ReleasedProjectReadError(TaskCenterError):
|
||||
"""Project read failed, with positive evidence that its CAD session released."""
|
||||
|
||||
|
||||
def _read_failure(root, message):
|
||||
error_type = TaskCenterError
|
||||
if (root / 'project-checkpoint.json').exists():
|
||||
from .batch_runtime import project_checkpoint_released
|
||||
if project_checkpoint_released(root):
|
||||
error_type = ReleasedProjectReadError
|
||||
return error_type('quick_run_sheet_discovery', message)
|
||||
try:
|
||||
record = json.loads((root / 'private-desktop.json').read_text(encoding='utf-8'))
|
||||
cleanup = record['desktop_cleanup']
|
||||
if (record['state'] == 'released' and record['clean'] is True
|
||||
and record['cad_exit_code'] == 0 and record['quit_helper_exit_code'] == 0
|
||||
and cleanup['desktop_handle_closed'] is True and cleanup['errors'] == []
|
||||
and cleanup['desktop_confirmation']['released'] is True):
|
||||
error_type = ReleasedProjectReadError
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
pass # Absence of an uncertainty marker is never release evidence.
|
||||
return error_type('quick_run_sheet_discovery', message)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DiscoveryRuntime:
|
||||
"""Worker and evidence adapters shared with the execution entry point."""
|
||||
@@ -83,18 +108,23 @@ def discover_project(
|
||||
completed = runtime.process_runner(command, runtime.timeout)
|
||||
except (OSError, subprocess.SubprocessError) as error:
|
||||
_require_released_reader(root)
|
||||
check_cancelled(stop_event)
|
||||
raise TaskCenterError(
|
||||
"quick_run_sheet_discovery",
|
||||
failure = _read_failure(
|
||||
root,
|
||||
f"枚举工程图页时执行器未能安全结束:{project};{error}",
|
||||
) from error
|
||||
)
|
||||
if isinstance(failure, ReleasedProjectReadError):
|
||||
check_cancelled(stop_event)
|
||||
raise failure from error
|
||||
_require_released_reader(root)
|
||||
check_cancelled(stop_event)
|
||||
report_progress(on_progress, f'读取会话已结束,正在核对位号清单:{project.name}')
|
||||
(root / "worker-output.txt").write_text(completed.stdout + "\n" + completed.stderr, encoding="utf-8")
|
||||
if completed.returncode != 0:
|
||||
detail = completed.stderr.strip() or completed.stdout.strip() or "sheet discovery failed"
|
||||
raise TaskCenterError("quick_run_sheet_discovery", runtime.failure_message(detail))
|
||||
failure = _read_failure(root, runtime.failure_message(detail))
|
||||
if isinstance(failure, ReleasedProjectReadError):
|
||||
check_cancelled(stop_event)
|
||||
raise failure
|
||||
check_cancelled(stop_event)
|
||||
report_progress(on_progress, f'读取会话已结束,正在核对位号清单:{project.name}')
|
||||
sheets = runtime.read_sheets(root / "sheets.tsv", project)
|
||||
matches = runtime.read_matches(root / "inventory.tsv", project, sheets)
|
||||
if collect_components:
|
||||
@@ -107,6 +137,10 @@ def discover_project(
|
||||
|
||||
|
||||
def _require_released_reader(root):
|
||||
if (root / 'project-checkpoint.json').exists():
|
||||
from .batch_runtime import project_checkpoint_released
|
||||
if not project_checkpoint_released(root):
|
||||
raise TaskCenterError('inventory_cleanup', f'工程读取关闭检查点无效,请保留诊断记录:{root}')
|
||||
if (root / 'uncertain.flag').exists():
|
||||
raise TaskCenterError('inventory_cleanup', f'采集会话未能正常收尾,请检查诊断记录后重试:{root}')
|
||||
|
||||
@@ -33,7 +33,7 @@ class ProjectSessionWorker:
|
||||
The VBS worker owns CAD. This driver never attaches to COM or terminates CAD.
|
||||
A separate monitor thread retains the existing idle watchdog and worker job.
|
||||
"""
|
||||
def __init__(self, project: Path, root: Path, *, private_desktop=False):
|
||||
def __init__(self, project: Path, root: Path, *, private_desktop=False, runtime=None):
|
||||
self.project, self.root = Path(project), Path(root).resolve()
|
||||
self.token = uuid4().hex
|
||||
self._done, self._abort = Event(), Event()
|
||||
@@ -46,12 +46,13 @@ class ProjectSessionWorker:
|
||||
self._timeout = 150
|
||||
self.starts = 0
|
||||
self.private_desktop = private_desktop
|
||||
self.runtime = runtime
|
||||
|
||||
def _monitor(self, command, timeout):
|
||||
try:
|
||||
if self.private_desktop:
|
||||
if self.private_desktop or self.runtime is not None:
|
||||
from .isolated_cad import run_isolated_process
|
||||
self._result = run_isolated_process(command, timeout, abort_event=self._abort)
|
||||
self._result = run_isolated_process(command, timeout, abort_event=self._abort, **({'runtime': self.runtime} if self.runtime is not None else {}))
|
||||
else:
|
||||
self._result = run_monitored_process(command, timeout, abort_event=self._abort)
|
||||
except Exception as error:
|
||||
@@ -130,10 +131,11 @@ class ProjectSessionWorker:
|
||||
|
||||
class ProjectSessionScope:
|
||||
"""One serial task can visit projects, but can hold only one worker at a time."""
|
||||
def __init__(self, fallback, *, enabled, private_desktop=False):
|
||||
def __init__(self, fallback, *, enabled, private_desktop=False, runtime=None):
|
||||
self.fallback, self.enabled = fallback, enabled
|
||||
self.worker = None
|
||||
self.private_desktop = private_desktop
|
||||
self.runtime = runtime
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
@@ -143,7 +145,7 @@ class ProjectSessionScope:
|
||||
return self.fallback(command, timeout)
|
||||
self.prepare(project)
|
||||
if self.worker is None:
|
||||
self.worker = ProjectSessionWorker(Path(project), Path(command[3]), private_desktop=self.private_desktop)
|
||||
self.worker = ProjectSessionWorker(Path(project), Path(command[3]), private_desktop=self.private_desktop, runtime=self.runtime)
|
||||
return self.worker.run_step(command, timeout)
|
||||
|
||||
def prepare(self, project):
|
||||
|
||||
@@ -7,6 +7,7 @@ from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from functools import lru_cache
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -171,6 +172,7 @@ class QuickRunPreparer:
|
||||
*,
|
||||
copy_part_number_to_value: bool = False,
|
||||
before_sheet_discovery: Callable[[Path], None] | None = None,
|
||||
runtime=None,
|
||||
) -> QuickRunResult:
|
||||
"""Create a page-addressed queue after read-only sheet discovery."""
|
||||
all_selected_projects = _selected_projects(projects)
|
||||
@@ -201,7 +203,8 @@ class QuickRunPreparer:
|
||||
before_sheet_discovery(project.path)
|
||||
if self._sheet_discoverer is None:
|
||||
discovery = self.discover_project(project.path, old_part_numbers,
|
||||
**({'collect_components': True} if targeted else {}))
|
||||
**({'collect_components': True} if targeted else {}),
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
if targeted:
|
||||
from .inventory import validate_target_inventory
|
||||
validate_target_inventory(project.path, enabled_rules, discovery.components)
|
||||
@@ -244,6 +247,7 @@ class QuickRunPreparer:
|
||||
*,
|
||||
copy_part_number_to_value: bool = False,
|
||||
before_sheet_discovery: Callable[[Path], None] | None = None,
|
||||
runtime=None,
|
||||
) -> QuickRunBatchSession:
|
||||
enabled_rules = tuple(rule for rule in rules if rule.enabled)
|
||||
handoff = self.ensure_batch_handoff(
|
||||
@@ -252,6 +256,7 @@ class QuickRunPreparer:
|
||||
rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value,
|
||||
before_sheet_discovery=before_sheet_discovery,
|
||||
**({'runtime': runtime} if runtime is not None else {}),
|
||||
)
|
||||
source = self._required_script_source()
|
||||
if source.resolve() != handoff.script_path.resolve():
|
||||
@@ -301,13 +306,13 @@ class QuickRunPreparer:
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def project_session_scope(self, *, enabled=True):
|
||||
def project_session_scope(self, *, enabled=True, runtime=None):
|
||||
"""Retain only the built-in supervised worker; injected workers stay compatible."""
|
||||
from .project_session import ProjectSessionScope
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
try:
|
||||
with ProjectSessionScope(self._process_runner,
|
||||
enabled=enabled and self._process_runner is _run_process, private_desktop=True) as scope:
|
||||
enabled=enabled and self._process_runner is _run_process, private_desktop=True, runtime=runtime) as scope:
|
||||
yield scope
|
||||
except (WorkerMonitorError, subprocess.TimeoutExpired) as error:
|
||||
_write_uncertain_flag(self._data_root / 'quick-run/uncertain.flag', 'project_session_cleanup')
|
||||
@@ -325,6 +330,7 @@ class QuickRunPreparer:
|
||||
batch_mode: bool = False,
|
||||
batch_session: QuickRunBatchSession | None = None,
|
||||
project_session=None,
|
||||
runtime=None,
|
||||
) -> QuickRunStepResult:
|
||||
"""Prepare or resume a queue, then execute one replacement rule end to end."""
|
||||
uncertain_path = self._data_root / "quick-run" / "uncertain.flag"
|
||||
@@ -332,7 +338,8 @@ class QuickRunPreparer:
|
||||
raise TaskCenterError("quick_run_uncertain", "a previous quick-run worker timed out or returned an uncorrelated status; reconcile the project before retrying")
|
||||
if self._process_runner is _run_process and not batch_mode:
|
||||
from .single_step import prepare_private_single_step
|
||||
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value)
|
||||
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
batch_mode = True
|
||||
current_root = self._data_root / "quick-run"
|
||||
if batch_session is None and self.evidence_path(task_id) == current_root and (current_root / "status.tsv").is_file():
|
||||
@@ -367,7 +374,8 @@ class QuickRunPreparer:
|
||||
if handoff is None:
|
||||
if any(getattr(rule, "scope", "all") == "targets" for rule in enabled_rules):
|
||||
handoff = self.ensure_batch_handoff(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
copy_part_number_to_value=copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
else:
|
||||
handoff = self.prepare(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
@@ -407,6 +415,8 @@ class QuickRunPreparer:
|
||||
try:
|
||||
if project_session is not None:
|
||||
completed = project_session.run_step(Path(selected_command[2]), command, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
elif runtime is not None and self._process_runner is _run_process:
|
||||
completed = _run_process(command, IDLE_PROGRESS_TIMEOUT_SECONDS, runtime=runtime)
|
||||
else:
|
||||
completed = self._process_runner(command, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
except subprocess.TimeoutExpired as error:
|
||||
@@ -681,10 +691,14 @@ class QuickRunPreparer:
|
||||
self._discovery_cache.invalidate(project_path)
|
||||
|
||||
def discover_project(self, project_path: Path, old_part_numbers: tuple[str, ...], *,
|
||||
collect_components: bool = False, stop_event=None, on_progress=None) -> ProjectDiscovery:
|
||||
collect_components: bool = False, stop_event=None, on_progress=None, runtime=None) -> ProjectDiscovery:
|
||||
from .operation_control import check_cancelled, control_options, report_progress
|
||||
check_cancelled(stop_event)
|
||||
controls = control_options(stop_event, on_progress)
|
||||
if runtime is not None and any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in inspect.signature(self._discover_project_sheets).parameters.values()):
|
||||
controls['runtime'] = runtime
|
||||
attempt = 0
|
||||
def load():
|
||||
nonlocal attempt
|
||||
@@ -700,14 +714,15 @@ class QuickRunPreparer:
|
||||
Path(project_path), old_part_numbers, collect_components, load,
|
||||
**({'stop_event': stop_event} if stop_event is not None else {}))
|
||||
|
||||
def collect_inventory(self, project_path: Path, old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None) -> tuple:
|
||||
def collect_inventory(self, project_path: Path, old_part_numbers: tuple[str, ...], *, stop_event=None, on_progress=None, runtime=None) -> tuple:
|
||||
"""Collect validated component targets, retaining reusable sheet evidence."""
|
||||
if not old_part_numbers:
|
||||
raise TaskCenterError("target_inventory", "at least one old Part Number is required")
|
||||
requested = {part.casefold() for part in old_part_numbers}
|
||||
from .operation_control import control_options
|
||||
return tuple(component for component in self.discover_project(
|
||||
project_path, old_part_numbers, collect_components=True, **control_options(stop_event, on_progress)).components
|
||||
project_path, old_part_numbers, collect_components=True, **control_options(stop_event, on_progress),
|
||||
**({'runtime': runtime} if runtime is not None else {})).components
|
||||
if component.old_part_number.casefold() in requested)
|
||||
|
||||
def _discover_project_sheets(
|
||||
@@ -718,6 +733,7 @@ class QuickRunPreparer:
|
||||
collect_components: bool = False,
|
||||
stop_event=None,
|
||||
on_progress=None,
|
||||
runtime=None,
|
||||
) -> _ProjectDiscovery:
|
||||
"""Enumerate sheets and requested source parts while the owned project is open."""
|
||||
from .project_discovery import DiscoveryRuntime, discover_project
|
||||
@@ -726,14 +742,16 @@ class QuickRunPreparer:
|
||||
if collect_components and self._process_runner is _run_process:
|
||||
from .native_inventory import NativeInventoryReader
|
||||
native = NativeInventoryReader(self._data_root).read(
|
||||
project_path, old_part_numbers, stop_event=stop_event, on_progress=on_progress)
|
||||
project_path, old_part_numbers, stop_event=stop_event, on_progress=on_progress,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
if native is not None:
|
||||
return native
|
||||
|
||||
process_runner = self._process_runner
|
||||
if stop_event is not None and process_runner is _run_process:
|
||||
if (stop_event is not None or runtime is not None) and process_runner is _run_process:
|
||||
def process_runner(command, timeout):
|
||||
return _run_process(command, timeout, abort_event=stop_event)
|
||||
return _run_process(command, timeout, abort_event=stop_event,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
|
||||
return discover_project(project_path, old_part_numbers,
|
||||
data_root=self._data_root, script_source=self._required_script_source(),
|
||||
@@ -1751,12 +1769,13 @@ def _cached_native_helper_launch_arguments(frozen: bool, executable: str) -> tup
|
||||
return (resolved_executable, str(Path(__file__).with_name("application.py")))
|
||||
|
||||
|
||||
def _run_process(command: tuple[str, ...], timeout: int, *, abort_event=None) -> subprocess.CompletedProcess[str]:
|
||||
def _run_process(command: tuple[str, ...], timeout: int, *, abort_event=None, runtime=None) -> subprocess.CompletedProcess[str]:
|
||||
from .isolated_cad import run_isolated_process
|
||||
from .legacy_paths import validate_project_access
|
||||
|
||||
validate_project_access(Path(command[3]))
|
||||
result = run_isolated_process(command, timeout,
|
||||
**({'runtime': runtime} if runtime is not None else {}),
|
||||
**({'abort_event': abort_event, 'cooperative_abort': True} if abort_event is not None else {}))
|
||||
validate_project_access(Path(command[3]))
|
||||
return result
|
||||
|
||||
@@ -105,7 +105,7 @@ def require_current_cell_binding(snapshot, binding, project, metadata):
|
||||
return current
|
||||
|
||||
|
||||
def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=False, confirmed=False):
|
||||
def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=False, confirmed=False, runtime=None):
|
||||
if not confirmed:
|
||||
raise TaskCenterError('quick_run_recovery_confirmation', '核验恢复需要明确确认,不能作为自动重试执行。')
|
||||
maintain_quick_run_storage(preparer._data_root)
|
||||
@@ -227,7 +227,11 @@ def recover(preparer, task_id, projects, rules, *, copy_part_number_to_value=Fal
|
||||
launch = (str(preparer._cscript_resolver()), '//nologo', script_argument, root_argument,
|
||||
*_native_helper_launch_arguments(), job_id)
|
||||
try:
|
||||
result = preparer._process_runner(launch, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
from .quick_run import _run_process
|
||||
if runtime is not None and preparer._process_runner is _run_process:
|
||||
result = _run_process(launch, IDLE_PROGRESS_TIMEOUT_SECONDS, runtime=runtime)
|
||||
else:
|
||||
result = preparer._process_runner(launch, IDLE_PROGRESS_TIMEOUT_SECONDS)
|
||||
except (OSError, subprocess.SubprocessError) as error:
|
||||
raise TaskCenterError('quick_run_recovery', '恢复执行未能确认结束,原保护锁仍在;不要重新替换或删除证据。') from error
|
||||
(audit / 'worker-output.txt').write_text(result.stdout + '\n' + result.stderr, encoding='utf-8')
|
||||
|
||||
@@ -142,10 +142,11 @@ class TaskCenterService:
|
||||
return self._operation("export_bom_template", task_id,
|
||||
lambda: export_template(self._store, task_id, path))
|
||||
|
||||
def preview_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None):
|
||||
def preview_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, runtime=None):
|
||||
from .bom_batch import preview_batch
|
||||
return self._idle_quick_run_operation('preview_bom_batch', task_id,
|
||||
lambda: preview_batch(self, task_id, path, **control_options(stop_event, on_progress)))
|
||||
lambda: preview_batch(self, task_id, path, **control_options(stop_event, on_progress),
|
||||
**({'runtime': runtime} if runtime is not None else {})))
|
||||
|
||||
def save_bom_batch(self, preview, rows_by_task):
|
||||
from .bom_batch import save_batch
|
||||
@@ -388,10 +389,15 @@ class TaskCenterService:
|
||||
return self._operation("list_projects", task_id, lambda: self._store.list_projects(task_id))
|
||||
|
||||
def collect_target_inventory(self, task_id: str, old_part_numbers: tuple[str, ...] | None = None,
|
||||
*, stop_event=None, on_progress=None):
|
||||
*, stop_event=None, on_progress=None, runtime=None):
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('collect_inventory', lambda owned:
|
||||
self.collect_target_inventory(task_id, old_part_numbers,
|
||||
stop_event=stop_event, on_progress=on_progress, runtime=owned), stop_event, task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.collect_target_inventory(task_id, old_part_numbers, **control_options(stop_event, on_progress))
|
||||
return delegated.collect_target_inventory(task_id, old_part_numbers, runtime=runtime,
|
||||
**control_options(stop_event, on_progress))
|
||||
def collect():
|
||||
from .bom_batch import _assert_editable
|
||||
_assert_editable(self, task_id)
|
||||
@@ -409,7 +415,7 @@ class TaskCenterService:
|
||||
project_path=str(project[0].path)) as metric:
|
||||
report_progress(on_progress, f'正在采集工程:{project[0].display_name}')
|
||||
rows = collect_target_inventory(self._quick_run_preparer, project[0].path, parts,
|
||||
**control_options(stop_event, on_progress))
|
||||
runtime=runtime, **control_options(stop_event, on_progress))
|
||||
check_cancelled(stop_event)
|
||||
metric['completed_units'] = len(rows)
|
||||
with self._store.transaction():
|
||||
@@ -555,11 +561,14 @@ class TaskCenterService:
|
||||
|
||||
return self._operation("expected_probe_path", task_id, expected_path)
|
||||
|
||||
def prepare_quick_run(self, task_id: str) -> QuickRunResult:
|
||||
def prepare_quick_run(self, task_id: str, *, runtime=None) -> QuickRunResult:
|
||||
"""Create the legacy VBS handoff without starting a task-center trial."""
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('prepare_quick_run', lambda owned:
|
||||
self.prepare_quick_run(task_id, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.prepare_quick_run(task_id)
|
||||
return delegated.prepare_quick_run(task_id, runtime=runtime)
|
||||
def prepare() -> QuickRunResult:
|
||||
task = self._store.get_task(task_id)
|
||||
preparer = (self._quick_run_preparer.ensure_batch_handoff
|
||||
@@ -569,6 +578,7 @@ class TaskCenterService:
|
||||
self._store.list_projects(task_id),
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
**({'runtime': runtime} if task.target_plan_mode else {}),
|
||||
)
|
||||
append_task_event(
|
||||
self._logging,
|
||||
@@ -624,11 +634,14 @@ class TaskCenterService:
|
||||
with _quick_run_lease(self._logging.root):
|
||||
return self._operation("set_value_policy", task_id, update)
|
||||
|
||||
def execute_quick_run(self, task_id: str, *, expected_item: tuple[str, str, str, str] | None = None) -> QuickRunStepResult:
|
||||
def execute_quick_run(self, task_id: str, *, expected_item: tuple[str, str, str, str] | None = None, runtime=None) -> QuickRunStepResult:
|
||||
"""Execute one replacement rule through restoration and verification."""
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('quick_run_step', lambda owned:
|
||||
self.execute_quick_run(task_id, expected_item=expected_item, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.execute_quick_run(task_id, expected_item=expected_item)
|
||||
return delegated.execute_quick_run(task_id, expected_item=expected_item, runtime=runtime)
|
||||
def execute() -> QuickRunStepResult:
|
||||
task = self._store.get_task(task_id)
|
||||
result = self._quick_run_preparer.execute_step(
|
||||
@@ -637,6 +650,7 @@ class TaskCenterService:
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
**({"expected_item": expected_item} if expected_item is not None else {}),
|
||||
runtime=runtime,
|
||||
)
|
||||
self._synchronize_quick_run_progress(task_id)
|
||||
append_task_event(
|
||||
@@ -663,12 +677,22 @@ class TaskCenterService:
|
||||
stop_event: threading.Event,
|
||||
continue_after_backup_failure: bool = False,
|
||||
reuse_project_session: bool = True,
|
||||
runtime=None,
|
||||
):
|
||||
if runtime is None:
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(self._logging.root, 'quick_run_queue', stop_event) as owned_runtime:
|
||||
owned_runtime.bind_logs(self._logging, (task_id,))
|
||||
result = self.execute_quick_run_queue(task_id, expected_item=expected_item,
|
||||
stop_event=stop_event, continue_after_backup_failure=continue_after_backup_failure,
|
||||
reuse_project_session=reuse_project_session, runtime=owned_runtime)
|
||||
owned_runtime.result = result
|
||||
return result
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.execute_quick_run_queue(task_id, expected_item=expected_item,
|
||||
stop_event=stop_event, continue_after_backup_failure=continue_after_backup_failure,
|
||||
reuse_project_session=reuse_project_session)
|
||||
reuse_project_session=reuse_project_session, runtime=runtime)
|
||||
from .batch import execute_batch
|
||||
from .task_queue import assert_task_queue_idle
|
||||
|
||||
@@ -690,6 +714,7 @@ class TaskCenterService:
|
||||
stop_event=stop_event,
|
||||
continue_after_backup_failure=continue_after_backup_failure,
|
||||
reuse_project_session=reuse_project_session,
|
||||
runtime=runtime,
|
||||
),
|
||||
)
|
||||
metric['state'] = 'paused' if result.paused else 'failed' if result.progress.failed_items else 'completed'
|
||||
@@ -700,17 +725,22 @@ class TaskCenterService:
|
||||
if result.paused:
|
||||
self._pause_quick_run_queue(task_id, 'queue paused at item boundary')
|
||||
return result
|
||||
except Exception:
|
||||
except Exception as error:
|
||||
if getattr(error, 'code', None) == 'batch_resources_pending':
|
||||
raise
|
||||
self._pause_quick_run_queue(task_id, 'queue stopped after execution error')
|
||||
raise
|
||||
finally:
|
||||
self._batch_feedback = None
|
||||
|
||||
def prepare_failed_quick_run(self, task_id: str) -> QuickRunProgress:
|
||||
def prepare_failed_quick_run(self, task_id: str, *, runtime=None) -> QuickRunProgress:
|
||||
"""Reset only failed projects and discard the finished queue handoff."""
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('prepare_failed_quick_run', lambda owned:
|
||||
self.prepare_failed_quick_run(task_id, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.prepare_failed_quick_run(task_id)
|
||||
return delegated.prepare_failed_quick_run(task_id, runtime=runtime)
|
||||
def prepare_retry() -> QuickRunProgress:
|
||||
task = self._store.get_task(task_id)
|
||||
if self._quick_run_preparer.has_interrupted(task_id):
|
||||
@@ -756,6 +786,7 @@ class TaskCenterService:
|
||||
self._store.list_projects(task_id),
|
||||
self._execution_rules(task_id),
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
runtime=runtime,
|
||||
)
|
||||
return self._quick_run_preparer.progress(
|
||||
task_id,
|
||||
@@ -775,10 +806,31 @@ class TaskCenterService:
|
||||
return self._idle_quick_run_operation('start_new_round', task_id,
|
||||
lambda: start_new_round(self, task_id))
|
||||
|
||||
def inspect_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, on_phase=None):
|
||||
def _owned_cad_operation(self, kind, action, stop_event=None, *, task_id=None):
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(self._logging.root, kind, stop_event) as runtime:
|
||||
runtime.bind_logs(self._logging, (task_id,) if task_id is not None else ())
|
||||
result = action(runtime)
|
||||
runtime.result = result
|
||||
return result
|
||||
|
||||
def pending_batch_resources(self):
|
||||
from .batch_runtime import pending_resources
|
||||
return pending_resources(self._logging.root)
|
||||
|
||||
def cleanup_batch_resources(self):
|
||||
"""Retry owned-resource release without opening or changing any project."""
|
||||
from .batch_runtime import cleanup_pending_resources
|
||||
from .task_queue import assert_task_queue_idle
|
||||
assert_task_queue_idle(self._logging.root)
|
||||
with _quick_run_lease(self._logging.root):
|
||||
return self._operation('cleanup_batch_resources', None,
|
||||
lambda: cleanup_pending_resources(self._logging.root, logging_context=self._logging))
|
||||
|
||||
def inspect_bom_batch(self, task_id: str, path: Path, *, stop_event=None, on_progress=None, on_phase=None, runtime=None):
|
||||
from .bom_import_report import inspect_batch
|
||||
return self._idle_quick_run_operation('inspect_bom_batch', task_id,
|
||||
lambda: inspect_batch(self, task_id, path, on_phase=on_phase,
|
||||
lambda: inspect_batch(self, task_id, path, on_phase=on_phase, runtime=runtime,
|
||||
**control_options(stop_event, on_progress)))
|
||||
|
||||
def _pause_quick_run_queue(self, task_id: str, message: str) -> None:
|
||||
@@ -835,10 +887,13 @@ class TaskCenterService:
|
||||
return delegated.has_interrupted_quick_run(task_id)
|
||||
return self._quick_run_preparer.has_interrupted(task_id)
|
||||
|
||||
def recover_quick_run(self, task_id: str, *, confirmed: bool = False) -> QuickRunStepResult:
|
||||
def recover_quick_run(self, task_id: str, *, confirmed: bool = False, runtime=None) -> QuickRunStepResult:
|
||||
if runtime is None:
|
||||
return self._owned_cad_operation('recover_quick_run', lambda owned:
|
||||
self.recover_quick_run(task_id, confirmed=confirmed, runtime=owned), task_id=task_id)
|
||||
delegated = self._project_task_service(task_id)
|
||||
if delegated is not self:
|
||||
return delegated.recover_quick_run(task_id, confirmed=confirmed)
|
||||
return delegated.recover_quick_run(task_id, confirmed=confirmed, runtime=runtime)
|
||||
from .recovery import recover
|
||||
|
||||
def reconcile() -> QuickRunStepResult:
|
||||
@@ -852,7 +907,7 @@ class TaskCenterService:
|
||||
raise TaskCenterError("quick_run_recovery", "工程包含手动执行阶段或失败记录,不能混用自动队列恢复。")
|
||||
result = recover(self._quick_run_preparer, task_id, projects,
|
||||
self._execution_rules(task_id), copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
confirmed=confirmed)
|
||||
confirmed=confirmed, runtime=runtime)
|
||||
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED}:
|
||||
self._store.set_task_status(task_id, TaskStatus.READY, summary="interrupted quick-run reconciled")
|
||||
self._synchronize_quick_run_progress(task_id)
|
||||
|
||||
@@ -5,7 +5,19 @@ from .errors import TaskCenterError
|
||||
from .schematic_backup import create_schematic_backup
|
||||
|
||||
|
||||
def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_number_to_value):
|
||||
def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_number_to_value, *, runtime=None):
|
||||
from .quick_run import (_queue_value_mode, _value_mode, _read_status_details,
|
||||
_queue_has_mutation_evidence)
|
||||
root = preparer.data_root / 'quick-run'
|
||||
command, status = root / 'command.tsv', root / 'status.tsv'
|
||||
# Reject a frozen-policy conflict before even opening the project tree for
|
||||
# backup. This preflight only reads evidence and cannot archive/rewrite it.
|
||||
if (command.is_file() and status.is_file()
|
||||
and _queue_value_mode(command) != _value_mode(copy_part_number_to_value)
|
||||
and (_read_status_details(status) or any(root.glob('snapshot-*'))
|
||||
or _queue_has_mutation_evidence(root))):
|
||||
raise TaskCenterError('quick_run_pending',
|
||||
'recorded Value policy differs; reconcile the old queue before creating a new task')
|
||||
progress = preparer.progress(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
if progress.queue_complete or progress.next_project_path is None:
|
||||
@@ -15,4 +27,5 @@ def prepare_private_single_step(preparer, task_id, projects, rules, copy_part_nu
|
||||
create_schematic_backup(preparer.data_root, progress.next_project_path,
|
||||
preparer.data_root / 'quick-run-history' / ('batch-' + uuid4().hex))
|
||||
preparer.ensure_batch_handoff(task_id, projects, rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value)
|
||||
copy_part_number_to_value=copy_part_number_to_value,
|
||||
**({'runtime': runtime} if runtime is not None else {}))
|
||||
@@ -3,6 +3,7 @@
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import asdict, dataclass, is_dataclass
|
||||
import hashlib
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -119,8 +120,15 @@ def _save(path: Path, state: dict) -> None:
|
||||
|
||||
|
||||
def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
queue_id: str | None = None) -> TaskQueueResult:
|
||||
queue_id: str | None = None, runtime=None) -> TaskQueueResult:
|
||||
"""Execute the frozen order; uncertain state stops before the next project."""
|
||||
if runtime is None:
|
||||
from .batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(service._logging.root, 'task_queue', stop_event) as owned_runtime:
|
||||
result = execute_task_queue(service, task_ids, stop_event=stop_event,
|
||||
queue_id=queue_id, runtime=owned_runtime)
|
||||
owned_runtime.result = result
|
||||
return result
|
||||
ids = tuple(task_ids)
|
||||
if not ids or len(set(ids)) != len(ids):
|
||||
raise TaskCenterError('task_queue_selection', '请选择不同的单工程任务。')
|
||||
@@ -136,6 +144,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
project_paths.append(str(projects[0].path.resolve(strict=False)).casefold())
|
||||
if len(set(project_paths)) != len(ids):
|
||||
raise TaskCenterError('task_queue_selection', '不能在同一队列重复处理同一个工程。')
|
||||
runtime.bind_logs(service._logging, ids)
|
||||
hashes = {task_id: _plan_hash(child, task_id) for task_id, child in children.items()}
|
||||
token = queue_id or uuid4().hex
|
||||
if len(token) != 32 or any(char not in '0123456789abcdef' for char in token):
|
||||
@@ -183,8 +192,13 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
expected = (str(progress.next_project_path), progress.next_old_part_number,
|
||||
progress.next_new_part_number,
|
||||
'part_label' if child.get_task(task_id).copy_part_number_to_value else 'preserve')
|
||||
result = child.execute_quick_run_queue(task_id, expected_item=expected,
|
||||
stop_event=stop_event)
|
||||
execute = child.execute_quick_run_queue
|
||||
parameters = inspect.signature(execute).parameters.values()
|
||||
runtime_options = ({'runtime': runtime} if any(
|
||||
parameter.name == 'runtime' or parameter.kind is inspect.Parameter.VAR_KEYWORD
|
||||
for parameter in parameters) else {})
|
||||
result = execute(task_id, expected_item=expected, stop_event=stop_event,
|
||||
**runtime_options)
|
||||
if child.has_interrupted_quick_run(task_id) or (
|
||||
result.last_result is not None and result.last_result.state == 'batch_saved_unverified'
|
||||
):
|
||||
|
||||
@@ -88,6 +88,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
self.run_panel.stop_queue_button.clicked.connect(self._pause_queue)
|
||||
self.run_panel.retry_button.clicked.connect(self._retry_failed)
|
||||
self.run_panel.recover_button.clicked.connect(self._recover_quick_run)
|
||||
self.run_panel.cleanup_resources_button.clicked.connect(self._cleanup_batch_resources)
|
||||
self.run_panel.new_round_button.clicked.connect(self._start_new_round)
|
||||
self.run_panel.history_button.clicked.connect(self._open_round_history)
|
||||
self.run_panel.copy_value_check.toggled.connect(self._value_option_changed)
|
||||
@@ -285,15 +286,18 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
|
||||
def _update_gates(self):
|
||||
busy = bool(self._workers) or self._loading
|
||||
resources_pending = bool(self.service.pending_batch_resources())
|
||||
self.run_panel.cleanup_resources_button.setVisible(resources_pending)
|
||||
self.run_panel.cleanup_resources_button.setEnabled(resources_pending and not busy)
|
||||
state = execution_state(self._context, busy=busy, dirty=self.plan_panel.table.dirty)
|
||||
group_unsaved = self._group_has_unsaved_plans() if state.editable or state.new_round else False
|
||||
context = self._context
|
||||
self.run_panel.quick_run_button.setText(state.execute_text)
|
||||
self.run_panel.quick_run_button.setEnabled(state.execute)
|
||||
self.run_panel.quick_run_button.setEnabled(state.execute and not resources_pending)
|
||||
self.run_panel.retry_button.setVisible(bool(context and context.phase == 'retry'))
|
||||
self.run_panel.retry_button.setEnabled(state.retry)
|
||||
self.run_panel.retry_button.setEnabled(state.retry and not resources_pending)
|
||||
self.run_panel.recover_button.setVisible(bool(context and context.interrupted))
|
||||
self.run_panel.recover_button.setEnabled(state.recover)
|
||||
self.run_panel.recover_button.setEnabled(state.recover and not resources_pending)
|
||||
self.run_panel.new_round_button.setVisible(bool(context and context.task.status is TaskStatus.COMPLETED))
|
||||
self.run_panel.new_round_button.setEnabled(state.new_round and not group_unsaved)
|
||||
self.run_panel.history_button.setVisible(bool(context and
|
||||
@@ -305,6 +309,9 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
rule_editable = state.editable and not group_unsaved
|
||||
self.mapping_table.set_rules_editable(rule_editable)
|
||||
self.plan_panel.set_plan_editable(state.editable, busy=busy)
|
||||
if resources_pending:
|
||||
self.plan_panel.collect_button.setEnabled(False)
|
||||
self.plan_panel.import_bom_button.setEnabled(False)
|
||||
self.add_mapping_button.setEnabled(rule_editable)
|
||||
selected = self.mapping_table.current_rule_id() is not None
|
||||
for button in (self.edit_mapping_button, self.delete_mapping_button,
|
||||
@@ -331,7 +338,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
and t.status is not TaskStatus.DELETED}
|
||||
group_dirty = self.plan_panel.has_unsaved_plan(group_ids)
|
||||
group_active = any(t.status in {TaskStatus.RUNNING, TaskStatus.PAUSED} for t in self._tasks if t.task_id in group_ids)
|
||||
self.task_list.queue_button.setEnabled(grouped and not busy and not group_dirty and not bool(context.interrupted))
|
||||
self.task_list.queue_button.setEnabled(grouped and not busy and not group_dirty and not bool(context.interrupted) and not resources_pending)
|
||||
for button in (self.task_list.up_button, self.task_list.down_button):
|
||||
button.setEnabled(grouped and not busy and not group_active
|
||||
and self.task_list.sort.currentData() == 'execution'
|
||||
@@ -342,7 +349,7 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
if self._loading and not self._message:
|
||||
self.run_panel.operator_band.setText('正在读取任务,可继续选择其他任务。')
|
||||
if not busy:
|
||||
self.run_panel.execution_status.setText('任务读取失败' if read_failed else
|
||||
self.run_panel.execution_status.setText('资源待清理' if resources_pending else '任务读取失败' if read_failed else
|
||||
'结果待核验' if context and context.interrupted else '空闲')
|
||||
|
||||
def _group_has_unsaved_plans(self):
|
||||
@@ -573,6 +580,17 @@ class TrialTaskCenterWindow(QMainWindow):
|
||||
self._set_operator_message(f"任务「{name}」{'已暂停' if result.paused else '已结束'}:已保存 {p.saved_items},失败 {p.failed_items},未命中 {p.no_match_items}。",
|
||||
'warning' if result.paused or p.failed_items else 'success')
|
||||
|
||||
def _cleanup_batch_resources(self):
|
||||
if not self.run_panel.cleanup_resources_button.isEnabled():
|
||||
return
|
||||
def done(pending):
|
||||
self._set_operator_message(
|
||||
'后台资源仍待清理,请保留诊断记录后重试。' if pending else '后台资源已释放,可以继续。',
|
||||
'warning' if pending else 'success')
|
||||
self._update_gates()
|
||||
self._run_async(self.service.cleanup_batch_resources, done,
|
||||
self.run_panel.cleanup_resources_button)
|
||||
|
||||
def _retry_failed(self):
|
||||
if self.run_panel.retry_button.isEnabled():
|
||||
task_id = self._task_id
|
||||
|
||||
@@ -4,6 +4,8 @@ import json
|
||||
import re
|
||||
|
||||
ERROR_TEXT = {
|
||||
"batch_resources_pending": "已核验的替换结果已保留。后台资源尚未释放,请使用“重新清理资源”后再开始下一批。",
|
||||
"inventory_session": "原生读取会话异常,安全重试未完成,已停止本批处理。请查看诊断记录并排除原因后重试。",
|
||||
"task_required": "请先选择任务。",
|
||||
"directory_required": "请先选择工程目录。",
|
||||
"discovery_root_locked": "旧目录已有执行记录,不能在同一任务中更换目录;请新建任务。",
|
||||
|
||||
@@ -25,6 +25,8 @@ class RunPanel(QWidget):
|
||||
self.stop_queue_button = QPushButton('完成当前项后暂停', objectName='stopQueueButton')
|
||||
self.retry_button = QPushButton('重试失败项', objectName='retryTaskItems')
|
||||
self.recover_button = QPushButton('核验并恢复', objectName='recoverQuickRunButton')
|
||||
self.cleanup_resources_button = QPushButton('重新清理资源', objectName='cleanupBatchResources')
|
||||
self.cleanup_resources_button.setToolTip('只释放本程序拥有的后台资源,不执行工程替换。')
|
||||
self.new_round_button = QPushButton('开始新一轮', objectName='startNewRoundButton')
|
||||
self.history_button = QPushButton('查看轮次证据', objectName='roundHistoryButton')
|
||||
self.copy_value_check = QCheckBox('Value 策略:替换后将 Part Label 写入 Value(逐行自定义优先)', objectName='copyPartNumberToValue')
|
||||
@@ -37,7 +39,7 @@ class RunPanel(QWidget):
|
||||
self.log_dialog = None
|
||||
self.toggle_log_button.toggled.connect(self._toggle_log)
|
||||
buttons = QHBoxLayout()
|
||||
for button in (self.quick_run_button, self.retry_button, self.recover_button,
|
||||
for button in (self.quick_run_button, self.retry_button, self.recover_button, self.cleanup_resources_button,
|
||||
self.new_round_button, self.history_button, self.stop_queue_button):
|
||||
buttons.addWidget(button)
|
||||
buttons.addStretch()
|
||||
|
||||
@@ -1 +1 @@
|
||||
VERSION = "1.1.3"
|
||||
VERSION = "1.1.4-beta.2"
|
||||
@@ -33,6 +33,42 @@ def test_bom_resource_verifier_accepts_cli_only_reads_without_cad(native_bom_evi
|
||||
assert verifier.verify_bom_resources(root)=={'cad_sessions':0,'ready_native_cli_reads':2}
|
||||
|
||||
|
||||
def test_desktop_observer_discovers_batch_resource_records(tmp_path, monkeypatch):
|
||||
monkeypatch.syspath_prepend(str(Path(__file__).resolve().parents[2] / 'tools'))
|
||||
from private_desktop_observer import PrivateDesktopObserver
|
||||
|
||||
path = tmp_path / 'data/batch-resources/batch/session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(json.dumps({'desktop': 'dx-observer-test', 'cad_pid': None, 'state': 'ready'}))
|
||||
observer = PrivateDesktopObserver(tmp_path)
|
||||
observer._sample()
|
||||
assert observer.records['dx-observer-test']['evidence_path'] == str(path.relative_to(tmp_path))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault', [None, 'pending', 'unbound', 'workers_open'])
|
||||
def test_bom_resource_verifier_checks_batch_release_and_operation_binding(native_bom_evidence, fault):
|
||||
verifier, root, rows = native_bom_evidence
|
||||
operations = {}
|
||||
for i, row in enumerate(rows):
|
||||
path = root / f'data/tasks/{i}/quick-run-history/native-inventory-ok/native-inventory.json'
|
||||
row['cleanup'].update(scope='operation', workers_closed=True, desktop_retained_by_batch=True)
|
||||
path.write_text(json.dumps(row))
|
||||
operations[str(path.parent.resolve())] = {'workers_closed': fault != 'workers_open',
|
||||
'binding': {'session_id': 'batch', 'generation': 1, 'attempt_id': str(i)}}
|
||||
if fault == 'unbound':
|
||||
operations.clear()
|
||||
path = root / 'data/batch-resources/batch/session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(json.dumps(dict(state='blocked' if fault == 'pending' else 'released',
|
||||
session_id='batch', generation=1, native_operations=operations,
|
||||
cleanup=dict(desktop_handle_closed=True, desktop_confirmation={'released':True}, errors=[]))))
|
||||
if fault:
|
||||
with pytest.raises(AssertionError):
|
||||
verifier.verify_bom_resources(root)
|
||||
else:
|
||||
assert verifier.verify_bom_resources(root)['ready_native_cli_reads'] == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault',['uncertain','fallback_active_job','fallback_retained_workspace'])
|
||||
def test_bom_resource_verifier_rejects_bad_earlier_attempt_despite_clean_retries(native_bom_evidence,fault):
|
||||
verifier,root,rows=native_bom_evidence
|
||||
|
||||
@@ -4,6 +4,8 @@ from pathlib import Path
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import json
|
||||
import pytest
|
||||
|
||||
|
||||
def test_default_data_root_uses_windows_known_folder_without_environment(tmp_path: Path, monkeypatch) -> None:
|
||||
@@ -189,3 +191,26 @@ def test_icon_uses_package_location_for_frozen_script_entrypoint(tmp_path, monke
|
||||
assert not qapp.windowIcon().isNull()
|
||||
assert not qapp.windowIcon().pixmap(32, 32).isNull()
|
||||
assert qapp.applicationVersion() == application.VERSION
|
||||
|
||||
|
||||
@pytest.mark.parametrize('missing_dependency', [False, True])
|
||||
def test_runtime_smoke_tracks_real_process_and_releases_on_failure(tmp_path, monkeypatch, missing_dependency):
|
||||
from dx_productivity.trial_task_center.application import _smoke_batch_runtime
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
import win32process
|
||||
|
||||
if missing_dependency:
|
||||
def fail(_handle):
|
||||
raise ModuleNotFoundError("No module named 'win32timezone'")
|
||||
monkeypatch.setattr(win32process, 'GetProcessTimes', fail)
|
||||
with pytest.raises(ModuleNotFoundError, match='win32timezone'):
|
||||
_smoke_batch_runtime(tmp_path, configure_logging(tmp_path))
|
||||
else:
|
||||
_smoke_batch_runtime(tmp_path, configure_logging(tmp_path))
|
||||
paths = list((tmp_path / 'batch-resources').glob('*/session.json'))
|
||||
assert len(paths) == 1
|
||||
record = json.loads(paths[0].read_text(encoding='utf-8'))
|
||||
assert record['state'] == 'released' and record['cad_pid'] is None
|
||||
assert record['cleanup']['desktop_handle_closed']
|
||||
assert record['cleanup']['desktop_confirmation']['released']
|
||||
assert not record['cleanup']['errors']
|
||||
@@ -355,6 +355,7 @@ def test_batch_clears_worker_current_state_after_saved_completion(tmp_path):
|
||||
|
||||
def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp_path, monkeypatch):
|
||||
import dx_productivity.trial_task_center.batch as batch_module
|
||||
import dx_productivity.trial_task_center.quick_run as quick_run_module
|
||||
|
||||
project_count = 20
|
||||
sheet_count = 5
|
||||
@@ -396,6 +397,15 @@ def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp
|
||||
|
||||
initial = service.quick_run_progress(task.task_id)
|
||||
expected = (str(initial.next_project_path), 'OLD-00', 'NEW-00', 'preserve')
|
||||
reads = {'_read_commands': 0, '_read_status_details': 0}
|
||||
def observe(name):
|
||||
original = getattr(quick_run_module, name)
|
||||
def counted(*args, **kwargs):
|
||||
reads[name] += 1
|
||||
return original(*args, **kwargs)
|
||||
monkeypatch.setattr(quick_run_module, name, counted)
|
||||
for name in reads:
|
||||
observe(name)
|
||||
tracemalloc.start()
|
||||
started = time.perf_counter()
|
||||
result = service.execute_quick_run_queue(task.task_id, expected_item=expected, stop_event=Event())
|
||||
@@ -410,13 +420,17 @@ def test_thousand_item_multisheet_mixed_queue_has_bounded_runtime_and_memory(tmp
|
||||
'sheets_per_project': sheet_count, 'rules_per_sheet': rule_count,
|
||||
'steps': total, 'saved_steps': saved, 'elapsed_seconds': elapsed,
|
||||
'peak_bytes': peak, 'steps_per_second': total / elapsed,
|
||||
'full_queue_parses': reads,
|
||||
}, indent=2), encoding='utf-8')
|
||||
assert result.progress.queue_complete
|
||||
assert result.progress.total_items == total
|
||||
assert result.progress.saved_items == saved
|
||||
assert result.progress.no_match_items == total - saved
|
||||
assert len(calls) == total and len(set(calls)) == total
|
||||
assert elapsed < 30
|
||||
# Wall-clock budgets depend on the machine and tracemalloc overhead. Guard
|
||||
# the repeated full-file work directly; keep timings as benchmark evidence.
|
||||
assert reads['_read_commands'] <= 2
|
||||
assert reads['_read_status_details'] <= 2
|
||||
assert peak < 128 * 1024 * 1024
|
||||
|
||||
|
||||
|
||||
@@ -360,16 +360,15 @@ def test_execute_quick_step_launches_automatic_runtime_and_reports_completion(tm
|
||||
|
||||
result = preparer.execute_step("task", (project,), (rule,))
|
||||
|
||||
assert calls == [
|
||||
((
|
||||
"C:\\Windows\\SysWOW64\\cscript.exe",
|
||||
"//nologo",
|
||||
str(result.script_path),
|
||||
str(tmp_path / "quick-run"),
|
||||
*_native_helper_launch_arguments(),
|
||||
_command_field(result.command_path, "job_id"),
|
||||
), 150)
|
||||
]
|
||||
assert len(calls) == 1
|
||||
command, timeout = calls[0]
|
||||
assert timeout == 150
|
||||
assert command[:2] == ("C:\\Windows\\SysWOW64\\cscript.exe", "//nologo")
|
||||
# Long evidence paths legitimately use 8.3 names or a managed alias.
|
||||
assert Path(command[2]).resolve() == result.script_path.resolve()
|
||||
assert Path(command[3]).resolve() == (tmp_path / 'quick-run').resolve()
|
||||
assert command[4:] == (*_native_helper_launch_arguments(),
|
||||
_command_field(result.command_path, 'job_id'))
|
||||
assert result.state == "restored_ok"
|
||||
assert result.message == "Replaced and verified 2 current-sheet matches."
|
||||
|
||||
|
||||
@@ -78,14 +78,19 @@ def test_result_for_other_queued_project_is_rejected(tmp_path):
|
||||
assert (tmp_path / 'quick-run/uncertain.flag').exists()
|
||||
|
||||
|
||||
def test_policy_change_cannot_reinterpret_partial_queue(tmp_path):
|
||||
def test_policy_change_cannot_reinterpret_partial_queue(tmp_path, monkeypatch):
|
||||
projects, rules = inputs()
|
||||
preparer = QuickRunPreparer(tmp_path, script_source=SOURCE)
|
||||
result = preparer.prepare('task', projects, rules)
|
||||
job = result.command_path.read_text(encoding='utf-8').splitlines()[1].split('\t')[1]
|
||||
result.status_path.write_text(f'job_id\tstate\tcode\tmessage\n{job}\tawaiting_native_replace\t\tcaptured\n', encoding='utf-8')
|
||||
before = {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
from dx_productivity.trial_task_center import single_step
|
||||
monkeypatch.setattr(single_step, 'create_schematic_backup',
|
||||
lambda *a, **k: pytest.fail('frozen policy must be rejected before backup or CAD'))
|
||||
with pytest.raises(TaskCenterError, match='quick_run_pending'):
|
||||
preparer.execute_step('task', projects, rules, copy_part_number_to_value=True)
|
||||
assert before == {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
|
||||
|
||||
def test_service_freezes_setting_after_queue_is_prepared(tmp_path):
|
||||
@@ -189,7 +194,7 @@ def test_serial_three_project_two_rule_queue_pins_each_job(tmp_path):
|
||||
assert preparer.progress('task', projects, rules).queue_complete
|
||||
|
||||
|
||||
def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_path):
|
||||
def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_path, monkeypatch):
|
||||
projects, rules = inputs()
|
||||
preparer = QuickRunPreparer(tmp_path, script_source=SOURCE)
|
||||
result = preparer.prepare('task', projects, rules)
|
||||
@@ -200,8 +205,13 @@ def test_old_queue_snapshot_blocks_policy_reinterpretation_without_status(tmp_pa
|
||||
encoding='utf-8',
|
||||
)
|
||||
(result.command_path.parent / 'snapshot-old.tsv').write_text('retained evidence', encoding='utf-8')
|
||||
before = {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
from dx_productivity.trial_task_center import single_step
|
||||
monkeypatch.setattr(single_step, 'create_schematic_backup',
|
||||
lambda *a, **k: pytest.fail('legacy snapshot must be checked before backup or CAD'))
|
||||
with pytest.raises(TaskCenterError, match='quick_run_pending'):
|
||||
preparer.execute_step('task', projects, rules)
|
||||
assert before == {path.name: path.read_bytes() for path in result.command_path.parent.iterdir() if path.is_file()}
|
||||
|
||||
|
||||
def test_frozen_launcher_reserves_helper_script_slot(monkeypatch, tmp_path):
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
import json
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import batch_attempt as attempts
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def prepared(tmp_path, monkeypatch):
|
||||
path = tmp_path / 'job.json'
|
||||
record = attempts.prepare_attempt(path, session_id='session-a', generation=1,
|
||||
task_id='task-a', job_id='job-a', project_path=tmp_path / 'board.prj')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(path))
|
||||
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
|
||||
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_ID', record['attempt_id'])
|
||||
monkeypatch.setenv('DXPART_TASK_ID', 'task-a')
|
||||
monkeypatch.setenv('DXPART_JOB_ID', 'job-a')
|
||||
return path, record
|
||||
|
||||
|
||||
def test_only_current_prepared_attempt_can_retry(prepared):
|
||||
path, expected = prepared
|
||||
assert attempts.can_retry_attempt(path, expected)
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
assert attempts.read_attempt(path)['state'] == 'mutation_possible'
|
||||
assert not attempts.can_retry_attempt(path, expected)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,value', [('session_id', 'old'), ('generation', 2),
|
||||
('attempt_id', '0' * 32), ('task_id', 'old'), ('job_id', 'old'),
|
||||
('project_path', 'other.prj')])
|
||||
def test_retry_rejects_stale_identity(prepared, field, value):
|
||||
path, expected = prepared
|
||||
assert not attempts.can_retry_attempt(path, dict(expected, **{field: value}))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('contents', [None, '{truncated', '{}', '[]'])
|
||||
def test_unreadable_or_incomplete_proof_never_allows_retry(prepared, contents):
|
||||
path, expected = prepared
|
||||
if contents is None:
|
||||
path.unlink()
|
||||
else:
|
||||
path.write_text(contents)
|
||||
assert not attempts.can_retry_attempt(path, expected)
|
||||
with pytest.raises((OSError, ValueError)):
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('name,value', [('DXPART_BATCH_SESSION_ID', 'old'),
|
||||
('DXPART_BATCH_GENERATION', '2'), ('DXPART_ATTEMPT_ID', '0' * 32),
|
||||
('DXPART_TASK_ID', 'old'), ('DXPART_JOB_ID', 'old')])
|
||||
def test_marker_blocks_stale_process_identity(prepared, monkeypatch, name, value):
|
||||
path, _ = prepared
|
||||
monkeypatch.setenv(name, value)
|
||||
with pytest.raises(ValueError):
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_marker_fsync_failure_blocks_caller_without_publishing_mutation(prepared, monkeypatch):
|
||||
path, _ = prepared
|
||||
def fail(_):
|
||||
raise OSError('durability failed')
|
||||
monkeypatch.setattr(os, 'fsync', fail)
|
||||
with pytest.raises(OSError, match='durability failed'):
|
||||
attempts.mark_mutation_possible(job_id='job-a')
|
||||
assert json.loads(path.read_text())['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_legacy_no_environment_remains_supported(monkeypatch):
|
||||
for name in tuple(os.environ):
|
||||
if name.startswith('DXPART_') and ('ATTEMPT' in name or 'BATCH_' in name):
|
||||
monkeypatch.delenv(name)
|
||||
assert attempts.mark_mutation_possible() is None
|
||||
|
||||
|
||||
def test_blank_batch_path_is_not_legacy(monkeypatch):
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', '')
|
||||
with pytest.raises(ValueError):
|
||||
attempts.mark_mutation_possible()
|
||||
|
||||
|
||||
def test_preparation_is_durable_before_return(tmp_path, monkeypatch):
|
||||
fsynced = []
|
||||
real_fsync = os.fsync
|
||||
def fsync(fd):
|
||||
fsynced.append(True)
|
||||
return real_fsync(fd)
|
||||
monkeypatch.setattr(os, 'fsync', fsync)
|
||||
path = tmp_path / 'attempt.json'
|
||||
first = attempts.prepare_attempt(path, session_id='s', generation=1,
|
||||
task_id='t', job_id='j', project_path=tmp_path / 'board.prj')
|
||||
assert fsynced and attempts.read_attempt(path)['state'] == 'prepared'
|
||||
second = attempts.prepare_attempt(tmp_path / 'second.json', session_id='s', generation=1,
|
||||
task_id='t', job_id='j', project_path=tmp_path / 'board.prj')
|
||||
assert first['attempt_id'] != second['attempt_id']
|
||||
|
||||
|
||||
def test_reused_worker_marks_each_logical_job(tmp_path, monkeypatch):
|
||||
directory = tmp_path / 'attempt'
|
||||
for job in ('job-a', 'job-b'):
|
||||
attempts.prepare_attempt(directory / f'{job}.json', session_id='session-a', generation=1,
|
||||
task_id='task-a', job_id=job, project_path=tmp_path / 'board.prj', attempt_id='a' * 32)
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(directory))
|
||||
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
|
||||
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_ID', 'a' * 32)
|
||||
for job in ('job-a', 'job-b'):
|
||||
attempts.mark_mutation_possible(job_id=job, task_id='task-a', project_path=tmp_path / 'board.prj')
|
||||
assert attempts.read_attempt(directory / f'{job}.json')['state'] == 'mutation_possible'
|
||||
|
||||
|
||||
def test_native_guard_binds_snapshot_to_command_and_current_attempt(prepared, tmp_path):
|
||||
from dx_productivity.trial_task_center.native_helper import mark_snapshot_mutation
|
||||
path, _ = prepared
|
||||
snapshot = tmp_path / 'snapshot-job-a.tsv'
|
||||
snapshot.write_text(f'M\t{tmp_path / "board.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
|
||||
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
|
||||
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
|
||||
assert mark_snapshot_mutation(snapshot)['state'] == 'mutation_possible'
|
||||
assert attempts.read_attempt(path)['state'] == 'mutation_possible'
|
||||
|
||||
|
||||
def test_native_guard_rejects_other_project_before_marking(prepared, tmp_path):
|
||||
from dx_productivity.trial_task_center.native_helper import mark_snapshot_mutation
|
||||
path, _ = prepared
|
||||
snapshot = tmp_path / 'snapshot-job-a.tsv'
|
||||
snapshot.write_text(f'M\t{tmp_path / "other.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
|
||||
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
|
||||
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
|
||||
with pytest.raises(ValueError):
|
||||
mark_snapshot_mutation(snapshot)
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_marker_cli_fails_closed_on_stale_job(prepared):
|
||||
from dx_productivity.trial_task_center.application import main
|
||||
path, record = prepared
|
||||
assert main(['--mark-batch-attempt', '--attempt-job', 'other', '--attempt-task', 'task-a',
|
||||
'--attempt-project', record['project_path']]) == 1
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
|
||||
|
||||
def test_native_failure_does_not_reuse_previous_attempt_transaction(prepared, tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import native_helper as native
|
||||
path, _ = prepared
|
||||
snapshot = tmp_path / 'snapshot-job-a.tsv'
|
||||
snapshot.write_text(f'M\t{tmp_path / "board.prj"}\tOLD\tNEW\tsheet:1\nC\tu1\tR1\t0\t0\t0\t0\t0\n')
|
||||
(tmp_path / 'command.tsv').write_text('task_id\tjob_id\tproject_path\n'
|
||||
f'task-a\tjob-a\t{tmp_path / "board.prj"}\n')
|
||||
report_path = tmp_path / 'snapshot-job-a.tsv.native.json'
|
||||
report_path.write_text(json.dumps({'stage': 'native_replace_started',
|
||||
'transaction': {'old': 'unrelated'}, 'batch_attempt': {'attempt_id': 'old'}}))
|
||||
def unavailable_cad(snapshot, report):
|
||||
raise RuntimeError('CAD unavailable before mutation')
|
||||
monkeypatch.setattr(native, 'run_native', unavailable_cad)
|
||||
assert native.run(snapshot) == 1
|
||||
report = json.loads(report_path.read_text())
|
||||
assert report['stage'] == 'preflight' and 'transaction' not in report
|
||||
assert report['batch_attempt']['job_id'] == 'job-a'
|
||||
assert attempts.read_attempt(path)['state'] == 'prepared'
|
||||
@@ -0,0 +1,60 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from test_accuracy_vbs_guards import run_vbs
|
||||
from dx_productivity.trial_task_center.batch_attempt import prepare_attempt
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault', ['', 'missing', 'stale', 'wrong-project', 'missing-path'])
|
||||
def test_vbs_guard_prevents_property_write_without_durable_current_proof(tmp_path, monkeypatch, fault):
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
monkeypatch.setenv('PYTHONPATH', str(root / 'src'))
|
||||
directory = tmp_path / 'attempt'
|
||||
record_path = directory / 'job-a.json'
|
||||
record = prepare_attempt(record_path, session_id='session-a', generation=1,
|
||||
task_id='task-a', job_id='job-a', project_path=tmp_path / 'board.prj')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_FILE', str(directory))
|
||||
monkeypatch.setenv('DXPART_BATCH_SESSION_ID', 'session-a')
|
||||
monkeypatch.setenv('DXPART_BATCH_GENERATION', '1')
|
||||
monkeypatch.setenv('DXPART_ATTEMPT_ID', '0' * 32 if fault == 'stale' else record['attempt_id'])
|
||||
if fault == 'missing':
|
||||
record_path.unlink()
|
||||
if fault == 'missing-path':
|
||||
monkeypatch.delenv('DXPART_ATTEMPT_FILE')
|
||||
monkeypatch.delenv('DXPART_BATCH_GENERATION')
|
||||
monkeypatch.delenv('DXPART_ATTEMPT_ID')
|
||||
project = tmp_path / ('other.prj' if fault == 'wrong-project' else 'board.prj')
|
||||
output = run_vbs(tmp_path, ['QPMarkBatchMutation'], '''
|
||||
Dim Command, Attr, Failure
|
||||
Set Command = CreateObject("Scripting.Dictionary")
|
||||
Command.Add "job_id", "job-a": Command.Add "task_id", "task-a"
|
||||
Command.Add "project_path", "''' + str(project) + '''"
|
||||
Set Attr = New PropertySink
|
||||
On Error Resume Next
|
||||
GuardedWrite Command, Attr
|
||||
Failure = Err.Number
|
||||
On Error GoTo 0
|
||||
WScript.Echo CStr(Failure <> 0)
|
||||
WScript.Echo Attr.Value
|
||||
''', '''
|
||||
Dim QP_HelperExecutable, QP_HelperScript, QP_Fso
|
||||
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
|
||||
QP_HelperExecutable = "''' + sys.executable + '''"
|
||||
QP_HelperScript = "''' + str(root / 'src/dx_productivity/trial_task_center/application.py') + '''"
|
||||
Class PropertySink
|
||||
Public Value
|
||||
Private Sub Class_Initialize
|
||||
Value = "unchanged"
|
||||
End Sub
|
||||
End Class
|
||||
Sub GuardedWrite(ByRef Command, ByRef Attr)
|
||||
QPMarkBatchMutation Command
|
||||
Attr.Value = "changed"
|
||||
End Sub
|
||||
''')
|
||||
assert output.splitlines() == (['False', 'changed'] if not fault else ['True', 'unchanged'])
|
||||
if not fault:
|
||||
assert json.loads(record_path.read_text())['state'] == 'mutation_possible'
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Release diagnostics remain visible without changing cleanup decisions."""
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import batch_resources, batch_runtime
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.ui.messages import _format_task_log_lines
|
||||
from test_batch_runtime import Desktop
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_desktop():
|
||||
Desktop.created = []
|
||||
Desktop.fail_release = False
|
||||
|
||||
|
||||
def events(root, task='one'):
|
||||
return [json.loads(line) for line in (root / 'tasks' / task / 'events.jsonl')
|
||||
.read_text(encoding='utf-8').splitlines()]
|
||||
|
||||
|
||||
def runtime(root):
|
||||
result = batch_runtime.BatchCadRuntime(root, 'task_queue', desktop_factory=Desktop)
|
||||
result.bind_logs(configure_logging(root), ('one', 'two'))
|
||||
return result
|
||||
|
||||
|
||||
def test_batch_release_is_readable_and_visible_for_each_participating_task(tmp_path):
|
||||
with runtime(tmp_path) as owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
log = owner.path.with_name('resource-release.log').read_text(encoding='utf-8')
|
||||
assert '开始释放' in log and '资源释放成功' in log
|
||||
assert owner.session_id in log and owner.record['desktop'] in log
|
||||
assert 'CAD 未启动' in log
|
||||
for task in ('one', 'two'):
|
||||
rows = events(tmp_path, task)
|
||||
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_succeeded']
|
||||
details = rows[-1]['details']
|
||||
assert details['batch_id'] == owner.session_id and details['generation'] == 1
|
||||
assert details['desktop_handle_closed'] is True and details['desktop_disappeared'] is True
|
||||
assert details['elapsed_seconds'] >= 0 and details['record_path'] == str(owner.path)
|
||||
assert '资源释放成功' in '\n'.join(_format_task_log_lines(tuple(json.dumps(row) for row in rows)))
|
||||
assert '资源释放成功' in (tmp_path / 'logs/application.log').read_text(encoding='utf-8')
|
||||
|
||||
|
||||
def test_failed_release_and_manual_retry_keep_both_results_without_replay(tmp_path):
|
||||
Desktop.fail_release = True
|
||||
owner = runtime(tmp_path)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError):
|
||||
with owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
owner.result = 'saved and verified'
|
||||
failed = events(tmp_path)[-1]
|
||||
assert failed['operation'] == 'resource_release_failed'
|
||||
assert 'outside owner retains desktop' in failed['message']
|
||||
assert failed['details']['desktop_disappeared'] is False
|
||||
assert failed['details']['result'] == 'blocked'
|
||||
Desktop.fail_release = False
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
|
||||
rows = events(tmp_path)
|
||||
assert [row['sequence'] for row in rows] == [1, 2, 3, 4]
|
||||
assert rows[-1]['operation'] == 'resource_release_succeeded'
|
||||
assert rows[-1]['details']['reason'] == 'manual_retry'
|
||||
assert owner.result == 'saved and verified' and len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_release_log_failure_does_not_skip_cleanup_or_falsify_its_result(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_release_log
|
||||
def denied(*_args):
|
||||
raise OSError('release log locked')
|
||||
monkeypatch.setattr(batch_release_log, '_append_line', denied)
|
||||
with runtime(tmp_path) as owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
assert Desktop.created[0].released
|
||||
assert owner.record['state'] == 'released' and not batch_resources.pending_resources(tmp_path)
|
||||
persisted = json.loads(owner.path.read_text(encoding='utf-8'))
|
||||
assert any('release log locked' in entry for entry in persisted['release_log_errors'])
|
||||
assert events(tmp_path)[-1]['operation'] == 'resource_release_succeeded'
|
||||
|
||||
|
||||
def test_restart_recheck_logs_original_batch_and_tasks(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import isolated_desktop
|
||||
Desktop.fail_release = True
|
||||
owner = runtime(tmp_path)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError):
|
||||
with owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
batch_resources.LIVE_RESOURCES.pop(str(owner.path))
|
||||
class Api:
|
||||
def confirm_desktop_released(self, name):
|
||||
assert name == owner.record['desktop']
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path, logging_context=configure_logging(tmp_path)) == ()
|
||||
row = events(tmp_path)[-1]
|
||||
assert row['operation'] == 'resource_release_succeeded'
|
||||
assert row['details']['reason'] == 'restart_recheck'
|
||||
assert row['details']['batch_id'] == owner.session_id
|
||||
assert row['details']['desktop_disappeared'] is True
|
||||
assert len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_cached_batch_has_no_misleading_release_success(tmp_path):
|
||||
with runtime(tmp_path) as owner:
|
||||
pass
|
||||
assert not owner.path.exists()
|
||||
assert not owner.path.with_name('resource-release.log').exists()
|
||||
assert not (tmp_path / 'tasks/one/events.jsonl').exists()
|
||||
|
||||
|
||||
def test_cad_exit_and_cleanup_evidence_are_logged(tmp_path):
|
||||
with runtime(tmp_path) as owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
class Cad:
|
||||
record = {'state': 'session_closed', 'clean': True,
|
||||
'cad_exit_code': 0, 'quit_helper_exit_code': 0, 'shutdown_seconds': .01}
|
||||
def __exit__(self, *_):
|
||||
pass
|
||||
owner.cad = Cad()
|
||||
owner._save(cad_pid=123, cad_guid='owned')
|
||||
event = events(tmp_path)[-1]
|
||||
assert event['details']['cad_pid'] == 123
|
||||
assert event['details']['cad_exit_code'] == event['details']['quit_helper_exit_code'] == 0
|
||||
assert 'CAD 退出码=0' in event['message']
|
||||
|
||||
|
||||
def test_release_logger_crash_cannot_hide_final_cleanup_failure(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_release_log
|
||||
def crash(*_args, **_options):
|
||||
raise OSError('diagnostic unavailable')
|
||||
monkeypatch.setattr(batch_release_log, 'write_release_event', crash)
|
||||
Desktop.fail_release = True
|
||||
owner = runtime(tmp_path)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError):
|
||||
with owner:
|
||||
with owner.native_scope(tmp_path / 'native'):
|
||||
pass
|
||||
assert batch_resources.pending_resources(tmp_path)
|
||||
Desktop.fail_release = False
|
||||
batch_resources.LIVE_RESOURCES.pop(str(owner.path))
|
||||
from dx_productivity.trial_task_center import isolated_desktop
|
||||
class Api:
|
||||
def confirm_desktop_released(self, _name):
|
||||
pass
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
|
||||
record = json.loads(owner.path.read_text())
|
||||
assert record['state'] == 'released' and record['release_log_errors']
|
||||
@@ -0,0 +1,500 @@
|
||||
"""Batch resources are lazy, shared, and independently blocked after failed release."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
|
||||
|
||||
class Desktop:
|
||||
created = []
|
||||
fail_release = False
|
||||
|
||||
def __init__(self, name, **_options):
|
||||
self.name = name
|
||||
self.processes = []
|
||||
self.released = False
|
||||
self.cleanup_record = None
|
||||
|
||||
def __enter__(self):
|
||||
self.created.append(self)
|
||||
return self
|
||||
|
||||
def start_process(self, command, **_options):
|
||||
process = SimpleNamespace(pid=100 + len(self.processes), closed=False,
|
||||
cleanup_record={}, returncode=0)
|
||||
def close():
|
||||
process.closed = True
|
||||
process.cleanup_record = {'pid': process.pid, 'process_handle_closed': True,
|
||||
'job_handle_closed': True, 'errors': []}
|
||||
process.close = close
|
||||
process.poll = lambda: process.returncode
|
||||
self.processes.append(process)
|
||||
return process
|
||||
|
||||
def close(self):
|
||||
for process in self.processes:
|
||||
process.close()
|
||||
self.cleanup_record = {'desktop_handle_closed': True,
|
||||
'desktop_confirmation': {'released': not self.fail_release}, 'errors': []}
|
||||
if self.fail_release:
|
||||
raise OSError('outside owner retains desktop')
|
||||
self.released = True
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_desktop():
|
||||
Desktop.created = []
|
||||
Desktop.fail_release = False
|
||||
|
||||
|
||||
def test_cache_only_batch_does_not_create_desktop_or_cad(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop):
|
||||
pass
|
||||
assert not Desktop.created
|
||||
|
||||
|
||||
def test_native_operations_share_one_desktop_but_release_each_worker(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
names = []
|
||||
for name in ('first', 'second'):
|
||||
with runtime.native_scope(tmp_path / name) as scope:
|
||||
names.append(scope.name)
|
||||
process = scope.start_process(('inventory.exe',))
|
||||
assert not process.closed
|
||||
assert process.closed and not Desktop.created[0].released
|
||||
assert names[0] == names[1]
|
||||
assert len(Desktop.created) == 1 and Desktop.created[0].released
|
||||
|
||||
|
||||
def test_failed_final_release_blocks_next_batch_and_resource_cleanup_never_replays(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import (
|
||||
BatchCadRuntime, cleanup_pending_resources, guard_pending_resources, pending_resources)
|
||||
Desktop.fail_release = True
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
pass
|
||||
runtime.result = 'verified completed task'
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
assert caught.value.completed_result == 'verified completed task'
|
||||
assert pending_resources(tmp_path)
|
||||
with pytest.raises(TaskCenterError):
|
||||
guard_pending_resources(tmp_path)
|
||||
assert not list(tmp_path.rglob('uncertain.flag'))
|
||||
Desktop.fail_release = False
|
||||
assert cleanup_pending_resources(tmp_path) == ()
|
||||
guard_pending_resources(tmp_path)
|
||||
assert len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_cleanup_failure_takes_priority_over_cancel(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
Desktop.fail_release = True
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with BatchCadRuntime(tmp_path, 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
pass
|
||||
raise TaskCenterError('operation_cancelled', 'cancel requested')
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
Desktop.fail_release = False
|
||||
from dx_productivity.trial_task_center.batch_runtime import cleanup_pending_resources
|
||||
cleanup_pending_resources(tmp_path)
|
||||
|
||||
|
||||
class Cad:
|
||||
created = []
|
||||
|
||||
def __init__(self, root, *, abort_event, desktop):
|
||||
self.root, self.desktop, self.abort_event = root, desktop, abort_event
|
||||
self.binding = SimpleNamespace(pid=500, guid='owned')
|
||||
self.closed = False
|
||||
|
||||
def __enter__(self):
|
||||
self.desktop.__enter__()
|
||||
self.created.append(self)
|
||||
return self
|
||||
|
||||
def checkpoint(self, evidence):
|
||||
assert not self.closed
|
||||
|
||||
def start_worker(self, *args, **kwargs):
|
||||
raise AssertionError('monitor is injected')
|
||||
|
||||
def __exit__(self, *_exc):
|
||||
self.closed = True
|
||||
self.desktop.close()
|
||||
|
||||
|
||||
def command_for(tmp_path, name='one'):
|
||||
root = tmp_path / 'data' / name
|
||||
root.mkdir(parents=True)
|
||||
project = tmp_path / ('project-' + name) / 'test.prj'
|
||||
project.parent.mkdir()
|
||||
project.write_text('project identity')
|
||||
(root / 'command.tsv').write_text('task_id\tjob_id\tproject_path\told_part\tnew_part\n'
|
||||
f'task\tjob\t{project}\told\tnew\n')
|
||||
return ('cscript', '//nologo', 'worker.vbs', str(root), 'python', 'job'), project
|
||||
|
||||
|
||||
def worker_runtime(tmp_path, monkeypatch, monitor):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
from dx_productivity.trial_task_center import worker_monitor
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', monitor)
|
||||
Cad.created = []
|
||||
return BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=Desktop, cad_factory=Cad)
|
||||
|
||||
|
||||
def test_two_projects_reuse_cad_and_produce_distinct_close_checkpoints(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
|
||||
one, _ = command_for(tmp_path)
|
||||
two, _ = command_for(tmp_path, 'two')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with runtime:
|
||||
runtime.run_worker(one, 5)
|
||||
assert project_checkpoint_released(Path(one[3]))
|
||||
runtime.run_worker(two, 5)
|
||||
assert project_checkpoint_released(Path(two[3]))
|
||||
assert len(Cad.created) == len(Desktop.created) == 1
|
||||
assert Cad.created[0].closed and Desktop.created[0].released
|
||||
|
||||
|
||||
@pytest.mark.parametrize('second_fails', [False, True])
|
||||
def test_prewrite_failure_releases_old_desktop_and_retries_only_once(tmp_path, monkeypatch, second_fails):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, _ = command_for(tmp_path)
|
||||
calls = []
|
||||
def monitor(command, *args, **kwargs):
|
||||
calls.append(len(calls))
|
||||
if len(calls) == 2:
|
||||
assert Desktop.created[0].released
|
||||
return subprocess.CompletedProcess(command, 2 if len(calls) == 1 or second_fails else 0, '', 'startup')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
if second_fails:
|
||||
with pytest.raises(WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
else:
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(calls) == len(Desktop.created) == len(Cad.created) == 2
|
||||
assert runtime.generation == 2
|
||||
releases = json.loads(runtime.path.read_text())['generation_releases']
|
||||
assert [release['generation'] for release in releases] == [1, 2]
|
||||
assert [release['desktop'] for release in releases] == [desktop.name for desktop in Desktop.created]
|
||||
assert all(release['cad_pid'] == 500 and release['cad_guid'] == 'owned'
|
||||
and release['cleanup']['desktop_confirmation']['released'] is True
|
||||
and release['released_at'] > 0 for release in releases)
|
||||
|
||||
|
||||
def test_possible_mutation_prevents_automatic_replay(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, _ = command_for(tmp_path)
|
||||
calls = []
|
||||
def monitor(*args, **kwargs):
|
||||
calls.append(1)
|
||||
path = next((Path(command[3]) / 'batch-attempts').rglob('job.json'))
|
||||
record = json.loads(path.read_text())
|
||||
record['state'] = 'mutation_possible'
|
||||
path.write_text(json.dumps(record))
|
||||
raise WorkerMonitorError('lost connection after possible write')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime, pytest.raises(WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(calls) == len(Desktop.created) == 1
|
||||
|
||||
|
||||
def test_previous_checkpoint_invalidated_before_new_worker(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, _ = command_for(tmp_path)
|
||||
calls = []
|
||||
def monitor(*args, **kwargs):
|
||||
assert not project_checkpoint_released(Path(command[3]))
|
||||
calls.append(1)
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
assert project_checkpoint_released(Path(command[3]))
|
||||
def fail_checkpoint(*args):
|
||||
raise WorkerMonitorError('cannot certify empty project')
|
||||
runtime.cad.checkpoint = fail_checkpoint
|
||||
# No retry after evidence has been marked as possibly modified.
|
||||
path = next((Path(command[3]) / 'batch-attempts').rglob('job.json'))
|
||||
record = json.loads(path.read_text())
|
||||
record['state'] = 'mutation_possible'
|
||||
path.write_text(json.dumps(record))
|
||||
with pytest.raises(WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert not project_checkpoint_released(Path(command[3]))
|
||||
|
||||
|
||||
def test_attempt_publication_failure_never_launches_worker(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_attempt
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda *a, **k: pytest.fail('must not launch'))
|
||||
def fail(*args, **kwargs):
|
||||
raise OSError('disk unavailable')
|
||||
monkeypatch.setattr(batch_attempt, 'prepare_attempt', fail)
|
||||
with runtime, pytest.raises(OSError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert not Cad.created
|
||||
|
||||
|
||||
def test_boundary_pause_does_not_abort_current_cad_startup(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from threading import Event
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda command, *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
pause, worker_abort = Event(), Event()
|
||||
runtime.stop_event = pause
|
||||
original_enter = Cad.__enter__
|
||||
def enter(cad):
|
||||
pause.set() # User requests pause after the current task has begun.
|
||||
assert cad.abort_event is worker_abort
|
||||
assert not cad.abort_event.is_set()
|
||||
return original_enter(cad)
|
||||
monkeypatch.setattr(Cad, '__enter__', enter)
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5, abort_event=worker_abort)
|
||||
assert not list(tmp_path.rglob('uncertain.flag'))
|
||||
|
||||
|
||||
def test_cleanup_still_runs_when_resource_journal_write_fails(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_runtime
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'read'):
|
||||
pass
|
||||
real_write = batch_runtime.write_record
|
||||
def failing_write(path, record):
|
||||
if record['state'] == 'closing':
|
||||
raise OSError('journal unavailable')
|
||||
return real_write(path, record)
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', failing_write)
|
||||
assert Desktop.created[0].released
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', real_write)
|
||||
runtime.close()
|
||||
|
||||
|
||||
def test_real_startup_failure_with_released_lease_can_retry(tmp_path, monkeypatch):
|
||||
import io
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center import isolated_cad, worker_monitor
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
command, _ = command_for(tmp_path)
|
||||
starts = []
|
||||
def executable():
|
||||
starts.append(1)
|
||||
if len(starts) == 1:
|
||||
raise worker_monitor.WorkerMonitorError('registry temporarily unavailable')
|
||||
return 'cad.exe'
|
||||
class BootstrapDesktop(Desktop):
|
||||
def start_process(self, command, **options):
|
||||
process = super().start_process(command, **options)
|
||||
process.stderr = io.BytesIO()
|
||||
process.wait = lambda timeout: process.returncode
|
||||
if command[0] == 'cad.exe':
|
||||
self.cad = process
|
||||
process.returncode = None
|
||||
else:
|
||||
action, evidence, pid = command
|
||||
environment = options['env']
|
||||
binding = {'desktop': self.name, 'pid': pid, 'guid': 'owned',
|
||||
'token': environment['DXPART_SESSION_TOKEN']}
|
||||
if action == 'probe':
|
||||
Path(evidence).write_text(json.dumps(binding))
|
||||
elif action == 'checkpoint':
|
||||
Path(evidence).write_text(json.dumps({'binding': binding, 'project_path': '', 'project_closed': True}))
|
||||
elif action == 'quit':
|
||||
self.cad.returncode = 0
|
||||
return process
|
||||
monkeypatch.setattr(isolated_cad, '_designer_executable', executable)
|
||||
monkeypatch.setattr(isolated_cad, '_helper_command', lambda *args: args)
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', lambda *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=BootstrapDesktop) as runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(starts) == len(Desktop.created) == 2
|
||||
assert all(desktop.released for desktop in Desktop.created)
|
||||
|
||||
|
||||
def test_native_rebuild_requires_strict_source_and_worker_proof(tmp_path):
|
||||
import hashlib
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
command, project = command_for(tmp_path)
|
||||
database = project.parent / 'database.dat'
|
||||
database.write_bytes(b'database')
|
||||
evidence = Path(command[3]) / 'native'
|
||||
evidence.mkdir()
|
||||
proof = {'version': 1, 'native_read_only': True, 'workers_closed': True,
|
||||
'project_path': str(project.resolve()), 'evidence_root': str(evidence.resolve()),
|
||||
'source_hashes': {str(path.resolve()): hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
for path in (project, database)}}
|
||||
path = evidence / 'native-retry-proof.json'
|
||||
path.write_text(json.dumps(proof))
|
||||
with BatchCadRuntime(tmp_path / 'data', 'bom_import', desktop_factory=Desktop) as runtime:
|
||||
with runtime.native_scope(evidence) as scope:
|
||||
pass
|
||||
proof['binding'] = scope.identity
|
||||
path.write_text(json.dumps(proof))
|
||||
database.write_bytes(b'changed externally')
|
||||
assert not runtime.recover_native_failure(project, evidence, 'timeout')
|
||||
database.write_bytes(b'database')
|
||||
assert runtime.recover_native_failure(project, evidence, 'timeout')
|
||||
assert Desktop.created[0].released
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
with pytest.raises(WorkerMonitorError, match='stale'):
|
||||
scope.start_process(('late-old-worker.exe',))
|
||||
assert scope.identity['generation'] == 1 and runtime.generation == 2
|
||||
scope.close() # A late close cannot grant the old operation a new identity.
|
||||
assert str(evidence.resolve()) not in runtime.record['native_operations']
|
||||
assert not runtime.recover_native_failure(project, evidence, 'another timeout')
|
||||
assert len(Desktop.created) == 2
|
||||
|
||||
|
||||
def test_release_publication_failure_can_be_reconciled_without_replaying(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_runtime
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
real_write = batch_runtime.write_record
|
||||
def failing_write(path, record):
|
||||
if record['state'] == 'released':
|
||||
raise OSError('cannot publish release')
|
||||
return real_write(path, record)
|
||||
with pytest.raises(TaskCenterError):
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'read'):
|
||||
pass
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', failing_write)
|
||||
assert Desktop.created[0].released and batch_runtime.pending_resources(tmp_path)
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', real_write)
|
||||
assert batch_runtime.cleanup_pending_resources(tmp_path) == ()
|
||||
assert len(Desktop.created) == 1
|
||||
releases = json.loads(runtime.path.read_text())['generation_releases']
|
||||
assert len(releases) == 1 and releases[0]['generation'] == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize('record', [
|
||||
{'version': 1, 'session_id': 'old', 'state': 'blocked'},
|
||||
{'version': 1, 'session_id': 'old', 'state': 'released'},
|
||||
[],
|
||||
])
|
||||
def test_unknown_resource_evidence_never_authorizes_cleanup(tmp_path, record):
|
||||
from dx_productivity.trial_task_center.batch_resources import cleanup_pending_resources, pending_resources
|
||||
path = tmp_path / 'batch-resources' / 'old' / 'session.json'
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(json.dumps(record))
|
||||
assert pending_resources(tmp_path)
|
||||
assert cleanup_pending_resources(tmp_path)
|
||||
assert json.loads(path.read_text()) == record
|
||||
|
||||
|
||||
def test_stale_generation_and_unknown_checkpoint_schema_are_rejected(tmp_path, monkeypatch):
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center.batch_resources import project_checkpoint_released
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, lambda *a, **k:
|
||||
subprocess.CompletedProcess(command, 0, '', ''))
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
path = Path(command[3]) / 'project-checkpoint.json'
|
||||
checkpoint = json.loads(path.read_text())
|
||||
assert project_checkpoint_released(Path(command[3]))
|
||||
for changes in ({'version': 999}, {'generation': 999}, {'attempt_id': 'old'}, {'cad_guid': 'foreign'}):
|
||||
path.write_text(json.dumps(dict(checkpoint, **changes)))
|
||||
assert not project_checkpoint_released(Path(command[3]))
|
||||
|
||||
|
||||
def test_restart_reconciles_only_confirmed_disappearance_without_replacement(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources, isolated_desktop
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
Desktop.fail_release = True
|
||||
runtime = BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
with pytest.raises(TaskCenterError):
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'read'):
|
||||
pass
|
||||
# Simulate restarting the UI: there are no retained owned handles to act on.
|
||||
batch_resources.LIVE_RESOURCES.pop(str(runtime.path))
|
||||
present = [True]
|
||||
class Api:
|
||||
def confirm_desktop_released(self, name):
|
||||
assert name == runtime.record['desktop']
|
||||
if present[0]:
|
||||
raise OSError('desktop still held by external thread')
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', Api)
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path)
|
||||
present[0] = False
|
||||
assert batch_resources.cleanup_pending_resources(tmp_path) == ()
|
||||
assert len(Desktop.created) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize('conflict', [
|
||||
'bootstrap_binding', 'bootstrap_json', 'bootstrap_schema', 'helper_identity',
|
||||
'checkpoint_binding', 'checkpoint_json', 'worker_identity',
|
||||
])
|
||||
def test_identity_conflict_never_rebuilds_or_launches_another_worker(tmp_path, monkeypatch, conflict):
|
||||
import io
|
||||
import subprocess
|
||||
from dx_productivity.trial_task_center import isolated_cad, worker_monitor
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
command, _ = command_for(tmp_path)
|
||||
workers = []
|
||||
|
||||
class IdentityDesktop(Desktop):
|
||||
def start_process(self, launch, **options):
|
||||
process = super().start_process(launch, **options)
|
||||
process.stderr = io.BytesIO(b'CAD COM GUID does not match its owned session')
|
||||
process.wait = lambda timeout: process.returncode
|
||||
if launch[0] == 'cad.exe':
|
||||
self.cad = process
|
||||
process.returncode = None
|
||||
return process
|
||||
action, evidence, pid = launch
|
||||
binding = {'desktop': self.name, 'pid': pid, 'guid': 'owned',
|
||||
'token': options['env']['DXPART_SESSION_TOKEN']}
|
||||
if action == 'probe':
|
||||
if conflict == 'helper_identity':
|
||||
process.returncode = 4 # Private helper identity-conflict exit.
|
||||
return process
|
||||
if conflict == 'bootstrap_binding':
|
||||
binding['pid'] = pid + 1
|
||||
if conflict == 'bootstrap_schema':
|
||||
binding.pop('token')
|
||||
Path(evidence).write_text('{' if conflict == 'bootstrap_json' else json.dumps(binding))
|
||||
elif action == 'checkpoint':
|
||||
closing = Path(evidence).name.startswith('cad-closed-')
|
||||
if closing and conflict == 'checkpoint_binding':
|
||||
binding['guid'] = 'foreign'
|
||||
record = {'binding': binding, 'project_path': '', 'project_closed': True}
|
||||
Path(evidence).write_text('{' if closing and conflict == 'checkpoint_json' else json.dumps(record))
|
||||
elif action == 'quit':
|
||||
self.cad.returncode = 0
|
||||
return process
|
||||
|
||||
monkeypatch.setattr(isolated_cad, '_designer_executable', lambda: 'cad.exe')
|
||||
monkeypatch.setattr(isolated_cad, '_helper_command', lambda *args: args)
|
||||
def monitor(*args, **kwargs):
|
||||
workers.append(1)
|
||||
if conflict == 'worker_identity':
|
||||
return subprocess.CompletedProcess(command, 2, '', 'private CAD GUID mismatch; no project was changed')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', monitor)
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=IdentityDesktop)
|
||||
with runtime, pytest.raises(worker_monitor.WorkerMonitorError):
|
||||
runtime.run_worker(command, 5)
|
||||
assert len(Desktop.created) == 1
|
||||
assert len(workers) == (1 if conflict.startswith('checkpoint') or conflict == 'worker_identity' else 0)
|
||||
assert runtime.generation == 1 and not runtime.record['retries']
|
||||
assert Desktop.created[0].released
|
||||
@@ -0,0 +1,122 @@
|
||||
"""Batch entry points own resource lifetime independently of task outcomes."""
|
||||
import sys
|
||||
from types import SimpleNamespace, ModuleType
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import task_queue
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from test_task_queue import Service
|
||||
from test_bom_batch import batch, records_for
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def runtime_module(monkeypatch):
|
||||
module = ModuleType('dx_productivity.trial_task_center.batch_runtime')
|
||||
scopes = []
|
||||
class CleanupError(TaskCenterError):
|
||||
def __init__(self):
|
||||
super().__init__('batch_resources_pending', 'resource cleanup blocked')
|
||||
class Runtime:
|
||||
result = None
|
||||
fail_cleanup = False
|
||||
def __init__(self, root, kind, stop_event=None):
|
||||
self.root, self.kind = root, kind
|
||||
self.closed = False
|
||||
scopes.append(self)
|
||||
def __enter__(self):
|
||||
return self
|
||||
def bind_logs(self, context, task_ids):
|
||||
self.log_task_ids = tuple(task_ids)
|
||||
def __exit__(self, *args):
|
||||
self.closed = True
|
||||
if self.fail_cleanup:
|
||||
error = CleanupError()
|
||||
error.completed_result = self.result
|
||||
raise error
|
||||
module.BatchCadRuntime = Runtime
|
||||
module.BatchResourceCleanupError = CleanupError
|
||||
monkeypatch.setitem(sys.modules, module.__name__, module)
|
||||
return Runtime, scopes
|
||||
|
||||
|
||||
def test_task_queue_owns_one_runtime_and_passes_it_to_every_child(tmp_path, runtime_module):
|
||||
Runtime, scopes = runtime_module
|
||||
service = Service(tmp_path)
|
||||
original = service.for_project_task
|
||||
seen = []
|
||||
def child(task_id):
|
||||
result = original(task_id)
|
||||
execute = result.execute_quick_run_queue
|
||||
def run(task_id, *, runtime, **options):
|
||||
assert not runtime.closed
|
||||
seen.append(runtime)
|
||||
return execute(task_id, **options)
|
||||
result.execute_quick_run_queue = run
|
||||
return result
|
||||
service.for_project_task = child
|
||||
result = task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event())
|
||||
assert [entry.state for entry in result.entries] == ['completed', 'completed']
|
||||
assert len(scopes) == 1 and scopes[0].closed
|
||||
assert seen == [scopes[0], scopes[0]]
|
||||
assert scopes[0].log_task_ids == ('A', 'B')
|
||||
|
||||
|
||||
def test_final_cleanup_failure_preserves_completed_tasks_and_resume_does_not_repeat(tmp_path, runtime_module):
|
||||
Runtime, scopes = runtime_module
|
||||
Runtime.fail_cleanup = True
|
||||
service = Service(tmp_path)
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event())
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
result = caught.value.completed_result
|
||||
assert [entry.state for entry in result.entries] == ['completed', 'completed']
|
||||
Runtime.fail_cleanup = False
|
||||
task_queue.execute_task_queue(service, ('A', 'B'), stop_event=Event(), queue_id=result.queue_id)
|
||||
assert service.calls == [('start', 'A'), ('end', 'A'), ('start', 'B'), ('end', 'B')]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('current', [True, False])
|
||||
def test_reader_requires_current_batch_project_checkpoint(tmp_path, runtime_module, current):
|
||||
from dx_productivity.trial_task_center.project_discovery import (
|
||||
_read_failure, _require_released_reader, ReleasedProjectReadError)
|
||||
module = sys.modules['dx_productivity.trial_task_center.batch_runtime']
|
||||
module.project_checkpoint_released = lambda root: current
|
||||
(tmp_path / 'project-checkpoint.json').write_text('{}')
|
||||
if current:
|
||||
_require_released_reader(tmp_path)
|
||||
assert isinstance(_read_failure(tmp_path, 'read failed'), ReleasedProjectReadError)
|
||||
else:
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
_require_released_reader(tmp_path)
|
||||
assert caught.value.code == 'inventory_cleanup'
|
||||
assert not isinstance(_read_failure(tmp_path, 'read failed'), ReleasedProjectReadError)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('read_error', [False, True])
|
||||
def test_bom_cleanup_failure_preserves_completed_or_partial_report(batch, tmp_path, runtime_module, read_error):
|
||||
from dx_productivity.trial_task_center.bom_import_report import inspect_batch
|
||||
from dx_productivity.trial_task_center.bom_plan import write_bom_template
|
||||
Runtime, scopes = runtime_module
|
||||
Runtime.fail_cleanup = True
|
||||
service, store, tasks, targets, calls = batch
|
||||
if read_error:
|
||||
def fail(*args, **kwargs):
|
||||
raise TaskCenterError('inventory_cleanup', 'reader did not close')
|
||||
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = fail
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
inspect_batch(service, tasks[0].task_id,
|
||||
write_bom_template(tmp_path / 'cleanup.xlsx', records_for(tasks, targets)))
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
report = caught.value.bom_report
|
||||
assert report is not None and report.total_rows == 2
|
||||
assert report.failed_rows == (1 if read_error else 0)
|
||||
assert report.passed_rows == (0 if read_error else 2)
|
||||
assert report.preview is None
|
||||
original = caught.value.__context__.bom_report if read_error else scopes[0].result
|
||||
assert report is not original
|
||||
assert (original.preview is not None) == (not read_error)
|
||||
if caught.value.completed_result is not None:
|
||||
assert caught.value.completed_result.preview is None
|
||||
assert len(scopes) == 1 and scopes[0].closed
|
||||
@@ -0,0 +1,58 @@
|
||||
"""An external Windows reference blocks a batch without replaying its results."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.batch_runtime import (
|
||||
BatchCadRuntime, BatchResourceCleanupError, cleanup_pending_resources, guard_pending_resources)
|
||||
from dx_productivity.trial_task_center.cad_identity import current_desktop_name
|
||||
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api, build_startup_info
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop object lifetime')
|
||||
def test_external_reference_preserves_completed_result_until_resource_only_cleanup(tmp_path):
|
||||
before = current_desktop_name()
|
||||
runtime = BatchCadRuntime(tmp_path, 'execution')
|
||||
runtime.bind_logs(configure_logging(tmp_path), ('windows-resource-test',))
|
||||
external = None
|
||||
try:
|
||||
with pytest.raises(BatchResourceCleanupError) as caught:
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
|
||||
# The test owns this independent actor, outside runtime's
|
||||
# process/job list. The runtime must leave it alive.
|
||||
external = _Win32Api().create_process((sys.executable, '-u', '-c',
|
||||
'import win32service,time,sys; '
|
||||
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000); '
|
||||
'd.SetThreadDesktop();print("ready",flush=True);time.sleep(30)', scope.name),
|
||||
build_startup_info(scope.name).lpDesktop, None, None,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
assert external.stdout.readline() == b'ready\r\n'
|
||||
runtime.result = 'already saved and independently verified'
|
||||
assert caught.value.completed_result == runtime.result
|
||||
assert external.poll() is None
|
||||
assert json.loads(runtime.path.read_text())['state'] == 'blocked'
|
||||
log = runtime.path.with_name('resource-release.log').read_text(encoding='utf-8')
|
||||
assert '资源释放失败' in log and '桌面消失:未确认' in log
|
||||
with pytest.raises(BatchResourceCleanupError):
|
||||
guard_pending_resources(tmp_path)
|
||||
assert not list(Path(tmp_path).rglob('uncertain.flag'))
|
||||
assert current_desktop_name() == before
|
||||
finally:
|
||||
if external is not None:
|
||||
external.close() # Only the test closes its own independent actor.
|
||||
assert cleanup_pending_resources(tmp_path) == ()
|
||||
assert json.loads(runtime.path.read_text())['state'] == 'released'
|
||||
assert runtime.result == 'already saved and independently verified'
|
||||
assert runtime.generation == 1
|
||||
rows = [json.loads(line) for line in (tmp_path / 'tasks/windows-resource-test/events.jsonl')
|
||||
.read_text(encoding='utf-8').splitlines()]
|
||||
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_failed',
|
||||
'resource_release_started', 'resource_release_succeeded']
|
||||
assert rows[-1]['details']['desktop_disappeared'] is True
|
||||
guard_pending_resources(tmp_path)
|
||||
@@ -35,6 +35,26 @@ def test_reports_all_matching_errors_and_success_is_read_only(batch, tmp_path):
|
||||
assert all(not store.list_target_plan(task.task_id) for task in tasks)
|
||||
|
||||
|
||||
def test_bom_batch_passes_one_runtime_to_all_project_reads(batch, tmp_path, monkeypatch):
|
||||
from types import SimpleNamespace
|
||||
service, store, tasks, targets, calls = batch
|
||||
bound = []
|
||||
runtime = SimpleNamespace(bind_logs=lambda context, task_ids: bound.append((context, task_ids)))
|
||||
seen = []
|
||||
for task in tasks:
|
||||
preparer = service.for_project_task(task.task_id)._quick_run_preparer
|
||||
original = preparer.collect_inventory
|
||||
def read(project, parts, *, runtime=None, _read=original, **options):
|
||||
seen.append(runtime)
|
||||
return _read(project, parts, **options)
|
||||
monkeypatch.setattr(preparer, 'collect_inventory', read)
|
||||
report = inspect_batch(service, tasks[0].task_id,
|
||||
write_bom_template(tmp_path / 'shared.xlsx', records_for(tasks, targets)), runtime=runtime)
|
||||
assert report.passed_projects == 2
|
||||
assert seen == [runtime, runtime]
|
||||
assert bound == [(service._logging, tuple(task.task_id for task in tasks))]
|
||||
|
||||
|
||||
def test_duplicate_marks_both_rows_and_cross_row_chain_rejected(batch, tmp_path):
|
||||
service, store, tasks, targets, calls = batch
|
||||
record = records_for(tasks, targets)[0]
|
||||
@@ -111,6 +131,45 @@ def test_cancel_and_native_cleanup_errors_propagate_without_next_project(batch,
|
||||
assert not calls
|
||||
|
||||
|
||||
def test_released_project_read_failure_reports_rows_and_checks_next_project(batch, tmp_path):
|
||||
from dx_productivity.trial_task_center.project_discovery import ReleasedProjectReadError
|
||||
service, store, tasks, targets, calls = batch
|
||||
def failed_reader(*args, **kwargs):
|
||||
raise ReleasedProjectReadError('quick_run_sheet_discovery', 'project read failed', error_id='READ-1')
|
||||
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = failed_reader
|
||||
report = inspect_batch(service, tasks[0].task_id,
|
||||
write_bom_template(tmp_path / 'read-failure.xlsx', records_for(tasks, targets)))
|
||||
assert (report.failed_rows, report.passed_rows, report.unchecked_rows) == (1, 1, 0)
|
||||
assert 'READ-1' in report.rows[0].reason and 'project read failed' in report.rows[0].reason
|
||||
assert report.preview is None
|
||||
assert all(not store.list_target_plan(task.task_id) for task in tasks)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('cancel_requested', [False, True])
|
||||
@pytest.mark.parametrize('failure_first', [False, True])
|
||||
@pytest.mark.parametrize('code', ['inventory_cleanup', 'quick_run_sheet_discovery'])
|
||||
def test_unsafe_reader_failure_carries_full_report_without_opening_later_project(batch, tmp_path, cancel_requested, failure_first, code):
|
||||
service, store, tasks, targets, calls = batch
|
||||
records = list(records_for(tasks, targets))
|
||||
records = ([records[0], dict(records[0], refdes='R2'), records[1]] if failure_first else
|
||||
[records[1], records[0], dict(records[0], refdes='R2')])
|
||||
stop = Event()
|
||||
def failed_reader(*args, **kwargs):
|
||||
if cancel_requested:
|
||||
stop.set()
|
||||
raise TaskCenterError(code, 'desktop retained', error_id='CLEANUP-1')
|
||||
service.for_project_task(tasks[0].task_id)._quick_run_preparer.collect_inventory = failed_reader
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
inspect_batch(service, tasks[0].task_id,
|
||||
write_bom_template(tmp_path / 'unsafe.xlsx', records), stop_event=stop)
|
||||
assert caught.value.code == code and caught.value.error_id == 'CLEANUP-1'
|
||||
report = caught.value.bom_report
|
||||
assert (report.passed_rows, report.failed_rows, report.unchecked_rows) == ((0, 2, 1) if failure_first else (1, 2, 0))
|
||||
assert len(calls) == (0 if failure_first else 1)
|
||||
assert report.preview is None and 'CLEANUP-1' in report.rows[1].reason
|
||||
assert all(not store.list_target_plan(task.task_id) for task in tasks)
|
||||
|
||||
|
||||
|
||||
def test_cancellation_after_reader_prevents_opening_next_project(batch, tmp_path):
|
||||
service, store, tasks, targets, calls = batch
|
||||
@@ -202,3 +261,38 @@ def test_ui_report_is_persistent_and_failure_does_not_stage(batch, tmp_path, qtb
|
||||
assert '失败 1 行' in panel.report_text
|
||||
assert all(not store.list_target_plan(task.task_id) for task in tasks)
|
||||
window.close()
|
||||
|
||||
|
||||
def test_ui_cleanup_failure_keeps_details_and_retry_clears_previous_result(batch, tmp_path, qtbot, monkeypatch):
|
||||
from PySide6.QtWidgets import QFileDialog
|
||||
from dx_productivity.trial_task_center.ui.main_window import TrialTaskCenterWindow
|
||||
from dx_productivity.trial_task_center.bom_import_state import BomImportPhase
|
||||
service, store, tasks, targets, calls = batch
|
||||
preparer = service.for_project_task(tasks[0].task_id)._quick_run_preparer
|
||||
original_reader = preparer.collect_inventory
|
||||
def failed_reader(*args, **kwargs):
|
||||
raise TaskCenterError('inventory_cleanup', 'desktop retained', error_id='UI-CLEANUP-1')
|
||||
preparer.collect_inventory = failed_reader
|
||||
path = write_bom_template(tmp_path / 'cleanup-ui.xlsx', records_for(tasks, targets))
|
||||
window = TrialTaskCenterWindow(service)
|
||||
qtbot.addWidget(window)
|
||||
window.open_task(tasks[0].task_id)
|
||||
qtbot.waitUntil(lambda: not window._loading and not window._context_query.busy)
|
||||
monkeypatch.setattr(QFileDialog, 'getOpenFileName', lambda *args: (str(path), 'Excel (*.xlsx)'))
|
||||
panel = window.plan_panel
|
||||
panel.import_bom_button.click()
|
||||
qtbot.waitUntil(lambda: not window._workers)
|
||||
assert panel.bom_import_state.phase is BomImportPhase.FAILED
|
||||
assert panel.last_bom_report.failed_rows == 1 and panel.last_bom_report.unchecked_rows == 1
|
||||
assert 'UI-CLEANUP-1' in panel.report_text and panel.report_toggle.isVisibleTo(panel)
|
||||
assert not panel._bom_batches and not calls
|
||||
qtbot.waitUntil(lambda: not window._loading and not window._context_query.busy)
|
||||
preparer.collect_inventory = original_reader
|
||||
panel.import_bom_button.click()
|
||||
assert panel.last_bom_report is None and not panel.report_toggle.isVisibleTo(panel)
|
||||
qtbot.waitUntil(lambda: not window._workers)
|
||||
assert panel.bom_import_state.phase is BomImportPhase.PREVIEW_READY
|
||||
assert panel.last_bom_report.passed_rows == 2 and 'UI-CLEANUP-1' not in panel.report_text
|
||||
assert all(not store.list_target_plan(task.task_id) for task in tasks)
|
||||
panel.discard_all()
|
||||
window.close()
|
||||
@@ -64,3 +64,28 @@ def test_ready_preview_is_separate_from_saved_or_discarded_plan():
|
||||
machine.release_preview(saved=True)
|
||||
assert machine.state.phase is Phase.PREVIEW_READY
|
||||
assert machine.state.report.preview is None and '已保存' in machine.state.notice
|
||||
|
||||
|
||||
def test_failed_import_preserves_attached_row_report_and_retry_resets_it():
|
||||
from dx_productivity.trial_task_center.bom_import_report import BomInspectionError
|
||||
machine = BomImportMachine()
|
||||
attempt = machine.start()
|
||||
report = BomImportReport((), (), errors=('resources uncertain',))
|
||||
error = BomInspectionError(TaskCenterError('inventory_cleanup', 'resources uncertain'), report)
|
||||
machine.finish(attempt, error=error)
|
||||
assert machine.state.phase is Phase.FAILED and machine.state.report is report
|
||||
machine.start()
|
||||
assert machine.state.report is None and machine.state.phase is Phase.READING
|
||||
|
||||
|
||||
def test_resource_cleanup_failure_preserves_bom_row_report_after_cancellation():
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchResourceCleanupError
|
||||
machine = BomImportMachine()
|
||||
attempt = machine.start()
|
||||
machine.cancel(attempt)
|
||||
report = BomImportReport((), (), errors=('project read failed',))
|
||||
error = BatchResourceCleanupError('desktop cleanup failed')
|
||||
error.bom_report = report
|
||||
machine.finish(attempt, error=error)
|
||||
assert machine.state.phase is Phase.FAILED and machine.state.report is report
|
||||
assert 'desktop cleanup failed' in machine.state.message
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Desktop diagnosis must find threads without windows and leave their owner alone."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.isolated_desktop import PrivateDesktop, _Win32Api, build_startup_info
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop diagnosis')
|
||||
def test_diagnostic_finds_windowless_desktop_thread_without_changing_owner(tmp_path):
|
||||
from dx_productivity.trial_task_center.cad_identity import current_desktop_name
|
||||
before = current_desktop_name()
|
||||
desktop = PrivateDesktop('dx-diagnostic-' + uuid4().hex, enabled=True)
|
||||
desktop.__enter__()
|
||||
# Explicitly connect a thread without creating any windows.
|
||||
child = _Win32Api().create_process((sys.executable, '-u', '-c',
|
||||
'import win32service,time,sys; '
|
||||
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000);d.SetThreadDesktop(); '
|
||||
'print("ready",flush=True);time.sleep(30)', desktop.name), build_startup_info(desktop.name).lpDesktop,
|
||||
None, None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
try:
|
||||
assert child.stdout.readline() == b'ready\r\n'
|
||||
record = tmp_path / 'private-desktop.json'
|
||||
record.write_text(json.dumps({'desktop': desktop.name}))
|
||||
original = record.read_bytes()
|
||||
output = tmp_path / 'diagnostic.json'
|
||||
script = Path(__file__).resolve().parents[2] / 'tools/diagnose_private_desktop.py'
|
||||
result = subprocess.run((sys.executable, str(script), '--record', str(record), '--output', str(output)),
|
||||
capture_output=True, timeout=20)
|
||||
assert result.returncode == 0, result.stderr.decode(errors='replace')
|
||||
report = json.loads(output.read_text(encoding='utf-8'))
|
||||
assert any(row['pid'] == child.pid for row in report['thread_scan']['threads'])
|
||||
assert report['windows'] == [] and report['desktop_still_listed']
|
||||
assert child.poll() is None and current_desktop_name() == before
|
||||
assert record.read_bytes() == original
|
||||
finally:
|
||||
child.close()
|
||||
desktop.close()
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop diagnosis')
|
||||
def test_no_thread_owner_is_not_mistaken_for_released_desktop(tmp_path):
|
||||
with PrivateDesktop('dx-diagnostic-' + uuid4().hex, enabled=True) as desktop:
|
||||
output = tmp_path / 'diagnostic.json'
|
||||
script = Path(__file__).resolve().parents[2] / 'tools/diagnose_private_desktop.py'
|
||||
result = subprocess.run((sys.executable, str(script), '--desktop', desktop.name, '--output', str(output)),
|
||||
capture_output=True, timeout=20)
|
||||
assert result.returncode == 0, result.stderr.decode(errors='replace')
|
||||
report = json.loads(output.read_text(encoding='utf-8'))
|
||||
assert report['thread_scan']['threads'] == []
|
||||
assert report['desktop_still_listed'] and report['windows'] == []
|
||||
# A leaked query handle would make this context's strict close fail.
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows desktop diagnosis')
|
||||
@pytest.mark.parametrize('hardlink', [False, True])
|
||||
def test_output_cannot_overwrite_input_record_even_through_hardlink(tmp_path, hardlink):
|
||||
record = tmp_path / 'private-desktop.json'
|
||||
original = json.dumps({'desktop': 'dx-no-such-' + uuid4().hex}).encode()
|
||||
record.write_bytes(original)
|
||||
output = record
|
||||
if hardlink:
|
||||
output = tmp_path / 'diagnostic.json'
|
||||
os.link(record, output)
|
||||
script = Path(__file__).resolve().parents[2] / 'tools/diagnose_private_desktop.py'
|
||||
result = subprocess.run((sys.executable, str(script), '--record', str(record), '--output', str(output)),
|
||||
capture_output=True, timeout=20)
|
||||
assert result.returncode != 0
|
||||
assert record.read_bytes() == original
|
||||
@@ -1,4 +1,5 @@
|
||||
from dataclasses import replace
|
||||
import json
|
||||
from pathlib import Path
|
||||
from subprocess import CompletedProcess
|
||||
from threading import Event
|
||||
@@ -93,6 +94,13 @@ def test_default_discovery_forwards_abort_to_private_runner_and_waits_cleanup(tm
|
||||
assert cooperative_abort
|
||||
stop.set()
|
||||
cleanup.append('released')
|
||||
# Match the private runner's durable proof after its owned cleanup.
|
||||
(Path(command[3]) / 'private-desktop.json').write_text(json.dumps({
|
||||
'state': 'released', 'clean': True, 'cad_exit_code': 0,
|
||||
'quit_helper_exit_code': 0,
|
||||
'desktop_cleanup': {'desktop_handle_closed': True, 'errors': [],
|
||||
'desktop_confirmation': {'released': True}},
|
||||
}), encoding='utf-8')
|
||||
return CompletedProcess(command, 2, '', 'cancelled')
|
||||
monkeypatch.setattr(isolated_cad, 'run_isolated_process', run)
|
||||
monkeypatch.setattr(quick_run, '_native_helper_launch_arguments', lambda: ())
|
||||
|
||||
@@ -314,6 +314,19 @@ def test_released_startup_failure_returns_retryable_pre_mutation_code(monkeypatc
|
||||
assert not (tmp_path / 'uncertain.flag').exists()
|
||||
|
||||
|
||||
def test_released_startup_identity_conflict_is_not_a_retryable_exit(monkeypatch, tmp_path):
|
||||
class FailedIdentity:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
def __enter__(self):
|
||||
raise runtime.PrivateCadIdentityError('bootstrap returned a foreign identity')
|
||||
def __exit__(self, *args):
|
||||
pytest.fail('failed identity must not enter execution')
|
||||
monkeypatch.setattr(runtime, 'IsolatedCadSession', FailedIdentity)
|
||||
with pytest.raises(runtime.CadIdentityConflict):
|
||||
runtime.run_isolated_process(('cscript.exe', '//nologo', 'helper.vbs', str(tmp_path)), 1)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', ['quit', 'cad_exit', 'unexpected_exit'])
|
||||
def test_uncertain_cleanup_is_durable_before_forced_owned_release(monkeypatch, tmp_path, failure):
|
||||
events, cad = fake_desktop(monkeypatch, tmp_path,
|
||||
|
||||
@@ -33,12 +33,17 @@ def test_release_waits_for_a_temporary_external_reference_without_stopping_its_o
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='native Windows desktop lifetime')
|
||||
def test_release_does_not_report_success_while_another_handle_keeps_desktop_alive():
|
||||
@pytest.mark.parametrize('with_window', [False, True])
|
||||
def test_release_does_not_report_success_while_another_handle_keeps_desktop_alive(with_window):
|
||||
desktop = PrivateDesktop('dx-test-' + uuid4().hex, enabled=True)
|
||||
desktop.__enter__()
|
||||
retained = subprocess.Popen((sys.executable, '-u', '-c',
|
||||
'import win32service,sys,time; h=win32service.OpenDesktop(sys.argv[1],0,False,1); print("ready",flush=True); time.sleep(30)',
|
||||
desktop.name), stdout=subprocess.PIPE)
|
||||
script = ('import win32service,sys,time; h=win32service.OpenDesktop(sys.argv[1],0,False,1); ' +
|
||||
('import win32gui; w=win32gui.CreateWindowEx(0,"STATIC","residue",0,0,0,1,1,0,0,0,None); '
|
||||
if with_window else '') + 'print("ready",flush=True); time.sleep(30)')
|
||||
# subprocess.STARTUPINFO ignores lpDesktop. Use the native launch boundary
|
||||
# so the diagnostic window really belongs to the requested desktop.
|
||||
retained = _Win32Api().create_process((sys.executable, '-u', '-c', script, desktop.name),
|
||||
build_startup_info(desktop.name).lpDesktop, None, None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
try:
|
||||
assert retained.stdout.readline() == b'ready\r\n'
|
||||
with pytest.raises(TaskCenterError, match='still exists'):
|
||||
@@ -46,11 +51,14 @@ def test_release_does_not_report_success_while_another_handle_keeps_desktop_aliv
|
||||
record = desktop.cleanup_record
|
||||
assert record['desktop_handle_closed']
|
||||
assert not record['desktop_confirmation']['released']
|
||||
windows = record['desktop_confirmation']['remaining_windows']
|
||||
assert {row['pid'] for row in windows} == ({retained.pid} if with_window else set())
|
||||
assert retained.poll() is None # Diagnostics must not terminate a foreign owner.
|
||||
assert record['desktop_confirmation']['elapsed_seconds'] >= 2
|
||||
finally:
|
||||
retained.terminate()
|
||||
retained.wait(timeout=5)
|
||||
retained.stdout.close()
|
||||
retained.close()
|
||||
desktop.close()
|
||||
assert desktop.cleanup_record['desktop_confirmation']['released']
|
||||
|
||||
|
||||
@@ -138,3 +138,15 @@ def test_component_cache_cannot_substitute_for_complete_page_discovery(tmp_path)
|
||||
def test_inventory_requires_effective_values_even_when_no_requested_base_part_matches(tmp_path):
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported):
|
||||
protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('NEW',))
|
||||
|
||||
|
||||
def test_unrequested_reference_override_does_not_block_certified_target_inventory(tmp_path):
|
||||
flat = effective_report(tmp_path)
|
||||
flat.write_text(flat.read_text().replace('Prop Ref Designator R2', 'Prop Ref Designator LIB_R2'))
|
||||
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('NEW',),
|
||||
effective_report_path=flat)
|
||||
assert [(row.uid, row.refdes, row.old_part_number) for row in result.components] == [('$27I100', 'R1', 'NEW')]
|
||||
# The ambiguous component is still refused when its part is requested.
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported, match='位号与属性不一致'):
|
||||
protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('OLD',),
|
||||
effective_report_path=flat)
|
||||
@@ -73,6 +73,142 @@ def test_reader_uses_a_disposable_database_and_returns_certified_native_identity
|
||||
assert messages
|
||||
|
||||
|
||||
def test_native_reader_borrows_batch_desktop_and_closes_only_native_scope(tmp_path):
|
||||
project, reader, stop, calls, released = reader_fixture(tmp_path)
|
||||
borrowed = reader.desktop_factory('shared')
|
||||
evidence = []
|
||||
class Runtime:
|
||||
def native_scope(self, root):
|
||||
evidence.append(root)
|
||||
return borrowed
|
||||
def forbidden(*args, **kwargs):
|
||||
raise AssertionError('batch native read must not create another desktop')
|
||||
reader.desktop_factory = forbidden
|
||||
result = reader.read(project, ('OLD',), runtime=Runtime())
|
||||
assert [row.refdes for row in result.components] == ['R1']
|
||||
assert len(calls) == 2 and released == [True]
|
||||
assert len(evidence) == 1 and evidence[0].parent.name == 'quick-run-history'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fail_close', [False, True])
|
||||
def test_native_jobs_release_descendants_before_next_command_or_fallback(tmp_path, fail_close):
|
||||
project, reader, stop, calls, released = reader_fixture(tmp_path)
|
||||
factory = reader.desktop_factory
|
||||
events = []
|
||||
def desktop(*args, **options):
|
||||
value = factory(*args, **options)
|
||||
original = value.start_process
|
||||
def start(command, **kwargs):
|
||||
events.append('start')
|
||||
process = original(command, **kwargs)
|
||||
def close():
|
||||
events.append('close')
|
||||
if fail_close:
|
||||
raise OSError('native descendant remained alive')
|
||||
process.close = close
|
||||
return process
|
||||
value.start_process = start
|
||||
return value
|
||||
reader.desktop_factory = desktop
|
||||
if fail_close:
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
reader.read(project, ('OLD',))
|
||||
assert caught.value.code == 'inventory_cleanup'
|
||||
assert events == ['start', 'close']
|
||||
assert list((tmp_path / 'evidence').rglob('uncertain.flag'))
|
||||
else:
|
||||
assert reader.read(project, ('OLD',)).components
|
||||
assert events == ['start', 'close', 'start', 'close']
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', ['launch', 'timeout', 'crash', 'input_error', 'parser'])
|
||||
def test_only_transient_native_process_failure_requests_one_proven_rebuild(tmp_path, failure):
|
||||
project, reader, stop, calls, released = reader_fixture(tmp_path, corrupt=failure == 'parser')
|
||||
reader.timeout = 0
|
||||
scopes, recoveries = [], []
|
||||
class Runtime:
|
||||
def native_scope(self, evidence):
|
||||
value = reader.desktop_factory('shared')
|
||||
value.cleanup_record = {'workers_closed': True}
|
||||
value.identity = {'session_id': 'current-batch', 'generation': 1,
|
||||
'attempt_id': f'native-{len(scopes) + 1}'}
|
||||
original = value.start_process
|
||||
first = not scopes
|
||||
scopes.append(evidence)
|
||||
def start(command, **options):
|
||||
if first and failure == 'launch':
|
||||
raise OSError('temporary process launch failed')
|
||||
process = original(command, **options)
|
||||
if first and failure == 'timeout':
|
||||
process.poll = lambda: None
|
||||
elif first and failure in {'crash', 'input_error'}:
|
||||
process.poll = lambda: 0xC0000005 if failure == 'crash' else 2
|
||||
process.close = lambda: None
|
||||
return process
|
||||
value.start_process = start
|
||||
return value
|
||||
def recover_native_failure(self, path, evidence, reason):
|
||||
proof = json.loads((evidence / 'native-retry-proof.json').read_text())
|
||||
assert path == project and reason
|
||||
assert proof['native_read_only'] and proof['workers_closed']
|
||||
assert proof['binding'] == {'session_id': 'current-batch', 'generation': 1,
|
||||
'attempt_id': 'native-1'}
|
||||
assert proof['project_path'] == str(project.resolve())
|
||||
assert set(proof['source_hashes']) == {str(project), str(project.parent / 'database/icdb.dat')}
|
||||
assert released == [True]
|
||||
recoveries.append(evidence)
|
||||
return True
|
||||
result = reader.read(project, ('OLD',), runtime=Runtime())
|
||||
if failure in {'launch', 'timeout', 'crash'}:
|
||||
assert [row.refdes for row in result.components] == ['R1']
|
||||
assert len(scopes) == 2 and len(recoveries) == 1
|
||||
else:
|
||||
assert result is None and len(scopes) == 1 and not recoveries
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', ['timeout', 'crash'])
|
||||
@pytest.mark.parametrize('recovery_allowed', [True, False])
|
||||
def test_native_session_failure_stops_batch_after_retry_or_denied_budget_without_cad_fallback(
|
||||
tmp_path, monkeypatch, failure, recovery_allowed):
|
||||
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer
|
||||
project, reader, stop, calls, released = reader_fixture(tmp_path)
|
||||
reader.timeout = 0
|
||||
scopes, recoveries, cad_calls = [], [], []
|
||||
class BatchRuntime:
|
||||
def native_scope(self, evidence):
|
||||
scope = reader.desktop_factory('shared')
|
||||
scope.cleanup_record = {'workers_closed': True}
|
||||
scope.identity = {'session_id': 'batch', 'generation': 1,
|
||||
'attempt_id': f'native-{len(scopes) + 1}'}
|
||||
original = scope.start_process
|
||||
def start(command, **options):
|
||||
process = original(command, **options)
|
||||
process.poll = lambda: None if failure == 'timeout' else 0xC0000005
|
||||
process.close = lambda: None
|
||||
return process
|
||||
scope.start_process = start
|
||||
scopes.append(evidence)
|
||||
return scope
|
||||
def recover_native_failure(self, project, evidence, reason):
|
||||
recoveries.append(evidence)
|
||||
return recovery_allowed
|
||||
def run_worker(self, *args, **kwargs):
|
||||
cad_calls.append(args)
|
||||
raise AssertionError('native session failure must stop before CAD fallback')
|
||||
monkeypatch.setattr(runtime, 'NativeInventoryReader', lambda *args, **kwargs: reader)
|
||||
preparer = QuickRunPreparer(tmp_path / 'cad-data')
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
preparer._discover_project_sheets(project, ('OLD',), collect_components=True, runtime=BatchRuntime())
|
||||
assert caught.value.code == 'inventory_session'
|
||||
assert len(scopes) == len(released) == (2 if recovery_allowed else 1)
|
||||
assert len(recoveries) == 1 and not cad_calls
|
||||
record = json.loads((scopes[-1] / 'native-inventory.json').read_text(encoding='utf-8'))
|
||||
assert record['state'] == 'session_failed' and record['source_hashes_unchanged']
|
||||
assert record['cleanup']['workers_closed']
|
||||
proof = json.loads((scopes[-1] / 'native-retry-proof.json').read_text(encoding='utf-8'))
|
||||
assert proof['workers_closed'] and proof['binding']['attempt_id'] == f'native-{len(scopes)}'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', ['malformed', 'timeout'])
|
||||
def test_safe_native_failure_allows_com_fallback_only_after_desktop_release(tmp_path, failure):
|
||||
project, reader, stop, calls, released = reader_fixture(
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
"""Discovery orchestration uses injected workers and validates their evidence."""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import subprocess
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -73,6 +75,20 @@ def test_discovery_without_components_never_requests_or_requires_component_file(
|
||||
assert result.components == ()
|
||||
|
||||
|
||||
def test_discovery_reuses_explicit_runtime_on_built_in_process_path(discovery, monkeypatch):
|
||||
project, roots, prepare = discovery
|
||||
shared = object()
|
||||
seen = []
|
||||
def process(command, timeout, *, runtime=None, **options):
|
||||
seen.append(runtime)
|
||||
write_evidence(roots[-1], project)
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
monkeypatch.setattr(quick_run, '_run_process', process)
|
||||
result = prepare(process).discover_project(project, ('OLD',), runtime=shared)
|
||||
assert result.sheets == (('MAIN', '1'),)
|
||||
assert seen == [shared]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', [OSError('launch failed'), subprocess.TimeoutExpired('worker', 150)])
|
||||
def test_discovery_worker_failures_keep_stable_error_code(discovery, failure):
|
||||
project, roots, prepare = discovery
|
||||
@@ -103,3 +119,30 @@ def test_discovery_rejects_foreign_sheet_evidence(discovery):
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
with pytest.raises(TaskCenterError, match='another project'):
|
||||
prepare(worker)._discover_project_sheets(project, ('OLD',))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('release', ['confirmed', 'missing', 'uncertain', 'malformed'])
|
||||
@pytest.mark.parametrize('cancelled', [False, True])
|
||||
def test_only_positive_release_evidence_certifies_failed_reader(discovery, release, cancelled):
|
||||
from dx_productivity.trial_task_center.project_discovery import ReleasedProjectReadError
|
||||
project, roots, prepare = discovery
|
||||
stop = Event()
|
||||
def worker(command, timeout):
|
||||
if release != 'missing':
|
||||
record = {'state': 'released' if release == 'confirmed' else 'uncertain',
|
||||
'clean': True, 'cad_exit_code': 0, 'quit_helper_exit_code': 0,
|
||||
'desktop_cleanup': {'desktop_handle_closed': True, 'errors': [],
|
||||
'desktop_confirmation': {'released': True}}}
|
||||
(roots[-1] / 'private-desktop.json').write_text(
|
||||
'{' if release == 'malformed' else json.dumps(record))
|
||||
# No uncertain.flag exists, including when its write would have failed.
|
||||
if cancelled:
|
||||
stop.set()
|
||||
raise PermissionError('uncertain.flag cannot be written')
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
prepare(worker)._discover_project_sheets(project, ('OLD',), stop_event=stop)
|
||||
if cancelled and release == 'confirmed':
|
||||
assert caught.value.code == 'operation_cancelled'
|
||||
else:
|
||||
assert caught.value.code == 'quick_run_sheet_discovery'
|
||||
assert isinstance(caught.value, ReleasedProjectReadError) == (release == 'confirmed')
|
||||
@@ -51,6 +51,22 @@ def test_same_project_steps_retain_one_supervised_worker_and_close(tmp_path):
|
||||
assert not (tmp_path / 'project-session.request').exists()
|
||||
|
||||
|
||||
def test_batch_runtime_retains_same_project_worker_without_owning_cad(tmp_path):
|
||||
from dx_productivity.trial_task_center.worker_monitor import run_monitored_process
|
||||
calls = []
|
||||
class Runtime:
|
||||
def run_worker(self, command, timeout, **options):
|
||||
calls.append(command)
|
||||
return run_monitored_process(command, timeout, **options)
|
||||
runtime = Runtime()
|
||||
command = toy_worker(tmp_path)
|
||||
with ProjectSessionScope(None, enabled=True, private_desktop=True, runtime=runtime) as scope:
|
||||
assert scope.run_step(tmp_path / 'board.prj', command, 3).returncode == 0
|
||||
assert scope.run_step(tmp_path / 'board.prj', (*command[:-1], 'job2'), 3).returncode == 0
|
||||
assert len(calls) == 1
|
||||
assert (tmp_path / 'closed.txt').exists()
|
||||
|
||||
|
||||
def test_session_progress_keeps_monitor_identity_and_resets_each_step():
|
||||
watch = ProgressWatch('token', 'first', 10, 0)
|
||||
first = b'DX_PROGRESS\ttoken\tfirst\t1\topen_project\t0\t1\t1\tfirst\n'
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
from dataclasses import replace
|
||||
import json
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
@@ -73,3 +74,32 @@ def test_cancelled_service_inventory_does_not_save_any_targets(group):
|
||||
service.collect_target_inventory(task_id, ('OLD',), stop_event=stop)
|
||||
assert error.value.code == 'operation_cancelled'
|
||||
assert not store.list_target_plan(task_id) and not store.get_task(task_id).target_plan_mode
|
||||
|
||||
|
||||
@pytest.mark.parametrize('entry', ['collect', 'prepare', 'execute', 'retry', 'recover', 'preview', 'inspect'])
|
||||
def test_pending_resources_block_every_standalone_cad_service_before_target_or_worker_changes(group, tmp_path, entry):
|
||||
service, store, tasks, targets = group
|
||||
task_id = tasks[0].task_id
|
||||
calls = []
|
||||
def collect(*args, **kwargs):
|
||||
calls.append('read')
|
||||
return (targets[task_id],)
|
||||
service.for_project_task(task_id)._quick_run_preparer.collect_inventory = collect
|
||||
record = service._logging.root / 'batch-resources/previous/session.json'
|
||||
record.parent.mkdir(parents=True)
|
||||
record.write_text(json.dumps({'version': 1, 'session_id': 'previous', 'state': 'blocked'}))
|
||||
path = write_bom_template(tmp_path / 'scope.xlsx', ({'project_path': str(targets[task_id].project_path),
|
||||
'old_part_number': 'OLD', 'new_part_number': 'NEW', 'refdes': 'R1'},))
|
||||
invoke = {
|
||||
'collect': lambda: service.collect_target_inventory(task_id, ('OLD',)),
|
||||
'prepare': lambda: service.prepare_quick_run(task_id),
|
||||
'execute': lambda: service.execute_quick_run(task_id),
|
||||
'retry': lambda: service.prepare_failed_quick_run(task_id),
|
||||
'recover': lambda: service.recover_quick_run(task_id, confirmed=True),
|
||||
'preview': lambda: service.preview_bom_batch(task_id, path),
|
||||
'inspect': lambda: service.inspect_bom_batch(task_id, path),
|
||||
}[entry]
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
invoke()
|
||||
assert caught.value.code == 'batch_resources_pending'
|
||||
assert not calls and not store.list_target_plan(task_id)
|
||||
@@ -10,6 +10,28 @@ from test_ui_batch_retry import open_project
|
||||
from test_ui_remediation import window
|
||||
|
||||
|
||||
def test_pending_batch_resources_block_new_work_and_cleanup_does_not_replace(window, qtbot, tmp_path, monkeypatch):
|
||||
open_project(window, qtbot, tmp_path)
|
||||
pending = [{'state': 'cleanup_pending'}]
|
||||
monkeypatch.setattr(window.service, 'pending_batch_resources', lambda: tuple(pending))
|
||||
calls = []
|
||||
def cleanup():
|
||||
calls.append('cleanup')
|
||||
pending.clear()
|
||||
return ()
|
||||
monkeypatch.setattr(window.service, 'cleanup_batch_resources', cleanup)
|
||||
window._update_gates()
|
||||
assert not window.run_panel.quick_run_button.isEnabled()
|
||||
assert not window.plan_panel.collect_button.isEnabled()
|
||||
assert not window.plan_panel.import_bom_button.isEnabled()
|
||||
assert window.run_panel.cleanup_resources_button.isEnabled()
|
||||
window.run_panel.cleanup_resources_button.click()
|
||||
qtbot.waitUntil(lambda: not window._workers)
|
||||
assert calls == ['cleanup']
|
||||
assert not window.run_panel.cleanup_resources_button.isEnabled()
|
||||
assert '已释放' in window.run_panel.operator_band.text()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('cleanup_failure', [False, True])
|
||||
def test_collection_cancel_waits_for_cleanup_then_shows_terminal_state(window, qtbot, tmp_path, monkeypatch, cleanup_failure):
|
||||
open_project(window, qtbot, tmp_path)
|
||||
|
||||
@@ -99,17 +99,33 @@ def verify_bom_resources(root):
|
||||
path = root / 'desktop-observation-bom.json'
|
||||
verify_observation(path, require_sessions=False)
|
||||
observation = json.loads(path.read_text(encoding='utf-8'))
|
||||
records = [json.loads(path.read_text(encoding='utf-8')) for path in
|
||||
(root / 'data/tasks').glob('*/quick-run-history/native-inventory-*/native-inventory.json')]
|
||||
paths = list((root / 'data/tasks').glob('*/quick-run-history/native-inventory-*/native-inventory.json'))
|
||||
records = [json.loads(path.read_text(encoding='utf-8')) for path in paths]
|
||||
batches = [json.loads(path.read_text(encoding='utf-8')) for path in
|
||||
(root / 'data/batch-resources').glob('*/session.json')]
|
||||
ready = [row for row in records if row['state'] == 'ready']
|
||||
for row in records:
|
||||
for path, row in zip(paths, records):
|
||||
assert row['state'] in {'ready', 'fallback'} and not row.get('cleanup_error')
|
||||
assert row['backend'] == 'native_cli'
|
||||
if row['state'] == 'ready':
|
||||
assert row['source_hashes_unchanged']
|
||||
assert not Path(row['workspace']).exists()
|
||||
cleanup = row['cleanup']
|
||||
assert cleanup['desktop_handle_closed'] and cleanup['desktop_confirmation']['released']
|
||||
if cleanup.get('scope') == 'operation':
|
||||
assert cleanup['workers_closed'] and cleanup['desktop_retained_by_batch']
|
||||
owners = [(batch, operation) for batch in batches for evidence, operation in
|
||||
batch.get('native_operations', {}).items() if Path(evidence).resolve() == path.parent.resolve()]
|
||||
assert len(owners) == 1, 'Native operation must have exactly one batch owner'
|
||||
batch, operation = owners[0]
|
||||
assert batch['state'] == 'released' and operation['workers_closed']
|
||||
assert operation['binding']['session_id'] == batch['session_id']
|
||||
assert operation['binding']['generation'] == batch['generation']
|
||||
assert operation['binding']['attempt_id']
|
||||
final = batch['cleanup']
|
||||
assert final['desktop_handle_closed'] and final['desktop_confirmation']['released']
|
||||
assert not final['errors']
|
||||
else:
|
||||
assert cleanup['desktop_handle_closed'] and cleanup['desktop_confirmation']['released']
|
||||
assert not cleanup['errors']
|
||||
if row['state'] == 'ready':
|
||||
assert cleanup['processes']
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
"""Read-only desktop thread diagnosis, with borrowed handles confined to a child.
|
||||
|
||||
Windows contracts: GetThreadDesktop handles must NOT be closed; Toolhelp
|
||||
snapshots must be closed. A missing thread/window owner does not prove that
|
||||
no other process holds an OpenDesktop handle. See the accompanying research.
|
||||
"""
|
||||
import argparse
|
||||
from collections import Counter
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'src'))
|
||||
|
||||
|
||||
def scan_threads(desktop_name):
|
||||
"""Run only in the short-lived scan child, never in the application UI."""
|
||||
import win32service
|
||||
|
||||
class ThreadEntry(ctypes.Structure):
|
||||
_fields_ = [('dwSize', wintypes.DWORD), ('cntUsage', wintypes.DWORD),
|
||||
('th32ThreadID', wintypes.DWORD), ('th32OwnerProcessID', wintypes.DWORD),
|
||||
('tpBasePri', wintypes.LONG), ('tpDeltaPri', wintypes.LONG),
|
||||
('dwFlags', wintypes.DWORD)]
|
||||
|
||||
kernel = ctypes.WinDLL('kernel32', use_last_error=True)
|
||||
user = ctypes.WinDLL('user32', use_last_error=True)
|
||||
snapshot = kernel.CreateToolhelp32Snapshot
|
||||
snapshot.argtypes, snapshot.restype = [wintypes.DWORD, wintypes.DWORD], wintypes.HANDLE
|
||||
first, next_thread = kernel.Thread32First, kernel.Thread32Next
|
||||
for function in (first, next_thread):
|
||||
function.argtypes, function.restype = [wintypes.HANDLE, ctypes.POINTER(ThreadEntry)], wintypes.BOOL
|
||||
kernel.CloseHandle.argtypes, kernel.CloseHandle.restype = [wintypes.HANDLE], wintypes.BOOL
|
||||
user.GetThreadDesktop.argtypes, user.GetThreadDesktop.restype = [wintypes.DWORD], wintypes.HANDLE
|
||||
user.GetUserObjectInformationW.argtypes = [wintypes.HANDLE, ctypes.c_int, ctypes.c_void_p,
|
||||
wintypes.DWORD, ctypes.POINTER(wintypes.DWORD)]
|
||||
user.GetUserObjectInformationW.restype = wintypes.BOOL
|
||||
kernel.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD]
|
||||
kernel.OpenProcess.restype = wintypes.HANDLE
|
||||
kernel.QueryFullProcessImageNameW.argtypes = [wintypes.HANDLE, wintypes.DWORD, wintypes.LPWSTR,
|
||||
ctypes.POINTER(wintypes.DWORD)]
|
||||
kernel.QueryFullProcessImageNameW.restype = wintypes.BOOL
|
||||
images = {}
|
||||
|
||||
def image(pid):
|
||||
if pid not in images:
|
||||
process = kernel.OpenProcess(0x1000, False, pid) # QUERY_LIMITED_INFORMATION
|
||||
if not process:
|
||||
images[pid] = {'image_error': ctypes.get_last_error()}
|
||||
else:
|
||||
try:
|
||||
value, length = ctypes.create_unicode_buffer(32768), wintypes.DWORD(32768)
|
||||
images[pid] = ({'image': value.value} if kernel.QueryFullProcessImageNameW(
|
||||
process, 0, value, ctypes.byref(length)) else {'image_error': ctypes.get_last_error()})
|
||||
finally:
|
||||
kernel.CloseHandle(process)
|
||||
return images[pid]
|
||||
|
||||
handle = snapshot(0x4, 0) # TH32CS_SNAPTHREAD, read-only system snapshot
|
||||
if handle == ctypes.c_void_p(-1).value:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
threads, unreadable, total = [], Counter(), 0
|
||||
target = None
|
||||
try:
|
||||
# On Windows a READOBJECTS-only handle still made GetThreadDesktop
|
||||
# return ACCESS_DENIED for another process on this desktop. Open with
|
||||
# the owner's full access, but perform queries only. Never switch to it.
|
||||
try:
|
||||
target = win32service.OpenDesktop(desktop_name, 0, False, 0x10000000)
|
||||
except Exception as error:
|
||||
code = getattr(error, 'winerror', error.args[0] if error.args else None)
|
||||
if code == 2:
|
||||
return {'threads': [], 'snapshot_threads': 0, 'unreadable_by_winerror': {}, 'target_gone': True}
|
||||
raise
|
||||
entry = ThreadEntry()
|
||||
entry.dwSize = ctypes.sizeof(entry)
|
||||
valid = first(handle, ctypes.byref(entry))
|
||||
while valid:
|
||||
total += 1
|
||||
desktop = user.GetThreadDesktop(entry.th32ThreadID)
|
||||
if not desktop:
|
||||
unreadable[str(ctypes.get_last_error())] += 1
|
||||
else:
|
||||
name, required = ctypes.create_unicode_buffer(256), wintypes.DWORD()
|
||||
if not user.GetUserObjectInformationW(desktop, 2, name, ctypes.sizeof(name), ctypes.byref(required)):
|
||||
unreadable[str(ctypes.get_last_error())] += 1
|
||||
elif name.value.casefold() == desktop_name.casefold():
|
||||
pid = entry.th32OwnerProcessID
|
||||
threads.append({'pid': pid, 'thread_id': entry.th32ThreadID, **image(pid)})
|
||||
# GetThreadDesktop is borrowed. CloseDesktop is forbidden here.
|
||||
valid = next_thread(handle, ctypes.byref(entry))
|
||||
if ctypes.get_last_error() != 18: # ERROR_NO_MORE_FILES
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
finally:
|
||||
if target is not None:
|
||||
target.CloseDesktop()
|
||||
if not kernel.CloseHandle(handle):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
return {'threads': threads, 'snapshot_threads': total, 'unreadable_by_winerror': dict(unreadable)}
|
||||
|
||||
|
||||
def diagnose(desktop_name):
|
||||
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api
|
||||
|
||||
# The child exits before we recheck desktop lifetime. GetThreadDesktop can
|
||||
# never make the long-lived supervisor responsible for borrowed handles.
|
||||
completed = subprocess.run((sys.executable, str(Path(__file__).resolve()), '--scan-worker', desktop_name),
|
||||
capture_output=True, text=True, encoding='utf-8', timeout=15, creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
completed.check_returncode()
|
||||
report = {'desktop': desktop_name, 'thread_scan': json.loads(completed.stdout)}
|
||||
api = _Win32Api()
|
||||
listed = {name.casefold() for name in api.service.GetProcessWindowStation().EnumDesktops()}
|
||||
report['desktop_still_listed'] = desktop_name.casefold() in listed
|
||||
if report['desktop_still_listed']:
|
||||
try:
|
||||
report['windows'] = api._desktop_windows(desktop_name)
|
||||
except Exception as error:
|
||||
report['window_query_error'] = str(error)
|
||||
else:
|
||||
report['windows'] = []
|
||||
report['scope'] = 'Thread/window scan only; external handle-only owners may require Microsoft Sysinternals Handle.'
|
||||
return report
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
source = parser.add_mutually_exclusive_group(required=True)
|
||||
source.add_argument('--record', type=Path, help='Existing private-desktop.json (read only)')
|
||||
source.add_argument('--desktop', help='Owned dx- desktop name')
|
||||
source.add_argument('--scan-worker', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--output', type=Path)
|
||||
args = parser.parse_args()
|
||||
if os.name != 'nt':
|
||||
parser.error('Windows is required')
|
||||
desktop = args.scan_worker or args.desktop
|
||||
if args.record:
|
||||
desktop = json.loads(args.record.read_text(encoding='utf-8-sig'))['desktop']
|
||||
if args.output and (args.output.resolve() == args.record.resolve()
|
||||
or args.output.exists() and args.output.samefile(args.record)):
|
||||
parser.error('Diagnostic output must not overwrite its input record')
|
||||
if not isinstance(desktop, str) or not re.fullmatch(r'dx-[A-Za-z0-9-]{1,100}', desktop):
|
||||
parser.error('Only a valid owned dx- desktop name is accepted')
|
||||
report = scan_threads(desktop) if args.scan_worker else diagnose(desktop)
|
||||
text = json.dumps(report, ensure_ascii=True, indent=2)
|
||||
if args.output:
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(text, encoding='utf-8')
|
||||
print(text)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -47,6 +47,7 @@ class PrivateDesktopObserver:
|
||||
paths += list((self.output / 'data' / 'tasks').glob('*/quick-run-history/sheet-discovery-*/private-desktop.json'))
|
||||
paths += list((self.output / 'data' / 'tasks').glob('*/quick-run-history/recovery-*/worker/private-desktop.json'))
|
||||
paths += list((self.output / 'independent-read' / 'quick-run-history').glob('sheet-discovery-*/private-desktop.json'))
|
||||
paths += list((self.output / 'data' / 'batch-resources').glob('*/session.json'))
|
||||
for path in paths:
|
||||
# A Windows atomic replacement can briefly deny a simultaneous
|
||||
# open. Retry briefly; persistent failures still fail the probe.
|
||||
@@ -58,6 +59,8 @@ class PrivateDesktopObserver:
|
||||
if attempt == 3:
|
||||
raise
|
||||
time.sleep(0.01)
|
||||
if not value.get('desktop'):
|
||||
continue # A resource-free batch has not created a desktop.
|
||||
self.records[value['desktop']] = dict(value, evidence_path=str(path.relative_to(self.output)))
|
||||
pid = value.get('cad_pid')
|
||||
if pid and pid not in self.handles:
|
||||
|
||||
@@ -0,0 +1,344 @@
|
||||
"""One-shot, two-project BOM/replace comparison on newly copied projects only.
|
||||
|
||||
The same tool imports either checkout via --repo. A failed run is retained and
|
||||
cannot be resumed: completed writes are never automatically repeated.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from dataclasses import asdict
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from threading import Event
|
||||
import time
|
||||
import traceback
|
||||
|
||||
|
||||
OLD, NEW = '4100-CA0000-T7', '4100-CA0000-T0'
|
||||
NAMES = ('Audio_D2128', 'Audio_T605A')
|
||||
|
||||
|
||||
def save(path, value):
|
||||
path.write_text(json.dumps(value, ensure_ascii=False, indent=2, default=str), encoding='utf-8')
|
||||
|
||||
|
||||
def hashes(root):
|
||||
return {str(path.relative_to(root)): hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
for path in root.rglob('*') if path.is_file()}
|
||||
|
||||
|
||||
def inspect_native(spec_path):
|
||||
"""Read the saved sheet directly, independently of the replacement snapshot."""
|
||||
import pythoncom
|
||||
import win32com.client
|
||||
from dx_productivity.trial_task_center.cad_identity import project_file_path, verify_application
|
||||
from dx_productivity.trial_task_center.native_helper import attach, attribute_value, components_on_sheet
|
||||
spec = json.loads(spec_path.read_text(encoding='utf-8'))
|
||||
assert Path(spec['project']).resolve().is_relative_to(Path(__file__).resolve().parents[1] / 'build')
|
||||
pythoncom.CoInitialize()
|
||||
app = document = None
|
||||
try:
|
||||
app = win32com.client.GetActiveObject('Viewdraw.Application')
|
||||
verify_application(app, require_view=False)
|
||||
assert not project_file_path(app), 'reader must start with empty CAD'
|
||||
assert app.OpenProject(spec['project'])
|
||||
rows = {}
|
||||
for schematic, sheet in spec['sheets']:
|
||||
document = app.SchematicSheetDocuments().Open(schematic, sheet)
|
||||
document.Activate()
|
||||
app, api = attach()
|
||||
assert Path(project_file_path(app)).resolve() == Path(spec['project']).resolve()
|
||||
pins = {}
|
||||
for item in app.ActiveView.Query(4096, 0):
|
||||
pin = api.IVdCmpPin(item._oleobj_)
|
||||
point = pin.GetLocation()
|
||||
connection = pin.Connection
|
||||
net = None if connection is None else connection.Net
|
||||
assert connection is None or net is not None
|
||||
pins.setdefault(str(pin.Component.UID), []).append(dict(number=str(pin.Number),
|
||||
x=float(point.X), y=float(point.Y), net_uid=None if net is None else str(net.UID),
|
||||
net_name=None if net is None else str(net.LogicalNetName)))
|
||||
for component in components_on_sheet(app, api):
|
||||
uid = str(component.UID)
|
||||
point = component.GetLocation()
|
||||
attributes = {}
|
||||
for item in component.Attributes:
|
||||
attr = api.IVdAttr(item._oleobj_)
|
||||
location = attr.GetLocation()
|
||||
attributes[str(attr.Name)] = dict(value=str(attr.Value or ''),
|
||||
instance_value=str(attr.InstanceValue or ''), x=float(location.X), y=float(location.Y),
|
||||
color=int(attr.Color), font=int(attr.Font), size=float(attr.Size),
|
||||
orientation=int(attr.Orientation), visible=bool(attr.Visible))
|
||||
rows[f'{schematic}:{sheet}:{uid}'] = dict(uid=uid, refdes=str(component.Refdes),
|
||||
schematic=schematic, sheet=sheet, x=float(point.X), y=float(point.Y),
|
||||
orientation=int(component.Orientation), scale=float(component.Scale), attributes=attributes,
|
||||
part_number=attribute_value(component, 'Part Number'), value=attribute_value(component, 'Value'),
|
||||
part_label=attribute_value(component, 'Part Label'),
|
||||
pins=sorted(pins.get(uid, []), key=lambda pin: (pin['x'], pin['y'], pin['number'])))
|
||||
document.Close(False, '')
|
||||
document = None
|
||||
app.CloseProject()
|
||||
assert not project_file_path(app)
|
||||
save(Path(spec['destination']), rows)
|
||||
finally:
|
||||
if document is not None:
|
||||
document.Close(False, '')
|
||||
if app is not None and project_file_path(app):
|
||||
assert Path(project_file_path(app)).resolve() == Path(spec['project']).resolve()
|
||||
app.CloseProject()
|
||||
app = None
|
||||
pythoncom.CoUninitialize()
|
||||
|
||||
|
||||
def compare_saved(before, after):
|
||||
assert set(before) == set(after), 'component identity set changed'
|
||||
selected = 0
|
||||
for key, old in before.items():
|
||||
new = after[key]
|
||||
if old['refdes'] != 'RV05':
|
||||
assert old == new, f'non-target changed: {key}'
|
||||
continue
|
||||
selected += 1
|
||||
assert old['part_number'] == OLD and new['part_number'] == NEW
|
||||
assert new['part_label'] == '0R'
|
||||
# This fixture uses the unchecked copy option. Its native target has no
|
||||
# Value attribute; preserve means retaining that absence, not restoring
|
||||
# OLD's Value. See the 2026-10-14 project-tasks design and the production
|
||||
# QPVerifyValuePolicy existence/content check after native replacement.
|
||||
assert new['value'] == '' and 'Value' not in new['attributes']
|
||||
for field in ('uid', 'refdes', 'schematic', 'sheet', 'x', 'y', 'orientation', 'scale'):
|
||||
assert old[field] == new[field], (key, field)
|
||||
graph = lambda row: sorted((pin['x'], pin['y'], pin['net_uid'], pin['net_name']) for pin in row['pins'])
|
||||
assert graph(old) == graph(new) and new['pins'], f'target connectivity changed: {key}'
|
||||
assert all(pin['number'] for pin in new['pins']), f'unpromoted target pin: {key}'
|
||||
assert selected == 1
|
||||
return {'components': len(after), 'targets': selected, 'unchanged_components': len(after) - selected,
|
||||
'pin_connections': sum(len(row['pins']) for row in after.values()),
|
||||
'target_state': [{key: row[key] for key in ('uid', 'refdes', 'part_number', 'part_label', 'value')}
|
||||
| {'value_attribute_exists': 'Value' in row['attributes']}
|
||||
for row in after.values() if row['refdes'] == 'RV05']}
|
||||
|
||||
|
||||
def verify_existing(repo, root):
|
||||
"""Finish readback only; this branch has no task queue or replacement call."""
|
||||
import sqlite3
|
||||
import win32api
|
||||
from dx_productivity.trial_task_center.isolated_cad import IsolatedCadSession
|
||||
from private_desktop_observer import PrivateDesktopObserver
|
||||
root = root.resolve(strict=True)
|
||||
assert root.is_relative_to(Path(__file__).resolve().parents[1] / 'build')
|
||||
original = json.loads((root / 'report.json').read_text(encoding='utf-8'))
|
||||
with sqlite3.connect(root / 'tasks.db') as database:
|
||||
assert database.execute('select count(*) from tasks where status="completed" and copy_part_number_to_value=0').fetchone()[0] == 2
|
||||
verification = root / 'read-only-verification'
|
||||
verification.mkdir(exist_ok=False)
|
||||
report = {'mode': 'read_only_saved_verification', 'copy_part_number_to_value': False,
|
||||
'value_policy': 'preserve native target Value; this target has no Value attribute', 'projects': []}
|
||||
try:
|
||||
with PrivateDesktopObserver(verification) as observer:
|
||||
for raw_project in original['projects']:
|
||||
project = Path(raw_project).resolve(strict=True)
|
||||
assert project.is_relative_to(root / 'projects')
|
||||
before = json.loads((root / f'before-{project.stem}.json').read_text(encoding='utf-8'))
|
||||
destination = root / f'after-{project.stem}.json'
|
||||
if not destination.exists():
|
||||
evidence = verification / project.stem
|
||||
evidence.mkdir()
|
||||
spec = evidence / 'spec.json'
|
||||
save(spec, {'project': str(project), 'sheets': sorted({(row['schematic'], row['sheet']) for row in before.values()}),
|
||||
'destination': str(destination)})
|
||||
with IsolatedCadSession(evidence) as cad:
|
||||
observer.records[cad.desktop.name] = dict(cad.record)
|
||||
observer.handles[cad.cad.pid] = win32api.OpenProcess(0x100000, False, cad.cad.pid)
|
||||
process = cad.start_worker((sys.executable, str(Path(__file__).resolve()), '--repo', str(repo),
|
||||
'--inspect-file', str(spec), '--run-native'), stdout=-1, stderr=-1)
|
||||
try:
|
||||
code = process.wait(timeout=180)
|
||||
(evidence / 'worker-output.txt').write_bytes(process.stdout.read() + b'\n' + process.stderr.read())
|
||||
assert code == 0, (evidence / 'worker-output.txt').read_text(errors='replace')
|
||||
finally:
|
||||
process.close()
|
||||
after = json.loads(destination.read_text(encoding='utf-8'))
|
||||
report['projects'].append({'project': str(project), **compare_saved(before, after)})
|
||||
observation = json.loads((verification / 'desktop-observation.json').read_text(encoding='utf-8'))
|
||||
assert not observation['observer_errors']
|
||||
assert observation['input_desktop_before'] == observation['input_desktop_after']
|
||||
assert not observation['owned_cad_became_foreground'] and not observation['owned_cad_on_input_desktop']
|
||||
assert all(observation['cad_exit_confirmed'].values()) and all(observation['desktop_absent'].values())
|
||||
report['passed'] = True
|
||||
finally:
|
||||
save(verification / 'saved-verification.json', report)
|
||||
print(json.dumps(report, ensure_ascii=False), flush=True)
|
||||
return 0 if report.get('passed') else 1
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--repo', type=Path, required=True)
|
||||
parser.add_argument('--source-root', type=Path)
|
||||
parser.add_argument('--library', type=Path)
|
||||
parser.add_argument('--output', type=Path)
|
||||
parser.add_argument('--run-native', action='store_true', required=True)
|
||||
parser.add_argument('--inspect-file', type=Path)
|
||||
parser.add_argument('--verify-existing', type=Path, help='Read saved completed run only; never execute replacement')
|
||||
args = parser.parse_args()
|
||||
repo = args.repo.resolve(strict=True)
|
||||
sys.path.insert(0, str(repo / 'src'))
|
||||
sys.path.insert(0, str(repo / 'tools'))
|
||||
if args.inspect_file:
|
||||
inspect_native(args.inspect_file)
|
||||
return 0
|
||||
if args.verify_existing:
|
||||
return verify_existing(repo, args.verify_existing)
|
||||
assert args.output and args.source_root and args.library
|
||||
output = args.output.resolve()
|
||||
assert output.is_relative_to(Path(__file__).resolve().parents[1] / 'build')
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
source_manifest = {'git_head': subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=repo, text=True).strip(),
|
||||
'files': {str(path.relative_to(repo)): hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
for folder, pattern in (('src', '*.py'), ('integration', '*.vbs'))
|
||||
for path in (repo / folder).rglob(pattern)}}
|
||||
save(output / 'source-under-test-manifest.json', source_manifest)
|
||||
library = args.library.resolve(strict=True)
|
||||
library_hash = hashlib.sha256(library.read_bytes()).hexdigest()
|
||||
source_root = args.source_root.resolve(strict=True)
|
||||
source_hashes, projects = {}, []
|
||||
for name in NAMES:
|
||||
source = source_root / name
|
||||
assert (source / (name + '.prj')).is_file()
|
||||
source_hashes[name] = hashes(source)
|
||||
folder = output / 'projects' / name
|
||||
shutil.copytree(source, folder)
|
||||
project = folder / (name + '.prj')
|
||||
text, count = re.subn(r'(?m)^KEY CentralLibrary .*$',
|
||||
lambda _: f'KEY CentralLibrary "{library.as_posix()}"', project.read_text(encoding='utf-8-sig'))
|
||||
assert count == 1
|
||||
project.write_text(text, encoding='utf-8')
|
||||
projects.append(project)
|
||||
from dx_productivity.trial_task_center.bom_plan import write_bom_template
|
||||
from dx_productivity.trial_task_center.isolated_cad import IsolatedCadSession
|
||||
from dx_productivity.trial_task_center.isolated_desktop import PrivateDesktop
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
from dx_productivity.trial_task_center.task_queue import execute_task_queue
|
||||
from private_desktop_observer import PrivateDesktopObserver
|
||||
report = {'repo': str(repo), 'library': str(library), 'source_root': str(source_root),
|
||||
'projects': list(map(str, projects)), 'input': {'old': OLD, 'new': NEW, 'refdes': 'RV05',
|
||||
'copy_part_number_to_value': False, 'value_mode': 'preserve'},
|
||||
'steps': [], 'desktop_starts': [], 'cad_starts': [], 'source_hashes': source_hashes}
|
||||
save(output / 'report.json', report)
|
||||
phase = 'setup'
|
||||
observer = PrivateDesktopObserver(output)
|
||||
original_desktop_enter, original_cad_enter = PrivateDesktop.__enter__, IsolatedCadSession.__enter__
|
||||
def enter_desktop(desktop):
|
||||
started = time.monotonic()
|
||||
try:
|
||||
return original_desktop_enter(desktop)
|
||||
finally:
|
||||
report['desktop_starts'].append({'phase': phase, 'desktop': desktop.name,
|
||||
'seconds': time.monotonic() - started})
|
||||
def enter_cad(cad):
|
||||
started = time.monotonic()
|
||||
try:
|
||||
return original_cad_enter(cad)
|
||||
finally:
|
||||
entry = {'phase': phase, 'desktop': cad.desktop.name, 'seconds': time.monotonic() - started,
|
||||
'pid': None if cad.cad is None else cad.cad.pid}
|
||||
report['cad_starts'].append(entry)
|
||||
if entry['pid']:
|
||||
import win32api
|
||||
observer.records[entry['desktop']] = dict(cad.record)
|
||||
try:
|
||||
observer.handles[entry['pid']] = win32api.OpenProcess(0x100000, False, entry['pid'])
|
||||
except Exception:
|
||||
pass
|
||||
PrivateDesktop.__enter__, IsolatedCadSession.__enter__ = enter_desktop, enter_cad
|
||||
store = TaskStore(output / 'tasks.db')
|
||||
service = TaskCenterService(store, configure_logging(output / 'data'))
|
||||
def record(stage, started, **details):
|
||||
entry = {'stage': stage, 'seconds': time.monotonic() - started, **details}
|
||||
report['steps'].append(entry)
|
||||
save(output / 'report.json', report)
|
||||
print(json.dumps(entry, ensure_ascii=False, default=str), flush=True)
|
||||
def independent(project, sheets, label):
|
||||
name = project.stem
|
||||
root = output / 'independent-read' / 'quick-run-history' / f'sheet-discovery-{label}-{name}'
|
||||
root.mkdir(parents=True)
|
||||
destination, spec = output / f'{label}-{name}.json', root / 'spec.json'
|
||||
save(spec, {'project': str(project), 'sheets': sorted(sheets), 'destination': str(destination)})
|
||||
with IsolatedCadSession(root) as cad:
|
||||
process = cad.start_worker((sys.executable, str(Path(__file__).resolve()), '--repo', str(repo),
|
||||
'--inspect-file', str(spec), '--run-native'), stdout=-1, stderr=-1)
|
||||
try:
|
||||
code = process.wait(timeout=180)
|
||||
(root / 'worker-output.txt').write_bytes(process.stdout.read() + b'\n' + process.stderr.read())
|
||||
assert code == 0, (root / 'worker-output.txt').read_text(errors='replace')
|
||||
finally:
|
||||
process.close()
|
||||
return json.loads(destination.read_text(encoding='utf-8'))
|
||||
started_all = time.monotonic()
|
||||
try:
|
||||
with observer:
|
||||
tasks = service.create_project_tasks(output / 'projects', selected_paths=tuple(projects))
|
||||
assert len(tasks) == 2
|
||||
bom = write_bom_template(output / 'input.xlsx', ({'project_path': str(project),
|
||||
'old_part_number': OLD, 'new_part_number': NEW, 'refdes': 'RV05', 'enabled': True,
|
||||
'sync_nc': True} for project in projects))
|
||||
phase = 'bom_import'
|
||||
started = time.monotonic()
|
||||
imported = service.inspect_bom_batch(tasks[0].task_id, bom, on_progress=lambda message: print(message, flush=True))
|
||||
assert imported.passed_rows == 2 and imported.preview is not None and not imported.failed_rows
|
||||
service.save_bom_batch(imported.preview, {item.task_id: item.rows for item in imported.preview.projects})
|
||||
record(phase, started, passed_rows=imported.passed_rows)
|
||||
phase = 'independent_before'
|
||||
before = {}
|
||||
for project in projects:
|
||||
rows = next(item.rows for item in imported.preview.projects if item.project_path.resolve() == project.resolve())
|
||||
sheets = {(row.target.schematic_name, row.target.sheet_number) for row in rows}
|
||||
before[project.stem] = independent(project, sheets, 'before')
|
||||
phase = 'queue_execution'
|
||||
started = time.monotonic()
|
||||
result = execute_task_queue(service, tuple(task.task_id for task in tasks), stop_event=Event())
|
||||
assert all(entry.state == 'completed' for entry in result.entries), result
|
||||
record(phase, started, result=asdict(result))
|
||||
phase = 'independent_after'
|
||||
for project in projects:
|
||||
sheets = {(row['schematic'], row['sheet']) for row in before[project.stem].values()}
|
||||
started = time.monotonic()
|
||||
after = independent(project, sheets, 'after')
|
||||
record(f'independent-{project.stem}', started, **compare_saved(before[project.stem], after))
|
||||
observation = json.loads((output / 'desktop-observation.json').read_text(encoding='utf-8'))
|
||||
assert not observation['observer_errors']
|
||||
assert observation['input_desktop_before'] == observation['input_desktop_after']
|
||||
assert not observation['owned_cad_became_foreground'] and not observation['owned_cad_on_input_desktop']
|
||||
assert all(observation['cad_exit_confirmed'].values()) and all(observation['desktop_absent'].values())
|
||||
report['verified'] = True
|
||||
except BaseException as error:
|
||||
report['error'] = f'{type(error).__name__}: {error}'
|
||||
report['traceback'] = traceback.format_exc()
|
||||
traceback.print_exc()
|
||||
finally:
|
||||
report['total_seconds_including_independent_reads'] = time.monotonic() - started_all
|
||||
report['workflow_seconds'] = sum(step['seconds'] for step in report['steps'] if step['stage'] in {'bom_import', 'queue_execution'})
|
||||
report['source_unchanged'] = all(hashes(source_root / name) == source_hashes[name] for name in NAMES)
|
||||
report['library_lmc_unchanged'] = hashlib.sha256(library.read_bytes()).hexdigest() == library_hash
|
||||
PrivateDesktop.__enter__, IsolatedCadSession.__enter__ = original_desktop_enter, original_cad_enter
|
||||
store.close()
|
||||
from dx_productivity.trial_task_center.legacy_paths import cleanup_worker_aliases
|
||||
report['alias_cleanup_errors'] = list(map(str, cleanup_worker_aliases()))
|
||||
save(output / 'report.json', report)
|
||||
print(json.dumps({key: report.get(key) for key in ('verified', 'error', 'workflow_seconds', 'source_unchanged',
|
||||
'library_lmc_unchanged', 'desktop_starts', 'cad_starts')}, ensure_ascii=False), flush=True)
|
||||
return 0 if (report.get('verified') and report['source_unchanged'] and report['library_lmc_unchanged']
|
||||
and not report['alias_cleanup_errors']) else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,201 @@
|
||||
"""Replay an issue BOM through inspect_batch using only captured native reports.
|
||||
|
||||
All fixtures and the copied task database are created under --output. This tool
|
||||
never opens CAD and refuses to create a desktop or persist a target plan.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from collections import defaultdict
|
||||
from dataclasses import asdict
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
import zipfile
|
||||
|
||||
|
||||
def digest(path):
|
||||
return hashlib.sha256(Path(path).read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def save(path, value):
|
||||
Path(path).write_text(json.dumps(value, ensure_ascii=False, indent=2, default=str), encoding='utf-8')
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--issue-root', type=Path, required=True)
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
parser.add_argument('--repo', type=Path, default=Path(__file__).resolve().parents[1])
|
||||
args = parser.parse_args()
|
||||
issue, output, repo = args.issue_root.resolve(), args.output.resolve(), args.repo.resolve()
|
||||
assert not output.is_relative_to(issue), 'Original evidence is read-only'
|
||||
assert 'build' in output.parts, 'Replay output must stay inside a build directory'
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
sys.path.insert(0, str(repo / 'src'))
|
||||
|
||||
from openpyxl import load_workbook
|
||||
from dx_productivity.trial_task_center import isolated_cad, isolated_desktop, native_inventory
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
from dx_productivity.trial_task_center.bom_import_report import inspect_batch
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.native_inventory_protocol import read_ascii_discovery
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
from dx_productivity.trial_task_center.target_plan import ProjectIdentityResolver
|
||||
|
||||
identities = ProjectIdentityResolver()
|
||||
expected = json.loads((issue / 'reported-row-replay.json').read_text(encoding='utf-8'))
|
||||
replayed_rows = {row['excel_row'] for row in expected}
|
||||
replayed_projects = {row['project'] for row in expected}
|
||||
baseline_native = json.loads((issue / 'native-replay-report.json').read_text(encoding='utf-8'))
|
||||
native_names = {row['evidence'] for row in baseline_native if row['status'] == 'ready'}
|
||||
captured = defaultdict(list)
|
||||
original_files = [issue / name for name in (
|
||||
'reported-app.zip', 'reported-bom.xlsx', 'reported-row-replay.json', 'native-replay-report.json')]
|
||||
for evidence in (issue / 'evidence/tasks').glob('*/quick-run-history/native-inventory-*'):
|
||||
if evidence.name not in native_names:
|
||||
continue
|
||||
metadata = json.loads((evidence / 'native-inventory.json').read_text(encoding='utf-8'))
|
||||
project = Path(metadata['project'])
|
||||
if project.name not in replayed_projects:
|
||||
continue
|
||||
scope, effective = evidence / 'output/scope.txt', evidence / 'output/effective.txt'
|
||||
assert scope.is_file() and effective.is_file()
|
||||
captured[identities.project(project)].append(evidence)
|
||||
original_files.extend((evidence / 'native-inventory.json', scope, effective))
|
||||
assert len(captured) == len(replayed_projects) == 6
|
||||
before_sources = {str(path): digest(path) for path in original_files}
|
||||
|
||||
database = output / 'app-copy/task-center.db'
|
||||
database.parent.mkdir()
|
||||
with zipfile.ZipFile(issue / 'reported-app.zip') as archive:
|
||||
names = [name for name in archive.namelist() if name.endswith('/data/task-center.db')]
|
||||
assert len(names) == 1
|
||||
database.write_bytes(archive.read(names[0]))
|
||||
connection = sqlite3.connect(database)
|
||||
connection.row_factory = sqlite3.Row
|
||||
source_counts = {'tasks': connection.execute('SELECT count(*) FROM tasks').fetchone()[0],
|
||||
'target_plan': connection.execute('SELECT count(*) FROM target_plan').fetchone()[0]}
|
||||
assert source_counts == {'tasks': 428, 'target_plan': 0}
|
||||
project_rows = list(connection.execute('SELECT project_id,task_id,path FROM projects'))
|
||||
remap, source_by_local, task_by_source = {}, {}, {}
|
||||
for project_row in project_rows:
|
||||
source = Path(project_row['path'])
|
||||
local = output / 'projects' / project_row['project_id'] / source.name
|
||||
local.parent.mkdir(parents=True)
|
||||
local.write_text('KEY FrontEndSnapshot "DxD"\nKEY iCDBDir "database"\n', encoding='utf-8')
|
||||
(local.parent / 'database').mkdir()
|
||||
(local.parent / 'database/icdb.dat').write_bytes(b'captured read-only replay fixture')
|
||||
source_key, local_key = identities.project(source), identities.project(local)
|
||||
remap[source_key] = local
|
||||
source_by_local[local_key] = source_key
|
||||
task_by_source[source_key] = project_row['task_id']
|
||||
connection.execute('UPDATE projects SET path=?,path_identity=?,discovery_root=? WHERE project_id=?',
|
||||
(str(local), local_key, str(output / 'projects'), project_row['project_id']))
|
||||
connection.execute('UPDATE tasks SET discovery_root=?', (str(output / 'projects'),))
|
||||
connection.commit()
|
||||
connection.close()
|
||||
|
||||
workbook = load_workbook(issue / 'reported-bom.xlsx', data_only=False)
|
||||
sheet = workbook.active
|
||||
assert len(workbook.worksheets) == 1
|
||||
assert tuple(cell.value for cell in sheet[1]) == (
|
||||
'project_name', 'project_path', 'old_part_number', 'new_part_number', 'refdes',
|
||||
'schematic_name', 'sheet_number', 'uid', 'enabled', 'custom_value', 'sync_nc')
|
||||
original_rows, replacement_rows, replacement_projects = sheet.max_row - 1, [], set()
|
||||
for excel_row in range(2, sheet.max_row + 1):
|
||||
source_path = sheet.cell(excel_row, 2).value
|
||||
key = identities.project(source_path)
|
||||
assert key in remap
|
||||
sheet.cell(excel_row, 2, str(remap[key]))
|
||||
has_replacement = any(sheet.cell(excel_row, column).value not in (None, '') for column in (3, 4, 5))
|
||||
if has_replacement:
|
||||
replacement_rows.append(excel_row)
|
||||
replacement_projects.add(key)
|
||||
if excel_row not in replayed_rows:
|
||||
# Preserve row positions, routing columns, header and original cell
|
||||
# styles. Uncaptured replacements are explicitly outside this replay.
|
||||
for column in (3, 4, 5, 6, 7, 8, 10):
|
||||
sheet.cell(excel_row, column).value = None
|
||||
else:
|
||||
assert key in captured and has_replacement
|
||||
assert (original_rows, len(replacement_rows), len(replacement_projects)) == (763, 437, 102)
|
||||
replay_file = output / 'captured-scope-original-format.xlsx'
|
||||
workbook.save(replay_file)
|
||||
workbook.close()
|
||||
|
||||
store = TaskStore(database)
|
||||
service = TaskCenterService(store, configure_logging(output / 'data'))
|
||||
loads, phases, forbidden_calls = [], [], []
|
||||
def forbidden(name):
|
||||
def fail(*args, **kwargs):
|
||||
forbidden_calls.append(name)
|
||||
raise AssertionError(f'Read-only replay attempted {name}')
|
||||
return fail
|
||||
for source_key, evidence_paths in captured.items():
|
||||
child = service.for_project_task(task_by_source[source_key])
|
||||
def captured_loader(project, parts, *, collect_components=False, _key=source_key,
|
||||
_evidence=tuple(evidence_paths), **controls):
|
||||
assert source_by_local[identities.project(project)] == _key
|
||||
assert collect_components and controls.get('runtime') is not None
|
||||
reads = [read_ascii_discovery(project, evidence / 'output/scope.txt', parts,
|
||||
effective_report_path=evidence / 'output/effective.txt') for evidence in _evidence]
|
||||
assert all(read == reads[0] for read in reads), 'Repeated native captures disagree'
|
||||
loads.append({'project': str(project), 'original_project': _key, 'requested_parts': parts,
|
||||
'components': len(reads[0].components), 'evidence': list(_evidence)})
|
||||
return reads[0]
|
||||
child._quick_run_preparer._discover_project_sheets = captured_loader
|
||||
initial_dump = tuple(store._connection.iterdump())
|
||||
with (patch.object(BatchCadRuntime, '_ensure_desktop', forbidden('batch desktop')),
|
||||
patch.object(isolated_desktop.PrivateDesktop, '__enter__', forbidden('private desktop')),
|
||||
patch.object(isolated_cad.IsolatedCadSession, '__enter__', forbidden('CAD')),
|
||||
patch.object(native_inventory.NativeInventoryReader, 'read', forbidden('native process')),
|
||||
patch.object(subprocess, 'Popen', forbidden('process launch')),
|
||||
patch.object(store, 'record_target_inventory', forbidden('inventory persistence')),
|
||||
patch.object(store, 'replace_target_plan', forbidden('target-plan persistence'))):
|
||||
report = inspect_batch(service, task_by_source[next(iter(captured))], replay_file,
|
||||
on_phase=lambda phase, message: phases.append({'phase': phase.value, 'message': message}))
|
||||
assert tuple(store._connection.iterdump()) == initial_dump, 'Replay changed the copied task database'
|
||||
assert not forbidden_calls
|
||||
assert report.preview is None
|
||||
assert len(loads) == len(captured) == 6
|
||||
current = [{'excel_row': row.excel_row, 'project': Path(row.project_path).name,
|
||||
'status': row.status, 'reason': row.reason} for row in report.rows]
|
||||
expected_outcomes = [(row['excel_row'], row['project'], row['status']) for row in expected]
|
||||
current_outcomes = [(row['excel_row'], row['project'], row['status']) for row in current]
|
||||
assert current_outcomes == expected_outcomes
|
||||
assert (report.total_rows, report.passed_rows, report.failed_rows, report.unchecked_rows) == (31, 20, 11, 0)
|
||||
assert [row.excel_row for row in report.rows if row.status == 'failed'] == [3, 4, 6, 7, 11, 12, 13, 16, 17, 18, 19]
|
||||
assert {str(path): digest(path) for path in original_files} == before_sources
|
||||
save(output / 'report.json', asdict(report))
|
||||
save(output / 'row-comparison.json', [{'original': old, 'current': new,
|
||||
'row_project_status_equal': True, 'reason_text_equal': old['reason'] == new['reason']}
|
||||
for old, new in zip(expected, current)])
|
||||
save(output / 'captured-loads.json', loads)
|
||||
save(output / 'source-hashes.json', before_sources)
|
||||
save(output / 'phases.json', phases)
|
||||
summary = {'source_repo': str(repo), 'source_commit': subprocess.check_output(
|
||||
['git', 'rev-parse', 'HEAD'], cwd=repo, text=True).strip(),
|
||||
'application_tasks': 428, 'original_bom_rows': original_rows,
|
||||
'original_replacement_rows': len(replacement_rows), 'original_replacement_projects': 102,
|
||||
'captured_projects': 6, 'captured_report_files': sum(map(len, captured.values())),
|
||||
'replayed_rows': report.total_rows, 'passed_rows': report.passed_rows,
|
||||
'failed_rows': report.failed_rows, 'unchecked_rows': report.unchecked_rows,
|
||||
'unreplayed_replacement_rows': len(replacement_rows) - report.total_rows,
|
||||
'unreplayed_projects': 96, 'row_project_status_equal': True,
|
||||
'cad_desktop_or_process_creations': 0, 'target_plan_writes': 0,
|
||||
'copied_database_unchanged_during_inspection': True, 'original_evidence_hashes_unchanged': True,
|
||||
'limitations': 'Only the 31 rows covered by six captured project inventories were replayed. '
|
||||
'Original and current reason wording can differ; Excel row/project/status match exactly.'}
|
||||
save(output / 'summary.json', summary)
|
||||
store.close()
|
||||
print(json.dumps(summary, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,45 @@
|
||||
"""Verify the frozen supervisor, not a source-Python substitute for it."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
|
||||
from PyInstaller.archive.readers import CArchiveReader
|
||||
|
||||
|
||||
def verify(executable: Path, data_root: Path) -> dict:
|
||||
executable = executable.resolve(strict=True)
|
||||
if data_root.exists():
|
||||
raise ValueError('Use a fresh verification data directory')
|
||||
archive = CArchiveReader(str(executable))
|
||||
assert 'win32timezone' in archive.open_embedded_archive('PYZ.pyz').toc, 'Missing win32timezone in frozen archive'
|
||||
environment = dict(os.environ, QT_QPA_PLATFORM='offscreen')
|
||||
for key in ('PYTHONPATH', 'PYTHONHOME', 'DX_TASK_CENTER_HOME'):
|
||||
environment.pop(key, None)
|
||||
result = subprocess.run([str(executable), '--smoke-test', '--data-root', str(data_root.resolve())],
|
||||
env=environment, cwd=executable.parent, timeout=90,
|
||||
creationflags=subprocess.CREATE_NO_WINDOW, capture_output=True)
|
||||
assert result.returncode == 0, (result.returncode, result.stderr)
|
||||
log = (data_root / 'logs/application.log').read_text(encoding='utf-8')
|
||||
assert 'packaged_runtime_smoke_passed' in log
|
||||
assert 'application_clean_shutdown exit_code=0' in log
|
||||
records = [json.loads(p.read_text(encoding='utf-8'))
|
||||
for p in (data_root / 'batch-resources').glob('*/session.json')]
|
||||
assert len(records) == 1 and records[0]['state'] == 'released'
|
||||
cleanup = records[0]['cleanup']
|
||||
assert cleanup['desktop_handle_closed'] and cleanup['desktop_confirmation']['released']
|
||||
assert not cleanup['errors']
|
||||
return {'passed': True, 'frozen_supervisor': True, 'win32timezone_archived': True,
|
||||
'process_creation_time_read': True, 'resources_released': True,
|
||||
'executable': str(executable), 'data_root': str(data_root.resolve())}
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('executable', type=Path)
|
||||
parser.add_argument('data_root', type=Path)
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(verify(args.executable, args.data_root), ensure_ascii=False, indent=2))
|
||||