Merge pull request 'fix: 闭环提交批量替换结果,复用桌面并处理保存占锁' (#31) from codex/issue29-blocking-cleanup into main
This commit was merged in pull request #31.
This commit is contained in:
commit
03ae2cdaa8
117 files changed
+9992
-621
No files matched your search
@@ -0,0 +1,32 @@
|
||||
# 批量替换阻塞与状态闭环实施计划
|
||||
|
||||
按用户在本聊天确认的设计执行。保留此前 issue29 修复,不修改其他聊天的 independent-verification 文档。
|
||||
|
||||
## 用户已确定的规则
|
||||
|
||||
- 位号可明确定位时忽略 UID 缺失、异常、重复或变化;贯通清单、计划、执行、回读、恢复和网表。
|
||||
- 同工程统一保存/关闭重开/回读一次,暂停时提前提交;提交前不得记作 saved_ok。
|
||||
- Mentor 和全部自有写进程确认退出后,隔离残留桌面并放行新批次,不假报桌面已释放。
|
||||
- 结果不确定的工程保留待核验并隔离,同写入源一并隔离;释放全部自有写进程后继续独立工程。人工暂停停止整批。
|
||||
- 操作回执、任务、队列状态幂等收敛。已保存结果、网表验收、资源告警分别展示。
|
||||
- 补齐关键调用进入/返回和等待原因及诊断包,不让清理错误覆盖原错误。
|
||||
|
||||
## 实施任务
|
||||
|
||||
1. 位号身份:原生清单与目标键使用位号身份,保留原始 UID 证据;旧计划/成功操作编号不变,执行与网表允许位号关联,合法多单元及真实歧义分别处理。
|
||||
2. 工程提交/暂停:增加待保存及工程提交记录,工程结束或暂停统一保存回读,所有待提交项原子转完成;SafeStop 不包装成 Baseline,确认未写入暂停返回 pending/paused;未知保存仍 blocked。
|
||||
3. 资源隔离:新增隔离终态和明确阻断字段,持久化进程退出证明;残留桌面不重复 Quit/CloseDesktop,新会话可进入;未知写进程不得隔离放行。
|
||||
4. 工程与任务隔离:新增 BLOCKED,工程异常释放后继续独立工程,区分本地回收控制和用户暂停;共享工程/iCDB写入源隔离,旧回执不重放。
|
||||
5. 诊断与状态:明确停止原因/等待原因,记录关键操作进入返回;诊断包含执行日志、worker进度、原生报告和提交证据,保留脱敏/体积限制;UI正确显示 blocked/failed/paused/completed。
|
||||
6. 集成验收:按设计20种场景测试,回归现有队列/恢复/资源/UI/网表;本机CAD仅副本,不把fake和CLI协议验证称为真实CAD验收;记录每工程save次数及阶段计时。打包候选版但不自动发布旧Release。
|
||||
|
||||
## 接口约定
|
||||
|
||||
- 现有 @REFDES| 标识复用,不改写旧成功操作编号。
|
||||
- saved_ok 仅表示保存回读已通过。新增暂存状态必须由工程提交记录覆盖后才能转 verified。
|
||||
- 隔离桌面保持 desktop_released=false;写进程退出证据与桌面存在性分开。
|
||||
- BLOCKED是业务待核验,不等于全局资源未释放。
|
||||
|
||||
## 工作区与验证
|
||||
|
||||
当前分支 codex/issue29-blocking-cleanup,base b19afc6。已有未提交变更已完成此前回归,用户授权本轮继续实施。每个非平凡行为先RED后GREEN。阶段回归按实际修改选择,最终检查 tracked diff、打包产物和验收覆盖。
|
||||
@@ -0,0 +1,22 @@
|
||||
# 保存后原生数据核验实施计划
|
||||
|
||||
用户已确认:原生数据定位 → DxDesigner 仅负责目标器件替换、属性写入和保存 → 从已保存数据库导出数据核验。此轮修正上一轮没有移除的 CAD 全页核验,不增加第二套全页保护。
|
||||
|
||||
## 约束与接口
|
||||
|
||||
- 正常执行不调用 QPCurrentSemanticState、QPVerifyUntouched、QPNativeCapture/QPNativeRead,不为核验关闭重开 CAD。工程末保存关闭一次;下一工程复用 CAD 会话。
|
||||
- 写入所需的目标器件定位、原实例属性快照、属性恢复与引脚编号提升仍属于写入操作,限于目标器件;不读取非目标属性/样式/连接。禁止把全页检查改名后继续运行。
|
||||
- 新目标快照使用 `X\tsaved_netlist_v1` 标记;保留原 M/C/A 格式及旧操作编号。C.connections 使用 `NETLIST` 表示连接由保存后原生导出核验,不查询 CAD 连接。旧快照/旧恢复证据按原协议处理,不删除历史保护。
|
||||
- 新 CAD 工程收尾清单第一行第三列为 `project_saved`;只证明写入/保存/关闭已返回,不能当成 `project_verified`。各项保持 pending_save,原生导出核验通过后才提交 saved_ok/verified。
|
||||
- 原生 helper 新快照成功阶段为 `native_replace_applied`,保留事务/库件/尝试身份;旧快照仍保留 native_replace_verified。新正常路径不做全页前后 COM 验证。
|
||||
- Python 在确认工程关闭后,复用 native_netlist.capture_project/export_netlist 与原生证据校验,按工程/页/位号对齐全部暂存目标;核验料号、Value/NC、Cell 和可导出的库件属性。证据不完整、目标缺失、属性错误均不得提前完成,不回退 CAD。
|
||||
- 独立前后网表比较继续单独展示;尽量复用同一份保存后导出,不再次驱动 CAD。原生格式不覆盖文字位置/字体/颜色,明确覆盖范围,不伪造图形证明。
|
||||
- 新回执/重启/暂停/恢复不再要求 full_sheet_v1 或 CAD value-readback TSV;使用有保存源与原尝试绑定的原生导出证据。旧回执验证保留兼容。
|
||||
- 不改原工程、中心库、Excel;真实验证仅新副本。保留用户和其他聊天文档。当前功能分支继续,本轮基点 d9cf6a0。
|
||||
|
||||
## 实施与验证
|
||||
|
||||
1. VBS/native_helper:先写真实函数测试,令任何正常路径全页扫描调用都失败;实现仅目标写入、工程保存关闭及 project_saved 协议。测试两项目标、合法多单元、暂停前缀、旧恢复兼容。
|
||||
2. Python 原生核验/回执:先写缺失/错误目标、原生导出失败、无 CAD 的正例;接入工程提交回调和持久证据。不得以省略 baseline 的方式给旧回执放行。
|
||||
3. 集成:同步运行时关闭检查点、journal、恢复与独立网表导出复用。先 RED 后 GREEN;真实副本两项替换比较扫描次数与计时,确认全页 CAD 采集为零、无保存后重开。
|
||||
4. 针对改动回归、独立审查、记录真实限制并提交本地变更。此次先改执行链;上一轮候选程序保留,不将旧包冒充本轮源码。
|
||||
@@ -0,0 +1,28 @@
|
||||
# 固定私有桌面跨批复用 Implementation Plan
|
||||
|
||||
**Goal:** 外部 Agent 保留 HDESK 引用时,正常批次和重启继续复用同一个受管理桌面,停止按批次增加桌面对象。
|
||||
|
||||
**Architecture:** 用现有 supervisor 领取当前用户/登录会话/窗口站唯一的命名槽位。桌面名称稳定,租用编号、Job 和执行代次每批独立;先确认写进程退出,再归还槽位,保留对象未消失的事实。原批次资源与业务回执仍沿用现有规则。
|
||||
|
||||
**Tech Stack:** Python 3.12、pywin32、现有原子 JSON 记录和 Windows Job;无新依赖、常驻服务或桌面池扩容。
|
||||
|
||||
**Spec:** `docs/testing/2026-11-18-desktop-reuse-research.md`,用户已明确批准实施,沿用当前功能分支,不涉及校验工作台。
|
||||
|
||||
## 约束
|
||||
|
||||
- 原工程、中心库和 Excel 不作为故障注入对象;真实验证使用新副本。
|
||||
- 命名槽位跨 data-root/应用重启一致,跨用户/登录会话隔离。
|
||||
- 空窗口列表不能证明写进程退出;实际 Job 与精确进程身份是交接依据。
|
||||
- 崩溃中断的启动窗口不能假报零写入或自动重放;新槽位不能绕过旧写入者。
|
||||
- 对象复用与桌面消失分别记录;旧租用不能关闭新代次的 CAD。
|
||||
- 正常批次仍退出 Mentor,不关闭外部 Agent 或远程强关其句柄。
|
||||
- 缓存命中和纯 CLI 不获取桌面。
|
||||
|
||||
## 步骤
|
||||
|
||||
1. [x] 新增 `desktop_slot.py` 与 `test_desktop_slot.py`。先以跨 root 顺序租用、竞争、放弃的互斥量、存活旧 writer、挂起 launch、损坏记录与重复归还建立 RED;实现稳定名称、命名互斥、持久来源、启动前 Job 记录、不可变归还证明和释放。
|
||||
2. [x] 修改 `isolated_desktop.py`:显式 managed 模式;新批 Job 命名与启动阶段绑定;关闭本批进程/句柄后归还;外部残留不重复等待/关闭。同步 `batch_runtime.py`、独立 `isolated_cad.py`、`cell_catalog.py` 的实际入口,测试证明记录使用真实稳定名称、库读取严格验证进程释放且保留桌面残留。
|
||||
3. [x] `batch_resources.py` 兼容旧 UUID 记录,管理新租用的历史观察;补旧/新租用隔离、进程仍活跃和归还证明损坏的测试,维持回执及源文件保护。
|
||||
4. [x] 使用生产入口实测外部持有、20 次跨进程轮换、并发争用、owner 崩溃/子 Job 未退;副本工程跑两批 Mentor,核验原生数据和桌面数量。完成受影响回归、差异审查与本地提交,记录不覆盖的故障机条件。
|
||||
|
||||
验证临时目录使用 `D:/CodexValidation/managed-desktop-20261118/`。每个新分支先观察预期失败,完成后检查 XML/退出码;不将 Win32 探针作为业务样本。
|
||||
@@ -0,0 +1,109 @@
|
||||
# 批量执行闭环验收
|
||||
|
||||
本轮对应 `docs/superpowers/plans/2026-11-17-batch-execution-closure.md`。位号优先定位;工程内先暂存,工程统一保存回读后提交;不确定工程隔离,确认全部自有写进程退出后继续独立工程。工程执行、网表验收、资源告警分开记录。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[导入 BOM 并保存待执行计划] --> B[原生读取当前工程所需位号与网表]
|
||||
B --> C{工程/页/位号能否明确定位}
|
||||
C -- 否 --> D[记录明确的匹配错误]
|
||||
C -- 是 --> E[逐项原生替换和内存核验]
|
||||
E --> F[写 pending_save / 暂存回执]
|
||||
F --> G{工程还有项且未暂停}
|
||||
G -- 是 --> E
|
||||
G -- 否 --> H[统一保存页 / 关闭工程 / 重开回读 / 最终关闭]
|
||||
H --> I{全部前缀核验通过}
|
||||
I -- 是 --> J[原子提交已保存回执和任务进度]
|
||||
J --> K{人工暂停}
|
||||
K -- 是 --> L[结束整批 / 保留未执行项]
|
||||
K -- 否 --> M[继续下一独立工程]
|
||||
E -- 未知写入或异常 --> N[保留 uncertain / 工程待核验]
|
||||
I -- 否 --> N
|
||||
N --> O[回收该工程自有写进程]
|
||||
O --> P{写进程退出和自有句柄关闭已证实}
|
||||
P -- 否 --> Q[停止整批 / 等待资源核验]
|
||||
P -- 是 --> R[隔离工程及共享写入源]
|
||||
R --> S{旧桌面是否仍存在}
|
||||
S -- 是 --> T[保留隔离告警 / desktop_released=false]
|
||||
S -- 否 --> M
|
||||
T --> M
|
||||
J --> U[独立原生网表验收 / 单独显示结果]
|
||||
R --> V[共享源后续任务未派发即阻塞]
|
||||
V --> W[原依赖恢复后显式核验重新准入]
|
||||
```
|
||||
|
||||
## 20 类验收场景
|
||||
|
||||
下面是自动化验收矩阵,不代表 20 次真实 Mentor 替换。Python 测试替代 COM 边界;VBS 测试执行生产脚本函数并使用假 COM 对象。真实 Windows 桌面测试另列。
|
||||
|
||||
| # | 场景 | 需要确认的结果 | 自动化覆盖 |
|
||||
|---|---|---|---|
|
||||
| 1 | 器件 UID 缺失或格式异常 | 位号明确时可定位,原始 UID 留证 | `test_refdes_identity_closure` inventory cases |
|
||||
| 2 | 不同位号共享 UID | 不误绑定,不阻断无关器件 | 同模块 duplicate UID / unselected collision cases |
|
||||
| 3 | 替换或重开后 UID 变化 | 快照、回读、网表仍关联原位号,旧操作编号不变 | 同模块 snapshot/readback/netlist cases |
|
||||
| 4 | 原 UID 被另一位号复用 | 不把错误位号当成替换成功 | 同模块 reused UID mismatch cases |
|
||||
| 5 | 不同页或层级同位号 | 依据工程、页和层级定位,真实歧义拒绝 | 同模块 sheet/hierarchy cases |
|
||||
| 6 | 同一物理器件多个合法单元 | 全单元快照及回读,物理引脚连接仍严格比较 | 同模块 multiunit cases |
|
||||
| 7 | 工程中多项暂存 | 游标推进,但 saved/completed 不提前增加 | `test_project_commit_progress` |
|
||||
| 8 | 工程末统一保存 | 每个已修改页保存一次,工程关闭重开回读一次 | `test_project_commit_vbs` save/reopen case |
|
||||
| 9 | 暂存前缀后人工暂停 | 提交已修改前缀,未执行项不产生写入/不确定状态 | `test_project_commit_end_to_end` pause cases |
|
||||
| 10 | 写入前暂停 | 保留 SafeStop 来源,当前项保持 pending | `test_project_commit_vbs` / `test_resource_isolation_prewrite_pause` |
|
||||
| 11 | 回读失败或提交清单不完整 | 不提前提交任何暂存项 | `test_project_commit_journal` / `test_project_commit_proof` |
|
||||
| 12 | 保存成功后程序崩溃 | 从不可变提交证据幂等恢复,不重放替换 | `test_project_commit_journal` / `test_project_commit_public_recovery` |
|
||||
| 13 | 工程保存回读持续有进度且超过原等待时间 | 不因固定 60 秒误判收尾失败 | `test_project_commit_session` progressing commit case |
|
||||
| 14 | 自有进程退出但旧桌面仍被外部句柄持有 | 保留 desktop_released=false,隔离告警,新批次可进入 | `test_resource_isolation`;真实 Windows 探针 |
|
||||
| 15 | 仍有写进程或进程身份未知 | 不允许隔离放行 | `test_resource_isolation` active/unknown cases |
|
||||
| 16 | 隔离后重启,随后外部句柄消失 | 重验退出证明;被动确认释放,不重复 Quit | 同模块 restart/passive disappearance cases |
|
||||
| 17 | 工程结果未知,资源已收尾 | 保留 blocked 和 uncertain,继续独立工程,恢复队列不重放 | `test_project_isolation_queue` / `test_workspace_execution` |
|
||||
| 18 | 本地回收与人工停止 | 本地回收不污染下一工程;人工暂停/强制结束仍生效 | `test_project_isolation_queue` / workspace control cases |
|
||||
| 19 | 后续工程共享 iCDB 写入源 | 未派发即阻塞,原依赖核验完成后才重新准入 | `test_lifecycle_integration_fix` dependency cases |
|
||||
| 20 | 任务完成、失败、恢复及资源告警 | 实时分类;已保存结果与网表/资源结果分别展示;诊断含最后成功项和未返回调用 | `test_live_task_completion` / `test_failed_native_recovery_guard` / `test_diagnostics_closure` |
|
||||
|
||||
## 证据位置
|
||||
|
||||
- 本轮回归日志:`build/batch-execution-closure/`。
|
||||
- Windows 桌面实际持有、重启复核及新桌面准入:`build/batch-execution-closure/resource-isolation/run-4ace6aa7a36b4204a7bed568b5ce7505/evidence.json`。
|
||||
- 工单原始日志保留在 `outputs/issues-20261117/issue29/`,不修改用户上传证据。
|
||||
|
||||
## 自动化验证结果
|
||||
|
||||
主程序/UI 全量运行 2335 项,最初 2320 项通过、15 项失败。14 项仍断言“未知结果必须整批暂停”,已按已批准的工程隔离策略更新,同时保留同工程后项不派发、不重放和独立工程可继续的断言;另一项修复测试对尚未创建完成的原生弹窗按钮的读取竞态。随后对全部失败项和本轮新增/修改的调用链定点复验。
|
||||
|
||||
最终 JUnit 合并记录为 **2438 个唯一用例通过,0 个未解决失败**,证据见 `build/batch-execution-closure/verified-test-summary.json`。这是全量运行加定点复验的合并结果,不是另一次全部用例在最后源码上的全量运行。定点记录为 root delta 143 项、lifecycle delta 97 项、project recovery 29 项、identity delta 47 项、deleted guard 1 项,均通过;这些记录存在重叠,不应直接相加。
|
||||
|
||||
独立审查已复核位号/多单元、工程暂存及暂停、资源隔离、共享源恢复及公开工程提交恢复。工程已保存但数据库回执未完成时,公开恢复入口验证原生关闭检查点、保存源、回读文件及原尝试身份后恢复回执,不启动 CAD 或重复替换;源变化、其他未知写入、证据损坏及不匹配的中断记录继续保持待核验。
|
||||
|
||||
## 实机边界
|
||||
|
||||
工程级批量保存、UID 异常和多单元替换需要使用副本进行真实 Mentor 验收。自动化协议验证、真实桌面句柄探针和真实 CAD 写入必须分别报告;不得用前两者声称已完成原 100 工程队列。原工程、中心库、Excel 和历史 uncertain 记录不作为试验修改对象。
|
||||
|
||||
首轮真实 Mentor 探针 `build/batch-execution-closure/mentor-live/` 在 RL02 原生替换和内存回读后失败:新暂存入口未暂时清除内存 `restored_ok`,导致生产命令读取器认定没有待处理命令。该问题已有真实读取器回归;复测使用 `mentor-live-fixed/` 全新副本。失败副本及 uncertain 保留,不重放。首轮 Mentor 正常退出、桌面释放,无强制终止;原备份 34 个文件前后哈希一致。
|
||||
|
||||
第二轮 `mentor-live-fixed/` 发现未选中非 PCB 图形的空位号被写成相同别名,导致后续全页原生回读重复 UID;已对空位号跳过引用别名并保留其独立原始 UID。第三轮 `mentor-live-final/` 通过该位置并写出 `pending_save`,随后全页语义采集缺少逐器件进度,触发无进度停止;补充实际完成计数后使用 `mentor-live-progress/` 新副本复测。前三轮异常资源均确认释放,备份源文件前后哈希一致;不计为工程保存成功验收。
|
||||
|
||||
第四轮真实 Mentor 验收通过,证据为 `build/batch-execution-closure/mentor-live-progress/closure-live-summary.json`。RL02、RL05 两项替换先暂存,最后统一提交;日志确认一个 worker、一次保存、两次打开工程(首次打开及保存后重开)。两项 SQL 操作均为 `verified/generation=1`,工程提交 `verified`,任务 `completed`。独立原生清单读取复核 13 个位号:两项目标料号正确,其余 11 项保留原料号。Mentor 和桌面确认释放,无强制终止;备份源 34 个文件哈希未变化。
|
||||
|
||||
### 本机计时及限制
|
||||
|
||||
| 实际边界 | 秒 |
|
||||
|---|---:|
|
||||
| 第一项开始至暂存回执 | 472.664 |
|
||||
| 第二项派发至暂存回执,含前一项检查点核验 | 539.298 |
|
||||
| 工程收尾,含完整状态检查、统一保存、关闭重开和回读 | 431.863 |
|
||||
| 其中两次原生替换提交调用 | 0.631 / 0.424 |
|
||||
| Mentor 退出 | 5.093 |
|
||||
| 桌面释放 | 0.009 |
|
||||
| 整个队列 | 1450.341 |
|
||||
| 队列结束后的独立原生清单复核 | 1.505 |
|
||||
|
||||
“其中”调用嵌套于上层步骤,不能再次相加。本轮同时存在构建和回归负载,不能作为独占性能基准,也没有同条件旧版对照。全页状态反复采集仍是明显耗时项;本次证明阻塞闭环、保存次数与结果提交正确,**没有证明速度提升**。真实 CAD 仅验证一个副本工程、两次替换;多单元、UID 异常和 20 类异常场景主要依靠自动化协议/COM 边界测试,不能当成 20 个真实工程已通过。
|
||||
|
||||
候选包仅供本地测试,不自动覆盖旧 Release。
|
||||
|
||||
## 本地候选程序
|
||||
|
||||
入口:`D:\CodexValidation\batch-closure-20261117\candidate\dist\DxPartReplace\DxPartReplace.exe`。这是包含完整 Harness/Node/Qt 运行时的目录包,运行时文件须与程序一起保留。应用版本资源仍为 1.3.1,本轮候选代码通过冻结模块核对区分,不代表远端已经发布了新版本。
|
||||
|
||||
冻结包内 162 个应用模块与最终源码编译结果一致,两份 VBS 与源码逐字节一致。EXE SHA256:`4b19948b6a008912851f3ab674afa361bf4a28de4b74f20d3d04b80719200cc1`。打包程序本身已通过启动、冻结监督进程、进程身份读取、资源释放及正常退出验证。报告位于候选目录上级的 `source-verification.json` 与 `packaged-runtime-verification.json`。
|
||||
|
||||
实施决定:沿用已有功能分支保留先前授权修复;互不重叠的模块并行处理后统一审查;本地直接交付完整目录包。前两项若判断错误需集成返工,目录包如需转移到其他机器可另行归档。本轮未合并、推送或发布远端版本。
|
||||
@@ -0,0 +1,68 @@
|
||||
# 保存后原生数据核验
|
||||
|
||||
本轮修正正常执行链仍然依赖全页 CAD/COM 核验的问题。DxDesigner 负责目标器件替换、属性恢复、保存和关闭;保存后的目标数据与连接核验由 Mentor 原生命令行导出承担。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[原生数据发现目标:工程、页、位号] --> B[打开工程与目标页]
|
||||
B --> C[仅记录目标器件原属性]
|
||||
C --> D[替换目标器件,记录库件 Value 写入期望]
|
||||
D --> E[恢复属性与 NC,暂存 pending_save]
|
||||
E --> F{还有同工程目标?}
|
||||
F -- 有 --> C
|
||||
F -- 无或暂停 --> G[保存并关闭一次,发布 project_saved]
|
||||
G --> H[捕获保存数据库,原生命令行导出]
|
||||
H --> I{按工程、页、位号核验目标数据}
|
||||
I -- 通过 --> J[原子提交本工程回执,追加完成状态]
|
||||
I -- 不通过或导出失败 --> K[保留已保存待核验状态,不重放替换]
|
||||
K --> H
|
||||
J --> L[独立前后网表比对复用同一份保存后导出]
|
||||
```
|
||||
|
||||
## 开发项
|
||||
|
||||
- [x] 正常路径不调用全页语义状态、非目标属性、连接索引与原生 CAD 宿主快照。
|
||||
- [x] 新快照 `saved_netlist_v1` 仅含目标 C/A;旧快照继续使用原兼容路径。
|
||||
- [x] helper 使用 `native_replace_applied`,不将写入完成冒充核验通过。
|
||||
- [x] 写入前固定选定库件、Value/NC 与 Cell 期望,保留原 preserve 语义。
|
||||
- [x] 工程末保存关闭一次;不为核验重开 CAD。
|
||||
- [x] 原生导出按工程、页、位号匹配目标,UID 作为尝试关联证据。
|
||||
- [x] `project_saved` 仅授权后续原生验证,SQL 完成提交必须具备已验证导出。
|
||||
- [x] 暂停、重启与恢复沿用原尝试,已保存操作不重放。
|
||||
- [x] 回执只记录本项观察与共享导出引用,完整工程证明只存一份。
|
||||
- [x] 独立网表比对复用保存后导出。
|
||||
- [x] 完成状态以追加方式发布,保持活动批次的增量读取位置。
|
||||
- [x] 单步执行和非复用模式也按本次调用保存、原生核验、提交完成。
|
||||
- [x] 多批保存恢复只对最新保存状态比较当前源;历史已完成批次核对其固化证据。
|
||||
- [x] 修正真实工程端口名 `LED2/RXDLY` 被误识别为层级路径的问题。
|
||||
- [x] 审查发现的可变缓存图与 mixed no-match 提交问题均有失败复现和修复测试。
|
||||
|
||||
## 验收记录
|
||||
|
||||
真实验证只使用备份的新副本,原备份 34 个文件哈希保持不变。样本选择单页 RL02/RL05,料号从 `4100-CA3300-T7` 替换为 `4100-CA3300-2200`。
|
||||
|
||||
第一次副本完成替换、一次保存与关闭,但原生解析因带 `/` 的端口名误判而拒绝提交。修复解析后,新 Python 进程在 6.03 秒内完成导出与原子提交,两项仍为 generation 1,没有调用 CAD 或重放替换。以原备份独立导出作前基准,与已验证保存后导出比较,结果 verified、差异为空;后导出直接复用。
|
||||
|
||||
第二次副本暴露完成状态发布重建文件、破坏增量读取位置的问题。追加式发布修复后,第三次全新副本完整执行通过:任务表状态 completed,两项操作 verified 且 generation 仍为 1,CAD 进程正常退出。
|
||||
|
||||
| 指标 | 上一轮 | 本轮第三份副本 |
|
||||
| --- | ---: | ---: |
|
||||
| 两项任务总耗时 | 1450.341 秒 | 226.461 秒 |
|
||||
| 工程打开 | — | 1 次 |
|
||||
| 工程保存 | 1 次 | 1 次 |
|
||||
| 全页原生 CAD 快照 | 4 次 | 0 次 |
|
||||
| 保存后为核验重开 CAD | 存在 | 0 次 |
|
||||
|
||||
本轮无 `post-state-*.tsv`,两个目标快照均无 U/G 记录。独立原生器件清单读取耗时 1.312 秒,13 个器件料号全部符合预期;对第三份副本的完整原生前后网表比较同样 verified、差异为空,直接复用保存后导出。原备份 34 文件哈希不变。耗时降低约 84.4%,剩余耗时主要在 CAD 启动、目标定位、替换交互和属性写入。
|
||||
|
||||
本次只完成两项目标的实机功能与性能探针,不能代替此前要求的 20 个业务样本验收。
|
||||
|
||||
第四份新副本验证非复用分支:253.389 秒,两次打开/保存,两个工程提交均 verified,任务 completed,全页 CAD 原生采集仍为 0。两个公开执行事件的 Value 回执均为 verified。随后在该副本模拟“SQL 已提交但状态发布未完成”的保护记录,复现历史源指纹错误阻塞;修复后,新 Python 进程在 0.452 秒内完成恢复,尝试编号、generation、操作状态和历史条目数量均未改变,审计记录 `native_replacement_called=false`。CAD 进程已正常退出。
|
||||
|
||||
全量回归先得到 2412 通过、1 失败;失败来自抽取 VBS 函数的测试夹具漏声明全局新模式变量。修正夹具后该文件 37 项通过;最后一轮涉及执行、恢复、状态、网表与身份关联的回归 430 项通过。后续多批恢复专项 51 项通过,包含三个恢复时机与五个失败证据场景。将全量与增量报告按测试名称合并、以后续复验覆盖原失败结果,**2426 个不同用例均已验证通过**;这不是一次完整重跑的数量,也不将重复执行算作额外业务样本。
|
||||
|
||||
证据位于 `D:/CodexValidation/saved-netlist-20261117/`,自动化报告位于 `build/saved-netlist-validation/`。
|
||||
|
||||
## 覆盖限制
|
||||
|
||||
原生 ASCII 导出不证明 Value 属性的空值/缺失差异,也不覆盖 CAD 显示位置、字体和颜色;回执保留此限制,不伪造这些图形核验结果。本轮不更新旧候选安装包。
|
||||
@@ -0,0 +1,56 @@
|
||||
# 私有桌面累积:GitHub 参考与本地可行性验证
|
||||
|
||||
日期:2026-11-18。范围:工单 #20/#29 的 Win32 Desktop 生命周期;不涉及独立校验工作台。本轮为源码调研和一次性 Win32 探针,未修改产品执行逻辑,也未启动 Mentor。
|
||||
|
||||
## 结论
|
||||
|
||||
推荐采用“当前用户、当前 Windows 登录会话内的固定命名桌面,串行租用;每批独立进程 Job 与执行代次”。桌面身份与任务/批次身份解耦。默认只需一个槽位,因为现有 CAD 本身串行执行。
|
||||
|
||||
现有 `BatchCadRuntime._ensure_desktop()` 每次需要新运行代次就生成 `dx-<uuid>`。外部 Agent 留住旧桌面后,隔离记录允许后续任务继续,但新一代仍创建不同桌面;这处理了队列阻断,没有限制桌面对象数量。
|
||||
|
||||
固定对象可以在前一批自有写进程退出后重新打开,即使外部只读句柄仍存在。关闭自己的句柄、关闭 Mentor 和确认桌面对象消失是三件不同的事。复用不应把 `desktop_disappeared=false` 伪报为 true。
|
||||
|
||||
## GitHub 参考
|
||||
|
||||
| 项目 | 核对源码 | 借鉴内容及边界 |
|
||||
| --- | --- | --- |
|
||||
| Sandboxie Plus | [GuiServer.cpp](https://github.com/sandboxie-plus/Sandboxie/blob/f23a6cfd50f506413634e8f2906bfc2888d2b1c3/Sandboxie/core/svc/GuiServer.cpp#L1237) | `GetWindowStationAndDesktopName` 使用静态窗口站/桌面句柄和缓存名称,复用长期管理对象。这里只借鉴资源生命周期,不表示 Sandboxie 已证明解决本机 OCular 问题;它的 dummy desktop、驱动和权限模型不照搬。 |
|
||||
| System Informer | [native.c](https://github.com/winsiderss/systeminformer/blob/952a0e7a4efb45b5e1acc4c733916684a7803406/phlib/native.c#L2158)、[hndlinfo.c](https://github.com/winsiderss/systeminformer/blob/952a0e7a4efb45b5e1acc4c733916684a7803406/phlib/hndlinfo.c#L2256) | 系统句柄枚举、进程句柄快照、对象类型/名称查询,适合作为按需持有者诊断参考。权限不足必须保留未知,不以推断代替持有者身份,不把远端关闭句柄作为日常回收策略。 |
|
||||
| DesktopHeapDiag | [README](https://github.com/leonardomsft/DesktopHeapDiag/blob/7af5ffc3bbc499d3e0850c61326880bc95cb93a5/README.MD)、[main.cpp](https://github.com/leonardomsft/DesktopHeapDiag/blob/7af5ffc3bbc499d3e0850c61326880bc95cb93a5/main.cpp) | 说明桌面堆与 USER 对象的区别。该工具通过大量注册窗口类、逼近分配上限估算使用率,不适合嵌入产品每批自动运行;本轮只读源码,没有运行它。 |
|
||||
|
||||
另核对 UltraVNC 的桌面线程退出处理和 CreateDesktop 示例。普通 Windows Virtual Desktop 切换器使用另一套窗口分组接口,不能替代 Win32 HDESK 管理。未引入任何上述项目的二进制、驱动或依赖。
|
||||
|
||||
微软 [CreateDesktopW](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createdesktopw) 明确规定:同名桌面已存在时返回已有对象句柄。[CloseDesktop](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-closedesktop) 关闭调用方句柄;调用进程线程正在使用该句柄时会失败。重复 CloseDesktop 不能替外部进程释放其引用。
|
||||
|
||||
## 最小落地方案
|
||||
|
||||
1. 同一用户、同一 Windows 登录会话、同一窗口站使用一个稳定桌面名。不能由工作区路径或批次 UUID 决定,避免更换 data-root、重启程序或多份安装包又制造新桌面。
|
||||
2. 用同范围的命名互斥量串行领取桌面;桌面名称、访问权限与持久租用记录共同确认归属,不能收养任意历史 `dx-*` 对象。
|
||||
3. 每批仍建立新 Job、执行代次、尝试身份与新 Mentor 进程。前批自有写进程及其子树确认退出后,才能把桌面交给下批。已保存回执、源隔离和不重放规则沿用既有逻辑。
|
||||
4. 批次结束先收尾 Mentor/worker 和 Job,再关闭本批桌面句柄。对象消失则记录 released;外部句柄留住但已满足交接条件时记录可复用的空闲桌面及残留事实。不能继续依赖“名称存在=前批写入仍活跃”。
|
||||
5. 异常/重启后重新核对原进程身份与租用记录,再打开相同槽位。槽位仍有实际写入者或无法安全交接时,仅等待该槽位处理,不以新 UUID 桌面绕开。
|
||||
6. 历史资源记录引用租用代次,而不把名称相同误认成同一批 CAD。旧记录不能反复向新 Mentor 发 Quit,也不能因新批正在使用同名桌面而重判旧批清理失败。
|
||||
|
||||
首版无需常驻服务、桌面池扩容或新第三方管理器。使用现有 supervisor 持有本批租用即可。正常结束仍按用户要求退出 CAD;跨批复用的是桌面对象。
|
||||
|
||||
## 本地可行性探针
|
||||
|
||||
脚本:`build/desktop-research-20261118/probe_named_desktop_reuse.py`。结果:`build/desktop-research-20261118/named-desktop-reuse-result.json`。
|
||||
|
||||
- 一个明确由探针创建的独立进程持续 OpenDesktop,模拟外部 Agent 持有引用。
|
||||
- 创建进程关闭自己的句柄后,桌面继续存在,确实进入外部持有场景。
|
||||
- 20 个独立工作进程依次打开/创建同一个名称,每次实际启动另一个进程,在目标桌面创建并销毁 Win32 STATIC 窗口。
|
||||
- 其中 4 个工作进程在自己的 GUI 子进程正常退出后直接 `os._exit(0)`,验证其桌面句柄交给系统收尾的情况;没有测试 GUI 子进程仍活着时的崩溃接管。
|
||||
- 20 轮名称集合始终只比初始状态多同一个对象,新增桌面对象数为 1。
|
||||
- 外部持有者关闭句柄后,该对象消失。调用线程的桌面未变化,没有切换输入桌面。
|
||||
- 46.356 秒,探针 passed;所有探针子进程已结束,无 Mentor 进程启动。
|
||||
|
||||
该探针证明同名对象在外部引用保留下可跨进程复用,不证明真实 OCular 的持有方式完全相同,也不证明 Mentor 跨批 ROT、COM、许可、弹窗和退出兼容性。它更不代表 20 个业务样本通过。
|
||||
|
||||
## 产品改动范围与正式验收
|
||||
|
||||
预计涉及 `isolated_desktop.py` 的明确打开/创建与释放事实、`batch_runtime.py` 的槽位租用和代次、`batch_resources.py` 的历史租用解释,以及 supervisor 跨进程互斥范围。保留现有进程所有权、已保存回执和原生网表核验。
|
||||
|
||||
正式验收应包括外部引用持续存在下的多批真实 Mentor、正常退出/重启、owner 崩溃但 Job 子进程尚在、两份应用同时请求、证据中断、旧租用记录与新代次隔离,并检查桌面对象数不随批次数增长。至少两份工程轮换且分别核验最终原生数据。
|
||||
|
||||
已经存在的任意 UUID 桌面不会因采用稳定槽位自动消失。存量对象仍需外部持有方正常关句柄,或由系统在相关登录会话结束时回收;产品先控制新增对象数量。应在用户故障机上核对持有者和长期运行数据,才能宣布 #20 根治。
|
||||
@@ -0,0 +1,83 @@
|
||||
# 固定私有桌面复用:实现与验收
|
||||
|
||||
日期:2026-11-18。分支:`codex/issue29-blocking-cleanup`,基础提交:`d08a940`。范围为 #20/#29 的桌面累积和租用交接;不包含独立校验工作台。
|
||||
|
||||
## 实现结果
|
||||
|
||||
同一 Windows 用户 SID、登录 SID、窗口站产生稳定的 `dx-<32hex>` 桌面名,不再按批次 UUID 创建新桌面。批次、执行代次、租用编号、Mentor 进程及每个子进程的命名 Job 仍独立。批次结束仍退出 Mentor。
|
||||
|
||||
默认队列运行时、独立 CAD 和需要 GUI 的 LibraryManager 入口均启用 managed 模式。原始 `PrivateDesktop` 默认接口保留,缓存命中及纯 CLI 路径不额外领取桌面。无新依赖、服务或桌面池。
|
||||
|
||||
租用状态存于 `%LOCALAPPDATA%/DxPartReplace/desktop-slots/<scope>/`,使用命名互斥量跨进程串行交接。目录创建后取规范路径,以适应本机 Codex MSIX 路径虚拟化。一个短生命周期线程持有和释放 Win32 mutex,允许主线程领取、监控线程收尾,避免 Windows 互斥量的线程所有权错误。
|
||||
|
||||
启动前先持久化命名 Job;实际 Windows 启动通过 `PROC_THREAD_ATTRIBUTE_JOB_LIST` 在 CreateProcess 内原子分配 Job,避免 owner 在“创建子进程”和“加入 Job”之间崩溃留下无归属子进程。启动记录未发布时,仅原子分配且实际 Job 已空的情况允许接管。自定义未知启动边界继续保留未确认状态。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[请求执行批次] --> B[领取稳定名称的互斥租用]
|
||||
B --> C{前批 owner / Job 已退出?}
|
||||
C -- 否或未确认 --> D[保留等待状态,不创建其他桌面]
|
||||
C -- 是 --> E[创建或打开同名桌面]
|
||||
E --> F[独立命名 Job 启动本批 Mentor / worker]
|
||||
F --> G[执行、保存及原生数据核验]
|
||||
G --> H[正常退出或失败收尾本批进程]
|
||||
H --> I{本批写入者与句柄均已收尾?}
|
||||
I -- 否 --> J[保留租用,继续收尾]
|
||||
I -- 是 --> K[关闭本批桌面句柄,写不可变归还证明]
|
||||
K --> L{桌面对象已消失?}
|
||||
L -- 否 --> M[记录 reusable / isolated,允许下批复用]
|
||||
L -- 是 --> N[记录 released]
|
||||
```
|
||||
|
||||
外部句柄留住桌面时,记录 `desktop_released=false` 与 `desktop_release_state=reusable`,不把归还租用伪报成对象已消失。旧批次只校验自己的不可变归还证明;被动观察桌面消失也取得同一互斥量,不打开或关闭新批次的 Mentor。
|
||||
|
||||
## 验收证据
|
||||
|
||||
证据根目录:`D:/CodexValidation/managed-desktop-20261118/`。构建临时 XML 位于本仓库 `build/managed-desktop/`,不作为产品文件提交。
|
||||
|
||||
| 验证 | 结果 | 证据 |
|
||||
| --- | --- | --- |
|
||||
| 最终相关回归 | 133 passed,43.55 秒 | `build/managed-desktop/final-regression.xml` |
|
||||
| 更早运行的桌面、runtime、catalog 回归 | 143 passed | `build/managed-desktop/regression-1.xml` |
|
||||
| 更早运行的资源生命周期、恢复、保存证明回归 | 135 passed | `build/managed-desktop/fake-delta.xml` |
|
||||
| 最终原生循环 | 20 轮通过,34.781 秒 | `native-probe-04/report.json` |
|
||||
| 两批真实 Mentor | 均通过保存与原生数据回读 | `mentor-live-final/two-batches-result.json` |
|
||||
| 原始备份保护 | 34 个文件 SHA-256 均未变化 | 对照 `mentor-live/original-hashes.json` |
|
||||
|
||||
上述 pytest 集合存在交集,不能累加为独立样本数。先前一次回归与真实 CAD 同时争用默认槽位导致 3 项失败,最终已在 CAD 退出后串行重跑相关模块,包含这 3 项,133 项全过。
|
||||
|
||||
### 原生循环的判定范围
|
||||
|
||||
生产 `BatchCadRuntime` 入口运行 20 个独立进程轮次,外部 helper 全程 OpenDesktop 保持句柄,各轮实际创建和销毁 Win32 窗口。验证稳定桌面名、20 个不同租用编号、竞争实例拒绝领取、owner 崩溃后存活 Job 阻止接管、Job 退出后允许复用。分别覆盖进程身份发布后崩溃,以及原子 Job 已分配但身份尚未发布时崩溃。helper 释放后检查对象消失。
|
||||
|
||||
桌面数量断言针对本探针自己的资源记录:整个运行仅一个受管理名称。全系统枚举同时记录,但其他应用可能并发创建对象,不能把全系统变化归因于本产品。早期 probe-01 因该全系统断言失败的报告保留;probe-03 在改成按归属统计后通过,最终版本另跑 probe-04。原生 20 轮不是 20 个 BOM 业务样本。
|
||||
|
||||
最终 probe-04 全部判定通过:20 个独立租用共用一个名称;并发实例和崩溃后存活 Job 均阻止接管;两种崩溃边界在 Job 退出后恢复;外部 holder 退出后桌面消失。最终检查两批真实任务的全部已记录写进程身份均已退出,受管理桌面已不存在。
|
||||
|
||||
### 两批真实 Mentor
|
||||
|
||||
使用两个独立的新工程副本,同一个外部 helper 跨两批一直持有桌面引用。每批仅替换 `RL02`:`4100-CA3300-T7` → `4100-CA3300-2200`,并从保存后的原生数据回读确认 RL02 和其他未选位号。
|
||||
|
||||
| 项目 | 第一批 | 第二批 |
|
||||
| --- | --- | --- |
|
||||
| 队列执行耗时 | 201.373 秒 | 156.222 秒 |
|
||||
| saved / failed | 1 / 0 | 1 / 0 |
|
||||
| 任务状态 | completed | completed |
|
||||
| CAD 代次 | 1 | 1 |
|
||||
| 正常退出 / 强杀 | 正常 / 无 | 正常 / 无 |
|
||||
| CAD 退出收尾耗时 | 3.484 秒 | 3.156 秒 |
|
||||
| 租用编号 | `2a9dfc16c34e4510bc0df4710869ea02` | `62c4f185a1ac45ccad8a387369d7b5d3` |
|
||||
|
||||
两批桌面均为 `dx-5f48d43089f541634f609af9e3fbdfac`。批次刚结束、helper 仍持有时,Mentor 已释放,桌面未消失,租用已归还。helper 关闭后,两批历史资源记录经被动核对均为 released,待释放集合为空。整体驱动耗时 368.837 秒。
|
||||
|
||||
任务 ID:第一批 `e7fbebd9-c2fa-4554-be05-26f004fd0d02`,第二批 `bfdcaaf8-59ec-467f-81b5-bf061858e799`。各批 `report.json`、`tasks.db` 和 `data/batch-resources/*/session.json` 均保留于 `mentor-live-final/`。
|
||||
|
||||
## 审查和保留问题
|
||||
|
||||
独立源码审查发现并修复两处问题,均观察 RED/GREEN:归还证明临时读取失败后,历史记录不能恢复非阻塞状态;创建 Job 失败时,没有启动进程却留下未确认 launching 记录。随后审查原子 Job 分配及跨线程 mutex 交接,未发现新的具体问题。
|
||||
|
||||
较早真实运行保留在 `mentor-live/`:首批选择 RL02、RL05,两处替换以及 Save/CloseProject 返回后,`bind_project_commit_source` 读取 `database/icdb.dat` 遇到 PermissionError。正常 Quit 后仍需强制结束两个本批后代。业务结果保留失败/未确认,未重放。自有写进程退出后租用成功归还。本次没有修改 `execution_proof.py`,也没有放宽保存证明;数据库锁的根因仍未解决。最终两批单目标成功不能证明该失败已修复。
|
||||
|
||||
已有随机 UUID 桌面不会自动消失;存量仍依赖外部持有方正常关句柄或 Windows 登录会话结束。测试 helper 模拟的是外部 OpenDesktop 引用,真实 OCular 环境的长期行为仍需用户故障机验收。稳定名称只限制本产品新增受管理对象,不保证消除其他软件内部资源泄漏。
|
||||
|
||||
本次未构建安装包或发布新版本,没有操作原工程、中心库、系统配置或外部 Agent。
|
||||
@@ -0,0 +1,86 @@
|
||||
# 保存后数据库占用:修复与验收
|
||||
|
||||
日期:2026-11-18。分支:`codex/issue29-blocking-cleanup`,基础提交:`cdbc92f`。范围为保存后的源文件绑定、占锁交接、诊断与收尾错误;不涉及独立校验工作台。
|
||||
|
||||
## 问题与处理
|
||||
|
||||
先前 `mentor-live/first-project` 的 RL02、RL05 两处替换及 Save/CloseProject 已返回,随后 `bind_project_commit_source()` 读取 `database/icdb.dat` 时发生 PermissionError。该异常被归为通用错误,外层资源释放错误又掩盖了原始位置。历史日志缺少底层 Windows 代码和持锁者证据,仍不能认定当时的具体持锁进程。
|
||||
|
||||
现在先验证 CAD 身份及工程确实关闭,固定保存 marker、原始尝试和目标证据到 `project-save-pending.json`,再进行源文件绑定。只有绑定成功才发布完整 `project-checkpoint.json` 并登记工程检查点。等待期间显示“工程已保存,等待数据库释放后核验”;不是已经核验成功。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[替换、保存及关闭工程] --> B[验证空工程与本批身份]
|
||||
B --> C[保留保存 marker 和原始尝试证据]
|
||||
C --> D[读取保存源文件]
|
||||
D --> E{读取结果}
|
||||
E -- 正常 --> F[绑定稳定源文件及命令摘要]
|
||||
E -- Windows 32 / 33 --> G[记录错误及持锁观察,最多等待 3 秒]
|
||||
E -- 权限、缺失等 --> P[已保存但待核验,保留首发原因]
|
||||
G --> D
|
||||
G -- 仍占用 --> H[再检查暂停意图,正常退出本批 Mentor 一次]
|
||||
H -- 正常退出 --> I[再次读取,最多等待 3 秒]
|
||||
H -- 强杀或收尾未确认 --> P
|
||||
I -- 仍占用或暂停 --> P
|
||||
I -- 可稳定读取 --> F
|
||||
F --> J[从保存副本导出原生数据并核验目标]
|
||||
J --> K[提交真实回执,任务完成]
|
||||
```
|
||||
|
||||
Windows 读文件使用 CreateFile 保留原生错误码,使用与后续 CRT 校验一致的 READ/WRITE 共享模式。仅 32(共享冲突)、33(锁冲突)进入等待;5(权限不足)、2(文件缺失)等直接形成明确的 `saved_source_pending` 错误。源文件读前后的工程摘要必须一致,改变或不可确认时保留待核验。
|
||||
|
||||
退出回调只存在于受管理 runtime 的已关闭工程、已验证保存 marker 分支。正常可读时继续复用 CAD;异常才走现有身份绑定的正常 Quit。该路径不派发替换,也不重新打开原工程。退出失败或需要强杀时,不绑定成功证明;资源释放和业务待核验分别保留。暂停会阻止进入退出回调,并在诊断、读取恢复后重新检查。已请求普通暂停但未遇锁的安全保存收尾仍沿用原流程。
|
||||
|
||||
首次锁冲突使用 Windows Restart Manager 只读查询候选进程,记录 PID、创建时间和名称。结果是该接口返回的观察,不是穷举全部内核句柄;空结果或查询失败均不授权结束任何外部进程。查询后重读,防止用已经过时的锁状态触发 CAD 回收。
|
||||
|
||||
`saved-source-status.json` 保留同一 session token、同一工程的首发错误、持锁观察及回收事实,后续 SQL 回执再次绑定时不会覆盖它;新 token 重置诊断。该文件不是成功证明,实际源文件始终重新验证。`operation-events.jsonl` 保留绑定及 Mentor 退出调用的先后顺序。多层资源异常仍显示最初的保存读取错误;诊断写入失败通过附注保留,不能替换首发错误。
|
||||
|
||||
## 验收结果
|
||||
|
||||
证据根目录:`D:/CodexValidation/saved-source-20261118/`。测试 XML 在仓库 `build/managed-desktop/`,集合有交集,不累加为独立用例或业务样本数。
|
||||
|
||||
| 验证 | 结果 | 证据 |
|
||||
| --- | --- | --- |
|
||||
| 保存、runtime、回执、恢复与反馈相关回归 | 124 passed,108.18 秒 | `saved-source-regression-final.xml` |
|
||||
| 源文件交接及资源状态增量回归 | 64 passed,26.56 秒 | `saved-source-delta-final.xml` |
|
||||
| 最终专项,Windows 默认 GBK 环境 | 21 passed,5.52 秒 | `saved-source-handoff-final.xml` |
|
||||
| 最终原生文件锁探针 | 20 个样本通过,28.320 秒 | `native-final-02/report.json` |
|
||||
| 原失败双目标工程,正常运行 | 2 saved / 0 failed,原生回读通过 | `mentor-dual/report.json` |
|
||||
| 双目标工程,保存后注入真实锁 | 2 saved / 0 failed,正常退出后核验通过 | `mentor-dual-fault/report.json`、`fault-result.json` |
|
||||
|
||||
新逻辑均先观察失败再修复;覆盖权限/缺失不重试、短暂锁、持续外部锁、源文件变化、暂停在诊断中到达和在退出提示后到达、强杀/退出未确认不发布证明、日志写失败、嵌套异常、诊断再绑定历史、下一工程新代次和可选进度损坏。独立只读审查提出的具体问题已修复,最终复查无剩余可操作发现。
|
||||
|
||||
### 20 个原生文件锁样本
|
||||
|
||||
脚本:`tools/probe_saved_source_handoff.py`。使用合成工程与文件,不启动 CAD。五种场景各 4 次:短暂独占锁、退出持有者后读取、持续外部锁、等待中暂停、文件缺失。真实 CreateFile 独占锁返回 32;Restart Manager 查到已知 fixture 进程。仅前两类最终绑定成功,其余保留 pending,退出回调次数受限,均不执行替换。真正 ACL 拒绝不通过修改用户 ACL 制造,错误 5 由专项测试模拟。
|
||||
|
||||
探针中的退出回调只释放其自建 fixture 进程,不能当成真实 Mentor 回收测试。首轮 `native-01` 曾发现诊断耗时导致使用旧冲突触发退出,报告保留;补重读及回归后通过,最终又以默认编码环境运行。20 个文件锁样本不是 20 个 BOM 业务样本。
|
||||
|
||||
### 两次真实 Mentor 双目标验收
|
||||
|
||||
两个独立的新副本均替换 RL02、RL05:`4100-CA3300-T7` → `4100-CA3300-2200`。从保存后的原生数据独立读取两个目标及其余未选位号;每次仅一个 VBS worker、一次 OpenProject、一次保存,没有重放。
|
||||
|
||||
| 指标 | 正常双目标 | 注入锁双目标 |
|
||||
| --- | --- | --- |
|
||||
| 队列耗时 | 257.842 秒 | 271.420 秒 |
|
||||
| saved / failed | 2 / 0 | 2 / 0 |
|
||||
| 任务 / 操作状态 | completed / verified | completed / verified |
|
||||
| CAD 代次 | 1 | 1 |
|
||||
| 正常退出 / 强杀 | 正常 / 无 | 正常 / 无 |
|
||||
| CAD 退出耗时 | 16.625 秒 | 5.688 秒 |
|
||||
|
||||
正常运行没有复现历史锁。故障运行由 `build/managed-desktop/live_saved_source_fault.py` 在已保存关闭后,用一个自建独立进程持有副本数据库的只读独占句柄;产品调用正常退出 Mentor,驱动确认退出回调返回后才释放 fixture,然后由产品继续核验。`fault-result.json` 证明注入成功、退出回调确实调用且运行退出码为 0。外部 fixture 并非被产品杀死,不能据此声称 Mentor 的 Quit 可以关闭任意外部文件句柄。
|
||||
|
||||
故障时间线(本地时间):18:16:39.722 开始绑定,18:16:43.192 开始正常退出 Mentor,18:16:48.910 退出返回,18:16:49.806 绑定完成。两次总耗时差异包含 CAD 运行波动,不能直接视为等待逻辑的固定开销。
|
||||
|
||||
任务 ID:正常 `d9a1c9af-7a39-4659-a445-a682c8c41a91`;故障 `be954b05-2b8c-4990-a74f-e18655ca93af`。两次任务数据库均 completed、两个操作均 verified;全部已记录写进程身份退出,受管理桌面消失。原始备份 34 文件 SHA-256 全部未变化。
|
||||
|
||||
真实故障验收后发现并修正同 token 诊断被第二次绑定覆盖的问题;暂停、嵌套报错、编码及诊断保留的最终增量由专项和资源回归验证。真实工程写入场景未为这些诊断增量重复执行。
|
||||
|
||||
## 边界
|
||||
|
||||
历史失败的具体持锁者仍不可从旧日志还原,本次修复并验证的是这类冲突的交接处理。持续的外部锁、真正权限不足、文件变化及强杀后不确定写入继续保留待核验,不能算执行成功,也不会自动重替换。
|
||||
|
||||
如果在完成源文件绑定之前退出程序,只保留保存 marker 和原始尝试,现有恢复逻辑不会把未绑定的 `project-save-pending.json` 当成成功证明。该状态仍需核对原结果,不提供自动重放或自动成功恢复。
|
||||
|
||||
未构建安装包、推送或发布版本;未修改原工程、中心库、系统配置和外部 Agent。
|
||||
@@ -0,0 +1,71 @@
|
||||
{
|
||||
"verification_method": "Full main/UI run plus focused reruns for amended behavior; not a second whole-suite run.",
|
||||
"unique_verified_cases": 2438,
|
||||
"unresolved_failures": 0,
|
||||
"junit_sha256": {
|
||||
"final-regression.xml": "3e06b8c07bb057a039d67ce62eb2dcc133a52c0a24af1158c0f32cb8658dc277",
|
||||
"root-delta.xml": "9124ddb63d063f4c07319e83541d4b8da2dfe0376d03ba4307568e8631d98f00",
|
||||
"lifecycle-delta.xml": "1a753e128f82aad2c93ac6a56ff02201141bb7a2df178066d4376d8252a4cdb5",
|
||||
"project-recovery-delta.xml": "a6834285d840263bcddd908d86594c9f742f696490a52bb2b2b382e4c7ffecaf",
|
||||
"identity-final-delta.xml": "7990ce9459db3dec29c3ddbec3b6080569f5c527f3240bc8d896bb659e34329c",
|
||||
"lifecycle-deleted-delta.xml": "fbdc727e207b747f5d4f95db437e04f573572fe74828d9d85dc9f3c6ad1fc445"
|
||||
},
|
||||
"real_mentor": {
|
||||
"verified": true,
|
||||
"real_mentor": true,
|
||||
"targets": [
|
||||
"RL02",
|
||||
"RL05"
|
||||
],
|
||||
"native_inventory_readback_targets": 13,
|
||||
"source_unchanged": true,
|
||||
"source_files": 34,
|
||||
"save_calls": 1,
|
||||
"project_open_calls": 2,
|
||||
"worker_starts": 1,
|
||||
"saved_items": 2,
|
||||
"operations": [
|
||||
[
|
||||
"verified",
|
||||
1
|
||||
],
|
||||
[
|
||||
"verified",
|
||||
1
|
||||
]
|
||||
],
|
||||
"project_commits": [
|
||||
[
|
||||
"verified"
|
||||
]
|
||||
],
|
||||
"queue_seconds": 1450.341202999989,
|
||||
"independent_readback_seconds": 1.5047315999981947,
|
||||
"resource_state": "released",
|
||||
"forced_termination": false,
|
||||
"limitation": "Two replacement items on one copied project; concurrent tests/build affected timing; no speedup claim."
|
||||
},
|
||||
"review_reports": [
|
||||
"identity-review.md",
|
||||
"progress-review.md",
|
||||
"lifecycle-review.md",
|
||||
"final-review.md"
|
||||
],
|
||||
"candidate_published": false,
|
||||
"packaged_source_verification": {
|
||||
"passed": true,
|
||||
"compiled_application_modules": 162,
|
||||
"mismatches": [],
|
||||
"executable_sha256": "4b19948b6a008912851f3ab674afa361bf4a28de4b74f20d3d04b80719200cc1",
|
||||
"full_harness_present": true
|
||||
},
|
||||
"packaged_runtime_verification": {
|
||||
"passed": true,
|
||||
"frozen_supervisor": true,
|
||||
"win32timezone_archived": true,
|
||||
"process_creation_time_read": true,
|
||||
"resources_released": true,
|
||||
"executable": "D:\\CodexValidation\\batch-closure-20261117\\candidate\\dist\\DxPartReplace\\DxPartReplace.exe",
|
||||
"data_root": "D:\\CodexValidation\\batch-closure-20261117\\candidate\\runtime-smoke"
|
||||
}
|
||||
}
|
||||
@@ -249,6 +249,7 @@ Function QPNativeCollectMode(ByRef SelectedUids, ByVal Token, ByVal Project, ByV
|
||||
QPNativeRequireContext Project, Sheet, Guid
|
||||
QPInvalidatePinIndex
|
||||
Set Components = QP_App.ActiveView.Query(VDM_COMP, VD_ALL)
|
||||
QPBindSheetReferences Components
|
||||
Set Seen = CreateObject("Scripting.Dictionary")
|
||||
Set Found = CreateObject("Scripting.Dictionary")
|
||||
Text = "NR1" & vbTab & Token & vbTab & QPEncode(Project) & vbTab & QPEncode(Sheet) & vbCrLf
|
||||
@@ -306,6 +307,13 @@ Function QPBuildNativeReadScript(ByVal Source, ByRef SelectedUids, ByVal Token,
|
||||
For Each Uid In SelectedUids.Keys
|
||||
Body = Body & "QP_NR_Selected.Add " & QPNativeExpression(Uid) & ",True" & vbCrLf
|
||||
Next
|
||||
If IsObject(QP_ReferenceAliases) Then
|
||||
Body=Body & "Set QP_ReferenceAliases=CreateObject(""Scripting.Dictionary"")" & vbCrLf
|
||||
For Each Uid In QP_ReferenceAliases.Keys
|
||||
Body=Body & "QP_ReferenceAliases.Add " & QPNativeExpression(Uid) & "," & _
|
||||
QPNativeExpression(QP_ReferenceAliases(Uid)) & vbCrLf
|
||||
Next
|
||||
End If
|
||||
Body = Body & "On Error Resume Next" & vbCrLf & _
|
||||
"Set QP_App=GetObject(,""Viewdraw.Application"")" & vbCrLf & _
|
||||
"If Err.Number=0 Then QP_NR_Text=QPNativeCollectMode(QP_NR_Selected," & _
|
||||
@@ -315,7 +323,7 @@ Function QPBuildNativeReadScript(ByVal Source, ByRef SelectedUids, ByVal Token,
|
||||
"If QP_NR_Code<>0 Then QP_NR_Text=""ERROR"" & vbTab & " & QPNativeExpression(Token) & _
|
||||
" & vbTab & CStr(QP_NR_Code) & vbTab & QPEncode(QP_NR_Source) & vbTab & QPEncode(QP_NR_Description) & vbCrLf" & vbCrLf & _
|
||||
"QPWriteUtf8 " & QPNativeExpression(ResultPath) & ",QP_NR_Text" & vbCrLf & "On Error GoTo 0" & vbCrLf
|
||||
Names = Split("QPNativeCollectMode QPNativeRequireContext QPNativeReadInvalid QPSnapshotComponent QPUntouchedComponent QPUntouchedFullState QPUntouchedIdentity QPComponentPins QPComponentIdentityKey QPInvalidatePinIndex QPConnectionSignatureFromPins QPAddSortedSignature QPSnapshotAttribute QPReadOptionalAttributeValue QPOptionalAttributeField QPSortSnapshotLines QPLog QPAppendAscii QPEncode QPDecode QPProjectPath QPSheetIdentity QPCanonicalPath QPEnsureProjectAccess QPReadUtf8 QPWriteUtf8", " ")
|
||||
Names = Split("QPNativeCollectMode QPNativeRequireContext QPNativeReadInvalid QPSnapshotComponent QPUntouchedComponent QPUntouchedFullState QPUntouchedIdentity QPComponentPins QPComponentIdentityKey QPComponentUnitKey QPBindSheetReferences QPInvalidatePinIndex QPConnectionSignatureFromPins QPAddSortedSignature QPSnapshotAttribute QPReadOptionalAttributeValue QPOptionalAttributeField QPSortSnapshotLines QPLog QPAppendAscii QPEncode QPDecode QPProjectPath QPSheetIdentity QPCanonicalPath QPEnsureProjectAccess QPReadUtf8 QPWriteUtf8", " ")
|
||||
Pattern.Global = False
|
||||
For Each Name In Names
|
||||
Pattern.Pattern = "^(Function|Sub) " & Name & "\([^\n]*\n[\s\S]*?^End \1[ \t]*"
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -286,7 +286,8 @@ def _execute_incident_retry(service, root, task_id, run_id, *, expected_item, st
|
||||
raise TaskCenterError('run_scope', '原队列检查点身份或范围已变化;保留本次结果。')
|
||||
checkpoint['entries'][task_id].update(state=state, message=message)
|
||||
if path.parent.name == 'workspace-queues':
|
||||
checkpoint['entries'][task_id].update(released=not bool(service.pending_batch_resources()), problems_done=True)
|
||||
from ..trial_task_center.batch_resources import resource_blockers
|
||||
checkpoint['entries'][task_id].update(released=not bool(resource_blockers(service.pending_batch_resources())), problems_done=True)
|
||||
if reports:
|
||||
checkpoint['entries'][task_id]['netlist_verification'] = reports[task_id]
|
||||
elif reports:
|
||||
|
||||
@@ -316,7 +316,8 @@ class IncidentRunner:
|
||||
remaining = tuple(self.service.pending_batch_resources())
|
||||
cleanup['remaining_resources'] = _bounded_details(remaining)
|
||||
cleanup['remaining_resources_observed'] = True
|
||||
released = not remaining
|
||||
from ..trial_task_center.batch_resources import resource_blockers
|
||||
released = not resource_blockers(remaining)
|
||||
except Exception as error:
|
||||
cleanup['verification_error'] = scrub({'type': type(error).__name__, 'detail': str(error)[:1000]})
|
||||
cleanup['remaining_resources'], released = None, None
|
||||
|
||||
@@ -402,7 +402,8 @@ class RunJournal:
|
||||
blockers.append("preflight_blocked")
|
||||
except Exception:
|
||||
blockers.append("preflight_unavailable")
|
||||
if service.pending_batch_resources():
|
||||
from ..trial_task_center.batch_resources import resource_blockers
|
||||
if resource_blockers(service.pending_batch_resources()):
|
||||
blockers.append("resources_pending")
|
||||
blockers = list(dict.fromkeys(blockers))
|
||||
value = {**row, "queue_available": queue is not None, "tasks": tasks, "counters": counters,
|
||||
|
||||
@@ -246,7 +246,8 @@ class ScheduleStore:
|
||||
with self.execution_lease(schedule_id):
|
||||
assert_task_queue_idle(self.data_root)
|
||||
with _queue_lease(self.data_root):
|
||||
if service.pending_batch_resources():
|
||||
from ..trial_task_center.batch_resources import resource_blockers
|
||||
if resource_blockers(service.pending_batch_resources()):
|
||||
raise TaskCenterError('schedule_busy', '后台资源尚未释放,请先在任务中心核验和清理。')
|
||||
with self._connect() as db:
|
||||
db.execute('BEGIN IMMEDIATE')
|
||||
|
||||
@@ -50,7 +50,8 @@ def preflight(service, task_ids, *, allow_completed=False):
|
||||
from .impact import _scope
|
||||
task_ids = _scope(task_ids)
|
||||
tasks, paths, hashes = [], set(), {}
|
||||
pending = bool(service.pending_batch_resources())
|
||||
from ..trial_task_center.batch_resources import resource_blockers
|
||||
pending = bool(resource_blockers(service.pending_batch_resources()))
|
||||
for task_id in task_ids:
|
||||
context = _context(service, task_id)
|
||||
state = execution_state(context)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Serial, confirmed queue execution with safe between-item pausing."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, replace
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
@@ -121,6 +121,9 @@ def execute_batch(
|
||||
**({'runtime': runtime} if runtime is not None else {})) as project_session:
|
||||
while not progress.queue_complete:
|
||||
if stop_event.is_set():
|
||||
if project_session.enabled and project_session.worker is not None:
|
||||
progress = preparer.complete_project_session(task_id, project_session, session, projects)
|
||||
progress = service._synchronize_quick_run_progress(task_id, progress=progress)
|
||||
backups.commit_prefetched(progress)
|
||||
return BatchRunResult(
|
||||
progress, True, last_result, tuple(item_messages), tuple(backups.unbacked_project_paths)
|
||||
@@ -133,6 +136,9 @@ def execute_batch(
|
||||
backups.prepare(current, progress)
|
||||
service._batch_feedback = (task_id, progress, f'正在处理 {current.name}')
|
||||
if stop_event.is_set():
|
||||
if project_session.enabled and project_session.worker is not None:
|
||||
progress = preparer.complete_project_session(task_id, project_session, session, projects)
|
||||
progress = service._synchronize_quick_run_progress(task_id, progress=progress)
|
||||
backups.commit_prefetched(progress)
|
||||
return BatchRunResult(
|
||||
progress, True, last_result, tuple(item_messages), tuple(backups.unbacked_project_paths)
|
||||
@@ -194,10 +200,16 @@ def execute_batch(
|
||||
item_messages.append(last_result.message)
|
||||
if last_result.progress is None:
|
||||
raise TaskCenterError('quick_run_queue_progress', '批量执行器未返回增量进度,队列已停止。')
|
||||
failed_component = (task.primary_project_id and last_result.state in {BATCH_FAILED_STATE, 'batch_skipped'}
|
||||
and session.journal_rows.get(metric['job_id'], {}).get('execution_unit') == 'component')
|
||||
candidate = last_result.progress
|
||||
if failed_component and project_session.worker is not None:
|
||||
# Verify the retained prefix before admitting another component.
|
||||
candidate = preparer.complete_project_session(task_id, project_session, session, projects)
|
||||
last_result = replace(last_result, progress=candidate)
|
||||
project_completed = candidate.completed_project_paths != progress.completed_project_paths
|
||||
synchronize = (
|
||||
progress.completed_items == 0
|
||||
(progress.completed_items == 0 and last_result.state != 'pending_save')
|
||||
or project_completed
|
||||
or candidate.queue_complete
|
||||
or last_result.state in {BATCH_FAILED_STATE, BATCH_SAVED_UNVERIFIED_STATE}
|
||||
@@ -208,7 +220,8 @@ def execute_batch(
|
||||
persistence['completed_units'] = 1
|
||||
else:
|
||||
updated = candidate
|
||||
if updated.completed_items <= progress.completed_items or updated.total_items != progress.total_items:
|
||||
if ((updated.completed_items <= progress.completed_items and last_result.state != 'pending_save')
|
||||
or updated.total_items != progress.total_items):
|
||||
_write_uncertain_flag(preparer.data_root / 'quick-run/uncertain.flag', 'batch_progress_mismatch')
|
||||
raise TaskCenterError('quick_run_queue_progress', '执行器返回后队列进度异常,已停止,避免重复执行。')
|
||||
# Keep every verified step: one project can contain many sheets and rules.
|
||||
@@ -233,12 +246,7 @@ def execute_batch(
|
||||
progress = updated
|
||||
if task.primary_project_id and last_result.state in {BATCH_FAILED_STATE, 'batch_skipped'}:
|
||||
row = session.journal_rows.get(metric['job_id'], {})
|
||||
if row.get('execution_unit') == 'component':
|
||||
# A failed component is terminal; rebuild its session before
|
||||
# admitting another UID against the last verified baseline.
|
||||
if project_session.worker is not None:
|
||||
project_session.worker.close()
|
||||
project_session.worker = None
|
||||
if failed_component:
|
||||
from .quick_run import _run_process
|
||||
if journal is None:
|
||||
raise TaskCenterError('quick_run_uncertain', '缺少失败回执及工程健康证明,暂缓剩余位号。')
|
||||
@@ -266,7 +274,7 @@ def execute_batch(
|
||||
feedback = '未命中已跳过,继续下一项' if last_result.state == 'no_match' else (
|
||||
'失败已记录,继续下一项'
|
||||
if last_result.state == BATCH_FAILED_STATE
|
||||
else '已保存并校验,继续下一项'
|
||||
else '替换已暂存,等待工程统一保存回读' if last_result.state == 'pending_save' else '已保存并校验,继续下一项'
|
||||
)
|
||||
service._batch_feedback = (task_id, progress, feedback)
|
||||
return BatchRunResult(
|
||||
|
||||
@@ -31,7 +31,7 @@ def write_release_event(root, path, record, phase, *, context=None, reason='batc
|
||||
'generation': record.get('generation'), 'desktop': record.get('desktop'),
|
||||
'cad_pid': record.get('cad_pid'), 'cad_guid': record.get('cad_guid'),
|
||||
'reason': reason, 'result': 'releasing' if phase == 'started' else (
|
||||
'released' if phase == 'succeeded' else 'blocked'),
|
||||
'released' if phase == 'succeeded' else 'isolated' if phase == 'isolated' else 'blocked'),
|
||||
'elapsed_seconds': elapsed, 'record_path': str(path),
|
||||
'desktop_handle_closed': cleanup.get('desktop_handle_closed'),
|
||||
'desktop_disappeared': disappeared, 'cad_exit_code': cad.get('cad_exit_code'),
|
||||
@@ -39,7 +39,8 @@ def write_release_event(root, path, record, phase, *, context=None, reason='batc
|
||||
'cad_shutdown': cad, 'shutdown_warning': record.get('shutdown_warning'),
|
||||
'retries': record.get('retries', {}), 'cleanup': cleanup, 'error': str(error)}
|
||||
title = {'started': '开始释放后台资源', 'succeeded': '后台资源释放成功',
|
||||
'failed': '后台资源释放失败,资源释放待处理'}[phase]
|
||||
'failed': '后台资源释放失败,资源释放待处理',
|
||||
'isolated': '后台写入进程已退出,残留桌面已隔离(未确认释放)'}[phase]
|
||||
purpose = {'batch_end': '批次结束', 'rebuild': '安全重建', 'manual_retry': '重新清理',
|
||||
'restart_recheck': '重启后重新检查'}.get(reason, reason)
|
||||
cad_text = f"CAD PID={details['cad_pid']}" if details['cad_pid'] else (
|
||||
@@ -56,7 +57,7 @@ def write_release_event(root, path, record, phase, *, context=None, reason='batc
|
||||
if error:
|
||||
message += ' · 原因:' + str(error)
|
||||
message = message.replace('\r', '\\r').replace('\n', '\\n')
|
||||
level = 'ERROR' if phase == 'failed' else 'INFO'
|
||||
level = 'ERROR' if phase == 'failed' else 'WARNING' if phase == 'isolated' else 'INFO'
|
||||
timestamp = datetime.now(timezone.utc).isoformat(timespec='milliseconds')
|
||||
errors = []
|
||||
try:
|
||||
|
||||
@@ -19,6 +19,13 @@ LIVE_RESOURCES = {}
|
||||
|
||||
class BatchResourceCleanupError(TaskCenterError):
|
||||
def __init__(self, message, *, completed_result=None, primary_error=None, cleanup_error=None):
|
||||
original, seen = primary_error, set()
|
||||
while original is not None and id(original) not in seen:
|
||||
if getattr(original, 'code', None) in {'saved_source_pending', 'saved_source_changed'}:
|
||||
message = str(original) + ';资源收尾异常:' + message
|
||||
break
|
||||
seen.add(id(original))
|
||||
original = getattr(original, 'primary_error', None)
|
||||
super().__init__('batch_resources_pending', message)
|
||||
self.completed_result = completed_result
|
||||
self.primary_error = primary_error
|
||||
@@ -51,7 +58,7 @@ def _valid_record(record, path):
|
||||
and type(record.get('generation')) is int and record['generation'] >= 1
|
||||
and isinstance(record.get('desktop'), str)
|
||||
and re.fullmatch('dx-[0-9a-f]{32}', record['desktop']) is not None
|
||||
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released'}
|
||||
and record.get('state') in {'starting', 'ready', 'recycling', 'closing', 'blocked', 'released', 'isolated'}
|
||||
and isinstance(processes, list) and all(isinstance(item, dict)
|
||||
and type(item.get('pid')) is int and item['pid'] > 0
|
||||
and 'creation_time' in item and (item['creation_time'] is None
|
||||
@@ -66,14 +73,17 @@ def _valid_record(record, path):
|
||||
|
||||
def pending_resources(root):
|
||||
pending = []
|
||||
for path in sorted(internal_path(canonical_path(root) / 'batch-resources').glob('*/session.json')):
|
||||
directory = internal_path(canonical_path(root) / 'batch-resources')
|
||||
paths = (*directory.glob('*/session.json'), *directory.glob('*/isolated-generation-*.json'))
|
||||
for path in sorted(paths):
|
||||
path = canonical_path(path)
|
||||
try:
|
||||
record = json.loads(internal_path(path).read_text(encoding='utf-8'))
|
||||
if not _valid_record(record, path):
|
||||
raise ValueError('invalid resource ownership record')
|
||||
if record.get('state') != 'released' or not _released_evidence(record):
|
||||
pending.append(dict(record, record_path=str(path)))
|
||||
pending.append(dict(record, record_path=str(path),
|
||||
blocking=not (record.get('state') == 'isolated' and _isolation_evidence(record))))
|
||||
except (OSError, ValueError, TypeError) as error:
|
||||
pending.append({'state': 'blocked', 'record_path': str(path), 'error': str(error)})
|
||||
import sys
|
||||
@@ -109,9 +119,76 @@ def pending_resources(root):
|
||||
return tuple(pending)
|
||||
|
||||
|
||||
def resource_blockers(records):
|
||||
"""Only a validated isolated desktop warning may leave the resource gate open."""
|
||||
return tuple(item for item in records
|
||||
if not (item.get('state') == 'isolated' and item.get('blocking') is False))
|
||||
|
||||
|
||||
def blocking_resources(root):
|
||||
return resource_blockers(pending_resources(root))
|
||||
|
||||
|
||||
def writer_identities(record):
|
||||
identities = []
|
||||
for item in [*record.get('writer_identities', ()), *record.get('processes', ()),
|
||||
*([record['cad_process']] if record.get('cad_process') else [])]:
|
||||
if item not in identities:
|
||||
identities.append(item)
|
||||
if record.get('cad_pid') is not None and not any(
|
||||
item.get('pid') == record['cad_pid'] for item in identities):
|
||||
raise ValueError('CAD process identity is missing')
|
||||
if any(type(item.get('pid')) is not int or item['pid'] <= 0
|
||||
or not isinstance(item.get('creation_time'), str) or not item['creation_time']
|
||||
for item in identities):
|
||||
raise ValueError('Exact writer process identity is missing')
|
||||
return identities
|
||||
|
||||
|
||||
def isolation_proof(record):
|
||||
"""Certify handle closure and exact writer exit, never a desktop disappearance."""
|
||||
cleanup = record.get('cleanup') or {}
|
||||
if cleanup.get('desktop_lease_pending') is True:
|
||||
raise ValueError('Managed desktop lease is still held')
|
||||
if cleanup.get('desktop_lease'):
|
||||
from .desktop_slot import returned_lease_intact
|
||||
if not returned_lease_intact(cleanup['desktop_lease'], record['desktop']):
|
||||
raise ValueError('Managed desktop return proof changed')
|
||||
if (cleanup.get('desktop_handle_closed') is not True
|
||||
or cleanup.get('owned_cleanup_complete') is not True):
|
||||
raise ValueError('Owned resource cleanup is not proven')
|
||||
identities = writer_identities(record)
|
||||
if not all(_process_gone(item) for item in identities):
|
||||
raise ValueError('Owned writer process is still active')
|
||||
return {'session_id': record['session_id'], 'generation': record['generation'],
|
||||
'desktop': record['desktop'], 'desktop_handle_closed': True, 'desktop_released': False,
|
||||
'processes_exited': True, 'processes': identities, 'observed_at': time.time()}
|
||||
|
||||
|
||||
def _isolation_evidence(record):
|
||||
try:
|
||||
proof = record.get('isolation_proof', {})
|
||||
current = isolation_proof(record)
|
||||
return (isinstance(proof, dict) and all(proof.get(key) == current[key] for key in
|
||||
('session_id', 'generation', 'desktop', 'desktop_handle_closed', 'desktop_released',
|
||||
'processes_exited', 'processes')))
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _released_evidence(record):
|
||||
"""A state label alone must never reopen the physical resource gate."""
|
||||
proof = record.get('release_proof', {})
|
||||
cleanup = record.get('cleanup') or {}
|
||||
if (proof.get('generation') == record['generation'] and proof.get('desktop_not_acquired') is True
|
||||
and cleanup.get('desktop_not_acquired') is True and cleanup.get('desktop_handle_closed') is True
|
||||
and cleanup.get('owned_cleanup_complete') is True and not cleanup.get('errors')
|
||||
and record.get('cad_pid') is None and not writer_identities(record)):
|
||||
return True
|
||||
if cleanup.get('desktop_lease'):
|
||||
from .desktop_slot import returned_lease_intact
|
||||
if not returned_lease_intact(cleanup['desktop_lease'], record['desktop']):
|
||||
return False
|
||||
if (isinstance(proof, dict) and proof.get('generation') == record['generation']
|
||||
and proof.get('desktop_released') is True and proof.get('processes_exited') is True):
|
||||
return True
|
||||
@@ -126,7 +203,7 @@ def _released_evidence(record):
|
||||
|
||||
|
||||
def guard_pending_resources(root):
|
||||
if pending_resources(root):
|
||||
if blocking_resources(root):
|
||||
raise BatchResourceCleanupError('上一批资源尚未确认释放;请先重试释放资源,再开始新批次。')
|
||||
|
||||
|
||||
@@ -216,7 +293,7 @@ def cleanup_pending_resources(root, *, logging_context=None, retry_failed=False)
|
||||
if not record.get('session_id'):
|
||||
continue # Corrupt evidence cannot authorize resource actions.
|
||||
log('started')
|
||||
identities = list(record.get('processes', ()))
|
||||
identities = writer_identities(record)
|
||||
cad_identity = record.get('cad_process')
|
||||
if record.get('cad_pid') is not None:
|
||||
if (not isinstance(cad_identity, dict) or cad_identity.get('pid') != record['cad_pid']
|
||||
@@ -229,12 +306,36 @@ def cleanup_pending_resources(root, *, logging_context=None, retry_failed=False)
|
||||
log('failed', '记录中的进程仍未确认退出')
|
||||
continue
|
||||
desktop = record.get('desktop')
|
||||
if desktop:
|
||||
_Win32Api().confirm_desktop_released(desktop)
|
||||
if (record.get('cleanup') or {}).get('desktop_lease'):
|
||||
# A return belongs to this past lease, even while the same name
|
||||
# is now serving another batch. Never open or stop its new owner.
|
||||
proof = isolation_proof(record)
|
||||
from .desktop_slot import managed_desktop_disappeared
|
||||
if not managed_desktop_disappeared(record['cleanup']['desktop_lease']):
|
||||
record.pop('record_path', None)
|
||||
record.update(state='isolated', blocking=False, isolation_proof=proof)
|
||||
write_record(path, record)
|
||||
log('isolated')
|
||||
continue
|
||||
elif desktop:
|
||||
try:
|
||||
_Win32Api().confirm_desktop_released(desktop)
|
||||
except Exception as error:
|
||||
# A residual desktop name is only a warning after owned
|
||||
# handle closure and independently revalidated writer exit.
|
||||
proof = isolation_proof(record)
|
||||
record.pop('record_path', None)
|
||||
record.update(state='isolated', blocking=False, isolation_proof=proof,
|
||||
isolation_warning=str(error))
|
||||
write_record(path, record)
|
||||
log('isolated', str(error))
|
||||
continue
|
||||
# After restart only confirm disappearance. Never attach to COM,
|
||||
# close a project, or terminate a PID whose handles we no longer own.
|
||||
record.pop('record_path', None)
|
||||
record = dict(record, state='released', reconciled=True, released_at=time.time(),
|
||||
record = dict(record, state='released', blocking=False, reconciled=True, released_at=time.time(),
|
||||
release_results={'mentor_released': True, 'desktop_released': True},
|
||||
release_state={'mentor': 'released', 'desktop': 'released'}, wait_reason='',
|
||||
release_proof={'generation': record['generation'], 'desktop_released': True,
|
||||
'processes_exited': True, 'processes': identities, 'method': 'restart_observation'})
|
||||
write_record(path, record)
|
||||
|
||||
@@ -11,7 +11,8 @@ from threading import Lock
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from .batch_resources import (BatchResourceCleanupError, LIVE_RESOURCES, cleanup_pending_resources,
|
||||
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record)
|
||||
guard_pending_resources, pending_resources, process_identity, project_checkpoint_released, write_record,
|
||||
isolation_proof)
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path, legacy_path
|
||||
from .isolated_desktop import PrivateDesktop, ProcessNotStartedError
|
||||
@@ -192,7 +193,9 @@ class BatchCadRuntime:
|
||||
|
||||
def _save(self, **values):
|
||||
record = dict(self.record, **values)
|
||||
write_record(self.path, record)
|
||||
# Prelaunch retries can change runtime state before it owns resources.
|
||||
if record['generation']:
|
||||
write_record(self.path, record)
|
||||
self.record = record
|
||||
|
||||
def _timing(self, phase, started, **details):
|
||||
@@ -202,21 +205,39 @@ class BatchCadRuntime:
|
||||
def _ensure_desktop(self, abort_event=None):
|
||||
if self.record['state'] == 'blocked':
|
||||
raise BatchResourceCleanupError('当前批次资源已阻断,不能启动新的工作进程。')
|
||||
if self.record['state'] == 'isolated':
|
||||
if self.desktop is not None:
|
||||
raise BatchResourceCleanupError('隔离桌面已停止使用,请为后续工程创建新的运行会话。')
|
||||
write_record(self.path.parent / f'isolated-generation-{self.generation}.json', self.record)
|
||||
if self.desktop is None:
|
||||
check_cancelled(self.stop_event if abort_event is None else abort_event)
|
||||
self.generation += 1
|
||||
generation = self.generation + 1
|
||||
name = 'dx-' + uuid4().hex
|
||||
self._save(state='starting', generation=self.generation, desktop=name,
|
||||
processes=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
|
||||
desktop = (self.desktop_factory(name, enabled=True, managed=True)
|
||||
if self.desktop_factory is PrivateDesktop else self.desktop_factory(name, enabled=True))
|
||||
# The factory does not acquire a desktop. Commit the starting
|
||||
# record before assigning cleanup ownership or entering it.
|
||||
self._save(state='starting', generation=generation, desktop=desktop.name,
|
||||
processes=[], writer_identities=[], cad_pid=None, cad_guid=None, checkpoints={}, native_operations={},
|
||||
blocking=True, isolation_proof=None,
|
||||
stopping_reason='', wait_reason='', primary_error=None,
|
||||
cad_process=None, cad_token=None, cad_shutdown=None, cad_start_requested=False, cleanup=None,
|
||||
release_state={'mentor': 'not_started', 'desktop': 'pending'})
|
||||
self.generation = generation
|
||||
self.desktop = desktop
|
||||
LIVE_RESOURCES[str(self.path)] = self
|
||||
self.desktop = self.desktop_factory(name, enabled=True)
|
||||
started = time.monotonic()
|
||||
try:
|
||||
self.desktop.__enter__()
|
||||
except Exception:
|
||||
self._save(state='blocked', error='private desktop startup failed')
|
||||
if getattr(self.desktop, '_slot', None) is not None and not self.desktop._slot_acquired:
|
||||
self.desktop.close()
|
||||
self._save(state='released', blocking=False, cleanup=self.desktop.cleanup_record,
|
||||
release_proof=dict(generation=self.generation, desktop_not_acquired=True))
|
||||
self.desktop = None
|
||||
LIVE_RESOURCES.pop(str(self.path), None)
|
||||
else:
|
||||
self._save(state='blocked', error='private desktop startup failed')
|
||||
raise
|
||||
self._timing('desktop_start', started)
|
||||
self._save(state='ready')
|
||||
@@ -226,6 +247,8 @@ class BatchCadRuntime:
|
||||
identity = process_identity(process)
|
||||
if identity not in self.record['processes']:
|
||||
self.record['processes'].append(identity)
|
||||
if identity not in self.record.setdefault('writer_identities', []):
|
||||
self.record['writer_identities'].append(identity)
|
||||
self._save()
|
||||
|
||||
def _release_process(self, process, *, desktop=None, generation=None):
|
||||
@@ -245,6 +268,9 @@ class BatchCadRuntime:
|
||||
cad_process = getattr(self.cad, 'cad', None)
|
||||
if cad_process is not None and cad_process not in processes:
|
||||
processes.append(cad_process)
|
||||
for process in getattr(self.cad, '_retained_processes', ()):
|
||||
if process not in processes:
|
||||
processes.append(process)
|
||||
if not processes:
|
||||
cleanup = getattr(self.desktop, 'cleanup_record', None) or {}
|
||||
# Closing our handle follows owned-process cleanup. An external
|
||||
@@ -401,14 +427,107 @@ class BatchCadRuntime:
|
||||
def _checkpoint(self, root, project, attempt_id):
|
||||
self.cad.checkpoint(root / ('cad-closed-' + attempt_id + '.json'))
|
||||
key = str(root).casefold()
|
||||
self.record['checkpoints'][key] = attempt_id
|
||||
self._save(state='ready')
|
||||
write_record(root / 'project-checkpoint.json', {'version': 1, 'resource_record': str(self.path),
|
||||
checkpoint = {'version': 1, 'resource_record': str(self.path),
|
||||
'session_id': self.session_id, 'generation': self.generation, 'attempt_id': attempt_id,
|
||||
'evidence_root': str(root), 'project_path': str(project), 'project_closed': True,
|
||||
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']})
|
||||
'worker_closed': True, 'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid']}
|
||||
request_path = internal_path(root / 'project-session.request')
|
||||
if request_path.is_file():
|
||||
request = request_path.read_text(encoding='utf-8-sig').strip().split('\t')
|
||||
if (len(request) == 3 and len(request[0]) == 32
|
||||
and all(character in '0123456789abcdef' for character in request[0])
|
||||
and canonical_path(request[1]) == project):
|
||||
marker = internal_path(root / ('project-commit-' + request[0] + '.tsv'))
|
||||
if marker.is_file():
|
||||
from .execution_proof import capture_project_commit, bind_project_commit_source
|
||||
from .quick_run import _decode_vbs_field
|
||||
rows = [line.split('\t') for line in marker.read_text(encoding='utf-8-sig').splitlines() if line]
|
||||
if (not rows or len(rows[0]) != 3 or rows[0][0] != request[0]
|
||||
or rows[0][2] not in {'project_verified', 'project_saved'}
|
||||
or canonical_path(_decode_vbs_field(rows[0][1])) != project
|
||||
or any(len(row) != 2 for row in rows[1:])):
|
||||
raise WorkerMonitorError('native project commit marker is not bound to the closed project')
|
||||
jobs = [row[0] for row in rows[1:]]
|
||||
expected_jobs = [record['job_id'] for _, record in self._current_attempts]
|
||||
expected_jobs = expected_jobs[expected_jobs.index(self.record['active_job']):]
|
||||
if not jobs or len(set(jobs)) != len(jobs) or jobs != expected_jobs[:len(jobs)]:
|
||||
raise WorkerMonitorError('native project commit marker does not cover an authorized prefix')
|
||||
saved_only = rows[0][2] == 'project_saved'
|
||||
allowed_outcomes = {'pending_save', 'no_match'} if saved_only else {'saved_ok', 'no_match'}
|
||||
if any(row[1] not in allowed_outcomes for row in rows[1:]):
|
||||
raise WorkerMonitorError('native project commit marker has an invalid outcome')
|
||||
proof = capture_project_commit(self.root, root, {'session_token': request[0],
|
||||
'project_path': str(project).casefold(), 'jobs_json': json.dumps(jobs)})
|
||||
proof['marker_sha256'] = hashlib.sha256(marker.read_bytes()).hexdigest()
|
||||
proof['files'].append({'path': str(canonical_path(marker)), 'sha256': proof['marker_sha256']})
|
||||
readback_names = []
|
||||
for job, outcome in proof['outcomes'].items():
|
||||
if saved_only:
|
||||
from .batch_attempt import read_attempt
|
||||
path, expected = next((path, expected) for path, expected in self._current_attempts
|
||||
if expected['job_id'] == job)
|
||||
actual = read_attempt(path)
|
||||
if any(actual.get(name) != value for name, value in expected.items() if name != 'state'):
|
||||
raise WorkerMonitorError('saved project attempt identity changed')
|
||||
if outcome == 'no_match' and actual['state'] != 'prepared':
|
||||
raise WorkerMonitorError('saved project no-match outcome has a possible native write')
|
||||
snapshot = internal_path(root / ('snapshot-' + job + '.tsv'))
|
||||
report_path = internal_path(str(snapshot) + '.native.json')
|
||||
if snapshot.exists():
|
||||
if b'X\tsaved_netlist_v1' not in snapshot.read_bytes().splitlines():
|
||||
raise WorkerMonitorError('saved project snapshot does not authorize native export validation')
|
||||
readback_names.append(snapshot.name)
|
||||
if outcome in {'saved_ok', 'pending_save'} or report_path.exists():
|
||||
report = json.loads(report_path.read_text(encoding='utf-8'))
|
||||
native = report.get('batch_attempt', {})
|
||||
if (report.get('ok') is not True
|
||||
or outcome in {'saved_ok', 'pending_save'} and report.get('stage') != 'native_replace_applied'
|
||||
or any(native.get(name) != expected[name] for name in
|
||||
('session_id', 'generation', 'attempt_id', 'task_id', 'job_id'))
|
||||
or canonical_path(native.get('project_path', '')) != project):
|
||||
raise WorkerMonitorError('saved project native write attempt is not bound')
|
||||
readback_names.append(report_path.name)
|
||||
if outcome in {'saved_ok', 'pending_save'}:
|
||||
if not snapshot.is_file() or actual['state'] != 'mutation_possible':
|
||||
raise WorkerMonitorError('saved project applied write lacks its original mutation attempt')
|
||||
readback_names.append('writer-expectations-' + job + '.tsv')
|
||||
proof.setdefault('batch_attempts', []).append({'path': str(path), 'expected': expected})
|
||||
proof['files'].append({'path': str(path), 'sha256': hashlib.sha256(internal_path(path).read_bytes()).hexdigest()})
|
||||
else:
|
||||
readback_names.append('post-state-' + job + '.tsv')
|
||||
if not saved_only and outcome == 'saved_ok':
|
||||
readback_names.extend(('snapshot-' + job + '.tsv', 'snapshot-' + job + '.tsv.native.json',
|
||||
'value-readback-' + job + '.tsv'))
|
||||
proof['readback_files'] = {name: hashlib.sha256(internal_path(root / name).read_bytes()).hexdigest()
|
||||
for name in readback_names}
|
||||
proof['native_attempt'] = {'session_id': self.session_id, 'generation': self.generation,
|
||||
'attempt_id': attempt_id}
|
||||
if saved_only:
|
||||
proof.update(marker_state='project_saved', saved_only=True, validation_required='saved_netlist_v1',
|
||||
native_outcomes=dict(rows[1:]), outcomes=dict(rows[1:]))
|
||||
# Keep the saved marker and attempt evidence before any file-lock wait.
|
||||
write_record(root / 'project-save-pending.json', dict(checkpoint, project_commit_pending=proof))
|
||||
proof = bind_project_commit_source(proof, self.root, root, project,
|
||||
stop_event=self.stop_event, release_mentor=self._release_saved_source_owner if saved_only else None)
|
||||
checkpoint['project_commit_source'] = proof
|
||||
write_record(root / 'project-checkpoint.json', checkpoint)
|
||||
self.record['checkpoints'][key] = attempt_id
|
||||
self._save()
|
||||
self._active_attempts = []
|
||||
|
||||
def _release_saved_source_owner(self, error):
|
||||
"""Only a closed project's read handoff may recycle an idle CAD here."""
|
||||
try:
|
||||
self.close(rebuilding=True, reason='saved_source_locked')
|
||||
except Exception as cleanup:
|
||||
self.record['primary_error'] = {'type': type(error).__name__, 'code': error.code, 'message': str(error)}
|
||||
try:
|
||||
self._save()
|
||||
except Exception as diagnostic:
|
||||
error.add_note('资源收尾记录写入失败:' + str(diagnostic))
|
||||
raise BatchResourceCleanupError(str(cleanup), primary_error=error, cleanup_error=cleanup) from cleanup
|
||||
self.desktop = self.cad = None
|
||||
|
||||
def _certify_unlaunched_worker(self, root, project, command, attempt_id, expected, before, inputs, *, failure=None,
|
||||
worker_creation_rejected=False):
|
||||
"""Certify no worker was created, using exact launch and release provenance."""
|
||||
@@ -514,6 +633,11 @@ class BatchCadRuntime:
|
||||
cooperative_abort=cooperative_abort, process_factory=start_worker,
|
||||
**({'owner_loss_path': self.path.parent / 'owner-loss.json'} if self.read_only else {}),
|
||||
**({'force_control': self.stop_event} if hasattr(self.stop_event, 'force_event') else {}))
|
||||
if result.returncode == 75 and not self.read_only:
|
||||
result.prewrite_pause = self._certify_prewrite_pause(
|
||||
root, project, attempt_id, expected, before, inputs,
|
||||
getattr(result, 'prewrite_pause', {}), abort_event)
|
||||
return result
|
||||
if result.returncode == 74 and not self.read_only:
|
||||
from .batch_attempt import can_retry_attempt
|
||||
pause = getattr(result, 'startup_pause', {})
|
||||
@@ -643,6 +767,92 @@ class BatchCadRuntime:
|
||||
self.close(rebuilding=True) # Prove disappearance before creating another.
|
||||
self.desktop = self.cad = None
|
||||
|
||||
def _certify_prewrite_pause(self, root, project, attempt_id, expected, before, inputs, pause, abort_event):
|
||||
from .batch_attempt import can_retry_attempt, read_attempt
|
||||
from .discovery_cache import project_fingerprint
|
||||
from .project_session_health import _source_parameters
|
||||
from .quick_run import _decode_vbs_field
|
||||
try:
|
||||
request_path = internal_path(root / 'project-session.request')
|
||||
request_bytes = request_path.read_bytes()
|
||||
request = request_bytes.decode('utf-8-sig').strip().split('\t')
|
||||
original = inputs['project-session.request'].decode('utf-8-sig').strip().split('\t')
|
||||
if (len(request) != 3 or len(original) != 3 or request[:2] != original[:2]
|
||||
or pause.get('session_token') != request[0] or len(request[0]) != 32
|
||||
or pause.get('job_id') != request[2] or canonical_path(request[1]) != project
|
||||
or canonical_path(pause.get('project_path', '')) != project
|
||||
or pause.get('worker_tree_exited') is not True
|
||||
or type(pause.get('sequence')) is not int or pause['sequence'] <= 0
|
||||
or not ((abort_event is not None and abort_event.is_set())
|
||||
or (self.stop_event is not None and self.stop_event.is_set()))
|
||||
or getattr(getattr(self.stop_event, 'force_event', None), 'is_set', lambda: False)()
|
||||
or internal_path(root / 'command.tsv').read_bytes() != inputs['command.tsv']):
|
||||
raise ValueError('prewrite pause ownership is not bound')
|
||||
selected = [(path, record) for path, record in expected if record['job_id'] == request[2]]
|
||||
if len(selected) != 1 or not can_retry_attempt(*selected[0]):
|
||||
raise ValueError('current paused job may have mutated')
|
||||
jobs = [record['job_id'] for _, record in expected]
|
||||
prefix_jobs = set(jobs[:jobs.index(request[2])])
|
||||
mutated = set()
|
||||
for path, record in expected:
|
||||
actual = read_attempt(path)
|
||||
if any(actual.get(key) != value for key, value in record.items() if key != 'state'):
|
||||
raise ValueError('batch attempt identity changed')
|
||||
if actual['state'] == 'mutation_possible':
|
||||
mutated.add(record['job_id'])
|
||||
marker_path = root / ('project-pause-' + request[0] + '.tsv')
|
||||
marker = internal_path(marker_path).read_text(encoding='utf-8-sig').strip().split('\t')
|
||||
if (len(marker) != 4 or marker[0] != request[0] or marker[2:] != [request[2], 'prewrite_closed']
|
||||
or canonical_path(_decode_vbs_field(marker[1])) != project):
|
||||
raise ValueError('prewrite project closure marker is missing or stale')
|
||||
manifest_path = root / ('project-commit-' + request[0] + '.tsv')
|
||||
committed = {}
|
||||
prefix_marker_state = None
|
||||
files = [marker_path, *(path for path, _ in expected)]
|
||||
if internal_path(manifest_path).exists():
|
||||
rows = [line.split('\t') for line in internal_path(manifest_path).read_text(encoding='utf-8-sig').splitlines()]
|
||||
if (not rows or len(rows[0]) != 3 or rows[0][0] != request[0]
|
||||
or canonical_path(_decode_vbs_field(rows[0][1])) != project
|
||||
or rows[0][2] not in {'project_verified', 'project_saved'}):
|
||||
raise ValueError('committed prefix manifest is not bound')
|
||||
prefix_marker_state = rows[0][2]
|
||||
allowed_outcomes = {'pending_save', 'no_match'} if prefix_marker_state == 'project_saved' else {'saved_ok', 'no_match'}
|
||||
for row in rows[1:]:
|
||||
if (len(row) != 2 or row[0] not in prefix_jobs or row[0] in committed
|
||||
or row[1] not in allowed_outcomes):
|
||||
raise ValueError('committed prefix manifest contains an unrelated result')
|
||||
committed[row[0]] = row[1]
|
||||
files.append(manifest_path)
|
||||
after = project_fingerprint(project, self.root)
|
||||
if (after is None or not mutated <= committed.keys()
|
||||
or not committed and (before is None or before != after)):
|
||||
raise ValueError('source changes lack a verified committed prefix')
|
||||
self._checkpoint(root, project, attempt_id)
|
||||
files.append(root / 'project-checkpoint.json')
|
||||
current = internal_path(root / 'current-command.tsv')
|
||||
status = internal_path(root / 'status.tsv')
|
||||
proof = dict(pause, phase='before_native_dispatch', task_id=selected[0][1]['task_id'],
|
||||
session_id=self.session_id, generation=self.generation, attempt_id=attempt_id,
|
||||
resource_record=str(self.path), project_closed=True, native_dispatch_not_started=True,
|
||||
source_sha256=hashlib.sha256(repr(after).encode()).hexdigest(), committed_prefix=committed,
|
||||
prefix_marker_state=prefix_marker_state, prefix_validation_required=prefix_marker_state == 'project_saved',
|
||||
pause_marker_path=str(marker_path),
|
||||
pause_marker_sha256=hashlib.sha256(internal_path(marker_path).read_bytes()).hexdigest(),
|
||||
commit_manifest_path=str(manifest_path) if committed else None,
|
||||
commit_manifest_sha256=hashlib.sha256(internal_path(manifest_path).read_bytes()).hexdigest() if committed else None,
|
||||
request_sha256=hashlib.sha256(request_bytes).hexdigest(),
|
||||
status_sha256=hashlib.sha256(status.read_bytes()).hexdigest() if status.exists() else None,
|
||||
current_command_sha256=hashlib.sha256(current.read_bytes()).hexdigest() if current.exists() else None,
|
||||
batch_attempts=[{'path': str(path), 'expected': record} for path, record in selected])
|
||||
proof.update(_source_parameters(project, self.root))
|
||||
proof_path = root / ('prewrite-pause-' + attempt_id + '.json')
|
||||
write_record(proof_path, proof)
|
||||
proof['files'] = [{'path': str(path), 'sha256': hashlib.sha256(internal_path(path).read_bytes()).hexdigest()}
|
||||
for path in [proof_path, *files]]
|
||||
return proof
|
||||
except (OSError, ValueError, TypeError, KeyError, AttributeError) as error:
|
||||
raise WorkerMonitorError('prewrite pause lacks bound current-job and committed-prefix proof: ' + str(error)) from error
|
||||
|
||||
def _retain_uncertain(self, root, project, expected, error):
|
||||
if self.read_only:
|
||||
return
|
||||
@@ -747,7 +957,7 @@ class BatchCadRuntime:
|
||||
def _close_once(self, *, rebuilding=False, reason=None):
|
||||
if self.desktop is None and not self.generation:
|
||||
return # A cache-only batch never acquired resources.
|
||||
if self.record['state'] == 'released':
|
||||
if self.record['state'] in {'released', 'isolated'}:
|
||||
return
|
||||
reason = reason or ('rebuild' if rebuilding else 'batch_end')
|
||||
if self._close_failure is not None:
|
||||
@@ -775,7 +985,7 @@ class BatchCadRuntime:
|
||||
completed_result=self.result, cleanup_error=self._close_failure)
|
||||
cleanup = self.desktop.cleanup_record or {}
|
||||
self.record['cleanup'] = cleanup
|
||||
if cleanup.get('desktop_confirmation', {}).get('released') is True:
|
||||
if (cleanup.get('desktop_confirmation') or {}).get('released') is True:
|
||||
self.record['release_results'] = {'mentor_released': True, 'desktop_released': True}
|
||||
self.record['release_state'] = {'mentor': 'released', 'desktop': 'released'}
|
||||
self._release_log('started', reason=reason)
|
||||
@@ -807,30 +1017,41 @@ class BatchCadRuntime:
|
||||
raise
|
||||
self._close_failure = None
|
||||
self._close_failure_rebuild = False
|
||||
self._release_log('succeeded', reason=reason, elapsed=time.monotonic() - started)
|
||||
self._release_log('isolated' if self.record['state'] == 'isolated' else 'succeeded',
|
||||
reason=reason, elapsed=time.monotonic() - started)
|
||||
|
||||
def _close_resources(self, *, rebuilding=False):
|
||||
from .operation_diagnostics import record_operation
|
||||
operation_root = self._active_evidence or self.path.parent
|
||||
started = time.monotonic()
|
||||
errors = []
|
||||
shutdown_errors = []
|
||||
retained_cad = False
|
||||
phases = {'mentor': 'closing' if self.cad is not None else 'released', 'desktop': 'waiting_for_mentor'}
|
||||
try:
|
||||
self._save(state='closing', release_state=phases)
|
||||
self._save(state='closing', release_state=phases,
|
||||
wait_reason='mentor_exit' if self.cad is not None else 'desktop_release',
|
||||
stopping_reason=getattr(self.stop_event, 'stopping_reason', ''),
|
||||
primary_error=self.record.get('primary_error') or getattr(self.stop_event, 'primary_error', ''))
|
||||
except Exception as error:
|
||||
errors.append('resource journal: ' + str(error))
|
||||
if self.cad is not None:
|
||||
cad = self.cad
|
||||
try:
|
||||
failure = getattr(self, '_cad_close_error', None)
|
||||
record_operation(operation_root, 'mentor_exit', 'enter')
|
||||
cad.__exit__(type(failure) if failure is not None else None, failure, None)
|
||||
record_operation(operation_root, 'mentor_exit', 'return')
|
||||
self.cad = None
|
||||
except Exception as error:
|
||||
record_operation(operation_root, 'mentor_exit', 'error', error=str(error))
|
||||
if rebuilding:
|
||||
self.record['shutdown_warning'] = str(error)
|
||||
# A rebuild must not proceed while the previous CAD owner has
|
||||
# an unconfirmed shutdown. Keep the warning for diagnostics,
|
||||
# but retain the failure in the resource gate as well.
|
||||
errors.append(str(error))
|
||||
shutdown_errors.append(str(error))
|
||||
from .read_recovery import process_tree_exited
|
||||
process = getattr(cad, 'cad', None)
|
||||
processes = [*getattr(cad, '_retained_processes', ()),
|
||||
@@ -853,9 +1074,13 @@ class BatchCadRuntime:
|
||||
phases['mentor'] = 'failed' if retained_cad else 'released'
|
||||
if self.desktop is not None and not retained_cad:
|
||||
phases['desktop'] = 'closing'
|
||||
self.record['wait_reason'] = 'desktop_release'
|
||||
try:
|
||||
record_operation(operation_root, 'desktop_release', 'enter')
|
||||
self.desktop.close()
|
||||
record_operation(operation_root, 'desktop_release', 'return')
|
||||
except Exception as error:
|
||||
record_operation(operation_root, 'desktop_release', 'error', error=str(error))
|
||||
errors.append(str(error))
|
||||
self._timing('batch_release', started)
|
||||
self.record['cleanup'] = getattr(self.desktop, 'cleanup_record', None)
|
||||
@@ -863,12 +1088,32 @@ class BatchCadRuntime:
|
||||
self.record['release_results'] = {
|
||||
'mentor_released': not retained_cad,
|
||||
'desktop_released': cleanup.get('desktop_handle_closed') is True
|
||||
and cleanup.get('desktop_confirmation', {}).get('released') is True}
|
||||
and (cleanup.get('desktop_confirmation') or {}).get('released') is True}
|
||||
phases['desktop'] = ('waiting_for_mentor' if retained_cad else
|
||||
'released' if self.record['release_results']['desktop_released'] else 'failed')
|
||||
self.record['release_state'] = phases
|
||||
self.record['cleanup_mode'] = getattr(self.stop_event, 'cleanup_mode', None)
|
||||
if not retained_cad and not self.record['release_results']['desktop_released']:
|
||||
try:
|
||||
proof = isolation_proof(self.record)
|
||||
if not self._other_writers_exited():
|
||||
raise ValueError('Retained writer ownership is still active')
|
||||
except Exception:
|
||||
pass
|
||||
else:
|
||||
phases['desktop'] = 'isolated'
|
||||
self._save(state='isolated', blocking=False, isolation_proof=proof,
|
||||
isolation_warning='; '.join(errors), release_state=phases, wait_reason='')
|
||||
LIVE_RESOURCES.pop(str(self.path), None)
|
||||
if shutdown_errors:
|
||||
raise BatchResourceCleanupError('; '.join(shutdown_errors), completed_result=self.result)
|
||||
return
|
||||
if errors:
|
||||
if (errors == shutdown_errors and not retained_cad and self.owned_processes_exited()
|
||||
and self._other_writers_exited() and self.record['release_results']['desktop_released']):
|
||||
self.record['shutdown_warning'] = '; '.join(shutdown_errors)
|
||||
self._record_release()
|
||||
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
|
||||
self._save(state='blocked', error='; '.join(errors))
|
||||
raise BatchResourceCleanupError('; '.join(errors), completed_result=self.result)
|
||||
self._record_release()
|
||||
@@ -879,5 +1124,57 @@ class BatchCadRuntime:
|
||||
'cad_pid': self.record['cad_pid'], 'cad_guid': self.record['cad_guid'],
|
||||
'cleanup': self.record['cleanup'], 'released_at': released_at}
|
||||
releases = [*self.record.get('generation_releases', ()), release]
|
||||
self._save(state='released', released_at=released_at, processes=[], generation_releases=releases)
|
||||
self._save(state='released', blocking=False, released_at=released_at, processes=[], generation_releases=releases,
|
||||
wait_reason='')
|
||||
LIVE_RESOURCES.pop(str(self.path), None)
|
||||
|
||||
def _other_writers_exited(self):
|
||||
from .isolated_cad import RETAINED_CAD_SESSIONS
|
||||
from .read_recovery import process_tree_exited
|
||||
from .worker_monitor import RETAINED_WORKERS, _worker_tree_exited
|
||||
return (all(all(process_tree_exited(process) for process in session._owned_processes())
|
||||
for session in tuple(RETAINED_CAD_SESSIONS.values())
|
||||
if session.root.is_relative_to(self.root))
|
||||
and all(_worker_tree_exited(entry['process'], entry['owned'])
|
||||
for entry in tuple(RETAINED_WORKERS.values()) if entry['root'].is_relative_to(self.root)))
|
||||
|
||||
def release_failed_project(self, error, evidence_root):
|
||||
"""Release proven stopped writers; the failed project's business evidence stays untouched."""
|
||||
from .stop_control import protect_interruption
|
||||
protect_interruption(evidence_root, 'project_failure: ' + str(error))
|
||||
request = getattr(self.stop_event, 'request_failure_cleanup', None)
|
||||
if request is not None:
|
||||
request(evidence_root, error)
|
||||
if self.cad is not None:
|
||||
self.cad.stop_control = self.stop_event
|
||||
self._cad_close_error = error
|
||||
cleanup_warning = None
|
||||
try:
|
||||
self.close(reason='project_failure')
|
||||
except BatchResourceCleanupError as cleanup:
|
||||
from .batch_resources import blocking_resources
|
||||
if (self.record['state'] not in {'released', 'isolated'} or blocking_resources(self.root)
|
||||
or not self.owned_processes_exited() or not self._other_writers_exited()):
|
||||
raise
|
||||
cleanup_warning = str(cleanup)
|
||||
error.shutdown_warning = cleanup_warning
|
||||
self.record.setdefault('generation_shutdown_warnings', []).append({
|
||||
'generation': self.generation, 'error': cleanup_warning, 'cad_shutdown': self.record.get('cad_shutdown')})
|
||||
self._save(shutdown_warning=cleanup_warning)
|
||||
if ((self.generation and not self.owned_processes_exited()) or not self._other_writers_exited()):
|
||||
return False
|
||||
from .batch_resources import blocking_resources
|
||||
if blocking_resources(self.root):
|
||||
return False
|
||||
write_record(canonical_path(evidence_root) / 'project-resource-release.json', {
|
||||
'version': 1, 'session_id': self.session_id, 'generation': self.generation,
|
||||
'resource_record': str(self.path), 'writers_exited': True,
|
||||
'resource_state': self.record['state'], 'desktop_released':
|
||||
self.record.get('release_results', {}).get('desktop_released') is True,
|
||||
'business_uncertainty_preserved': True})
|
||||
self.desktop = self.cad = None
|
||||
self._active_evidence = None
|
||||
self._active_attempts = []
|
||||
self._close_failure = None
|
||||
self._cad_close_error = None
|
||||
return True
|
||||
@@ -305,9 +305,9 @@ def pending_supervisors(root):
|
||||
|
||||
def reconcile_supervisors(root):
|
||||
"""Recheck orphan ownership only; this never launches or replays a job."""
|
||||
from .batch_resources import _process_gone, pending_resources, write_record
|
||||
from .batch_resources import _process_gone, blocking_resources, write_record
|
||||
from .read_recovery import process_tree_exited
|
||||
physical = pending_resources(root)
|
||||
physical = blocking_resources(root)
|
||||
for record in pending_supervisors(root):
|
||||
owner = (_LIVE_SUPERVISORS.get(canonical_path(record['record_path']))
|
||||
or _ACTIVE_SUPERVISORS.get(canonical_path(record['record_path'])))
|
||||
@@ -348,21 +348,26 @@ def _environment():
|
||||
|
||||
def _finish_response(root, journal, response, context):
|
||||
"""A supervisor exit may release its last USER reference. Never repeat work."""
|
||||
from .batch_resources import cleanup_pending_resources, write_record
|
||||
from .batch_resources import cleanup_pending_resources, resource_blockers, write_record
|
||||
from .supervisor_result import validated_outcome, resource_pending_error, application_shutdown_error
|
||||
result, error, primary = validated_outcome(journal['operation'], response)
|
||||
pending = cleanup_pending_resources(root, logging_context=context)
|
||||
related = tuple(item for item in pending if item.get('session_id') == journal['batch_id'] or not item.get('session_id'))
|
||||
journal['state'] = 'blocked' if related else 'finished'
|
||||
journal['outcome'] = ('completed_resources_pending' if related and response.get('business_succeeded')
|
||||
blockers = resource_blockers(related)
|
||||
warnings = tuple(item for item in related if item not in blockers)
|
||||
journal['state'] = 'blocked' if blockers else 'finished'
|
||||
journal['outcome'] = ('completed_resources_pending' if blockers and response.get('business_succeeded')
|
||||
else response.get('status', 'interrupted'))
|
||||
journal['resource_recheck'] = {'pending': bool(related), 'checked_at': time.time()}
|
||||
journal['resource_recheck'] = {'pending': bool(blockers), 'warnings': list(warnings),
|
||||
'desktop_released': not related, 'owned_writers_exited': not blockers, 'checked_at': time.time()}
|
||||
write_record(Path(root) / 'batch-resources' / journal['batch_id'] / 'supervisor.json', journal)
|
||||
if related:
|
||||
if blockers:
|
||||
raise resource_pending_error(journal['operation'], result, error, primary)
|
||||
for failure in (error, primary):
|
||||
if failure is not None:
|
||||
failure.resources_released = True
|
||||
failure.resources_released = not warnings
|
||||
failure.owned_writers_exited = True
|
||||
failure.resource_warnings = warnings
|
||||
resource_path = internal_path(Path(root) / 'batch-resources' / journal['batch_id'] / 'session.json')
|
||||
def preserve_shutdown_failure(detail):
|
||||
try:
|
||||
@@ -373,7 +378,9 @@ def _finish_response(root, journal, response, context):
|
||||
journal['shutdown_record_error'] = str(record_error)
|
||||
def shutdown_failure():
|
||||
failure = application_shutdown_error(result, error, primary)
|
||||
failure.resources_released = True
|
||||
failure.resources_released = not warnings
|
||||
failure.owned_writers_exited = True
|
||||
failure.resource_warnings = warnings
|
||||
return failure
|
||||
try:
|
||||
resource = json.loads(resource_path.read_text(encoding='utf-8'))
|
||||
@@ -558,6 +565,7 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
on_phase(*payload)
|
||||
elif value['type'] == 'state':
|
||||
state = decode(value['payload'])
|
||||
service._supervisor_state = state
|
||||
business_watch.observe('state', state, time.monotonic())
|
||||
service._task_queue_feedback = getattr(service, '_workspace_queue_context', None) or state['queue']
|
||||
service._batch_feedback = state['batch']
|
||||
@@ -578,6 +586,7 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
service._task_queue_feedback = None
|
||||
service._batch_feedback = None
|
||||
service._supervisor_waiting = False
|
||||
service._supervisor_state = None
|
||||
service._supervisor_business_stalled = False
|
||||
service._supervisor_dialog_state = None
|
||||
try:
|
||||
@@ -616,10 +625,10 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
exit_code=getattr(process, 'returncode', None),
|
||||
stdout_chars=len(stdout_tail), stderr_chars=len(stderr_tail))
|
||||
if force_accepted:
|
||||
from .batch_resources import cleanup_pending_resources
|
||||
from .batch_resources import cleanup_pending_resources, resource_blockers
|
||||
try:
|
||||
pending = cleanup_pending_resources(root, logging_context=service._logging)
|
||||
if pending:
|
||||
if resource_blockers(pending):
|
||||
ownership_cleanup_error = TaskCenterError('batch_resources_pending', '强制结束后的资源仍待清理。')
|
||||
except Exception as error:
|
||||
ownership_cleanup_error = error
|
||||
@@ -628,7 +637,8 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
failure = TaskCenterError('mentor_release_required' if automatic else 'user_forced_stop',
|
||||
'执行失败,已自动收尾本批后台进程;已有结果保留,当前结果待核验。' if automatic else
|
||||
'已按确认强制结束;已保存的执行证据保留,当前结果待核验。请先核验恢复,不要重复替换。')
|
||||
failure.resources_released = ownership_cleanup_error is None
|
||||
failure.resources_released = ownership_cleanup_error is None and not pending
|
||||
failure.owned_writers_exited = ownership_cleanup_error is None
|
||||
failure.primary_error = transport_error
|
||||
if ownership_cleanup_error is not None:
|
||||
journal.update(force_state='confirmation_failed', force_cleanup_error=str(ownership_cleanup_error))
|
||||
@@ -688,7 +698,7 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
raise failure from protection_error
|
||||
# Recheck physical resources even when final business results are
|
||||
# missing; this never authorizes repeating the interrupted job.
|
||||
from .batch_resources import cleanup_pending_resources
|
||||
from .batch_resources import cleanup_pending_resources, resource_blockers
|
||||
outstanding = cleanup_pending_resources(root, logging_context=service._logging)
|
||||
failure = cause if isinstance(cause, TaskCenterError) else TaskCenterError(
|
||||
'supervisor_interrupted', '批次监督进程未留下可信结果;保留保护记录,不自动重试。' +
|
||||
@@ -697,6 +707,8 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
from .read_recovery import process_tree_exited
|
||||
failure.resources_released = (not outstanding and ownership_cleanup_error is None
|
||||
and (process is None or process_tree_exited(process)))
|
||||
failure.owned_writers_exited = (not resource_blockers(outstanding) and ownership_cleanup_error is None
|
||||
and (process is None or process_tree_exited(process)))
|
||||
raise failure from (transport_error or error)
|
||||
if transport_error is not None:
|
||||
if response['payload']['business_succeeded']:
|
||||
@@ -705,9 +717,13 @@ def run_supervised(service, operation, arguments, *, stop_event=None, on_progres
|
||||
else:
|
||||
_mark_interrupted(root, journal, transport_error)
|
||||
failure = resource_pending_error(operation, decoded_result, transport_error, decoded_primary)
|
||||
from .batch_resources import pending_resources
|
||||
from .batch_resources import pending_resources, resource_blockers
|
||||
from .read_recovery import process_tree_exited
|
||||
failure.resources_released = (ownership_cleanup_error is None and not pending_resources(root)
|
||||
pending = pending_resources(root)
|
||||
failure.resources_released = (ownership_cleanup_error is None and not pending
|
||||
and process is not None and process_tree_exited(process))
|
||||
failure.owned_writers_exited = (ownership_cleanup_error is None and not resource_blockers(pending)
|
||||
and process is not None and process_tree_exited(process))
|
||||
failure.resource_warnings = tuple(row for row in pending if row not in resource_blockers(pending))
|
||||
raise failure
|
||||
return _finish_response(root, journal, response['payload'], service._logging)
|
||||
@@ -14,6 +14,14 @@ from uuid import UUID
|
||||
from .path_access import internal_path, canonical_path
|
||||
from .supervisor_protocol import VERSION, OPERATIONS, valid_journal, encode, decode, encode_error
|
||||
|
||||
|
||||
def _persist_result(directory, envelope):
|
||||
from .batch_resources import write_record
|
||||
from .operation_diagnostics import operation_span
|
||||
with operation_span(directory, 'result_persistence', item_id=envelope['attempt_id']):
|
||||
write_record(directory / 'result.json', envelope)
|
||||
|
||||
|
||||
def _output_stream():
|
||||
if sys.stdout is not None:
|
||||
return sys.stdout, False
|
||||
@@ -85,7 +93,7 @@ def consume_manual_dialog_actions(directory, runtime, batch_id, attempt_id):
|
||||
write_record(claimed.with_suffix('.result'), result)
|
||||
|
||||
|
||||
def _wait_for_owned_writes(runtime, control):
|
||||
def _wait_for_owned_writes(runtime, control) -> Exception | None:
|
||||
"""Process exit would close kill-on-close Jobs, so it is also a write boundary."""
|
||||
if runtime.read_only:
|
||||
return
|
||||
@@ -103,15 +111,21 @@ def _wait_for_owned_writes(runtime, control):
|
||||
return False
|
||||
return all(_worker_tree_exited(entry['process'], entry['owned'])
|
||||
for entry in tuple(RETAINED_WORKERS.values()) if entry['root'].is_relative_to(runtime.root))
|
||||
if not exited():
|
||||
try:
|
||||
wait_for_resolution(control, runtime.path.parent,
|
||||
'supervisor exit still has owned write processes', resolved=exited, release_on_failure=True)
|
||||
except MentorReleaseRequired:
|
||||
cleanup_retained_workers(runtime.root, force=True)
|
||||
cleanup_retained_workers(runtime.root)
|
||||
if runtime.desktop is not None or runtime.cad is not None:
|
||||
runtime.close(reason='resource_recheck')
|
||||
try:
|
||||
if not exited():
|
||||
try:
|
||||
wait_for_resolution(control, runtime.path.parent,
|
||||
'supervisor exit still has owned write processes', resolved=exited, release_on_failure=True)
|
||||
except MentorReleaseRequired:
|
||||
cleanup_retained_workers(runtime.root, force=True)
|
||||
cleanup_retained_workers(runtime.root)
|
||||
if runtime.desktop is not None or runtime.cad is not None:
|
||||
runtime.close(reason='resource_recheck')
|
||||
except Exception as error:
|
||||
# Desktop references may remain after writers exit; publish that failure.
|
||||
if not exited():
|
||||
raise
|
||||
return error
|
||||
|
||||
|
||||
def run(request_path):
|
||||
@@ -154,7 +168,7 @@ def run(request_path):
|
||||
'generation': max(1, runtime.generation), 'sequence': sequence,
|
||||
'type': kind, 'payload': payload}
|
||||
if persist:
|
||||
write_record(directory / 'result.json', envelope)
|
||||
_persist_result(directory, envelope)
|
||||
try:
|
||||
output.write(json.dumps(envelope, ensure_ascii=True, allow_nan=False) + '\n')
|
||||
output.flush()
|
||||
@@ -175,9 +189,13 @@ def run(request_path):
|
||||
emit('heartbeat')
|
||||
feedback = getattr(service, '_task_queue_feedback', None)
|
||||
emit('state', encode({'phase': runtime.record['state'], 'queue': feedback,
|
||||
'release_state': runtime.record.get('release_state'),
|
||||
'batch': service.batch_feedback(),
|
||||
'active_project': runtime.record.get('active_project'),
|
||||
'business_progress': runtime.business_progress(),
|
||||
'stopping_reason': runtime.record.get('stopping_reason') or getattr(runtime.stop_event, 'stopping_reason', ''),
|
||||
'wait_reason': runtime.record.get('wait_reason') or getattr(runtime.stop_event, 'wait_reason', ''),
|
||||
'primary_error': runtime.record.get('primary_error') or getattr(runtime.stop_event, 'primary_error', ''),
|
||||
'dialog_supervision': runtime.dialog_state(),
|
||||
'dialog_binding': {'batch_id': batch_id, 'attempt_id': attempt_id,
|
||||
'generation': runtime.generation, 'session_token': getattr(runtime.cad, 'token', None)},
|
||||
@@ -249,7 +267,15 @@ def run(request_path):
|
||||
context.report_exception('batch_supervisor', error)
|
||||
response.update(status='error', error=encode_error(error))
|
||||
finally:
|
||||
_wait_for_owned_writes(runtime, stop)
|
||||
cleanup_error = _wait_for_owned_writes(runtime, stop)
|
||||
if cleanup_error is not None:
|
||||
context.report_exception('batch_supervisor_resource_recheck', cleanup_error)
|
||||
response['status'] = 'error'
|
||||
response.setdefault('error', encode_error(cleanup_error))
|
||||
if runtime.record.get('state') == 'isolated':
|
||||
response['resource_warning'] = {'state': 'isolated', 'blocking': False,
|
||||
'desktop': runtime.record['desktop'], 'desktop_released': False,
|
||||
'owned_writers_exited': True}
|
||||
evidence.try_append('worker_closing', status=response.get('status'), business_succeeded=response.get('business_succeeded'))
|
||||
finished.set()
|
||||
thread.join(3)
|
||||
|
||||
@@ -40,8 +40,9 @@ def retained_bom_rows(pending, existing_rows):
|
||||
candidates = tuple(index for identity in dict.fromkeys(((record.old.casefold(), ref.casefold()),
|
||||
(record.new.casefold(), ref.casefold()))) for index in by_identity.get(identity, ())
|
||||
if (not record.schematic or existing_rows[index].target.schematic_name.casefold() == record.schematic.casefold())
|
||||
and (not record.sheet or existing_rows[index].target.sheet_number.casefold() == record.sheet.casefold())
|
||||
and (not record.uid or existing_rows[index].target.uid == record.uid))
|
||||
and (not record.sheet or existing_rows[index].target.sheet_number.casefold() == record.sheet.casefold()))
|
||||
if len(candidates) > 1 and record.uid:
|
||||
candidates = tuple(index for index in candidates if existing_rows[index].target.uid == record.uid)
|
||||
if len(candidates) > 1:
|
||||
raise TaskCenterError('bom_scope_ambiguous', f'位号 {ref} 无法唯一匹配已保存位号;请填写原理图、图页或 UID:'
|
||||
f'{request.source_file} 第 {request.source_row} 行')
|
||||
|
||||
@@ -25,7 +25,7 @@ _HEADER_HELP = (
|
||||
'位号,必填。一行可填 R1,R2,R3,支持逗号、分号、空格或顿号分隔。',
|
||||
'原理图名称,可选。位号重复时用于精确匹配。',
|
||||
'图页,可选。位号跨页重复时应填写图页或 UID。',
|
||||
'器件 UID,可选;填写时必须与采集结果一致。',
|
||||
'器件 UID,可选。工程、图页和位号已唯一匹配时忽略;仅用于区分仍有多个候选的位号。',
|
||||
'是否替换:TRUE/FALSE 或 是/否;留空默认替换。未列出的位号保留。',
|
||||
'自定义 Value,可选;留空沿用工具原有 Value 策略。',
|
||||
'同步 NC:TRUE/FALSE 或 是/否;留空默认开启。',
|
||||
@@ -209,8 +209,9 @@ class BomMatcher:
|
||||
for key, target in self.by_identity.get(identity, ())
|
||||
if (not record.project_path or self.identities.project(record.project_path) == key[0])
|
||||
and (not record.schematic or target.schematic_name.casefold() == record.schematic.casefold())
|
||||
and (not record.sheet or target.sheet_number.casefold() == record.sheet.casefold())
|
||||
and (not record.uid or target.uid == record.uid)]
|
||||
and (not record.sheet or target.sheet_number.casefold() == record.sheet.casefold())]
|
||||
if len(candidates) > 1 and record.uid:
|
||||
candidates = [(key, target) for key, target in candidates if target.uid == record.uid]
|
||||
if len(candidates) != 1:
|
||||
raise TaskCenterError('bom_import', f'位号 {refdes} 无法唯一匹配当前采集清单')
|
||||
key, target = candidates[0]
|
||||
|
||||
@@ -156,7 +156,10 @@ def _catalog_from_evidence(evidence, library, part, partition, fingerprint, path
|
||||
def _require_cleanup(cleanup):
|
||||
def pending(message):
|
||||
raise TaskCenterError('batch_resources_pending', message)
|
||||
if not isinstance(cleanup, dict) or cleanup.get('errors') or not cleanup.get('desktop_handle_closed') or not cleanup.get('desktop_confirmation', {}).get('released'):
|
||||
from .desktop_slot import returned_lease_intact
|
||||
returned = isinstance(cleanup, dict) and returned_lease_intact(cleanup.get('desktop_lease', {}), cleanup.get('desktop'))
|
||||
if (not isinstance(cleanup, dict) or cleanup.get('errors') or not cleanup.get('desktop_handle_closed')
|
||||
or not ((cleanup.get('desktop_confirmation') or {}).get('released') or returned)):
|
||||
pending('catalog private desktop cleanup was not proven')
|
||||
if not cleanup.get('processes'):
|
||||
pending('catalog owned process cleanup is missing')
|
||||
@@ -419,7 +422,7 @@ def _run_owned_reader(clone, part, partition, root):
|
||||
from .isolated_desktop import PrivateDesktop
|
||||
from .legacy_paths import _AliasPool, _fits, _short_path
|
||||
exe = _library_manager_executable()
|
||||
desktop = PrivateDesktop('dx-cell-' + uuid.uuid4().hex, enabled=True)
|
||||
desktop = PrivateDesktop('dx-cell-' + uuid.uuid4().hex, enabled=True, managed=True)
|
||||
aliases = _AliasPool()
|
||||
report_path = root / 'worker-result.json'
|
||||
clone_fingerprint = _content_fingerprint(_library_manifest(clone))
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
"""One desktop per user/logon, with independent durable leases and named Jobs."""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
from threading import Event, Thread
|
||||
from uuid import uuid4
|
||||
|
||||
from .batch_resources import write_record
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path
|
||||
|
||||
|
||||
def _fail(message):
|
||||
raise TaskCenterError('desktop_slot_busy', message)
|
||||
|
||||
|
||||
def _context():
|
||||
import win32api
|
||||
import win32con
|
||||
import win32security
|
||||
import win32service
|
||||
token = win32security.OpenProcessToken(win32api.GetCurrentProcess(), win32con.TOKEN_QUERY)
|
||||
try:
|
||||
sid = win32security.GetTokenInformation(token, win32security.TokenUser)[0]
|
||||
groups = win32security.GetTokenInformation(token, win32security.TokenGroups)
|
||||
logon = [value for value, flags in groups if flags & 0xC0000000 == 0xC0000000]
|
||||
if len(logon) != 1:
|
||||
_fail('无法确认当前 Windows 登录会话,未创建桌面。')
|
||||
return dict(sid=win32security.ConvertSidToStringSid(sid),
|
||||
logon=win32security.ConvertSidToStringSid(logon[0]),
|
||||
station=win32service.GetUserObjectInformation(win32service.GetProcessWindowStation(), 2))
|
||||
finally:
|
||||
token.Close()
|
||||
|
||||
|
||||
class _Windows:
|
||||
def lock(self, name):
|
||||
import win32event
|
||||
mutex = win32event.CreateMutex(None, False, name)
|
||||
# A mutex is recursive on one thread; separate desktop owners are not.
|
||||
if name in _held:
|
||||
mutex.Close()
|
||||
_fail('当前进程已有桌面租用,等待本批收尾后再领取。')
|
||||
ready, release = Event(), Event()
|
||||
result = {}
|
||||
def own_mutex():
|
||||
try:
|
||||
result['acquired'] = win32event.WaitForSingleObject(mutex, 0) in (0, 0x80)
|
||||
except Exception as error:
|
||||
result['error'] = error
|
||||
finally:
|
||||
ready.set()
|
||||
if result.get('acquired'):
|
||||
release.wait()
|
||||
try: win32event.ReleaseMutex(mutex)
|
||||
except Exception as error: result['error'] = error
|
||||
# Runtime setup and worker-monitor cleanup occur on different threads;
|
||||
# keep Windows mutex ownership on one short-lived dedicated thread.
|
||||
thread = Thread(target=own_mutex, name='desktop-slot-mutex', daemon=True)
|
||||
thread.start(); ready.wait()
|
||||
if not result.get('acquired'):
|
||||
thread.join(); mutex.Close()
|
||||
if 'error' in result: raise result['error']
|
||||
_fail('桌面正在由另一批任务使用,请等待其收尾。')
|
||||
_held.add(name)
|
||||
return mutex, name, release, thread, result
|
||||
|
||||
def unlock(self, value):
|
||||
handle, name, release, thread, result = value
|
||||
release.set(); thread.join()
|
||||
try:
|
||||
if 'error' in result:
|
||||
raise result['error']
|
||||
finally:
|
||||
try:
|
||||
handle.Close()
|
||||
finally:
|
||||
_held.discard(name)
|
||||
|
||||
def desktop_exists(self, name):
|
||||
import win32service
|
||||
return name.casefold() in {s.casefold() for s in win32service.GetProcessWindowStation().EnumDesktops()}
|
||||
|
||||
def process_gone(self, identity):
|
||||
from .batch_resources import _process_gone
|
||||
return _process_gone(identity)
|
||||
|
||||
def job_empty(self, name):
|
||||
import win32job
|
||||
try:
|
||||
handle = win32job.OpenJobObject(4, False, name) # JOB_OBJECT_QUERY
|
||||
except Exception as error:
|
||||
if getattr(error, 'winerror', error.args[0] if error.args else None) == 2:
|
||||
return True # No handle remains; KILL_ON_JOB_CLOSE has been applied.
|
||||
raise
|
||||
try:
|
||||
return win32job.QueryInformationJobObject(handle, win32job.JobObjectBasicAccountingInformation)['ActiveProcesses'] == 0
|
||||
finally:
|
||||
handle.Close()
|
||||
|
||||
def current_process(self):
|
||||
import win32api
|
||||
import win32process
|
||||
return dict(pid=os.getpid(), creation_time=str(win32process.GetProcessTimes(win32api.GetCurrentProcess())['CreationTime']))
|
||||
|
||||
|
||||
_held = set()
|
||||
|
||||
|
||||
def returned_lease_intact(receipt, name):
|
||||
"""Read an immutable *past* return; never query or close a newer lease."""
|
||||
try:
|
||||
data = internal_path(receipt['path']).read_bytes()
|
||||
record = json.loads(data)
|
||||
return (receipt['returned'] is True and receipt['name'] == name
|
||||
and hashlib.sha256(data).hexdigest() == receipt['sha256']
|
||||
and record['lease_id'] == receipt['lease_id'] and record['scope'] == receipt['scope']
|
||||
and record['name'] == name and record['state'] == 'idle' and record['writers_exited'] is True)
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
return False
|
||||
|
||||
|
||||
def managed_desktop_disappeared(receipt):
|
||||
"""Observe under the same mutex; an active newer lease is left untouched."""
|
||||
backend = _Windows()
|
||||
try:
|
||||
mutex = backend.lock('Local\\DxPartReplace-Desktop-' + receipt['scope'])
|
||||
except TaskCenterError:
|
||||
return False
|
||||
try:
|
||||
return not backend.desktop_exists(receipt['name'])
|
||||
finally:
|
||||
backend.unlock(mutex)
|
||||
|
||||
|
||||
class DesktopSlot:
|
||||
def __init__(self, *, root=None, context=None, backend=None):
|
||||
self.context = _context() if context is None else context
|
||||
digest = hashlib.sha256(json.dumps(self.context, sort_keys=True).encode()).hexdigest()[:32]
|
||||
self.name = 'dx-' + digest
|
||||
self.scope = digest
|
||||
self.backend = _Windows() if backend is None else backend
|
||||
base = Path(root) if root is not None else Path(os.environ['LOCALAPPDATA']) / 'DxPartReplace/desktop-slots'
|
||||
internal_path(base).mkdir(parents=True, exist_ok=True)
|
||||
base = canonical_path(base)
|
||||
self.path = base / digest / 'slot.json'
|
||||
self.lease_id = uuid4().hex
|
||||
self.return_path = self.path.parent / 'leases' / (self.lease_id + '.json')
|
||||
self.handle = None
|
||||
self.receipt = None
|
||||
self.record = None
|
||||
|
||||
def _save(self, record):
|
||||
write_record(self.path, record)
|
||||
self.record = record
|
||||
|
||||
def _validate(self, record):
|
||||
if (record.get('version') != 1 or record.get('context') != self.context
|
||||
or record.get('name') != self.name or record.get('scope') != self.scope
|
||||
or not re.fullmatch('[0-9a-f]{32}', record.get('lease_id', ''))
|
||||
or record.get('state') not in {'leased', 'idle'} or not isinstance(record.get('launches'), list)):
|
||||
_fail('桌面租用记录不完整,未打开桌面。')
|
||||
identities = [record['owner'], *(r['process'] for r in record['launches'] if r.get('process'))]
|
||||
if any(type(item.get('pid')) is not int or not isinstance(item.get('creation_time'), str)
|
||||
or not item['creation_time'] for item in identities):
|
||||
_fail('桌面原进程身份不完整,未打开桌面。')
|
||||
for launch in record['launches']:
|
||||
recoverable = launch.get('state') in {'started', 'closed'} or (
|
||||
launch.get('state') == 'launching' and launch.get('atomic_job') is True)
|
||||
if (not recoverable
|
||||
or not re.fullmatch(r'Local\\DxPartReplace-Job-[0-9a-f]{32}', launch.get('job', ''))
|
||||
or not self.backend.job_empty(launch['job'])
|
||||
or launch.get('process') and not self.backend.process_gone(launch['process'])):
|
||||
_fail('前批进程或启动尝试尚未确认退出,保留原桌面租用。')
|
||||
|
||||
def acquire(self):
|
||||
if self.handle is not None or self.receipt is not None:
|
||||
_fail('该租用已领取或归还,不能重复派发。')
|
||||
self.handle = self.backend.lock('Local\\DxPartReplace-Desktop-' + self.scope)
|
||||
try:
|
||||
if internal_path(self.path).exists():
|
||||
previous = json.loads(internal_path(self.path).read_text(encoding='utf-8'))
|
||||
self._validate(previous)
|
||||
if previous['state'] == 'leased' and not self.backend.process_gone(previous['owner']):
|
||||
_fail('前批桌面持有者尚未退出,保留原租用。')
|
||||
elif self.backend.desktop_exists(self.name):
|
||||
_fail('同名桌面缺少归属记录,未收养现有桌面。')
|
||||
self._save(dict(version=1, context=self.context, name=self.name, scope=self.scope,
|
||||
lease_id=self.lease_id, owner=self.backend.current_process(), state='leased', launches=[]))
|
||||
except Exception as error:
|
||||
handle, self.handle = self.handle, None
|
||||
self.backend.unlock(handle)
|
||||
if isinstance(error, TaskCenterError): raise
|
||||
raise TaskCenterError('desktop_slot_busy', '桌面租用记录无法核对,未创建新桌面。') from error
|
||||
return self
|
||||
|
||||
def prepare_launch(self, *, atomic_job=False):
|
||||
if self.handle is None or self.receipt is not None:
|
||||
_fail('桌面租用已结束,禁止派发。')
|
||||
job = 'Local\\DxPartReplace-Job-' + uuid4().hex
|
||||
record = dict(self.record, launches=[*self.record['launches'],
|
||||
dict(job=job, state='launching', process=None, atomic_job=atomic_job)])
|
||||
self._save(record)
|
||||
return job
|
||||
|
||||
def launched(self, job, process):
|
||||
launches = [dict(item, state='started', process=process) if item['job'] == job else item
|
||||
for item in self.record['launches']]
|
||||
self._save(dict(self.record, launches=launches))
|
||||
|
||||
def launch_closed(self, job):
|
||||
launches = [dict(item, state='closed') if item['job'] == job else item for item in self.record['launches']]
|
||||
self._save(dict(self.record, launches=launches))
|
||||
|
||||
def release(self, *, desktop_disappeared):
|
||||
if self.receipt is not None: return self.receipt
|
||||
if self.handle is None: _fail('桌面租用未领取。')
|
||||
self._validate(self.record)
|
||||
record = dict(self.record, state='idle', writers_exited=True, desktop_disappeared=desktop_disappeared)
|
||||
write_record(self.return_path, record)
|
||||
self._save(record)
|
||||
receipt = dict(returned=True, name=self.name, scope=self.scope, lease_id=self.lease_id,
|
||||
desktop_disappeared=desktop_disappeared, path=str(self.return_path),
|
||||
sha256=hashlib.sha256(internal_path(self.return_path).read_bytes()).hexdigest())
|
||||
handle, self.handle = self.handle, None
|
||||
self.receipt = receipt
|
||||
self.backend.unlock(handle)
|
||||
return receipt
|
||||
@@ -9,16 +9,20 @@ import stat
|
||||
import zipfile
|
||||
|
||||
from .path_access import canonical_path, internal_path
|
||||
from .operation_diagnostics import summarize_operations
|
||||
|
||||
MAX_FILE_BYTES = 4 * 1024 * 1024
|
||||
MAX_BUNDLE_BYTES = 32 * 1024 * 1024
|
||||
MAX_FILES = 512
|
||||
_BATCH_NAMES = {'session.json', 'supervisor.json', 'request.json', 'result.json',
|
||||
'business-result.json', 'supervisor.log', 'resource-release.log', 'cancel.json',
|
||||
'process.json', 'lifecycle.jsonl', 'owner-loss.json', 'claim.json', 'force-request.json'}
|
||||
'process.json', 'lifecycle.jsonl', 'owner-loss.json', 'claim.json', 'force-request.json',
|
||||
'operation-events.jsonl'}
|
||||
_QUICK_NAMES = {'private-desktop.json', 'checkpoint.json', 'status.json',
|
||||
'uncertain.flag', 'resolved-uncertain.flag', 'protection.json', 'cleanup.json',
|
||||
'project-session.json', 'project-checkpoint.json', 'project-read-release.json', 'status.tsv'}
|
||||
'project-session.json', 'project-checkpoint.json', 'project-read-release.json', 'status.tsv',
|
||||
'worker-progress.json', 'quick-part-replace.log', 'native-inventory.json', 'operation-events.jsonl',
|
||||
'recovery-commit.json', 'retry-preparation.json'}
|
||||
_SECRET_KEY = re.compile(r'(?:password|passwd|(?:client[_-]?)?secret|credentials?|authorization|api[_-]?key|access[_-]?token|refresh[_-]?token|token)', re.I)
|
||||
_USER_KEY = re.compile(r'(?:user(?:[_-]?name)?|login[_-]?name)', re.I)
|
||||
_SECRET_TEXT = re.compile(r'''(?ix)(\b(?:password|passwd|(?:client[_-]?)?secret|credentials?|authorization|api[_-]?key|access[_-]?token|refresh[_-]?token|token)\b["']?\s*[=:]\s*)(?:"[^"\r\n]*"|'[^'\r\n]*'|[^\s,;}]+)''')
|
||||
@@ -52,6 +56,15 @@ def _scrub_json(value):
|
||||
|
||||
|
||||
def _allowed(parts):
|
||||
if parts[0] == 'execution-proof' or (len(parts) > 3 and parts[0] == 'tasks' and parts[2] == 'execution-proof'):
|
||||
return (parts[-1] == 'completion.json'
|
||||
or bool(re.fullmatch(r'project-(?:commit|pause)-[a-f0-9]{32}\.tsv', parts[-1]))
|
||||
or bool(re.fullmatch(r'(?:post-state|value-readback)-[A-Za-z0-9_.-]+\.(?:tsv|json)', parts[-1])))
|
||||
if (len(parts) >= 5 and parts[:1] == ('tasks',) and parts[2] == 'netlist-checks'):
|
||||
return parts[-1] in {'verification.json', 'netlist.json', 'capture.json', 'export-cache.json'}
|
||||
if (any(re.fullmatch(r'native-inventory-[A-Za-z0-9-]+', part) for part in parts)
|
||||
and len(parts) >= 3 and parts[-2] == 'output'):
|
||||
return parts[-1] in {'scope.txt', 'effective.txt'} or bool(re.fullmatch(r'icdb2ascii-(?:scope|effective)\.(?:stdout|stderr)\.txt', parts[-1], re.I))
|
||||
if re.fullmatch(r'cad-dialogs-[a-f0-9]{32}\.json', parts[-1]):
|
||||
return ((len(parts) == 4 and parts[0] == 'batch-resources' and re.fullmatch(r'generation-\d+', parts[2])) or
|
||||
any(value in parts for value in ('quick-run', 'quickrun', 'quick-run-history')))
|
||||
@@ -71,7 +84,10 @@ def _allowed(parts):
|
||||
if len(parts) >= 2 and parts[0] in {'queue', 'queues', 'task-queues', 'workspace-queues'}:
|
||||
return parts[-1].endswith('.json')
|
||||
if len(parts) >= 2 and any(value in parts for value in ('quick-run', 'quickrun', 'quick-run-history')):
|
||||
return parts[-1] in _QUICK_NAMES or bool(re.fullmatch(r'retained-worker-[a-f0-9]{32}\.json', parts[-1]))
|
||||
return (parts[-1] in _QUICK_NAMES
|
||||
or bool(re.fullmatch(r'retained-worker-[a-f0-9]{32}\.json', parts[-1]))
|
||||
or bool(re.fullmatch(r'project-(?:commit|pause)-[a-f0-9]{32}\.tsv', parts[-1]))
|
||||
or bool(re.fullmatch(r'(?:post-state|value-readback)-[A-Za-z0-9_.-]+\.(?:tsv|json)', parts[-1])))
|
||||
return False
|
||||
|
||||
|
||||
@@ -84,11 +100,13 @@ def export_diagnostic_bundle(data_root, destination: Path) -> Path:
|
||||
root, destination = canonical_path(data_root), canonical_path(destination)
|
||||
if destination.suffix.lower() != '.zip':
|
||||
raise ValueError('diagnostic destination must have a .zip suffix')
|
||||
manifest = {'version': 1, 'included': [], 'omissions': [], 'unknowns': [
|
||||
'External desktop handle holders are unknown; empty window/thread lists do not prove release.',
|
||||
manifest = {'version': 2, 'included': [], 'omissions': [], 'truncated': [], 'unknowns': [
|
||||
'Active logs and metadata are point-in-time reads; no source CAD, libraries, BOM, database or environment configuration is included.'],
|
||||
'limits': {'max_file_bytes': MAX_FILE_BYTES, 'max_bundle_bytes': MAX_BUNDLE_BYTES, 'max_files': MAX_FILES}}
|
||||
total = 0
|
||||
candidates = []
|
||||
operations = {}
|
||||
items = {}
|
||||
physical = internal_path(root)
|
||||
# os.walk does not descend symlinks; explicitly prune Windows junctions too.
|
||||
with zipfile.ZipFile(internal_path(destination), 'x', compression=zipfile.ZIP_DEFLATED) as archive:
|
||||
@@ -101,7 +119,7 @@ def export_diagnostic_bundle(data_root, destination: Path) -> Path:
|
||||
info = entry.lstat()
|
||||
if stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 0x400:
|
||||
manifest['omissions'].append({'path': _scrub_text(relative), 'reason': 'link_or_reparse_point'})
|
||||
elif base == physical and name not in {'logs', 'batch-resources', 'attempts', 'tasks', 'queue', 'queues', 'task-queues', 'workspace-queues', 'quick-run', 'quick-run-history'}:
|
||||
elif base == physical and name not in {'logs', 'batch-resources', 'attempts', 'tasks', 'queue', 'queues', 'task-queues', 'workspace-queues', 'quick-run', 'quick-run-history', 'execution-proof'}:
|
||||
manifest['omissions'].append({'path': _scrub_text(relative), 'reason': 'outside_allowlist'})
|
||||
else:
|
||||
safe_folders.append(name)
|
||||
@@ -119,28 +137,94 @@ def export_diagnostic_bundle(data_root, destination: Path) -> Path:
|
||||
reason = 'link_or_reparse_point'
|
||||
elif not stat.S_ISREG(info.st_mode):
|
||||
reason = 'not_regular_file'
|
||||
elif info.st_size > MAX_FILE_BYTES or total + info.st_size > MAX_BUNDLE_BYTES or len(manifest['included']) >= MAX_FILES:
|
||||
reason = 'size_or_count_limit'
|
||||
else:
|
||||
with path.open('rb') as source:
|
||||
payload = source.read(MAX_FILE_BYTES + 1)
|
||||
if len(payload) > MAX_FILE_BYTES or b'\x00' in payload:
|
||||
reason = 'binary_or_size_limit'
|
||||
else:
|
||||
text = payload.decode('utf-8-sig')
|
||||
if relative.suffix == '.json':
|
||||
text = json.dumps(_scrub_json(json.loads(text)), ensure_ascii=False, indent=2)
|
||||
elif '.jsonl' in relative.name:
|
||||
text = '\n'.join(json.dumps(_scrub_json(json.loads(line)), ensure_ascii=False)
|
||||
for line in text.splitlines() if line.strip()) + '\n'
|
||||
else:
|
||||
text = _scrub_text(text)
|
||||
archive.writestr(label, text)
|
||||
total += len(payload)
|
||||
manifest['included'].append(label)
|
||||
candidates.append((info.st_mtime_ns, path, relative, label, info.st_size))
|
||||
except (OSError, UnicodeError, ValueError):
|
||||
reason = 'unreadable_or_invalid_text'
|
||||
if reason:
|
||||
manifest['omissions'].append({'path': label, 'reason': reason})
|
||||
# Current failures must survive a busy workspace's file/count budget.
|
||||
for _, path, relative, label, size in sorted(candidates, key=lambda row: (-row[0], row[3])):
|
||||
reason = None
|
||||
budget = min(MAX_FILE_BYTES, MAX_BUNDLE_BYTES - total)
|
||||
try:
|
||||
tail = bool(re.search(r'\.(?:log|jsonl)(?:\.\d+)?$', relative.name))
|
||||
truncated = size > budget
|
||||
if len(manifest['included']) >= MAX_FILES or budget <= 0 or (truncated and (not tail or budget < 64)):
|
||||
reason = 'size_or_count_limit'
|
||||
else:
|
||||
info = path.lstat()
|
||||
if stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 0x400 or info.st_nlink != 1:
|
||||
manifest['omissions'].append({'path': label, 'reason': 'link_or_reparse_point'})
|
||||
continue
|
||||
with path.open('rb') as source:
|
||||
if truncated:
|
||||
source.seek(max(0, size - budget))
|
||||
source.readline(budget + 1) # Discard clipped credentials and UTF-8 lines.
|
||||
payload = source.read(budget + 1)
|
||||
if len(payload) > budget or b'\x00' in payload:
|
||||
reason = 'binary_or_size_limit'
|
||||
elif not payload and size:
|
||||
reason = 'no_complete_tail_line'
|
||||
else:
|
||||
try:
|
||||
content = payload.decode('utf-8-sig')
|
||||
except UnicodeDecodeError:
|
||||
if relative.name not in {'scope.txt', 'effective.txt'}:
|
||||
raise
|
||||
content = payload.decode('mbcs' if os.name == 'nt' else 'cp1252')
|
||||
if relative.suffix == '.json':
|
||||
content = json.dumps(_scrub_json(json.loads(content)), ensure_ascii=False, indent=2)
|
||||
elif '.jsonl' in relative.name:
|
||||
records, invalid = [], 0
|
||||
for line in content.splitlines():
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
records.append(_scrub_json(json.loads(line)))
|
||||
except ValueError:
|
||||
invalid += 1
|
||||
content = '\n'.join(json.dumps(record, ensure_ascii=False) for record in records) + '\n'
|
||||
if invalid:
|
||||
manifest['omissions'].append({'path': label, 'reason': 'invalid_jsonl_lines', 'lines': invalid})
|
||||
else:
|
||||
content = _scrub_text(content)
|
||||
encoded = content.encode('utf-8')
|
||||
if tail and budget >= 64 and len(encoded) > budget:
|
||||
# Redaction may grow a short secret. Trim complete oldest lines again.
|
||||
offset, remaining = 0, len(encoded)
|
||||
for line in encoded.splitlines(keepends=True):
|
||||
if remaining <= budget:
|
||||
break
|
||||
offset += len(line)
|
||||
remaining -= len(line)
|
||||
encoded = encoded[offset:]
|
||||
content = encoded.decode('utf-8')
|
||||
truncated = True
|
||||
if '.jsonl' in relative.name:
|
||||
records = [json.loads(line) for line in content.splitlines() if line.strip()]
|
||||
if len(encoded) > budget:
|
||||
reason = 'size_or_count_limit'
|
||||
else:
|
||||
archive.writestr(label, encoded)
|
||||
total += len(encoded)
|
||||
manifest['included'].append(label)
|
||||
if truncated:
|
||||
manifest['truncated'].append({'path': label, 'mode': 'tail', 'source_bytes': size})
|
||||
if relative.name == 'operation-events.jsonl':
|
||||
operations[_scrub_text(relative.parent.as_posix())] = summarize_operations(records)
|
||||
elif relative.name == 'status.tsv':
|
||||
rows = [line.split('\t') for line in content.splitlines() if line.strip()]
|
||||
rows = [row for row in rows if len(row) >= 4 and row[0] != 'job_id']
|
||||
if rows:
|
||||
saved = [row[0] for row in rows if row[1] in {'saved_ok', 'restored_ok'}]
|
||||
items[_scrub_text(relative.parent.as_posix())] = {
|
||||
'last_successful_item': saved[-1] if saved else None,
|
||||
'latest_item': {'job_id': rows[-1][0], 'state': rows[-1][1]}}
|
||||
except (OSError, UnicodeError, ValueError):
|
||||
reason = 'unreadable_or_invalid_text'
|
||||
if reason:
|
||||
manifest['omissions'].append({'path': label, 'reason': reason})
|
||||
archive.writestr('closure-summary.json', json.dumps({'operations': operations, 'items': items}, ensure_ascii=False, indent=2))
|
||||
archive.writestr('manifest.json', json.dumps(manifest, ensure_ascii=False, indent=2))
|
||||
return destination
|
||||
@@ -14,25 +14,43 @@ class ExecutionFeedback:
|
||||
stage: str = ''
|
||||
stage_seconds: float = 0.0
|
||||
waiting_for_safe_completion: bool = False
|
||||
stopping_reason: str = ''
|
||||
wait_reason: str = ''
|
||||
primary_error: str = ''
|
||||
|
||||
|
||||
def wait_reason_for_stage(stage):
|
||||
return {'save_and_close': 'save', 'verify_saved_project': 'readback', 'saved_source_wait': 'readback',
|
||||
'native_snapshot_read': 'readback', 'capture_semantic_state': 'readback',
|
||||
'private_shutdown': 'desktop_release',
|
||||
'awaiting_reconciliation': 'reconciliation', 'mentor_exit': 'mentor_exit',
|
||||
'desktop_release': 'desktop_release'}.get(stage,
|
||||
'native_call' if stage in {'native_replace', 'restore_attributes',
|
||||
'restore_attribute_fields', 'verify_restoration', 'verify_attribute_fields'} else '')
|
||||
|
||||
|
||||
def read_feedback(task_id, position, total, message, root: Path, started_after: float):
|
||||
private_state = ''
|
||||
private_seconds = 0.0
|
||||
private = root / 'quick-run' / 'private-desktop.json'
|
||||
try:
|
||||
modified = private.stat().st_mtime
|
||||
if modified >= started_after:
|
||||
state = json.loads(private.read_text(encoding='utf-8')).get('state')
|
||||
if state in {'starting', 'closing'}:
|
||||
return ExecutionFeedback(task_id, position, total, message,
|
||||
'private_startup' if state == 'starting' else 'private_shutdown', max(0, time.time() - modified))
|
||||
private_state = 'private_startup' if state == 'starting' else 'private_shutdown'
|
||||
private_seconds = max(0, time.time() - modified)
|
||||
except (OSError, ValueError, TypeError, AttributeError):
|
||||
pass
|
||||
path = root / 'quick-run' / 'worker-progress.json'
|
||||
stage, seconds, waiting = '', 0.0, False
|
||||
record = {}
|
||||
try:
|
||||
modified = path.stat().st_mtime
|
||||
if modified >= started_after:
|
||||
record = json.loads(path.read_text(encoding='utf-8'))
|
||||
if not isinstance(record, dict):
|
||||
record = {}
|
||||
waiting = record.get('waiting_for_safe_completion') is True
|
||||
candidate = record.get('stage', '')
|
||||
if candidate and candidate != 'finished':
|
||||
@@ -40,4 +58,14 @@ def read_feedback(task_id, position, total, message, root: Path, started_after:
|
||||
stage = candidate
|
||||
except (OSError, ValueError, TypeError, AttributeError):
|
||||
pass # Display feedback is optional; it never changes execution state.
|
||||
return ExecutionFeedback(task_id, position, total, message, stage, seconds, waiting)
|
||||
if private_state:
|
||||
stage, seconds = private_state, private_seconds
|
||||
stopping_reason = record.get('stopping_reason', '')
|
||||
if not stopping_reason and record.get('stop_requested'):
|
||||
stopping_reason = 'inactivity'
|
||||
wait_reason = record.get('wait_reason') or wait_reason_for_stage(stage)
|
||||
if private_state and wait_reason not in {'mentor_exit', 'desktop_release'}:
|
||||
wait_reason = wait_reason_for_stage(stage)
|
||||
return ExecutionFeedback(task_id, position, total, message, stage, seconds, waiting,
|
||||
stopping_reason, wait_reason,
|
||||
record.get('primary_error') or record.get('resolution_error') or '')
|
||||
@@ -33,6 +33,11 @@ def create_execution_schema(connection):
|
||||
connection.execute('''CREATE TABLE IF NOT EXISTS execution_target_claims (
|
||||
task_id TEXT NOT NULL, round_id TEXT NOT NULL, target_key TEXT NOT NULL,
|
||||
operation_key TEXT NOT NULL, PRIMARY KEY(task_id,round_id,target_key))''')
|
||||
connection.execute('''CREATE TABLE IF NOT EXISTS execution_project_commits (
|
||||
task_id TEXT NOT NULL, round_id TEXT NOT NULL, session_token TEXT NOT NULL,
|
||||
project_path TEXT NOT NULL, evidence_root TEXT NOT NULL, state TEXT NOT NULL,
|
||||
jobs_json TEXT NOT NULL, proof_json TEXT,
|
||||
PRIMARY KEY(task_id,round_id,session_token))''')
|
||||
|
||||
|
||||
def _json(value):
|
||||
@@ -115,7 +120,7 @@ class ExecutionJournal:
|
||||
current=self._round(task_id)
|
||||
if current:
|
||||
unsafe=self.store._connection.execute('''SELECT 1 FROM execution_operations
|
||||
WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain') LIMIT 1''',
|
||||
WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain','pending_save') LIMIT 1''',
|
||||
(task_id,current['round_id'])).fetchone()
|
||||
if unsafe:raise TaskCenterError('quick_run_uncertain','原执行项尚待核验,不能准备重试。')
|
||||
self.store._connection.execute('DELETE FROM execution_retry_selections WHERE task_id=?',(task_id,))
|
||||
@@ -154,7 +159,7 @@ class ExecutionJournal:
|
||||
(task_id, current['round_id']))}
|
||||
for key in json.loads(current['plan_json']):
|
||||
record = records.get(key)
|
||||
if record is None or record['state'] in {'prepared', 'uncertain'}:return False
|
||||
if record is None or record['state'] in {'prepared', 'uncertain', 'pending_save'}:return False
|
||||
if record['state'] == 'pending':
|
||||
command = json.loads(record['command_json'])
|
||||
return bool(self.can_continue_component(task_id, command))
|
||||
@@ -392,13 +397,18 @@ class ExecutionJournal:
|
||||
if record is None or not self.certified_startup_pause(task_id, record):
|
||||
return
|
||||
proof = json.loads(record['receipt_json'] or '{}').get('startup_pause', {})
|
||||
status = internal_path(root / 'status.tsv').read_bytes()
|
||||
status_size = proof.get('status_size')
|
||||
if proof.get('phase') == 'before_native_dispatch' and type(status_size) is int and 0 < status_size <= len(status):
|
||||
# A delayed marker retirement can follow appends to the certified stream.
|
||||
status = status[:status_size]
|
||||
if (proof.get('task_id') != task_id or proof.get('job_id') != row['job_id']
|
||||
or proof.get('native_dispatch_not_started') is not True
|
||||
or proof.get('project_closed') is not True or proof.get('worker_tree_exited') is not True
|
||||
or not proof.get('files') or not proof_intact(proof)
|
||||
or _source_digest(proof) != proof.get('source_sha256')
|
||||
or hashlib.sha256(marker.read_bytes()).hexdigest() != proof.get('current_command_sha256')
|
||||
or hashlib.sha256(internal_path(root / 'status.tsv').read_bytes()).hexdigest() != proof.get('status_sha256')
|
||||
or hashlib.sha256(status).hexdigest() != proof.get('status_sha256')
|
||||
# Scope cleanup removes this request after the durable pause;
|
||||
# a surviving request must still match the certified one.
|
||||
or internal_path(root / 'project-session.request').exists() and hashlib.sha256(
|
||||
@@ -430,7 +440,7 @@ class ExecutionJournal:
|
||||
with self.store._lock:
|
||||
current=self._round(task_id)
|
||||
return bool(current and self.store._connection.execute('''SELECT 1 FROM execution_operations
|
||||
WHERE task_id=? AND round_id=? AND state='prepared' LIMIT 1''',
|
||||
WHERE task_id=? AND round_id=? AND state IN ('prepared','pending_save') LIMIT 1''',
|
||||
(task_id,current['round_id'])).fetchone())
|
||||
|
||||
def unresolved_jobs(self,task_id,rows):
|
||||
@@ -438,7 +448,7 @@ class ExecutionJournal:
|
||||
current=self._round(task_id)
|
||||
if current is None:return ()
|
||||
keys={record[0] for record in self.store._connection.execute('''SELECT operation_key
|
||||
FROM execution_operations WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain')''',
|
||||
FROM execution_operations WHERE task_id=? AND round_id=? AND state IN ('prepared','uncertain','pending_save')''',
|
||||
(task_id,current['round_id']))}
|
||||
return tuple(row['job_id'] for row in rows if _hash(command_identity(row)) in keys)
|
||||
|
||||
@@ -506,6 +516,8 @@ class ExecutionJournal:
|
||||
elif record['state']=='failed':
|
||||
restored[row['job_id']]=tuple(json.loads(record['receipt_json'])[name] for name in ('native_state','code','message'))
|
||||
continue
|
||||
elif record['state']=='pending_save':
|
||||
raise TaskCenterError('quick_run_uncertain','工程替换尚待统一保存回读,禁止重放;请先核验工程提交记录。')
|
||||
outcome=latest.get(row['job_id'])
|
||||
if outcome and outcome[0] in _TERMINAL:
|
||||
record=connection.execute('''SELECT * FROM execution_operations
|
||||
@@ -546,6 +558,186 @@ class ExecutionJournal:
|
||||
WHERE task_id=? AND round_id=? AND operation_key=?''',(identifier,generation,task_id,current['round_id'],key))
|
||||
return JournalAttempt(task_id,current['round_id'],key,identifier,generation,row['job_id'])
|
||||
|
||||
def stage_project_result(self, attempt, row, *, session_token, root, proof):
|
||||
"""Retain an applied operation without granting a saved receipt or retry."""
|
||||
from .execution_proof import proof_intact
|
||||
if not session_token or not proof_intact(proof):
|
||||
raise TaskCenterError('quick_run_uncertain', '工程暂存证据不完整。')
|
||||
with self.store.transaction():
|
||||
connection = self.store._connection
|
||||
record = connection.execute('''SELECT * FROM execution_operations
|
||||
WHERE task_id=? AND round_id=? AND operation_key=?''',
|
||||
(attempt.task_id, attempt.round_id, attempt.operation_key)).fetchone()
|
||||
if (record is None or record['state'] != 'prepared' or record['attempt_id'] != attempt.attempt_id
|
||||
or record['generation'] != attempt.generation or _hash(command_identity(row)) != attempt.operation_key):
|
||||
raise TaskCenterError('quick_run_uncertain', '工程暂存结果不属于当前执行声明。')
|
||||
project = command_identity(row)['project_path']
|
||||
commit = connection.execute('''SELECT * FROM execution_project_commits
|
||||
WHERE task_id=? AND round_id=? AND session_token=?''',
|
||||
(attempt.task_id, attempt.round_id, session_token)).fetchone()
|
||||
if commit and (commit['state'] != 'pending_save' or commit['project_path'] != project
|
||||
or commit['evidence_root'] != str(canonical_path(root))):
|
||||
raise TaskCenterError('quick_run_uncertain', '工程提交身份发生变化。')
|
||||
jobs = json.loads(commit['jobs_json']) if commit else []
|
||||
if row['job_id'] in jobs:
|
||||
raise TaskCenterError('quick_run_uncertain', '工程暂存操作重复。')
|
||||
jobs.append(row['job_id'])
|
||||
connection.execute('''INSERT INTO execution_project_commits VALUES (?,?,?,?,?, 'pending_save',?,NULL)
|
||||
ON CONFLICT(task_id,round_id,session_token) DO UPDATE SET jobs_json=excluded.jobs_json''',
|
||||
(attempt.task_id, attempt.round_id, session_token, project, str(canonical_path(root)), _json(jobs)))
|
||||
receipt = dict(native_state='pending_save', code='session_staged', message='替换已暂存,等待工程统一保存回读。',
|
||||
job_id=row['job_id'], session_token=session_token, row=row, proof=proof,
|
||||
operation_attempt=[attempt.operation_key, attempt.attempt_id, attempt.generation])
|
||||
connection.execute('''UPDATE execution_operations SET state='pending_save',receipt_json=?
|
||||
WHERE task_id=? AND round_id=? AND operation_key=?''',
|
||||
(_json(receipt), attempt.task_id, attempt.round_id, attempt.operation_key))
|
||||
|
||||
def pending_project_commits(self, task_id):
|
||||
with self.store._lock:
|
||||
current = self._round(task_id)
|
||||
if not current:return ()
|
||||
return tuple(dict(row) for row in self.store._connection.execute('''SELECT * FROM execution_project_commits
|
||||
WHERE task_id=? AND round_id=? AND state='pending_save' ''', (task_id, current['round_id'])))
|
||||
|
||||
def project_commit_records(self, task_id):
|
||||
with self.store._lock:
|
||||
current = self._round(task_id)
|
||||
return tuple(dict(row) for row in self.store._connection.execute('''SELECT * FROM execution_project_commits
|
||||
WHERE task_id=? AND round_id=?''', (task_id, current['round_id']))) if current else ()
|
||||
|
||||
def project_commit_attempts(self, task_id, session_token, *, require_recorded=False):
|
||||
with self.store._lock:
|
||||
current = self._round(task_id)
|
||||
if not current:return {}
|
||||
result = {}
|
||||
for row in self.store._connection.execute('''SELECT * FROM execution_operations
|
||||
WHERE task_id=? AND round_id=?''', (task_id, current['round_id'])):
|
||||
receipt = json.loads(row['receipt_json'] or '{}')
|
||||
token = receipt.get('session_token') or receipt.get('proof', {}).get('project_commit', {}).get('session_token')
|
||||
if token == session_token:
|
||||
actual = [row['operation_key'], row['attempt_id'], row['generation']]
|
||||
original = receipt.get('operation_attempt') or receipt.get('proof', {}).get('project_commit', {}).get('operation_attempts', {}).get(receipt['job_id'])
|
||||
if original != actual and (require_recorded or original is not None):
|
||||
raise TaskCenterError('quick_run_uncertain', '原工程暂存尝试与当前执行记录不同。')
|
||||
result[receipt['job_id']] = actual
|
||||
return result
|
||||
|
||||
def project_commit_recovery_attempts(self, task_id):
|
||||
"""Include actual attempts outside the frozen command rows as well."""
|
||||
with self.store._lock:
|
||||
current = self._round(task_id)
|
||||
if not current:return ()
|
||||
return tuple([row['operation_key'], row['attempt_id'], row['generation']]
|
||||
for row in self.store._connection.execute('''SELECT operation_key,attempt_id,generation
|
||||
FROM execution_operations WHERE task_id=? AND round_id=?
|
||||
AND (attempt_id IS NOT NULL OR generation>0)''', (task_id, current['round_id'])))
|
||||
|
||||
def bind_project_commit_proof(self, task_id, session_token, proof):
|
||||
with self.store.transaction():
|
||||
current = self._round(task_id)
|
||||
self.store._connection.execute('''UPDATE execution_project_commits SET proof_json=?
|
||||
WHERE task_id=? AND round_id=? AND session_token=? AND state='pending_save' AND proof_json IS NULL''',
|
||||
(_json(proof), task_id, current['round_id'], session_token))
|
||||
|
||||
def promote_project_commit_proof(self, task_id, session_token, expected_proof, verified_proof):
|
||||
"""Attach native verification to the same saved attempt without replacing its source binding."""
|
||||
from .execution_proof import proof_intact
|
||||
with self.store.transaction():
|
||||
current = self._round(task_id)
|
||||
record = self.store._connection.execute('''SELECT * FROM execution_project_commits
|
||||
WHERE task_id=? AND round_id=? AND session_token=?''',
|
||||
(task_id, current['round_id'], session_token)).fetchone() if current else None
|
||||
original = {key: value for key, value in expected_proof.items() if key not in {'files', 'outcomes', 'saved_netlist'}}
|
||||
if (record is None or record['state'] != 'pending_save'
|
||||
or expected_proof.get('session_token') != session_token
|
||||
or expected_proof.get('validation_required') != 'saved_netlist_v1'
|
||||
or verified_proof.get('saved_netlist', {}).get('status') != 'verified'
|
||||
or any(verified_proof.get(key) != value for key, value in original.items())
|
||||
or any(item not in verified_proof.get('files', []) for item in expected_proof.get('files', []))
|
||||
or not proof_intact(verified_proof)
|
||||
or json.loads(record['proof_json'] or 'null') not in (expected_proof, verified_proof)):
|
||||
raise TaskCenterError('quick_run_uncertain', '原工程保存证明发生变化,不能覆盖核验结果。')
|
||||
self.store._connection.execute('''UPDATE execution_project_commits SET proof_json=?
|
||||
WHERE task_id=? AND round_id=? AND session_token=?''',
|
||||
(_json(verified_proof), task_id, current['round_id'], session_token))
|
||||
|
||||
def staged_project_rows(self, task_id, session_token):
|
||||
with self.store._lock:
|
||||
current = self._round(task_id)
|
||||
if not current:return ()
|
||||
records = self.store._connection.execute('''SELECT receipt_json FROM execution_operations
|
||||
WHERE task_id=? AND round_id=? AND state='pending_save' ''', (task_id, current['round_id']))
|
||||
return tuple(receipt for record in records
|
||||
if (receipt := json.loads(record[0])).get('session_token') == session_token)
|
||||
|
||||
def commit_project_results(self, task_id, session_token, proofs, commit_proof):
|
||||
"""Commit a whole staged prefix and its history in one SQL transaction."""
|
||||
from .execution_proof import proof_intact
|
||||
statuses = []
|
||||
with self.store.transaction():
|
||||
connection = self.store._connection
|
||||
current = self._round(task_id)
|
||||
commit = connection.execute('''SELECT * FROM execution_project_commits
|
||||
WHERE task_id=? AND round_id=? AND session_token=?''', (task_id, current['round_id'], session_token)).fetchone() if current else None
|
||||
if commit is None:
|
||||
raise TaskCenterError('quick_run_uncertain', '缺少原工程暂存提交记录。')
|
||||
if commit['state'] == 'verified':return ()
|
||||
jobs = json.loads(commit['jobs_json'])
|
||||
if (commit_proof.get('session_token') != session_token
|
||||
or str(canonical_path(commit_proof.get('project_path', ''))).casefold() != commit['project_path']
|
||||
or commit_proof.get('jobs') != jobs or set(proofs) != set(jobs)
|
||||
or not commit_proof.get('files') or not proof_intact(commit_proof)
|
||||
or any(not proof.get('files') or not proof_intact(proof) for proof in proofs.values())):
|
||||
raise TaskCenterError('quick_run_uncertain', '工程保存回读证明未覆盖全部暂存项。')
|
||||
saved_protocol = commit_proof.get('marker_state') == 'project_saved' or commit_proof.get('validation_required') == 'saved_netlist_v1'
|
||||
saved = commit_proof.get('saved_netlist', {})
|
||||
if saved_protocol and (commit_proof.get('validation_required') != 'saved_netlist_v1'
|
||||
or saved.get('status') != 'verified' or set(saved.get('jobs', {})) != set(jobs)
|
||||
or set(commit_proof.get('native_outcomes', {})) != set(jobs)
|
||||
or any(state not in {'pending_save', 'no_match'} for state in commit_proof['native_outcomes'].values())
|
||||
or commit_proof.get('outcomes') != {job: 'saved_ok' if state == 'pending_save' else state
|
||||
for job, state in commit_proof['native_outcomes'].items()}
|
||||
or not saved.get('files') or not proof_intact(saved)):
|
||||
raise TaskCenterError('quick_run_uncertain', '工程已保存,尚未完成原生数据核验。')
|
||||
records = connection.execute('''SELECT * FROM execution_operations
|
||||
WHERE task_id=? AND round_id=? AND state='pending_save' ''', (task_id, current['round_id'])).fetchall()
|
||||
records = [record for record in records if json.loads(record['receipt_json']).get('session_token') == session_token]
|
||||
if {json.loads(record['receipt_json'])['job_id'] for record in records} != set(jobs):
|
||||
raise TaskCenterError('quick_run_uncertain', '工程暂存前缀与提交证明不一致。')
|
||||
for record in records:
|
||||
receipt = json.loads(record['receipt_json'])
|
||||
job = receipt['job_id']
|
||||
proof = dict(proofs[job])
|
||||
if proof.get('native_attempt') != receipt.get('proof', {}).get('native_attempt'):
|
||||
raise TaskCenterError('quick_run_uncertain', '保存回读与原暂存原生尝试不一致。')
|
||||
if saved_protocol:
|
||||
proof['project_commit'] = dict(session_token=session_token, project_path=commit_proof['project_path'],
|
||||
validation_required='saved_netlist_v1',
|
||||
operation_attempts={job: [record['operation_key'], record['attempt_id'], record['generation']]},
|
||||
saved_netlist={key: saved[key] for key in ('status', 'export_path', 'export_sha256')})
|
||||
command = json.loads(record['command_json'])
|
||||
proof['baseline'] = dict(project_path=command['project_path'], schematic_name=command['schematic_name'],
|
||||
sheet_number=command['sheet_number'], validation_mode='saved_netlist_v1',
|
||||
state='N\tsaved_netlist_v1\t' + saved['export_sha256'])
|
||||
shared = [{'path': saved['export_path'], 'sha256': saved['export_sha256']}]
|
||||
if saved.get('verification_path'):
|
||||
shared.append({'path': saved['verification_path'], 'sha256': saved['verification_sha256']})
|
||||
proof['files'] = list(proof['files']) + [item for item in shared if item not in proof['files']]
|
||||
else:
|
||||
proof['project_commit'] = commit_proof
|
||||
proof['files'] = list(proof['files']) + list(commit_proof['files'])
|
||||
attempt = JournalAttempt(task_id, current['round_id'], record['operation_key'],
|
||||
record['attempt_id'], record['generation'], job)
|
||||
connection.execute('''UPDATE execution_operations SET state='prepared'
|
||||
WHERE task_id=? AND round_id=? AND operation_key=?''', (task_id, current['round_id'], record['operation_key']))
|
||||
native_state = commit_proof.get('outcomes', {}).get(job, 'saved_ok')
|
||||
message = '工程统一保存,原生数据核验通过。' if saved_protocol else '工程统一保存、关闭重开及回读校验通过。'
|
||||
self.finish(attempt, native_state, 'project_committed', message, proof=proof)
|
||||
statuses.append((job, native_state, 'project_committed', message))
|
||||
connection.execute('''UPDATE execution_project_commits SET state='verified',proof_json=?
|
||||
WHERE task_id=? AND round_id=? AND session_token=?''', (_json(commit_proof), task_id, current['round_id'], session_token))
|
||||
return tuple(statuses)
|
||||
|
||||
def finish(self,attempt,native_state,code,message,*,progress=None,proof=None):
|
||||
if native_state not in _TERMINAL:raise TaskCenterError('quick_run_uncertain','不能提交未知的执行结果。')
|
||||
state=({'saved_ok':'verified','no_match':'no_match','restored_ok':'restored',
|
||||
@@ -682,7 +874,7 @@ class ExecutionJournal:
|
||||
from .execution_proof import proof_intact
|
||||
if not proof_intact(receipt.get('proof',{})):return True
|
||||
if not outcome or tuple(outcome[:2])!=(receipt['native_state'],receipt['code']):return True
|
||||
elif record['state']=='uncertain':return True
|
||||
elif record['state'] in {'uncertain','pending_save'}:return True
|
||||
elif record['state']=='prepared' and (not outcome or outcome[0] not in _TERMINAL):return True
|
||||
return False
|
||||
|
||||
|
||||
@@ -17,14 +17,35 @@ def proof_intact(proof):
|
||||
except (OSError,ValueError,KeyError,TypeError):return False
|
||||
|
||||
|
||||
def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
|
||||
def capture_result(data_root,root,row,outcome,*,strict=False,persist=True,project_commit=None):
|
||||
from .quick_run import _decode_vbs_field, _safe_run_token
|
||||
from .native_helper import read_snapshot, require_resumable_transaction, snapshot_target_cell
|
||||
from .native_helper import read_snapshot, require_resumable_transaction, snapshot_target_cell, saved_netlist_snapshot
|
||||
state,code,_=outcome
|
||||
proof={'files':[]}
|
||||
paths=[]
|
||||
try:
|
||||
_safe_run_token(row['job_id'])
|
||||
snapshot = internal_path(root/f'snapshot-{row["job_id"]}.tsv')
|
||||
saved_protocol = ((snapshot.is_file() and saved_netlist_snapshot(snapshot))
|
||||
or bool(project_commit and project_commit.get('validation_required') == 'saved_netlist_v1'))
|
||||
if saved_protocol:
|
||||
proof['validation_required'] = 'saved_netlist_v1'
|
||||
if state in {'saved_ok', 'no_match'}:
|
||||
saved = (project_commit or {}).get('saved_netlist', {})
|
||||
if saved.get('status') != 'verified' or row['job_id'] not in saved.get('jobs', {}):
|
||||
raise ValueError('saved native netlist proof is required')
|
||||
proof['saved_netlist'] = {key:saved[key] for key in ('status','export_path','export_sha256')}
|
||||
proof['value_readback'] = saved['jobs'][row['job_id']]
|
||||
if state == 'no_match' and persist:
|
||||
destination = internal_path(data_root/'execution-proof'/uuid4().hex)
|
||||
destination.mkdir(parents=True,exist_ok=False)
|
||||
target = destination/f'no-match-{row["job_id"]}.json'
|
||||
reference = dict(command=dict(row),session_token=project_commit['session_token'],
|
||||
**{key:saved[key] for key in ('export_path','export_sha256','verification_path','verification_sha256')})
|
||||
data = json.dumps(reference,ensure_ascii=False,sort_keys=True).encode('utf-8')
|
||||
with target.open('xb') as stream:
|
||||
stream.write(data);stream.flush();os.fsync(stream.fileno())
|
||||
proof['files'].append({'path':str(canonical_path(target)),'sha256':hashlib.sha256(data).hexdigest()})
|
||||
if row.get('execution_unit') == 'inventory_empty':
|
||||
from .component_scope import validate_empty_command
|
||||
if state != 'batch_skipped' or not validate_empty_command(root, row):
|
||||
@@ -33,13 +54,13 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
|
||||
checked_manifest = manifest.read_bytes()
|
||||
paths.append(manifest)
|
||||
proof['inventory_empty'] = True
|
||||
if strict and state in {'saved_ok','no_match'} and code not in {'batch_closed','session_verified'}:
|
||||
if strict and state in {'saved_ok','no_match'} and code not in {'batch_closed','session_verified','project_committed'}:
|
||||
raise ValueError('save/close was not verified')
|
||||
checked={}
|
||||
if row.get('execution_unit') == 'inventory_empty':
|
||||
checked[manifest] = checked_manifest
|
||||
baseline=internal_path(root/f'post-state-{row["job_id"]}.tsv')
|
||||
if baseline.is_file():
|
||||
if baseline.is_file() and not saved_protocol:
|
||||
checked[baseline]=baseline.read_bytes()
|
||||
with baseline.open(encoding='utf-8-sig',newline='') as stream:
|
||||
rows=list(csv.DictReader(stream,delimiter='\t'))
|
||||
@@ -53,9 +74,9 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
|
||||
proof['baseline']={'project_path':str(canonical_path(row['project_path'])).casefold(),
|
||||
'schematic_name':actual['schematic_name'],'sheet_number':actual['sheet_number'],'state':actual['semantic_state']}
|
||||
paths.append(baseline)
|
||||
elif strict and state in {'saved_ok','no_match','restored_ok','recovery_readback'}:
|
||||
elif strict and not saved_protocol and state in {'saved_ok','no_match','restored_ok','recovery_readback'}:
|
||||
raise ValueError('missing verified semantic state')
|
||||
if state=='saved_ok' or (state=='restored_ok' and code in {'recovery_verified','recovery_saved_verified'}):
|
||||
if state=='saved_ok' or (state=='pending_save' and code == 'session_staged') or (state=='restored_ok' and code in {'recovery_verified','recovery_saved_verified'}):
|
||||
snapshot=internal_path(root/f'snapshot-{row["job_id"]}.tsv')
|
||||
report_path=Path(str(snapshot)+'.native.json')
|
||||
if strict or snapshot.is_file() or report_path.is_file():
|
||||
@@ -74,7 +95,8 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
|
||||
raise ValueError('snapshot command changed')
|
||||
attempt=report.get('batch_attempt')
|
||||
if strict:
|
||||
if report.get('stage')!='native_replace_verified' or report.get('ok') is not True:
|
||||
required_stage = 'native_replace_applied' if saved_protocol else 'native_replace_verified'
|
||||
if report.get('stage')!=required_stage or report.get('ok') is not True:
|
||||
raise ValueError('native result is not verified')
|
||||
if (not isinstance(attempt,dict) or attempt.get('job_id')!=row['job_id'] or
|
||||
attempt.get('task_id')!=row['task_id'] or
|
||||
@@ -84,6 +106,10 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
|
||||
raise ValueError('native attempt differs from this job')
|
||||
proof['native_attempt']=attempt
|
||||
paths.extend((snapshot,report_path))
|
||||
if saved_protocol:
|
||||
writer = internal_path(root/f'writer-expectations-{row["job_id"]}.tsv')
|
||||
checked[writer] = writer.read_bytes()
|
||||
paths.append(writer)
|
||||
if any(path.read_bytes()!=data for path,data in checked.items()):
|
||||
raise ValueError('execution evidence changed during receipt binding')
|
||||
if persist and paths:
|
||||
@@ -101,6 +127,57 @@ def capture_result(data_root,root,row,outcome,*,strict=False,persist=True):
|
||||
raise TaskCenterError('quick_run_uncertain','原生回执、尝试身份或工程语义基线无法核验,已停止以避免重复替换。') from error
|
||||
|
||||
|
||||
def capture_project_commit(data_root, root, commit):
|
||||
"""Bind a durable project marker to the exact staged prefix before SQL commit."""
|
||||
from .quick_run import _decode_vbs_field
|
||||
try:
|
||||
path = internal_path(Path(root) / f'project-commit-{commit["session_token"]}.tsv')
|
||||
data = path.read_bytes()
|
||||
lines = data.decode('utf-8-sig').splitlines()
|
||||
token, project, state = lines[0].split('\t')
|
||||
outcomes = [line.split('\t') for line in lines[1:] if line]
|
||||
jobs = json.loads(commit['jobs_json'])
|
||||
saved_only = state == 'project_saved'
|
||||
allowed = {'pending_save', 'no_match'} if saved_only else {'saved_ok', 'no_match'}
|
||||
if (token != commit['session_token'] or state not in {'project_verified', 'project_saved'}
|
||||
or str(canonical_path(_decode_vbs_field(project))).casefold() != commit['project_path']
|
||||
or [outcome[0] for outcome in outcomes] != jobs
|
||||
or any(len(outcome) != 2 or outcome[1] not in allowed for outcome in outcomes)):
|
||||
raise ValueError('project commit does not cover staged prefix')
|
||||
destination = internal_path(data_root / 'execution-proof' / uuid4().hex)
|
||||
destination.mkdir(parents=True, exist_ok=False)
|
||||
target = destination / path.name
|
||||
with target.open('xb') as stream:
|
||||
stream.write(data); stream.flush(); os.fsync(stream.fileno())
|
||||
if path.read_bytes() != data:
|
||||
raise ValueError('project marker changed during binding')
|
||||
proof = dict(session_token=token, project_path=commit['project_path'], jobs=jobs,
|
||||
outcomes=dict(outcomes), files=[dict(path=str(canonical_path(target)), sha256=hashlib.sha256(data).hexdigest())])
|
||||
if saved_only:
|
||||
proof.update(marker_state=state, saved_only=True, validation_required='saved_netlist_v1', native_outcomes=dict(outcomes))
|
||||
return proof
|
||||
except (OSError, UnicodeError, ValueError, KeyError, TypeError, IndexError) as error:
|
||||
raise TaskCenterError('quick_run_uncertain', '工程统一保存或回读尚未得到完整证明,保留全部暂存项待核验。') from error
|
||||
|
||||
|
||||
def bind_project_commit_source(proof, data_root, root, project, *, stop_event=None, release_mentor=None):
|
||||
"""Capture saved bytes only at the completed native-close callback."""
|
||||
from .saved_source import bind_sources
|
||||
return bind_sources(proof, data_root, root, project,
|
||||
stop_event=stop_event, release_mentor=release_mentor)
|
||||
|
||||
|
||||
def validate_project_commit_source(proof, root):
|
||||
from .project_session_health import _source_digest
|
||||
try:
|
||||
return (proof.get('source_sha256') and _source_digest(proof) == proof['source_sha256']
|
||||
and proof.get('command_sha256') == hashlib.sha256(internal_path(root / 'command.tsv').read_bytes()).hexdigest()
|
||||
and bool(proof.get('saved_sources')) and all(
|
||||
hashlib.sha256(internal_path(source).read_bytes()).hexdigest() == digest
|
||||
for source, digest in proof['saved_sources'].items()) and proof_intact(proof))
|
||||
except (OSError, KeyError, TypeError, ValueError):return False
|
||||
|
||||
|
||||
def capture_recovery(data_root,audit,row,outcome,*,strict=False):
|
||||
"""The recovery audit proves readback; it never asserts a second replacement."""
|
||||
worker=audit/'worker'
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
"""Read-only Restart Manager observations; never shut down registered applications."""
|
||||
import ctypes
|
||||
from ctypes import wintypes as w
|
||||
import os
|
||||
|
||||
from .path_access import strip_extended_prefix
|
||||
|
||||
|
||||
def locking_processes(path):
|
||||
result = dict(method='windows_restart_manager', processes=[], complete=False)
|
||||
if os.name != 'nt':
|
||||
return dict(result, error='unsupported platform')
|
||||
class UniqueProcess(ctypes.Structure):
|
||||
_fields_ = [('pid', w.DWORD), ('started', w.FILETIME)]
|
||||
class ProcessInfo(ctypes.Structure):
|
||||
_fields_ = [('process', UniqueProcess), ('name', w.WCHAR*256), ('service', w.WCHAR*64),
|
||||
('type', ctypes.c_int), ('status', w.ULONG), ('session', w.DWORD), ('restartable', w.BOOL)]
|
||||
session = w.DWORD()
|
||||
api = ctypes.WinDLL('rstrtmgr')
|
||||
api.RmStartSession.argtypes = [ctypes.POINTER(w.DWORD), w.DWORD, w.LPWSTR]
|
||||
api.RmRegisterResources.argtypes = [w.DWORD, w.UINT, ctypes.POINTER(w.LPCWSTR), w.UINT,
|
||||
ctypes.POINTER(UniqueProcess), w.UINT, ctypes.POINTER(w.LPCWSTR)]
|
||||
api.RmGetList.argtypes = [w.DWORD, ctypes.POINTER(w.UINT), ctypes.POINTER(w.UINT),
|
||||
ctypes.POINTER(ProcessInfo), ctypes.POINTER(w.DWORD)]
|
||||
api.RmEndSession.argtypes = [w.DWORD]
|
||||
started = False
|
||||
try:
|
||||
code = api.RmStartSession(ctypes.byref(session), 0, ctypes.create_unicode_buffer(33))
|
||||
if code: raise OSError(code, 'RmStartSession')
|
||||
started = True
|
||||
files = (w.LPCWSTR*1)(strip_extended_prefix(path))
|
||||
code = api.RmRegisterResources(session, 1, files, 0, None, 0, None)
|
||||
if code: raise OSError(code, 'RmRegisterResources')
|
||||
needed, count, reboot = w.UINT(), w.UINT(), w.DWORD()
|
||||
code = api.RmGetList(session, ctypes.byref(needed), ctypes.byref(count), None, ctypes.byref(reboot))
|
||||
for _ in range(3):
|
||||
if code != 234: break # ERROR_MORE_DATA
|
||||
count.value = needed.value
|
||||
items = (ProcessInfo*count.value)()
|
||||
code = api.RmGetList(session, ctypes.byref(needed), ctypes.byref(count), items, ctypes.byref(reboot))
|
||||
if code == 0:
|
||||
result['processes'] = [dict(pid=item.process.pid, name=item.name,
|
||||
creation_filetime=(item.process.started.dwHighDateTime << 32) | item.process.started.dwLowDateTime)
|
||||
for item in items[:count.value]]
|
||||
if code: raise OSError(code, 'RmGetList')
|
||||
result['complete'] = True # Complete RM result, not an exhaustive kernel-handle scan.
|
||||
except Exception as error:
|
||||
result['error'] = str(error)
|
||||
finally:
|
||||
if started: api.RmEndSession(session)
|
||||
return result
|
||||
@@ -99,7 +99,7 @@ class IsolatedCadSession:
|
||||
self.root = canonical_path(root)
|
||||
self.token = uuid4().hex
|
||||
self.borrowed_desktop = desktop is not None
|
||||
self.desktop = desktop if desktop is not None else PrivateDesktop('dx-' + self.token, enabled=True)
|
||||
self.desktop = desktop if desktop is not None else PrivateDesktop('dx-' + self.token, enabled=True, managed=True)
|
||||
self.cad = None
|
||||
self.binding = None
|
||||
self.started = time.monotonic()
|
||||
|
||||
@@ -87,6 +87,7 @@ def build_startup_info(desktop_name: str) -> StartupInfo:
|
||||
|
||||
|
||||
class _Win32Api:
|
||||
atomic_job_launch = True
|
||||
def __init__(self) -> None:
|
||||
try:
|
||||
import win32api, win32con, win32event, win32job, win32process, win32service
|
||||
@@ -101,6 +102,12 @@ class _Win32Api:
|
||||
def create_desktop(self, name: str):
|
||||
return self.service.CreateDesktop(name, 0, self.con.GENERIC_ALL, None)
|
||||
|
||||
def validate_desktop_owner(self, handle, sid):
|
||||
import win32security
|
||||
security = win32security.GetUserObjectSecurity(handle, win32security.OWNER_SECURITY_INFORMATION)
|
||||
if win32security.ConvertSidToStringSid(security.GetSecurityDescriptorOwner()) != sid:
|
||||
raise TaskCenterError('desktop_slot_busy', '同名桌面的所有者不匹配,未启动工作进程。')
|
||||
|
||||
def confirm_desktop_released(self, name: str):
|
||||
started = time.monotonic()
|
||||
# USER references can outlive the owned processes briefly. Allow a
|
||||
@@ -166,7 +173,7 @@ class _Win32Api:
|
||||
return windows
|
||||
|
||||
def create_process(self, command: Sequence[str], desktop: str, cwd: Path | None, env: Mapping[str, str] | None,
|
||||
*, stdout=None, stderr=None):
|
||||
*, stdout=None, stderr=None, job_name=None):
|
||||
# Windows venv python.exe redirects to a second process. Launch the
|
||||
# interpreter directly with its venv identity, so the handle we own is
|
||||
# also the process that admits the request and creates desktop windows.
|
||||
@@ -176,18 +183,23 @@ class _Win32Api:
|
||||
command = (sys._base_executable, *command[1:])
|
||||
env = dict(os.environ if env is None else env)
|
||||
env['__PYVENV_LAUNCHER__'] = sys.executable
|
||||
startup = self.process.STARTUPINFO()
|
||||
startup.lpDesktop = desktop
|
||||
flags = self.con.CREATE_NO_WINDOW | self.con.CREATE_SUSPENDED
|
||||
job = self.job.CreateJobObject(None, "")
|
||||
try:
|
||||
startup = self.process.STARTUPINFO()
|
||||
startup.lpDesktop = desktop
|
||||
flags = self.con.CREATE_NO_WINDOW | self.con.CREATE_SUSPENDED
|
||||
job = self.job.CreateJobObject(None, job_name or "")
|
||||
except Exception as error:
|
||||
error.desktop_launch_cleanup_proven = True # CreateProcess has not been called.
|
||||
raise
|
||||
process = None
|
||||
try:
|
||||
info = self.job.QueryInformationJobObject(job, self.job.JobObjectExtendedLimitInformation)
|
||||
info["BasicLimitInformation"]["LimitFlags"] |= self.job.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
|
||||
self.job.SetInformationJobObject(job, self.job.JobObjectExtendedLimitInformation, info)
|
||||
streams = (None, None)
|
||||
if stdout is not None or stderr is not None:
|
||||
(handle, thread, pid, _), streams = _create_redirected_process(command, desktop, cwd, env, flags, stdout, stderr)
|
||||
if stdout is not None or stderr is not None or job_name:
|
||||
(handle, thread, pid, _), streams = _create_redirected_process(command, desktop, cwd, env, flags, stdout, stderr,
|
||||
**({'job': job} if job_name else {}))
|
||||
else:
|
||||
handle, thread, pid, _ = self.process.CreateProcess(
|
||||
str(command[0]), subprocess.list2cmdline([str(item) for item in command]),
|
||||
@@ -197,7 +209,7 @@ class _Win32Api:
|
||||
open_process=getattr(self, 'open_process', None), process_ids=self.process_ids)
|
||||
process.stdout, process.stderr = streams
|
||||
process.thread = thread
|
||||
self.job.AssignProcessToJobObject(job, handle)
|
||||
if not job_name: self.job.AssignProcessToJobObject(job, handle)
|
||||
process.job_assigned = True
|
||||
if not self.job.IsProcessInJob(handle, job):
|
||||
raise TaskCenterError('private_desktop', 'Owned process Job membership was not confirmed')
|
||||
@@ -218,6 +230,7 @@ class _Win32Api:
|
||||
self._pending_cleanup = pending = []
|
||||
pending.append(process if process is not None else job)
|
||||
raise TaskCenterError('private_desktop', f'launch failed: {error}; cleanup failed: {cleanup_error}') from error
|
||||
error.desktop_launch_cleanup_proven = True
|
||||
raise
|
||||
return process
|
||||
|
||||
@@ -260,7 +273,7 @@ def _stop_and_wait(process):
|
||||
return errors
|
||||
|
||||
|
||||
def _create_redirected_process(command, desktop, cwd, env, flags, stdout, stderr):
|
||||
def _create_redirected_process(command, desktop, cwd, env, flags, stdout, stderr, *, job=None):
|
||||
"""Use an explicit inheritance list; never inherit unrelated parent handles."""
|
||||
import msvcrt
|
||||
import pywintypes
|
||||
@@ -323,13 +336,19 @@ def _create_redirected_process(command, desktop, cwd, env, flags, stdout, stderr
|
||||
handles = list(dict.fromkeys((stdin_handle, stdout_handle, stderr_handle)))
|
||||
inherited = (wintypes.HANDLE * len(handles))(*handles)
|
||||
size = ctypes.c_size_t()
|
||||
initialize(None, 1, 0, ctypes.byref(size))
|
||||
count = 2 if job is not None else 1
|
||||
initialize(None, count, 0, ctypes.byref(size))
|
||||
attributes = ctypes.create_string_buffer(size.value)
|
||||
if not initialize(attributes, 1, 0, ctypes.byref(size)):
|
||||
if not initialize(attributes, count, 0, ctypes.byref(size)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
if not update(attributes, 0, 0x20002, inherited, ctypes.sizeof(inherited), None, None):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if job is not None:
|
||||
jobs = (wintypes.HANDLE * 1)(int(job))
|
||||
# Assign before the initial thread exists, including owner-loss windows.
|
||||
if not update(attributes, 0, 0x2000D, jobs, ctypes.sizeof(jobs), None, None):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
startup = ExtendedStartup()
|
||||
startup.StartupInfo.cb = ctypes.sizeof(startup)
|
||||
startup.StartupInfo.lpDesktop = desktop
|
||||
@@ -570,8 +589,13 @@ class _Win32Process:
|
||||
class PrivateDesktop:
|
||||
"""Own a private Windows desktop and processes started on it."""
|
||||
|
||||
def __init__(self, name: str, *, enabled: bool = False, api=None) -> None:
|
||||
self.name = name
|
||||
def __init__(self, name: str, *, enabled: bool = False, api=None, managed=False, slot=None) -> None:
|
||||
if managed and slot is None and os.name == 'nt':
|
||||
from .desktop_slot import DesktopSlot
|
||||
slot = DesktopSlot()
|
||||
self._slot = slot
|
||||
self._slot_acquired = False
|
||||
self.name = slot.name if slot is not None else name
|
||||
self.enabled = enabled
|
||||
self._handle = None
|
||||
self._processes: list[subprocess.Popen[str]] = []
|
||||
@@ -583,7 +607,16 @@ class PrivateDesktop:
|
||||
def __enter__(self) -> "PrivateDesktop":
|
||||
if not self.enabled:
|
||||
raise TaskCenterError("private_desktop_opt_in", "private desktop backend requires explicit opt-in")
|
||||
self._create()
|
||||
if self._slot is not None:
|
||||
self._slot.acquire()
|
||||
self._slot_acquired = True
|
||||
try:
|
||||
self._create()
|
||||
if self._slot is not None and hasattr(self._api, 'validate_desktop_owner'):
|
||||
self._api.validate_desktop_owner(self._handle, self._slot.context['sid'])
|
||||
except Exception:
|
||||
self.close()
|
||||
raise
|
||||
if self._handle is None:
|
||||
raise TaskCenterError("private_desktop", "private desktop could not be created")
|
||||
return self
|
||||
@@ -638,8 +671,23 @@ class PrivateDesktop:
|
||||
if self._api is None:
|
||||
raise TaskCenterError("private_desktop", "private desktop process API is unavailable")
|
||||
options = {} if stdout is None and stderr is None else {'stdout': stdout, 'stderr': stderr}
|
||||
process = self._api.create_process(command, build_startup_info(self.name).lpDesktop, cwd, env, **options)
|
||||
job = self._slot.prepare_launch(atomic_job=bool(getattr(self._api, 'atomic_job_launch', False))) if self._slot is not None else None
|
||||
if job is not None: options['job_name'] = job
|
||||
try:
|
||||
process = self._api.create_process(command, build_startup_info(self.name).lpDesktop, cwd, env, **options)
|
||||
except Exception as error:
|
||||
if job is not None and getattr(error, 'desktop_launch_cleanup_proven', False):
|
||||
self._slot.launch_closed(job)
|
||||
raise
|
||||
self._processes.append(process)
|
||||
if job is not None:
|
||||
from .batch_resources import process_identity
|
||||
try:
|
||||
self._slot.launched(job, process_identity(process))
|
||||
except Exception:
|
||||
process.close()
|
||||
self._slot.launch_closed(job)
|
||||
raise
|
||||
self._process_history.append({'pid': getattr(process, 'pid', None),
|
||||
'owner_pid': os.getpid(), 'owner_thread_id': get_native_id(),
|
||||
'started_time_ns': time.time_ns(), 'process_handle': _native_handle(getattr(process, 'handle', None)),
|
||||
@@ -666,6 +714,12 @@ class PrivateDesktop:
|
||||
self._remember_process_cleanup(process, forgotten=True)
|
||||
|
||||
def close(self) -> None:
|
||||
if self._slot is not None and not self._slot_acquired:
|
||||
self.cleanup_record = dict(desktop=self.name, desktop_not_acquired=True,
|
||||
desktop_handle_closed=True, owned_cleanup_complete=True, processes=[], errors=[])
|
||||
return
|
||||
if self._slot is not None and self._slot.receipt is not None:
|
||||
return # A past lease must never observe or close a newer owner.
|
||||
errors, remaining = [], []
|
||||
self.cleanup_record = {'desktop': self.name, 'processes': []}
|
||||
for process in reversed(self._processes):
|
||||
@@ -689,6 +743,7 @@ class PrivateDesktop:
|
||||
close_pending()
|
||||
except Exception as error:
|
||||
errors.append(error)
|
||||
self.cleanup_record['owned_cleanup_complete'] = not errors and not self._processes
|
||||
try:
|
||||
self.release_desktop()
|
||||
except Exception as error:
|
||||
@@ -703,6 +758,10 @@ class PrivateDesktop:
|
||||
errors = []
|
||||
if self.cleanup_record is None:
|
||||
self.cleanup_record = {'desktop': self.name, 'processes': []}
|
||||
if self._slot is not None and self._slot.receipt is not None:
|
||||
return
|
||||
if self._slot is not None and self.cleanup_record.get('owned_cleanup_complete') is not True:
|
||||
raise TaskCenterError('desktop_waiting_for_mentor', '自有进程和句柄尚未收尾,桌面租用保持。')
|
||||
from .read_recovery import process_tree_exited
|
||||
if any(not process_tree_exited(process) for process in self._processes):
|
||||
self.cleanup_record.update(desktop_release_state='waiting_for_mentor',
|
||||
@@ -720,20 +779,35 @@ class PrivateDesktop:
|
||||
event['error'] = str(error)
|
||||
self._desktop_handle_history.append(event)
|
||||
confirm = getattr(self._api, 'confirm_desktop_released', None)
|
||||
if confirm and self._handle is None:
|
||||
confirmation = None
|
||||
if self._slot is not None and self._handle is None and not errors:
|
||||
try:
|
||||
disappeared = not self._slot.backend.desktop_exists(self.name)
|
||||
self._slot.release(desktop_disappeared=disappeared)
|
||||
confirmation = dict(desktop=self.name, released=disappeared,
|
||||
method='managed_lease_return', elapsed_seconds=0.0)
|
||||
except Exception as error:
|
||||
errors.append(error)
|
||||
if self._slot.receipt is not None:
|
||||
self.cleanup_record['desktop_lease'] = self._slot.receipt
|
||||
elif confirm and self._handle is None:
|
||||
try:
|
||||
confirm(self.name)
|
||||
except Exception as error:
|
||||
errors.append(error)
|
||||
confirmation = getattr(self._api, 'desktop_release_check', None)
|
||||
self.cleanup_record.update(desktop_handle_closed=self._handle is None,
|
||||
desktop_release_state='failed' if errors else 'released',
|
||||
desktop_confirmation=getattr(self._api, 'desktop_release_check', None),
|
||||
desktop_confirmation=confirmation,
|
||||
desktop_lease_pending=self._slot is not None and self._slot.receipt is None,
|
||||
desktop_handle_history=deepcopy(self._desktop_handle_history),
|
||||
process_history=deepcopy(self._process_history),
|
||||
diagnostic_scope={'external_handle_holders': 'unknown',
|
||||
'thread_desktops': 'unknown unless observed through residual windows; no exhaustive holder scan',
|
||||
'empty_windows_prove_no_holders': False},
|
||||
errors=[str(error) for error in errors])
|
||||
if confirmation and self._slot is not None and not confirmation['released']:
|
||||
self.cleanup_record['desktop_release_state'] = 'reusable'
|
||||
if errors:
|
||||
self.cleanup_record['owned_resource_diagnostics'] = self._owned_resource_diagnostics()
|
||||
_raise_cleanup_errors(errors)
|
||||
|
||||
@@ -15,6 +15,7 @@ class TaskStatus(str, Enum):
|
||||
PAUSED = "paused"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
BLOCKED = "blocked"
|
||||
DELETED = "deleted"
|
||||
|
||||
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
"""Native current-sheet replacement adapter for the existing QuickPartReplace runner."""
|
||||
import argparse
|
||||
from collections import Counter
|
||||
from contextlib import contextmanager
|
||||
from contextvars import ContextVar
|
||||
from functools import wraps
|
||||
import faulthandler
|
||||
import ctypes
|
||||
import csv
|
||||
@@ -15,6 +18,46 @@ import time
|
||||
import traceback
|
||||
|
||||
|
||||
_SNAPSHOT_REFERENCE_KEYS = ContextVar('snapshot_reference_keys', default={})
|
||||
|
||||
|
||||
@contextmanager
|
||||
def snapshot_identity_scope(snapshot):
|
||||
"""Keep original receipt keys while resolving live COM objects by reference."""
|
||||
rows = [line.split('\t') for line in Path(snapshot).read_text(encoding='utf-8-sig').splitlines()]
|
||||
references, owners, spelling = {}, {}, {}
|
||||
for row in rows:
|
||||
if len(row) >= 3 and row[0] in {'C', 'U'}:
|
||||
uid, ref = decode(row[1]), decode(row[2]).strip().casefold()
|
||||
spelling[ref] = decode(row[2]).strip()
|
||||
transform = tuple(decode(value) for value in row[3:7]) if row[0] == 'C' and len(row) == 8 else None
|
||||
references.setdefault(ref, []).append((uid, transform))
|
||||
owners.setdefault(uid, set()).add(ref)
|
||||
for row in rows:
|
||||
if len(row) == 3 and row[0] == 'G':
|
||||
headers = [line.split('\t') for line in decode(row[2]).splitlines() if line.startswith('C\t')]
|
||||
if len(headers) == 1 and len(headers[0]) == 8:
|
||||
header = [decode(value) for value in headers[0]]
|
||||
ref = header[2].strip().casefold()
|
||||
references[ref] = [(uid, tuple(header[3:7]) if uid == header[1] else transform)
|
||||
for uid, transform in references.get(ref, ())]
|
||||
keys = {ref: [(uid if len(owners[uid]) == 1 else '@REFDES|' + spelling[ref], transform)
|
||||
for uid, transform in values] for ref, values in references.items() if ref}
|
||||
token = _SNAPSHOT_REFERENCE_KEYS.set(keys)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_SNAPSHOT_REFERENCE_KEYS.reset(token)
|
||||
|
||||
|
||||
def _bound_snapshot_identity(action):
|
||||
@wraps(action)
|
||||
def bound(snapshot, *args, **kwargs):
|
||||
with snapshot_identity_scope(snapshot):
|
||||
return action(snapshot, *args, **kwargs)
|
||||
return bound
|
||||
|
||||
|
||||
def snapshot_attempt(snapshot):
|
||||
"""Bind batch evidence to the actual snapshot and its confirmed queue row."""
|
||||
from .batch_attempt import _bound_attempt
|
||||
@@ -99,6 +142,22 @@ def read_snapshot(path):
|
||||
return metadata[0], components
|
||||
|
||||
|
||||
def saved_netlist_snapshot(snapshot):
|
||||
rows = [line.split('\t') for line in Path(snapshot).read_text(encoding='utf-8-sig').splitlines() if line]
|
||||
markers = [row for row in rows if row[0] == 'X']
|
||||
if ['X', 'saved_netlist_v1'] not in markers:
|
||||
return False
|
||||
if (markers != [['X', 'saved_netlist_v1']] or any(row[0] not in {'M', 'C', 'A', 'X', 'L'} for row in rows)
|
||||
or not any(row[0] == 'C' for row in rows)
|
||||
or any(len(row) != 8 or row[7] != 'NETLIST' for row in rows if row[0] == 'C')):
|
||||
raise ValueError('Invalid saved native data snapshot scope')
|
||||
_, components = read_snapshot(snapshot)
|
||||
selected = {row[1] for row in components}
|
||||
if any(len(row) != 12 or decode(row[1]) not in selected for row in rows if row[0] == 'A'):
|
||||
raise ValueError('Invalid saved native data snapshot attributes')
|
||||
return True
|
||||
|
||||
|
||||
def snapshot_value_mode(metadata):
|
||||
return metadata[5] if len(metadata) >= 6 else "part_label"
|
||||
|
||||
@@ -392,16 +451,35 @@ def components_on_sheet(app, api):
|
||||
|
||||
|
||||
def _component_uid(component):
|
||||
value = getattr(component, 'UID', None)
|
||||
refdes = str(getattr(component, 'Refdes', '') or '').strip()
|
||||
identities = _SNAPSHOT_REFERENCE_KEYS.get().get(refdes.casefold(), ())
|
||||
if len(identities) == 1:
|
||||
return identities[0][0]
|
||||
if identities:
|
||||
matches = [uid for uid, transform in identities if transform and _same_unit(component, transform)]
|
||||
if len(matches) != 1:
|
||||
raise ValueError('Snapshot physical unit association is missing or ambiguous')
|
||||
return matches[0]
|
||||
try:
|
||||
value = getattr(component, 'UID', None)
|
||||
except Exception:
|
||||
value = None
|
||||
uid = '' if value is None else str(value).strip()
|
||||
if uid:
|
||||
return uid
|
||||
refdes = str(getattr(component, 'Refdes', '') or '').strip()
|
||||
if not refdes:
|
||||
raise ValueError("Component UID and Refdes are empty; cannot safely identify the target")
|
||||
return '@REFDES|' + refdes
|
||||
|
||||
|
||||
def _same_unit(component, transform):
|
||||
point = component.GetLocation()
|
||||
return (abs(float(point.X) - float(transform[0])) <= 10
|
||||
and abs(float(point.Y) - float(transform[1])) <= 10
|
||||
and float(component.Orientation) == float(transform[2])
|
||||
and float(component.Scale) == float(transform[3]))
|
||||
|
||||
|
||||
def check_active_context(app, metadata):
|
||||
from .cad_identity import verify_application, project_file_path
|
||||
verify_application(app)
|
||||
@@ -418,18 +496,24 @@ def check_context(app, api, metadata, records, require_selection=True):
|
||||
by_uid = {_component_uid(component): component for component in components}
|
||||
if len(by_uid) != len(components):
|
||||
raise ValueError("Duplicate component UID on sheet")
|
||||
for record in records:
|
||||
component = by_uid.get(record[1])
|
||||
if component is None or str(component.Refdes).strip().casefold() != record[2].strip().casefold():
|
||||
raise ValueError('Snapshot reference differs from the selected live component')
|
||||
if len(metadata) == 10:
|
||||
selectors = _target_selectors(metadata[9])
|
||||
if set(selectors) != wanted:
|
||||
selected_refs = {ref.strip().casefold() for ref, _ in selectors.values()}
|
||||
if selected_refs != {record[2].strip().casefold() for record in records}:
|
||||
raise ValueError("Target selector differs from snapshot components")
|
||||
for uid, (refdes, old_value) in selectors.items():
|
||||
component = by_uid.get(uid)
|
||||
if (component is None or str(component.Refdes) != refdes or
|
||||
attribute_value(component, "Part Number", required=True).casefold() != metadata[2].casefold() or
|
||||
attribute_value(component, "Value") != old_value):
|
||||
candidates = [component for component in components if str(component.Refdes).strip().casefold() == refdes.strip().casefold()]
|
||||
if (not candidates or any(_component_uid(component) not in wanted
|
||||
or attribute_value(component, "Part Number", required=True).casefold() != metadata[2].casefold()
|
||||
or attribute_value(component, "Value") != old_value for component in candidates)):
|
||||
raise ValueError("Selected target changed since inventory")
|
||||
else:
|
||||
actual = {_component_uid(component) for component in components if attribute_value(component, "Part Number", required=True).casefold() == metadata[2].casefold()}
|
||||
scope = [component for component in components if _component_uid(component) in wanted] if all(row[7] == "NETLIST" for row in records) else components
|
||||
actual = {_component_uid(component) for component in scope if attribute_value(component, "Part Number", required=True).casefold() == metadata[2].casefold()}
|
||||
if actual != wanted:
|
||||
raise ValueError(f"Source batch changed since snapshot: expected={sorted(wanted)!r} actual={sorted(actual)!r}")
|
||||
if require_selection:
|
||||
@@ -545,6 +629,21 @@ def select_snapshot_batch(app, api, metadata, records):
|
||||
return components
|
||||
|
||||
|
||||
def applied_replacement_uids(components, records, target):
|
||||
"""Associate just the written targets while the native operation completes."""
|
||||
wanted = {row[1] for row in records}
|
||||
targets = {_component_uid(component): component for component in components
|
||||
if _component_uid(component) in wanted}
|
||||
if len(targets) != len(records):
|
||||
return None
|
||||
for record in records:
|
||||
component = targets[record[1]]
|
||||
if (str(component.Refdes).strip().casefold() != record[2].strip().casefold()
|
||||
or attribute_value(component, 'Part Number', required=True).casefold() != target.casefold()):
|
||||
return None
|
||||
return [record[1] for record in records]
|
||||
|
||||
|
||||
def existing_replacement_uids(components, records, target, binding, require_part_label=True):
|
||||
matches = replacement_matches(components, records, target, require_part_label,
|
||||
library_row=binding['library_row'])
|
||||
@@ -553,7 +652,7 @@ def existing_replacement_uids(components, records, target, binding, require_part
|
||||
return matches
|
||||
|
||||
|
||||
def promote_pin_numbers(app, api, replacement_uids, slot_count=1, on_progress=None):
|
||||
def promote_pin_numbers(app, api, replacement_uids, slot_count=1, on_progress=None, *, verify_pins=True):
|
||||
def progress(phase):
|
||||
if on_progress:
|
||||
on_progress(phase)
|
||||
@@ -567,6 +666,8 @@ def promote_pin_numbers(app, api, replacement_uids, slot_count=1, on_progress=No
|
||||
for slot in range(1, int(slot_count) + 1):
|
||||
progress(f'promote_slot_{slot}')
|
||||
app.ActiveView.Block.PromoteSymbolNumbers(1, slot)
|
||||
if not verify_pins:
|
||||
return []
|
||||
progress('query_promoted_pins')
|
||||
promoted = []
|
||||
for item in app.ActiveView.Query(4096, 0):
|
||||
@@ -591,20 +692,26 @@ def _promotion_phase(snapshot, report, phase):
|
||||
|
||||
|
||||
def replacement_matches(components, records, target, require_part_label, *, library_row=None):
|
||||
current_by_uid = {_component_uid(component): component for component in components}
|
||||
if len(current_by_uid) != len(components):
|
||||
raise ValueError('Duplicate component UID on sheet')
|
||||
current_by_ref = {}
|
||||
for component in components:
|
||||
current_by_ref.setdefault(str(component.Refdes).strip().casefold(), []).append(component)
|
||||
expected_name = expected_label = None
|
||||
if library_row is not None:
|
||||
if (len(library_row) < 6 or str(library_row[1]).casefold() != target.casefold()
|
||||
or not str(library_row[4]).strip()):
|
||||
raise ValueError('Recorded target library identity is incomplete or inconsistent')
|
||||
expected_name, expected_label = str(library_row[4]), str(library_row[5])
|
||||
matches = []
|
||||
matches, used = [], set()
|
||||
for record in records:
|
||||
candidate = current_by_uid.get(record[1])
|
||||
if candidate is None:
|
||||
raise ValueError('Native replacement lost component UID identity')
|
||||
candidates = current_by_ref.get(str(record[2]).strip().casefold(), [])
|
||||
if len(candidates) > 1:
|
||||
candidates = [candidate for candidate in candidates if len(record) >= 7 and _same_unit(candidate, record[3:7])]
|
||||
if len(candidates) != 1:
|
||||
raise ValueError('Native replacement reference is missing or ambiguous')
|
||||
candidate = candidates[0]
|
||||
if id(candidate) in used:
|
||||
raise ValueError('Duplicate physical unit association')
|
||||
used.add(id(candidate))
|
||||
if attribute_value(candidate, "Part Number", required=True).casefold() != target.casefold():
|
||||
return None
|
||||
identity = str(candidate.Refdes).strip() or f"UID={candidate.UID}"
|
||||
@@ -622,7 +729,7 @@ def replacement_matches(components, records, target, require_part_label, *, libr
|
||||
if expected_name is not None and (part_name != expected_name
|
||||
or (expected_label and part_label != expected_label)):
|
||||
raise ValueError(f'Native replacement does not match the selected target library identity; uid={candidate.UID}')
|
||||
matches.append(_component_uid(candidate))
|
||||
matches.append(record[1])
|
||||
if len(set(matches)) != len(records):
|
||||
raise ValueError("Duplicate replacement association")
|
||||
return matches
|
||||
@@ -651,6 +758,7 @@ def verify_replacement_readback(snapshot, components, metadata, records, before_
|
||||
return verified
|
||||
|
||||
|
||||
@_bound_snapshot_identity
|
||||
def run_native(snapshot, report):
|
||||
import win32api
|
||||
import win32con
|
||||
@@ -659,6 +767,7 @@ def run_native(snapshot, report):
|
||||
from pywinauto import Desktop
|
||||
|
||||
metadata, records = read_snapshot(snapshot)
|
||||
native_mode = saved_netlist_snapshot(snapshot)
|
||||
target_cell = snapshot_target_cell(snapshot, required=False)
|
||||
require_part_label = snapshot_value_mode(metadata) == "part_label"
|
||||
app, api = attach()
|
||||
@@ -666,7 +775,7 @@ def run_native(snapshot, report):
|
||||
cell_proof = target_cell_catalog(app, snapshot, metadata, target_cell) if target_cell is not None else None
|
||||
components = components_on_sheet(app, api)
|
||||
wanted = {row[1] for row in records}
|
||||
if len(metadata) == 10:
|
||||
if len(metadata) == 10 and not native_mode:
|
||||
verify_unselected_snapshot(snapshot, components, wanted, api=api)
|
||||
source_uids = {
|
||||
_component_uid(component)
|
||||
@@ -677,6 +786,11 @@ def run_native(snapshot, report):
|
||||
binding = require_resumable_transaction(snapshot, report, metadata, records)
|
||||
if binding.get("cell_catalog") != cell_proof:
|
||||
raise ValueError("Current Cell catalog differs from recorded native transaction")
|
||||
if native_mode:
|
||||
if report.get("stage") == "native_replace_applied" and report.get("pin_promotion_phase") == "promotion_complete":
|
||||
report["already_replaced"] = True
|
||||
return
|
||||
raise ValueError("Native replacement may already have been applied; verify saved native data before continuing")
|
||||
report["replaced_uids"] = existing_replacement_uids(
|
||||
components, records, metadata[3], binding, require_part_label=require_part_label
|
||||
)
|
||||
@@ -696,7 +810,7 @@ def run_native(snapshot, report):
|
||||
return
|
||||
components = select_snapshot_batch(app, api, metadata, records)
|
||||
before_uids = {_component_uid(component) for component in components}
|
||||
untouched = None if len(metadata) == 10 else {_component_uid(component): (
|
||||
untouched = None if native_mode or len(metadata) == 10 else {_component_uid(component): (
|
||||
str(component.Refdes),
|
||||
*(attribute_value(component, name) for name in ("Part Number", "Value", "Part Name", "Part Label"))
|
||||
) for component in components if _component_uid(component) not in wanted}
|
||||
@@ -854,13 +968,17 @@ def run_native(snapshot, report):
|
||||
report["stage"] = "native_replace_started"
|
||||
save_report(snapshot, report)
|
||||
mark_snapshot_mutation(snapshot)
|
||||
click(dlg_item(dialog, 1695))
|
||||
from .operation_diagnostics import operation_span
|
||||
with operation_span(Path(snapshot).parent, 'native_replace', item_id=Path(snapshot).stem):
|
||||
click(dlg_item(dialog, 1695))
|
||||
|
||||
replacement_poll_count = 0
|
||||
|
||||
def selected_replaced():
|
||||
nonlocal replacement_poll_count
|
||||
replacement_poll_count += 1
|
||||
if native_mode:
|
||||
return applied_replacement_uids(components_on_sheet(app, api), records, metadata[3])
|
||||
selected = [api.IVdComp(component._oleobj_) for component in app.ActiveView.Query(128, 1)]
|
||||
if len(selected) == len(records):
|
||||
matches = replacement_matches(selected, records, metadata[3], require_part_label, library_row=columns)
|
||||
@@ -880,7 +998,7 @@ def run_native(snapshot, report):
|
||||
# Preserve the last observed identities without retrying a mutation.
|
||||
report['replacement_timeout'] = {'polls': replacement_poll_count}
|
||||
try:
|
||||
current = components_on_sheet(app, api)
|
||||
current = [] if native_mode else components_on_sheet(app, api)
|
||||
refs = {record[2] for record in records}
|
||||
report['replacement_timeout'].update(
|
||||
component_count=len(current),
|
||||
@@ -895,16 +1013,17 @@ def run_native(snapshot, report):
|
||||
except Exception as error:
|
||||
report['replacement_timeout']['target_control_error'] = str(error)
|
||||
raise
|
||||
current = components_on_sheet(app, api)
|
||||
verify_replacement_readback(snapshot, current, metadata, records, before_uids,
|
||||
untouched, require_part_label, columns, report['replaced_uids'], api=api)
|
||||
if not native_mode:
|
||||
current = components_on_sheet(app, api)
|
||||
verify_replacement_readback(snapshot, current, metadata, records, before_uids,
|
||||
untouched, require_part_label, columns, report['replaced_uids'], api=api)
|
||||
report["stage"] = "native_replace_applied"
|
||||
save_report(snapshot, report)
|
||||
report["stage"] = "pin_promotion_started"
|
||||
save_report(snapshot, report)
|
||||
report["promoted_pins"] = promote_pin_numbers(app, api, report["replaced_uids"], slot_count,
|
||||
lambda phase: _promotion_phase(snapshot, report, phase))
|
||||
report["stage"] = "native_replace_verified"
|
||||
lambda phase: _promotion_phase(snapshot, report, phase), verify_pins=not native_mode)
|
||||
report["stage"] = "native_replace_applied" if native_mode else "native_replace_verified"
|
||||
finally:
|
||||
if dialog and win32gui.IsWindow(dialog):
|
||||
close_button = dlg_item(dialog, 1696)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from collections import Counter
|
||||
from collections import Counter, defaultdict
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
@@ -38,38 +38,45 @@ def _require_report(text, mode):
|
||||
|
||||
|
||||
def _effective_properties(text):
|
||||
from .native_netlist import _is_pcb
|
||||
_require_report(text, 'FLAT BLOCKS')
|
||||
if len(re.findall(r'^Config .+', text, re.MULTILINE)) != 1:
|
||||
raise NativeInventoryUnsupported('展开报告必须具有唯一配置')
|
||||
result, attributes, paths = {}, {}, []
|
||||
inconsistent_refs = set()
|
||||
symbol = None
|
||||
symbol = symbol_type = None
|
||||
|
||||
def finish():
|
||||
if symbol is None:
|
||||
return
|
||||
uid, ref = symbol
|
||||
if not _UID.fullmatch(uid):
|
||||
if attributes.get('Part Number'):
|
||||
raise NativeInventoryUnsupported('展开报告的物料缺少唯一 UID')
|
||||
if symbol_type is not None and not _is_pcb({'symbol': symbol_type, 'attributes': attributes}):
|
||||
return
|
||||
if uid in result or paths != ['']:
|
||||
raise NativeInventoryUnsupported('展开报告含重复或嵌套实例,需由 CAD 核验')
|
||||
uid, ref, page = symbol
|
||||
if not ref:
|
||||
if attributes.get('Part Number') and symbol_type and _is_pcb({'symbol': symbol_type,'attributes':attributes}):
|
||||
raise NativeInventoryUnsupported('展开报告的请求物料缺少位号')
|
||||
return
|
||||
unit = symbol_type.split(' type ', 1)[0] if symbol_type else ''
|
||||
key = (ref.strip().casefold(), page.strip(), unit)
|
||||
if key in result or paths != ['']:
|
||||
raise NativeInventoryUnsupported('展开报告含重复位号或嵌套实例,无法明确定位')
|
||||
if ref != attributes.get('Ref Designator', ''):
|
||||
inconsistent_refs.add(uid)
|
||||
result[uid] = dict(attributes)
|
||||
inconsistent_refs.add(key)
|
||||
result[key] = (uid, dict(attributes))
|
||||
if len(result) > 100000:
|
||||
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
|
||||
|
||||
for line in text.splitlines():
|
||||
if line.startswith(' Symbol '):
|
||||
finish()
|
||||
match = re.fullmatch(r' Symbol (.+?)\tref des ([^\t]*)\tpage [^\t]*', line)
|
||||
match = re.fullmatch(r' Symbol (.*?)\tref des ([^\t]*)\tpage ([^\t]*)', line)
|
||||
if match is None:
|
||||
raise NativeInventoryUnsupported('展开报告含不明确的器件身份')
|
||||
symbol, attributes, paths = match.groups(), {}, []
|
||||
symbol, attributes, paths, symbol_type = match.groups(), {}, [], None
|
||||
elif symbol is not None and line.startswith('\tPath '):
|
||||
paths.append(line[6:])
|
||||
elif symbol is not None and line.startswith('\tSymbol name '):
|
||||
symbol_type = line[13:]
|
||||
elif symbol is not None:
|
||||
match = _PROPERTY.fullmatch(line)
|
||||
if match:
|
||||
@@ -82,39 +89,61 @@ def _effective_properties(text):
|
||||
|
||||
|
||||
def _parse_scope(text, requested, effective, inconsistent_refs):
|
||||
from .native_netlist import _is_pcb
|
||||
blocks, sheets, targets = set(), set(), set()
|
||||
seen_uids = set()
|
||||
block, symbol, attributes = '', None, {}
|
||||
physical_targets = {}
|
||||
by_ref = defaultdict(list)
|
||||
uid_counts = Counter(uid for uid, _ in effective.values())
|
||||
unit_counts = Counter(key[:2] for key in effective)
|
||||
for key in effective:
|
||||
by_ref[key[0]].append(key)
|
||||
block, symbol, attributes, symbol_type = '', None, {}, None
|
||||
def finish():
|
||||
if symbol is None:
|
||||
return
|
||||
uid, sheet = symbol
|
||||
if not _UID.fullmatch(uid):
|
||||
if attributes.get('Part Number'):
|
||||
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少唯一身份')
|
||||
if symbol_type is not None and not _is_pcb({'symbol': symbol_type, 'attributes': attributes}):
|
||||
return
|
||||
if uid in seen_uids or uid not in effective:
|
||||
raise NativeInventoryUnsupported('分层与展开报告的 UID 不完整或重复,需由 CAD 核验')
|
||||
seen_uids.add(uid)
|
||||
values = effective[uid]
|
||||
uid, sheet = symbol
|
||||
ref = attributes.get('Ref Designator','')
|
||||
if not ref:
|
||||
if attributes.get('Part Number','').casefold() in requested:
|
||||
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
|
||||
return
|
||||
unit = symbol_type.split(' type ', 1)[0] if symbol_type else ''
|
||||
candidates = [key for key in by_ref[ref.strip().casefold()]
|
||||
if (not key[1] or key[1] == sheet.strip())
|
||||
and (not key[2] or not unit or key[2] == unit)]
|
||||
if len(candidates) != 1 or candidates[0] in seen_uids:
|
||||
raise NativeInventoryUnsupported('分层与展开报告的器件范围不完整或位号重复')
|
||||
key = candidates[0]
|
||||
seen_uids.add(key)
|
||||
flat_uid, values = effective[key]
|
||||
if attributes.get('Part Number') and 'Part Number' not in values:
|
||||
raise NativeInventoryUnsupported('展开报告缺少物料生效属性')
|
||||
if 'Value' in attributes and 'Value' not in values:
|
||||
raise NativeInventoryUnsupported('展开报告缺少 Value 生效属性')
|
||||
if values.get('Part Number', '').casefold() in requested:
|
||||
# A library reference override on an unrelated part cannot affect
|
||||
# the requested identities. Still validate every UID/config/path,
|
||||
# the requested identities. Still validate every reference/config/path,
|
||||
# and require CAD whenever the ambiguous component is requested.
|
||||
if uid in inconsistent_refs:
|
||||
if key in inconsistent_refs:
|
||||
raise NativeInventoryUnsupported('展开报告的位号与属性不一致')
|
||||
ref, part = values.get('Ref Designator', ''), values['Part Number']
|
||||
if not ref or 'Ref Designator' not in attributes:
|
||||
raise NativeInventoryUnsupported('全工程报告中的请求物料缺少位号')
|
||||
target = (block, sheet, uid, ref, part, values.get('Value', ''),
|
||||
multiunit = unit_counts[key[:2]] > 1
|
||||
identity = uid if not multiunit and _UID.fullmatch(uid) and uid == flat_uid and uid_counts[uid] == 1 else '@REFDES|' + ref
|
||||
target = (block, sheet, identity, ref, part, values.get('Value', ''),
|
||||
values.get('Part Label'), values.get('Cell Name'))
|
||||
if target in targets:
|
||||
physical_key = block, sheet, ref
|
||||
prior = physical_targets.get(physical_key)
|
||||
if prior and prior[3:] != target[3:]:
|
||||
raise NativeInventoryUnsupported('Conflicting effective attributes across physical symbol units')
|
||||
if target in targets and not multiunit:
|
||||
raise NativeInventoryUnsupported('全工程报告含重复器件身份')
|
||||
targets.add(target)
|
||||
physical_targets[physical_key] = target
|
||||
if len(targets) > 100000:
|
||||
raise NativeInventoryUnsupported('原生器件数量超过读取上限')
|
||||
for line in text.splitlines():
|
||||
@@ -124,14 +153,16 @@ def _parse_scope(text, requested, effective, inconsistent_refs):
|
||||
if not name or name in blocks:
|
||||
raise NativeInventoryUnsupported('全工程报告含不明确的原理图身份')
|
||||
blocks.add(name)
|
||||
block, symbol, attributes = name, None, {}
|
||||
block, symbol, attributes, symbol_type = name, None, {}, None
|
||||
elif line.startswith(' Symbol '):
|
||||
finish()
|
||||
match = re.fullmatch(r' Symbol (.+?)\tpage (.+)', line)
|
||||
match = re.fullmatch(r' Symbol (.*?)\tpage (.+)', line)
|
||||
if not block or match is None:
|
||||
raise NativeInventoryUnsupported('全工程报告含不明确的图页身份')
|
||||
symbol, attributes = match.groups(), {}
|
||||
symbol, attributes, symbol_type = match.groups(), {}, None
|
||||
sheets.add((block, symbol[1]))
|
||||
elif symbol is not None and line.startswith('\tSymbol name '):
|
||||
symbol_type = line[13:]
|
||||
elif symbol is not None and line.startswith('\tProp '):
|
||||
# Pin properties have two tabs and must never override a component.
|
||||
match = _PROPERTY.fullmatch(line)
|
||||
@@ -149,49 +180,46 @@ def _parse_scope(text, requested, effective, inconsistent_refs):
|
||||
|
||||
|
||||
def _hierarchical_root_discovery(project, report_path, effective_path, requested, expected_root):
|
||||
# ComponentTarget has no instance path: certify only globally unique root selections.
|
||||
from .native_netlist import _parse_ascii, _check_hierarchy_graph, _is_pcb
|
||||
text = read_native_text(report_path)
|
||||
roots = re.findall(r'^Block ([^\t\r\n]+)\tRoot$', text, re.MULTILINE)
|
||||
root = expected_root or (roots[0].rstrip() if len(roots) == 1 else None)
|
||||
try:
|
||||
components, _ = _parse_ascii(effective_path, require_connections=False)
|
||||
_check_hierarchy_graph(report_path, {'components': components}, root)
|
||||
_, _, expanded = _check_hierarchy_graph(report_path, {'components': components}, root, include_rows=True)
|
||||
except ValueError as error:
|
||||
raise NativeInventoryUnsupported(str(error)) from error
|
||||
physical = {uid: row for uid, row in components.items() if _is_pcb(row)}
|
||||
source_uids = Counter(row.get('source_uid', uid) for uid, row in components.items())
|
||||
references = Counter(row['refdes'] for row in physical.values())
|
||||
scopes, sheets, block = {}, set(), None
|
||||
for line in text.splitlines():
|
||||
if line.startswith('Block '):
|
||||
block = line[6:].split('\t', 1)[0].rstrip()
|
||||
elif line.startswith(' Symbol '):
|
||||
match = re.fullmatch(r' Symbol (.+?)\tpage (.+)', line)
|
||||
if match is None or block is None:
|
||||
raise NativeInventoryUnsupported('分层报告图页身份不明确')
|
||||
scopes[(block, match[1])] = match[2]
|
||||
sheets.add((block, match[2]))
|
||||
targets = []
|
||||
for uid, row in physical.items():
|
||||
sheets = {(row['source_schematic'], row['source_page']) for row in expanded}
|
||||
counts = Counter(row.get('source_uid', uid) for uid, row in components.items())
|
||||
targets = {}
|
||||
for scope in expanded:
|
||||
row = components[scope['graph_key']]
|
||||
attributes = row['attributes']
|
||||
if attributes.get('Part Number', '').casefold() not in requested:
|
||||
if not _is_pcb(row) or attributes.get('Part Number', '').casefold() not in requested:
|
||||
continue
|
||||
if row.get('instance_path') or source_uids[uid] != 1 or references[row['refdes']] != 1:
|
||||
raise NativeInventoryUnsupported('请求物料含子实例或非唯一 UID/位号,当前写入选择器不能安全定位')
|
||||
if 'source_refdes' in row or row.get('ambiguous_attributes'):
|
||||
raise NativeInventoryUnsupported('请求物料的位号或生效属性存在歧义')
|
||||
sheet = scopes.get((root, uid))
|
||||
if not sheet:
|
||||
raise NativeInventoryUnsupported('请求物料缺少唯一根块图页')
|
||||
targets.append(ComponentTarget(Path(project), root, sheet, uid, row['refdes'],
|
||||
attributes['Part Number'], attributes.get('Value', ''), attributes.get('Part Label'),
|
||||
attributes.get('Cell Name')))
|
||||
if not scope.get('refdes'):
|
||||
raise NativeInventoryUnsupported('Hierarchical physical component is missing its reference observation')
|
||||
if 'Value' in scope['attributes'] and 'Value' not in attributes:
|
||||
raise NativeInventoryUnsupported('Effective physical component is missing its Value observation')
|
||||
if row.get('ambiguous_attributes'):
|
||||
raise NativeInventoryUnsupported('Requested physical component has ambiguous effective attributes')
|
||||
source_uid = row.get('source_uid', scope['source_uid'])
|
||||
identity = source_uid if (_UID.fullmatch(source_uid) and counts[source_uid] == 1
|
||||
and source_uid == scope['source_uid']) else '@REFDES|' + row['refdes']
|
||||
target = ComponentTarget(Path(project), scope['source_schematic'], scope['source_page'],
|
||||
identity, row['refdes'], attributes['Part Number'], attributes.get('Value', ''),
|
||||
attributes.get('Part Label'), attributes.get('Cell Name'))
|
||||
previous = targets.get(target.key)
|
||||
if previous and (previous.old_part_number, previous.old_value, previous.actual_part_label,
|
||||
previous.actual_cell) != (target.old_part_number, target.old_value,
|
||||
target.actual_part_label, target.actual_cell):
|
||||
raise NativeInventoryUnsupported('Conflicting effective attributes across physical symbol units')
|
||||
targets[target.key] = target
|
||||
matches = {sheet: set() for sheet in sheets}
|
||||
for target in targets:
|
||||
for target in targets.values():
|
||||
matches[(target.schematic_name, target.sheet_number)].add(target.old_part_number.casefold())
|
||||
return ProjectDiscovery(tuple(sorted(sheets)), {key: frozenset(value) for key, value in matches.items()},
|
||||
tuple(sorted(targets, key=lambda target: target.key)), complete_sheet_manifest=False)
|
||||
tuple(sorted(targets.values(), key=lambda target: target.key)), complete_sheet_manifest=False)
|
||||
|
||||
|
||||
def read_ascii_discovery(project, report_path, old_part_numbers, *, expected_root=None,
|
||||
|
||||
@@ -107,10 +107,11 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
|
||||
symbol = pin = net = conn = None
|
||||
for line in text.splitlines():
|
||||
if line.startswith(' Symbol '):
|
||||
match = re.fullmatch(r' Symbol (.+?)\tref des ([^\t]*)\tpage [^\t]*', line)
|
||||
match = re.fullmatch(r' Symbol (.*?)\tref des ([^\t]*)\tpage ([^\t]*)', line)
|
||||
if not match:
|
||||
raise ValueError('Invalid effective symbol identity')
|
||||
symbol = {'uid': match[1], 'refdes': match[2], 'attributes': {}, 'pins': {}, 'paths': []}
|
||||
symbol = {'uid': match[1], 'refdes': match[2], 'source_page': match[3].strip(),
|
||||
'attributes': {}, 'pins': {}, 'paths': []}
|
||||
symbols.append(symbol)
|
||||
pin = None
|
||||
elif symbol is not None:
|
||||
@@ -155,7 +156,7 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
|
||||
raise ValueError('Duplicate or empty net')
|
||||
expected[net] = Counter()
|
||||
elif line.startswith('\t\tConn '):
|
||||
match = re.fullmatch(r'\t\tConn (.+?) pin (.+?) page .*', line)
|
||||
match = re.fullmatch(r'\t\tConn (.*?) pin (.+?) page .*', line)
|
||||
if net is None or not match:
|
||||
raise ValueError('Invalid net endpoint')
|
||||
conn = match.groups()
|
||||
@@ -168,12 +169,15 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
|
||||
conn = None
|
||||
if conn is not None:
|
||||
raise ValueError('Endpoint missing path')
|
||||
raw_counts = Counter((row['uid'], tuple(row['paths'])) for row in symbols)
|
||||
endpoint_aliases = defaultdict(list)
|
||||
for symbol in symbols:
|
||||
uid = symbol.pop('uid')
|
||||
paths = symbol.pop('paths')
|
||||
if len(paths) != 1:
|
||||
raise ValueError('Symbol instance path is missing or ambiguous')
|
||||
source_uid = uid
|
||||
symbol['source_uid'] = source_uid
|
||||
uid = _instance_identity(uid, paths[0])
|
||||
attrs = symbol['attributes']
|
||||
if paths[0]:
|
||||
@@ -181,8 +185,16 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
|
||||
symbol['refdes'] = attrs.get('Ref Designator', '')
|
||||
elif attrs.get('Ref Designator') and attrs['Ref Designator'] != symbol['refdes']:
|
||||
symbol['source_refdes'], symbol['refdes'] = symbol['refdes'], attrs['Ref Designator']
|
||||
if _is_pcb(symbol) and (not re.fullmatch(r'\$\d+I\d+', source_uid) or not symbol['refdes'] or attrs.get('Ref Designator') != symbol['refdes']):
|
||||
# Port names can contain '/', so retain the reported path even at the root.
|
||||
symbol['instance_path'] = '' if paths[0] == 'root' else paths[0]
|
||||
if _is_pcb(symbol) and (not symbol['refdes'] or attrs.get('Ref Designator') != symbol['refdes']):
|
||||
raise ValueError(f'PCB component identity is ambiguous: {uid!r} / {symbol["refdes"]!r}')
|
||||
raw_identity = uid
|
||||
if _is_pcb(symbol) and (not re.fullmatch(r'\$\d+I\d+', source_uid)
|
||||
or raw_counts[(source_uid, tuple(paths))] > 1):
|
||||
slot = re.match(r'\S+\.(\d+) type ', symbol.get('symbol', ''))
|
||||
uid = _instance_identity('@REFDES|' + symbol['refdes'] + ('|slot=' + slot[1] if slot else ''), paths[0])
|
||||
endpoint_aliases[raw_identity].append(uid)
|
||||
symbol['other_attributes'] = sorted(symbol.get('other_attributes', []))
|
||||
if uid in components:
|
||||
old = components[uid]
|
||||
@@ -203,6 +215,16 @@ def _parse_ascii(path, *, hierarchy_boundaries=(), require_connections=True):
|
||||
for net in row['nets']:
|
||||
nets[net][(uid, pin)] += 1
|
||||
for name, endpoints in expected.items():
|
||||
normalized = Counter()
|
||||
for (uid, pin), count in endpoints.items():
|
||||
candidates = [key for key in endpoint_aliases.get(uid, [uid])
|
||||
if key in components and pin in components[key]['pins']
|
||||
and name in components[key]['pins'][pin]['nets']]
|
||||
if len(set(candidates)) == 1:
|
||||
uid = candidates[0]
|
||||
normalized[(uid, pin)] += count
|
||||
endpoints.clear()
|
||||
endpoints.update(normalized)
|
||||
for (uid, port), count in list(endpoints.items()):
|
||||
if uid not in hierarchy_boundaries:
|
||||
continue
|
||||
@@ -316,18 +338,19 @@ def read_graph(ascii_path, keyin_path, *, hierarchy_path=None, root_block=None):
|
||||
return graph
|
||||
|
||||
|
||||
def _check_hierarchy_graph(path, graph, root_block):
|
||||
"""Expand native child links independently; inventory UID selectors stay root-only."""
|
||||
def _check_hierarchy_graph(path, graph, root_block, *, include_rows=False):
|
||||
"""Bind expanded instances by reference/page/unit while retaining raw UID evidence."""
|
||||
text = read_native_text(path)
|
||||
_require_report(text, 'HIER BLOCKS')
|
||||
blocks, roots, block = {}, [], None
|
||||
blocks, roots, block, symbol = {}, [], None, None
|
||||
for line in text.splitlines():
|
||||
if line.startswith('Block '):
|
||||
fields = line[6:].split('\t')
|
||||
block = fields[0].rstrip()
|
||||
if not block or block in blocks:
|
||||
raise ValueError('Duplicate or missing native hierarchy block')
|
||||
blocks[block] = {'children': {}, 'symbols': Counter()}
|
||||
blocks[block] = {'children': {}, 'symbols': Counter(), 'rows': []}
|
||||
symbol = None
|
||||
if fields[1:] == ['Root']:
|
||||
roots.append(block)
|
||||
elif line.startswith('\tChild '):
|
||||
@@ -336,13 +359,23 @@ def _check_hierarchy_graph(path, graph, root_block):
|
||||
raise ValueError('Ambiguous native hierarchy child')
|
||||
blocks[block]['children'][match[2]] = match[1]
|
||||
elif line.startswith(' Symbol '):
|
||||
match = re.fullmatch(r' Symbol (.+?)\tpage (.+)', line)
|
||||
match = re.fullmatch(r' Symbol (.*?)\tpage (.+)', line)
|
||||
if block is None or match is None:
|
||||
raise ValueError('Ambiguous native hierarchy symbol')
|
||||
blocks[block]['symbols'][match[1]] += 1
|
||||
symbol = {'source_uid': match[1], 'source_page': match[2], 'attributes': {}}
|
||||
blocks[block]['rows'].append(symbol)
|
||||
elif symbol is not None and line.startswith('\tProp Ref Designator '):
|
||||
symbol['refdes'] = line[len('\tProp Ref Designator '):]
|
||||
elif symbol is not None and line.startswith('\tProp Part Number '):
|
||||
symbol['attributes']['Part Number'] = line[len('\tProp Part Number '):]
|
||||
elif symbol is not None and (line == '\tProp Value' or line.startswith('\tProp Value ')):
|
||||
symbol['attributes']['Value'] = line[len('\tProp Value '):]
|
||||
elif symbol is not None and line.startswith('\tSymbol name '):
|
||||
symbol['symbol'] = line[len('\tSymbol name '):]
|
||||
if roots != [root_block]:
|
||||
raise ValueError('Native hierarchy root differs from captured project')
|
||||
identities, boundaries = Counter(), set()
|
||||
identities, boundaries, expanded, source_sheets = Counter(), set(), [], set()
|
||||
|
||||
pending = [(root_block, '', set())]
|
||||
while pending:
|
||||
@@ -352,6 +385,11 @@ def _check_hierarchy_graph(path, graph, root_block):
|
||||
row = blocks[name]
|
||||
identities.update({_instance_identity(uid, path): count for uid, count in row['symbols'].items()
|
||||
if uid not in row['children']})
|
||||
for item in row['rows']:
|
||||
source_sheets.add((name, item['source_page']))
|
||||
if item['source_uid'] in row['children'] and not _is_pcb(item):
|
||||
continue
|
||||
expanded.append(dict(item, instance_path=path, source_schematic=name))
|
||||
for uid, child in row['children'].items():
|
||||
if uid not in row['symbols']:
|
||||
raise ValueError('Native hierarchy child symbol missing')
|
||||
@@ -359,10 +397,41 @@ def _check_hierarchy_graph(path, graph, root_block):
|
||||
boundaries.add(identity)
|
||||
pending.append((child, identity, ancestors | {name}))
|
||||
if graph is not None:
|
||||
effective = Counter({uid: len(row.get('annotation_instances', [row])) for uid, row in graph['components'].items()})
|
||||
if identities != effective:
|
||||
remaining = [(uid, item) for uid, row in graph['components'].items()
|
||||
for item in row.get('annotation_instances', [row])]
|
||||
indices, consumed = defaultdict(list), set()
|
||||
for index, (uid, row) in enumerate(remaining):
|
||||
instance = row.get('instance_path', uid.rsplit('/', 1)[0] if '/' in uid else '')
|
||||
ref = row.get('source_refdes', row.get('refdes', '')).strip().casefold()
|
||||
indices[(instance, 'ref', ref)].append(index)
|
||||
indices[(instance, 'uid', row.get('source_uid', uid.rsplit('/', 1)[-1]))].append(index)
|
||||
for item in expanded:
|
||||
ref = item.get('refdes', '').strip().casefold()
|
||||
key = item['instance_path'], 'ref' if ref else 'uid', ref or item['source_uid']
|
||||
candidates = [(index, remaining[index][0], remaining[index][1]) for index in indices[key]
|
||||
if index not in consumed and (not remaining[index][1].get('source_page')
|
||||
or remaining[index][1]['source_page'] == item['source_page'])]
|
||||
if len(candidates) > 1 and item.get('symbol'):
|
||||
unit = item['symbol'].split(' type ', 1)[0]
|
||||
candidates = [candidate for candidate in candidates if candidate[2].get('symbol', '').split(' type ', 1)[0] == unit]
|
||||
if len(candidates) > 1:
|
||||
exact = [candidate for candidate in candidates if candidate[2].get('source_uid') == item['source_uid']]
|
||||
if exact:
|
||||
candidates = exact
|
||||
# Identical nonphysical annotations are counted, not mistaken for devices.
|
||||
if not candidates or (len(candidates) > 1 and ref):
|
||||
raise ValueError('Hierarchical/effective instance coverage differs')
|
||||
index, uid, row = candidates[0]
|
||||
item['graph_key'] = uid
|
||||
if row:
|
||||
row['source_schematic'] = item['source_schematic']
|
||||
row['source_page'] = row.get('source_page') or item['source_page']
|
||||
consumed.add(index)
|
||||
if len(consumed) != len(remaining):
|
||||
raise ValueError('Hierarchical/effective instance coverage differs')
|
||||
return identities, boundaries
|
||||
# A page may contain only a child block absent from the flat component graph.
|
||||
graph['source_sheets'] = [list(sheet) for sheet in sorted(source_sheets)]
|
||||
return (identities, boundaries, expanded) if include_rows else (identities, boundaries)
|
||||
|
||||
|
||||
def export_netlist(capture: dict, output: Path, *, stop_event=None) -> dict:
|
||||
@@ -466,6 +535,80 @@ def _connections(graph):
|
||||
return sorted(nets)
|
||||
|
||||
|
||||
def reference_targets(graph, refdes, *, sheet='', schematic=''):
|
||||
"""Return every native unit for a physical reference within the requested sheet."""
|
||||
ref = refdes.strip().casefold()
|
||||
candidates = [(uid, row) for uid, row in graph['components'].items()
|
||||
if _is_pcb(row) and row.get('refdes', '').strip().casefold() == ref]
|
||||
pages_known = any(row.get('source_page') for _, row in candidates)
|
||||
blocks_known = any(row.get('source_schematic') for _, row in candidates)
|
||||
return tuple(uid for uid, row in candidates
|
||||
if (not sheet or not pages_known or row.get('source_page') == sheet)
|
||||
and (not schematic or not blocks_known or row.get('source_schematic') == schematic))
|
||||
|
||||
|
||||
def _reference_graph(graph):
|
||||
"""Normalize physical endpoints without changing the captured source evidence."""
|
||||
from copy import deepcopy
|
||||
result = deepcopy(graph)
|
||||
identities, components = {}, {}
|
||||
groups = defaultdict(list)
|
||||
for row in result['components'].values():
|
||||
if _is_pcb(row):
|
||||
groups[row.get('refdes', '').strip().casefold()].append(row)
|
||||
for uid, row in result['components'].items():
|
||||
if _is_pcb(row):
|
||||
ref = row.get('refdes', '').strip().casefold()
|
||||
if not ref:
|
||||
raise ValueError('Physical component has no reference designator')
|
||||
# Separate slots of the same physical device remain independently checked.
|
||||
peers = groups[ref]
|
||||
scopes = {(item.get('source_schematic', ''), item.get('source_page', '')) for item in peers}
|
||||
scope = (row.get('source_schematic', ''), row.get('source_page', ''))
|
||||
same_scope = sum((item.get('source_schematic', ''), item.get('source_page', '')) == scope for item in peers)
|
||||
slot = re.match(r'\S+\.(\d+) type ', row.get('symbol', ''))
|
||||
identity = '@REFDES|' + ref
|
||||
if len(scopes) > 1:
|
||||
identity += '|sheet=' + json.dumps(scope, ensure_ascii=False)
|
||||
if same_scope > 1 and slot:
|
||||
repeated_slot = sum(bool(re.match(r'\S+\.' + re.escape(slot[1]) + r' type ', item.get('symbol', '')))
|
||||
for item in peers if (item.get('source_schematic', ''), item.get('source_page', '')) == scope) > 1
|
||||
if repeated_slot:
|
||||
numbers = [pin.get('number') for pin in row.get('pins', {}).values()]
|
||||
if not numbers or any(not number for number in numbers):
|
||||
raise ValueError('Repeated symbol slots lack a unique physical pin partition')
|
||||
identity += '|pins=' + json.dumps(sorted(numbers), ensure_ascii=False)
|
||||
else:
|
||||
identity += '|slot=' + slot[1]
|
||||
row.pop('source_uid', None)
|
||||
row.pop('instance_path', None)
|
||||
else:
|
||||
identity = uid
|
||||
if identity in components:
|
||||
raise ValueError('Reference designator and symbol slot are ambiguous')
|
||||
identities[uid] = identity
|
||||
components[identity] = row
|
||||
result['components'] = components
|
||||
for net in result['nets']:
|
||||
net['endpoints'] = [[identities.get(uid, uid), pin] for uid, pin in net['endpoints']]
|
||||
return result, identities
|
||||
|
||||
|
||||
def _reference_expectations(graph, aliases, values):
|
||||
result = {}
|
||||
for uid, value in (values or {}).items():
|
||||
targets = (uid,) if uid in aliases else ()
|
||||
if not targets and uid.startswith('@REFDES|'):
|
||||
targets = reference_targets(graph, uid[len('@REFDES|'):])
|
||||
scopes = {(graph['components'][key].get('source_schematic', ''),
|
||||
graph['components'][key].get('source_page', '')) for key in targets}
|
||||
if len(scopes) > 1:
|
||||
raise ValueError('Expected reference requires a sheet-qualified native identity')
|
||||
for target in targets or (uid,):
|
||||
result[aliases.get(target, target)] = value
|
||||
return result
|
||||
|
||||
|
||||
def compare_netlists(before: dict, after: dict, *, expected_changes=None, expected_symbols=None) -> dict:
|
||||
"""Compare complete nonempty exports, allowing only declared attribute/symbol changes.
|
||||
|
||||
@@ -476,7 +619,13 @@ def compare_netlists(before: dict, after: dict, *, expected_changes=None, expect
|
||||
graph = result.get('graph', {})
|
||||
if result.get('status') != 'success' or result.get('complete') is not True or not graph.get('components') or not graph.get('nets'):
|
||||
return {'status': 'unavailable', 'verified': False, 'reason': 'Complete nonempty native graphs required'}
|
||||
left, right = before['graph'], after['graph']
|
||||
try:
|
||||
left, aliases = _reference_graph(before['graph'])
|
||||
right, _ = _reference_graph(after['graph'])
|
||||
expected_changes = _reference_expectations(before['graph'], aliases, expected_changes)
|
||||
expected_symbols = _reference_expectations(before['graph'], aliases, expected_symbols)
|
||||
except (KeyError, TypeError, ValueError) as error:
|
||||
return {'status': 'unavailable', 'verified': False, 'reason': str(error)}
|
||||
differences = []
|
||||
try:
|
||||
connections_changed = _connections(left) != _connections(right)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
"""Overlap native exports with replacement, keeping each saved input immutable."""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from concurrent.futures import Future, ThreadPoolExecutor
|
||||
from dataclasses import asdict
|
||||
import json
|
||||
import hashlib
|
||||
@@ -54,7 +54,7 @@ def library_from_receipt(command, receipt):
|
||||
if receipt.get('native_state') not in {'saved_ok', 'restored_ok'}:
|
||||
return {}
|
||||
from .execution_proof import proof_intact
|
||||
from .native_helper import read_snapshot, require_resumable_transaction
|
||||
from .native_helper import read_snapshot, require_resumable_transaction, saved_netlist_snapshot
|
||||
from .path_access import canonical_path, internal_path
|
||||
proof = receipt.get('proof', {})
|
||||
native_files = [Path(item['path']) for item in proof.get('files', ())
|
||||
@@ -70,13 +70,18 @@ def library_from_receipt(command, receipt):
|
||||
if str(snapshot) not in paths:
|
||||
raise ValueError('原生库件证据缺少配套快照')
|
||||
report = json.loads(internal_path(path).read_text(encoding='utf-8'))
|
||||
if report.get('ok') is not True or report.get('stage') != 'native_replace_verified':
|
||||
saved_native = saved_netlist_snapshot(snapshot)
|
||||
if saved_native and proof.get('saved_netlist', proof.get('project_commit', {}).get('saved_netlist', {})).get('status') != 'verified':
|
||||
raise ValueError('已保存的原生数据尚未核验,不能使用写入回执声明成功')
|
||||
stage = 'native_replace_applied' if saved_native else 'native_replace_verified'
|
||||
if report.get('ok') is not True or report.get('stage') != stage:
|
||||
raise ValueError('原生库件回执未通过实际替换核验')
|
||||
metadata, records = read_snapshot(snapshot)
|
||||
binding = require_resumable_transaction(snapshot, report, metadata, records)
|
||||
if (canonical_path(metadata[1]) != canonical_path(command['project_path'])
|
||||
or metadata[2:4] != [command['old_part'], command['new_part']]
|
||||
or metadata[4] != f"{command['schematic_name']}:{command['sheet_number']}"):
|
||||
or metadata[4] != f"{command['schematic_name']}:{command['sheet_number']}"
|
||||
or (metadata[9] if len(metadata) == 10 else '') != command.get('target_selectors', '')):
|
||||
raise ValueError('原生库件证据属于不同执行项')
|
||||
row = binding['library_row']
|
||||
if row[1].casefold() != command['new_part'].casefold() or not row[4]:
|
||||
@@ -95,12 +100,16 @@ def library_from_receipt(command, receipt):
|
||||
raise ValueError('Value 回读证据的器件范围不完整')
|
||||
for uid in binding['source_uids']:
|
||||
identity = {'attributes': {'Part Number': row[1], 'Part Name': row[4], 'Part Label': row[5]},
|
||||
'target_symbol': binding['target_symbol'], 'slot_count': binding['slot_count']}
|
||||
'target_symbol': binding['target_symbol'], 'slot_count': binding['slot_count'],
|
||||
'refdes': next(record[2] for record in records if record[1] == uid),
|
||||
'schematic_name': command.get('schematic_name',''), 'sheet_number': command.get('sheet_number','')}
|
||||
if uid in values:
|
||||
value = values[uid]
|
||||
native_empty = (saved_native and value['actual_value'] == '' and value['actual_value_present'] is None
|
||||
and value.get('value_presence_coverage') == 'native_ascii_empty_presence_unknown')
|
||||
if (value['actual_part_number'].casefold() != row[1].casefold()
|
||||
or not isinstance(value['actual_value'], str)
|
||||
or type(value['actual_value_present']) is not bool):
|
||||
or (type(value['actual_value_present']) is not bool and not native_empty)):
|
||||
raise ValueError('Value 回读证据与目标库件不符')
|
||||
identity.update(preserved_value=value['actual_value'],
|
||||
preserved_value_present=value['actual_value_present'])
|
||||
@@ -180,30 +189,86 @@ def _rule_rows(child, task_id, plan, before):
|
||||
if not identities:
|
||||
raise ValueError('规则替换缺少绑定实际 sheet/UID 的原生事务证据')
|
||||
if selector is not None:
|
||||
original = before['graph']['components'].get(selector[0], {})
|
||||
if (set(identities) != {selector[0]} or original.get('refdes') != selector[1]
|
||||
or original.get('attributes', {}).get('Value', '') != selector[2]):
|
||||
keys = native_netlist.reference_targets(before['graph'], selector[1],
|
||||
sheet=command.get('sheet_number',''),schematic=command.get('schematic_name',''))
|
||||
if ((set(identities) != {selector[0]} and not all(
|
||||
identity.get('refdes','').strip().casefold() == selector[1].strip().casefold()
|
||||
for identity in identities.values())) or not keys or any(
|
||||
before['graph']['components'][key].get('attributes', {}).get('Value', '') != selector[2]
|
||||
for key in keys)):
|
||||
raise ValueError('规则位号选择器与替换前器件及原生 UID 范围不一致')
|
||||
scopes.append({key: command.get(key, '') for key in
|
||||
('project_path', 'schematic_name', 'sheet_number', 'old_part', 'new_part')}
|
||||
| {'source_uids': sorted(identities)})
|
||||
for uid in identities:
|
||||
component = before['graph']['components'].get(uid)
|
||||
if component is None or component['attributes'].get('Part Number', '').casefold() != command['old_part'].casefold():
|
||||
identity = identities[uid]
|
||||
keys = _reference_keys(before['graph'], dict(identity, uid=uid,
|
||||
sheet_number=command.get('sheet_number',''), schematic_name=command.get('schematic_name','')))
|
||||
if not keys or any(before['graph']['components'][key]['attributes'].get('Part Number', '').casefold()
|
||||
!= command['old_part'].casefold() for key in keys):
|
||||
raise ValueError('原生 UID 无法唯一对应替换前的原料号;不能按同料号猜测图页')
|
||||
row = {'enabled': True, 'target': {'uid': uid, 'old_value': component['attributes'].get('Value', '')},
|
||||
for key in keys:
|
||||
component = before['graph']['components'][key]
|
||||
row = {'enabled': True, 'target': {'uid': key, 'refdes': component['refdes'],
|
||||
'schematic_name':command.get('schematic_name',''),'sheet_number':command.get('sheet_number',''),
|
||||
'old_value': component['attributes'].get('Value', '')},
|
||||
'new_part_number': command['new_part'], 'value_mode': command['value_mode'],
|
||||
'custom_value_enabled': command['value_mode'] == 'custom',
|
||||
'custom_value': command.get('custom_value', ''), 'sync_nc': command.get('sync_nc', '1') == '1',
|
||||
'target_cell': command.get('target_cell', '')}
|
||||
if uid in rows and rows[uid] != row:
|
||||
raise ValueError('同一 UID 对应多个不同规则结果,不能独立核验')
|
||||
rows[uid] = row
|
||||
if key in rows and rows[key] != row:
|
||||
raise ValueError('同一位号对应多个不同规则结果,不能独立核验')
|
||||
rows[key] = row
|
||||
if not rows:
|
||||
raise ValueError('已保存的规则替换缺少可核验的原生 UID 事务')
|
||||
return list(rows.values()), scopes
|
||||
|
||||
|
||||
def _reference_keys(graph, identity):
|
||||
ref = identity.get('refdes', '').strip().casefold()
|
||||
keys = native_netlist.reference_targets(graph, ref, sheet=identity.get('sheet_number',''),
|
||||
schematic=identity.get('schematic_name','')) if ref else ()
|
||||
uid = identity.get('uid')
|
||||
if not keys and uid in graph['components']:
|
||||
component = graph['components'][uid]
|
||||
constraints = ((ref, component.get('refdes','').strip().casefold()),
|
||||
(identity.get('sheet_number',''), component.get('source_page','')),
|
||||
(identity.get('schematic_name',''), component.get('source_schematic','')))
|
||||
if not any(expected and actual and expected != actual for expected, actual in constraints):
|
||||
keys = (uid,)
|
||||
return keys
|
||||
|
||||
|
||||
def _reference_plan(plan, before, library):
|
||||
from copy import deepcopy
|
||||
result, rebound = deepcopy(plan), {}
|
||||
graph = before['graph']
|
||||
rows, plan_keys = [], {}
|
||||
for row in result['rows']:
|
||||
target = row['target']
|
||||
keys = _reference_keys(graph, target)
|
||||
if not keys and row['enabled']:
|
||||
raise ValueError('独立网表无法定位计划中的位号')
|
||||
plan_keys[target['uid']] = keys
|
||||
for key in keys:
|
||||
item = deepcopy(row)
|
||||
item['target']['uid'] = key
|
||||
rows.append(item)
|
||||
for uid, identity in library.items():
|
||||
keys = _reference_keys(graph, dict(identity, uid=uid))
|
||||
if not keys and not identity.get('refdes'):
|
||||
keys = plan_keys.get(uid, ())
|
||||
if not keys:
|
||||
raise ValueError('独立网表无法定位回执中的位号或图页')
|
||||
exact = tuple(key for key in keys if key == uid or graph['components'][key].get('source_uid') == uid)
|
||||
for key in exact or keys:
|
||||
if key in rebound and rebound[key] != identity:
|
||||
raise ValueError('独立网表无法区分同一位号的原生单元回执')
|
||||
rebound[key] = identity
|
||||
result['rows'] = rows
|
||||
return result, rebound
|
||||
|
||||
|
||||
class NetlistVerification:
|
||||
def __init__(self, root, run_id, stop_event):
|
||||
self.root, self.run_id, self.stop = Path(root), run_id, stop_event
|
||||
@@ -305,6 +370,18 @@ class NetlistVerification:
|
||||
return
|
||||
try:
|
||||
record['library'] = _library_expectations(record['child'], task_id)
|
||||
preparer = getattr(record['child'], '_quick_run_preparer', None)
|
||||
if preparer is not None:
|
||||
from .saved_netlist_verification import load_saved_export
|
||||
saved = load_saved_export(preparer.data_root / 'quick-run',
|
||||
Path(record['before_capture']['project']))
|
||||
if saved is not None:
|
||||
exported = Future()
|
||||
exported.set_result(saved['result'])
|
||||
record.update(after_capture=saved['capture'],
|
||||
after_export=Path(saved['export_path']).parent,
|
||||
after=exported)
|
||||
return
|
||||
directory = record['directory']
|
||||
attempt = uuid4().hex
|
||||
capture = native_netlist.capture_project(Path(record['before_capture']['project']),
|
||||
@@ -398,6 +475,7 @@ class NetlistVerification:
|
||||
rows, scopes = _rule_rows(record['child'], task_id, plan, before)
|
||||
plan = dict(plan, rows=rows)
|
||||
result['rule_scopes'] = scopes
|
||||
plan, library = _reference_plan(plan, before, library)
|
||||
expected = self._expected(plan, after, library)
|
||||
symbols = {uid: {key: item[key] for key in ('target_symbol', 'slot_count')}
|
||||
for uid, item in library.items() if uid in expected}
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Small append-only call timeline; diagnostics never replace an execution error."""
|
||||
from datetime import datetime, timezone
|
||||
from contextlib import contextmanager
|
||||
import json
|
||||
import os
|
||||
|
||||
from .path_access import internal_path
|
||||
|
||||
|
||||
def record_operation(root, operation, event, *, item_id='', error=''):
|
||||
if event not in {'enter', 'return', 'error'}:
|
||||
raise ValueError('operation event must be enter, return or error')
|
||||
try:
|
||||
record = {'timestamp_utc': datetime.now(timezone.utc).isoformat(),
|
||||
'operation': operation, 'event': event, 'item_id': item_id}
|
||||
if error:
|
||||
record['error'] = str(error)
|
||||
line = json.dumps(record, ensure_ascii=False) + '\n'
|
||||
path = internal_path(root) / 'operation-events.jsonl'
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open('a', encoding='utf-8') as stream:
|
||||
stream.write(line)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
@contextmanager
|
||||
def operation_span(root, operation, *, item_id=''):
|
||||
"""Trace a real call without letting diagnostics change its outcome."""
|
||||
def publish(event, error=''):
|
||||
try:
|
||||
record_operation(root, operation, event, item_id=item_id, error=error)
|
||||
except Exception:
|
||||
pass
|
||||
publish('enter')
|
||||
try:
|
||||
yield
|
||||
except BaseException as error:
|
||||
publish('error', error)
|
||||
raise
|
||||
else:
|
||||
publish('return')
|
||||
|
||||
|
||||
def summarize_operations(records):
|
||||
active, errors, last_item = {}, [], None
|
||||
for record in records:
|
||||
if not isinstance(record, dict) or not all(isinstance(record.get(name, ''), str)
|
||||
for name in ('operation', 'item_id', 'event')):
|
||||
continue
|
||||
key = record.get('operation'), record.get('item_id', '')
|
||||
if record.get('event') == 'enter':
|
||||
active[key] = record
|
||||
elif record.get('event') in {'return', 'error'}:
|
||||
active.pop(key, None)
|
||||
if record['event'] == 'error':
|
||||
errors.append(record)
|
||||
elif record.get('operation') == 'item' and record.get('item_id'):
|
||||
last_item = record['item_id']
|
||||
return {'last_successful_item': last_item, 'incomplete_operations': list(active.values()),
|
||||
'errors': errors}
|
||||
@@ -0,0 +1,159 @@
|
||||
"""Keep uncertain engineering sources quarantined without stopping unrelated work."""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import re
|
||||
|
||||
from .path_access import canonical_path, internal_path
|
||||
|
||||
|
||||
def write_sources(project):
|
||||
project = canonical_path(project)
|
||||
sources = {str(project).casefold()}
|
||||
try:
|
||||
from .native_inventory_protocol import read_native_text
|
||||
text = read_native_text(internal_path(project))
|
||||
match = re.search(r'^\s*KEY\s+iCDBDir\s+"([^"\r\n]+)"', text, re.MULTILINE | re.IGNORECASE)
|
||||
if match:
|
||||
directory = Path(match[1])
|
||||
if not directory.is_absolute():
|
||||
directory = project.parent / directory
|
||||
sources.add(str(canonical_path(directory / 'icdb.dat')).casefold())
|
||||
except (OSError, UnicodeError, ValueError):
|
||||
pass
|
||||
return sources
|
||||
|
||||
|
||||
def record_task_outcome(child, task_id, state, message):
|
||||
store = getattr(child, '_store', None)
|
||||
if store is None:
|
||||
return
|
||||
from .models import TaskStatus
|
||||
target = {'blocked': TaskStatus.BLOCKED, 'failed': TaskStatus.FAILED, 'paused': TaskStatus.PAUSED}.get(state)
|
||||
if target is None:
|
||||
return # Successful completion is committed with its actual execution receipts.
|
||||
with store.transaction():
|
||||
task = store.get_task(task_id)
|
||||
if task.status not in {TaskStatus.COMPLETED, TaskStatus.DELETED}:
|
||||
store.set_task_status(task_id, target, summary=message)
|
||||
|
||||
|
||||
def isolate_project(runtime, child, task_id, error):
|
||||
release = getattr(runtime, 'release_failed_project', None)
|
||||
if release is None:
|
||||
return False
|
||||
preparer = getattr(child, '_quick_run_preparer', None)
|
||||
root = preparer.data_root / 'quick-run' if preparer is not None else runtime.root / 'tasks' / task_id / 'quick-run'
|
||||
released = release(error, root)
|
||||
return released
|
||||
|
||||
|
||||
def dependency_path(root, task_id):
|
||||
return canonical_path(root) / 'dependency-blocks' / (task_id + '.json')
|
||||
|
||||
|
||||
def dependency_block(root, task_id):
|
||||
from .errors import TaskCenterError
|
||||
try:
|
||||
value = json.loads(internal_path(dependency_path(root, task_id)).read_text(encoding='utf-8'))
|
||||
if (not isinstance(value, dict) or value.get('version') != 1 or value.get('task_id') != task_id or value.get('undispatched') is not True
|
||||
or value.get('state') not in {'blocked', 'readmission_prepared', 'readmitted'}
|
||||
or not isinstance(value.get('dependency_task_id'), str)
|
||||
or Path(value['dependency_task_id']).name != value['dependency_task_id']
|
||||
or not isinstance(value.get('sources'), list) or not value['sources']
|
||||
or not all(isinstance(source, str) and Path(source).is_absolute() for source in value['sources'])
|
||||
or not isinstance(value.get('plan_hash'), str) or not re.fullmatch('[0-9a-f]{64}', value['plan_hash'])
|
||||
or not isinstance(value.get('queue_id'), str) or not re.fullmatch('[0-9a-f]{32}', value['queue_id'])
|
||||
or not isinstance(value.get('project_path'), str) or not Path(value['project_path']).is_absolute()):
|
||||
raise ValueError('invalid dependency block')
|
||||
return value
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except (OSError, ValueError, TypeError) as error:
|
||||
raise TaskCenterError('quick_run_dependency', '共享数据源隔离记录无法核验,请保留证据。') from error
|
||||
|
||||
|
||||
def block_dependency(root, child, task_id, dependency_task_id, sources, plan_hash, queue_id):
|
||||
from .batch_resources import write_record
|
||||
projects = child.projects(task_id)
|
||||
write_record(dependency_path(root, task_id), {'version': 1, 'task_id': task_id,
|
||||
'dependency_task_id': dependency_task_id, 'sources': sorted(sources), 'plan_hash': plan_hash,
|
||||
'project_path': str(canonical_path(projects[0].path)).casefold(),
|
||||
'queue_id': queue_id, 'state': 'blocked', 'undispatched': True})
|
||||
record_task_outcome(child, task_id, 'blocked', f'共享数据源依赖 {dependency_task_id} 尚待核验;本任务未派发替换。')
|
||||
|
||||
|
||||
def _validate_dependency(service, task_id, block):
|
||||
from .batch_resources import resource_blockers
|
||||
from .errors import TaskCenterError
|
||||
from .models import TaskStatus
|
||||
from .task_queue import _plan_hash
|
||||
child = service.for_project_task(task_id)
|
||||
dependency = service.for_project_task(block['dependency_task_id'])
|
||||
projects = child.projects(task_id)
|
||||
dependency_projects = dependency.projects(block['dependency_task_id'])
|
||||
queue_path = next((path for directory in ('task-queues', 'workspace-queues')
|
||||
if internal_path(path := canonical_path(service._logging.root) / directory / (block['queue_id'] + '.json')).is_file()), None)
|
||||
try:
|
||||
queue = json.loads(internal_path(queue_path).read_text(encoding='utf-8')) if queue_path else {}
|
||||
queue_bound = (queue['plan_hashes'][task_id] == block['plan_hash']
|
||||
and queue['dependency_tasks'][task_id] == block['dependency_task_id']
|
||||
and queue['entries'][task_id]['state'] == 'blocked')
|
||||
except (OSError, ValueError, TypeError, KeyError):
|
||||
queue_bound = False
|
||||
store = getattr(child, '_store', None)
|
||||
with store._lock:
|
||||
dispatched = store._connection.execute('''SELECT 1 FROM execution_operations o
|
||||
JOIN execution_rounds r ON r.task_id=o.task_id AND r.round_id=o.round_id
|
||||
WHERE o.task_id=? AND (o.attempt_id IS NOT NULL OR o.generation>0 OR o.state!='pending') LIMIT 1''',
|
||||
(task_id,)).fetchone()
|
||||
evidence = child._quick_run_preparer.data_root / 'quick-run'
|
||||
absent_attempt = not dispatched and not any(internal_path(evidence).glob('snapshot-*'))
|
||||
if (len(projects) != 1 or len(dependency_projects) != 1
|
||||
or not queue_bound or not absent_attempt
|
||||
or str(canonical_path(projects[0].path)).casefold() != block['project_path']
|
||||
or _plan_hash(child, task_id) != block['plan_hash']
|
||||
or not set(block['sources']) <= write_sources(projects[0].path) & write_sources(dependency_projects[0].path)
|
||||
or child.has_interrupted_quick_run(task_id)
|
||||
or dependency.has_interrupted_quick_run(block['dependency_task_id'])
|
||||
or dependency.get_task(block['dependency_task_id']).status not in {TaskStatus.READY, TaskStatus.COMPLETED}
|
||||
or resource_blockers(service.pending_batch_resources())):
|
||||
raise TaskCenterError('quick_run_dependency', '原依赖工程尚未核验、写入资源未退出或冻结计划已变化,不能解除隔离。')
|
||||
|
||||
|
||||
def recover_dependency(service, task_id, *, confirmed):
|
||||
from .batch_resources import write_record
|
||||
from .errors import TaskCenterError
|
||||
from .models import TaskStatus
|
||||
from .quick_run_protocol import QuickRunStepResult
|
||||
root = service._logging.root
|
||||
block = dependency_block(root, task_id)
|
||||
if not confirmed:
|
||||
raise TaskCenterError('quick_run_dependency', '请确认核验共享数据源依赖后解除未派发任务的隔离。')
|
||||
_validate_dependency(service, task_id, block)
|
||||
path = dependency_path(root, task_id)
|
||||
write_record(path, dict(block, state='readmission_prepared'))
|
||||
with service._store.transaction():
|
||||
task = service._store.get_task(task_id)
|
||||
if task.status is TaskStatus.BLOCKED:
|
||||
service._store.set_task_status(task_id, TaskStatus.READY, summary='共享数据源依赖已核验;原计划未派发,可重新确认执行。')
|
||||
elif task.status not in {TaskStatus.READY, TaskStatus.COMPLETED}:
|
||||
raise TaskCenterError('quick_run_dependency', '当前任务状态已变化,请重新读取隔离记录。')
|
||||
write_record(path, dict(block, state='readmitted'))
|
||||
child = service.for_project_task(task_id)
|
||||
progress = child.quick_run_progress(task_id)
|
||||
handoff = child._quick_run_preparer.data_root / 'quick-run'
|
||||
return QuickRunStepResult(handoff / 'QuickPartReplace.vbs', handoff / 'command.tsv', handoff / 'status.tsv', path,
|
||||
'pending', '共享数据源依赖已核验;本任务从未派发替换,已恢复原待执行计划。',
|
||||
progress.next_project_path, progress.next_old_part_number, progress.next_new_part_number,
|
||||
progress.next_schematic_name, progress.next_sheet_number, progress)
|
||||
|
||||
|
||||
def dependency_readmitted(service, task_id, plan_hash):
|
||||
from .models import TaskStatus
|
||||
block = dependency_block(service._logging.root, task_id)
|
||||
if not block or block['state'] not in {'readmission_prepared', 'readmitted'} or block['plan_hash'] != plan_hash:
|
||||
return False
|
||||
if service.get_task(task_id).status not in {TaskStatus.READY, TaskStatus.COMPLETED}:
|
||||
return False
|
||||
_validate_dependency(service, task_id, block)
|
||||
return True
|
||||
@@ -2,12 +2,14 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import json
|
||||
import subprocess
|
||||
import time
|
||||
from threading import Event, Thread
|
||||
from uuid import uuid4
|
||||
|
||||
from .worker_monitor import WorkerMonitorError, run_monitored_process
|
||||
from .operation_diagnostics import operation_span
|
||||
|
||||
|
||||
def _write_control(path: Path, value: str):
|
||||
@@ -28,7 +30,7 @@ def _write_control(path: Path, value: str):
|
||||
|
||||
|
||||
class ProjectSessionWorker:
|
||||
"""Only a saved-and-reopened checkpoint releases the caller for its next step.
|
||||
"""Retain applied steps; engineering close performs the save/readback commit.
|
||||
|
||||
The VBS worker owns CAD. This driver never attaches to COM or terminates CAD.
|
||||
A separate monitor thread retains the existing idle watchdog and worker job.
|
||||
@@ -61,6 +63,10 @@ class ProjectSessionWorker:
|
||||
self._done.set()
|
||||
|
||||
def run_step(self, command: tuple[str, ...], timeout: float):
|
||||
with operation_span(self.root, 'native_step', item_id=command[-1]):
|
||||
return self._run_step(command, timeout)
|
||||
|
||||
def _run_step(self, command: tuple[str, ...], timeout: float):
|
||||
if self._closed:
|
||||
raise WorkerMonitorError('project session is already closed')
|
||||
if Path(command[3]).resolve() != self.root.resolve():
|
||||
@@ -82,6 +88,7 @@ class ProjectSessionWorker:
|
||||
if not self._waiting:
|
||||
raise WorkerMonitorError('project session has no verified checkpoint')
|
||||
self._waiting = False
|
||||
_write_control(self.root / 'project-session.request', f'{self.token}\t{self.project}\t{job}')
|
||||
_write_control(self.root / 'project-session-next.tsv', f'{self.token}\t{job}')
|
||||
while True:
|
||||
if self._done.is_set():
|
||||
@@ -92,8 +99,8 @@ class ProjectSessionWorker:
|
||||
raise self._error
|
||||
if self._result.returncode == 0:
|
||||
raise WorkerMonitorError('project session exited before authorized closing')
|
||||
if self._result.returncode == 74:
|
||||
pause = getattr(self._result, 'startup_pause', {})
|
||||
if self._result.returncode in {74, 75}:
|
||||
pause = getattr(self._result, 'startup_pause' if self._result.returncode == 74 else 'prewrite_pause', {})
|
||||
if (pause.get('session_token') != self.token or pause.get('job_id') != job
|
||||
or Path(pause.get('project_path', '')).resolve() != self.project.resolve()
|
||||
or pause.get('worker_tree_exited') is not True):
|
||||
@@ -110,13 +117,20 @@ class ProjectSessionWorker:
|
||||
except (FileNotFoundError, PermissionError):
|
||||
fields = ()
|
||||
if len(fields) == 4 and fields[:2] == [self.token, job]:
|
||||
if fields[2] not in {'saved_ok', 'no_match'} or fields[3] != 'session_verified':
|
||||
if (fields[2], fields[3]) not in {('saved_ok', 'session_verified'), ('no_match', 'session_verified'),
|
||||
('pending_save', 'session_staged'), ('pending_save', 'session_staged_no_match')}:
|
||||
raise WorkerMonitorError('project session returned an invalid checkpoint')
|
||||
self._waiting = True
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
self._done.wait(0.025)
|
||||
|
||||
def close(self):
|
||||
if not self._closed and self._thread is not None and not self._done.is_set():
|
||||
with operation_span(self.root, 'close_project', item_id=self.token):
|
||||
return self._close()
|
||||
return self._close()
|
||||
|
||||
def _close(self):
|
||||
if self._closed:
|
||||
return
|
||||
try:
|
||||
@@ -128,25 +142,33 @@ class ProjectSessionWorker:
|
||||
from .stop_control import StopControl
|
||||
interactive = isinstance(getattr(self.runtime, 'stop_event', None), StopControl)
|
||||
close_grace = 60 if self.private_desktop else min(30, max(10, self._timeout))
|
||||
if interactive:
|
||||
# A failed close has its own deadline, separate from pause.
|
||||
force_event = getattr(getattr(self.runtime, 'stop_event', None), 'force_event', None)
|
||||
last_notice = time.monotonic()
|
||||
deadline = last_notice + close_grace
|
||||
while not self._done.wait(.1):
|
||||
if force_event is not None and force_event.is_set():
|
||||
self._abort.set()
|
||||
break
|
||||
if time.monotonic() >= deadline:
|
||||
force_event = getattr(getattr(self.runtime, 'stop_event', None), 'force_event', None)
|
||||
last_notice = time.monotonic()
|
||||
deadline = last_notice + close_grace
|
||||
last_progress = (-1, -1)
|
||||
while not self._done.wait(.1):
|
||||
if force_event is not None and force_event.is_set():
|
||||
self._abort.set()
|
||||
break
|
||||
try:
|
||||
native = json.loads((self.root / 'worker-progress.json').read_text(encoding='utf-8'))
|
||||
progress = (native['sequence'], native.get('native_progress_frames', 0))
|
||||
if all(type(value) is int for value in progress) and progress > last_progress:
|
||||
last_progress = progress
|
||||
deadline = time.monotonic() + close_grace
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
pass
|
||||
if time.monotonic() >= deadline:
|
||||
if interactive:
|
||||
self.runtime.stop_event.request_failure_cleanup(self.root,
|
||||
'project session did not acknowledge closing within its grace period')
|
||||
self._abort.set()
|
||||
break
|
||||
if time.monotonic() - last_notice >= 15:
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(
|
||||
'project_session_awaiting_close token=%s', self.token)
|
||||
last_notice = time.monotonic()
|
||||
self._abort.set()
|
||||
break
|
||||
if time.monotonic() - last_notice >= 15:
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(
|
||||
'project_session_awaiting_close token=%s', self.token)
|
||||
last_notice = time.monotonic()
|
||||
if not self._done.wait(close_grace):
|
||||
self._abort.set()
|
||||
if not self._done.wait(30):
|
||||
@@ -174,6 +196,7 @@ class ProjectSessionScope:
|
||||
self.worker = None
|
||||
self.private_desktop = private_desktop
|
||||
self.runtime = runtime
|
||||
self.on_commit = None
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
@@ -189,13 +212,25 @@ class ProjectSessionScope:
|
||||
def prepare(self, project):
|
||||
if self.worker is not None and (self.worker.project.resolve() != Path(project).resolve()
|
||||
or self.worker._done.is_set()):
|
||||
self.worker.close()
|
||||
self.worker = None
|
||||
self.close()
|
||||
|
||||
def close(self):
|
||||
if self.worker is None:return
|
||||
worker = self.worker
|
||||
worker.close()
|
||||
if self.on_commit is not None:
|
||||
self.on_commit(worker)
|
||||
self.on_commit = None
|
||||
self.worker = None
|
||||
|
||||
def __exit__(self, error_type, error, traceback):
|
||||
if self.worker is not None:
|
||||
try:
|
||||
self.worker.close()
|
||||
except Exception:
|
||||
self.close()
|
||||
except Exception as cleanup_error:
|
||||
self.worker._abort.set()
|
||||
if error is not None:
|
||||
if hasattr(error, 'add_note'):
|
||||
error.add_note('Project session cleanup: ' + str(cleanup_error))
|
||||
return False
|
||||
raise
|
||||
@@ -373,24 +373,289 @@ class QuickRunPreparer:
|
||||
if journal is None or not journal.needs_reconciliation(task_id):
|
||||
return
|
||||
root = self._data_root/'quick-run'
|
||||
flag = root/'uncertain.flag'
|
||||
if _io_path(flag).exists():
|
||||
raise TaskCenterError('quick_run_uncertain', '工程已保留待核验保护,请先核验原执行证据。')
|
||||
from .handoff_guard import assert_handoff_intact
|
||||
assert_handoff_intact(root, task_id)
|
||||
try:
|
||||
self._commit_staged_project(task_id)
|
||||
self._bind_journal_handoff(task_id, QuickRunResult(root/'QuickPartReplace.vbs',
|
||||
root/'command.tsv', root/'status.tsv', root/'quick-part-replace.log', self.vbs_root_compatible))
|
||||
except TaskCenterError:
|
||||
_write_uncertain_flag(root/'uncertain.flag','uncommitted_receipt_requires_verification')
|
||||
raise
|
||||
except (OSError, ValueError, KeyError, TypeError) as error:
|
||||
_write_uncertain_flag(flag, 'uncommitted_receipt_requires_verification')
|
||||
if isinstance(error, TaskCenterError):
|
||||
raise
|
||||
raise TaskCenterError('quick_run_uncertain', '无法核验原工程提交回执;请保留执行证据。') from error
|
||||
|
||||
def result_proof(self, root, row, outcome, *, persist=True):
|
||||
def result_proof(self, root, row, outcome, *, persist=True, project_commit=None):
|
||||
from .execution_proof import capture_result
|
||||
return capture_result(self._data_root,root,row,outcome,
|
||||
strict=self._process_runner is _run_process,persist=persist)
|
||||
strict=self._process_runner is _run_process,persist=persist,project_commit=project_commit)
|
||||
|
||||
def _commit_staged_project(self, task_id, *, session_token=None, bind_source=False):
|
||||
journal = getattr(self, 'journal', None)
|
||||
if journal is None:return
|
||||
from .execution_proof import capture_project_commit, validate_project_commit_source
|
||||
from .operation_diagnostics import record_operation
|
||||
for commit in journal.pending_project_commits(task_id):
|
||||
if session_token is not None and commit['session_token'] != session_token:continue
|
||||
root = Path(commit['evidence_root'])
|
||||
record_operation(root, 'readback_proof', 'enter', item_id=commit['session_token'])
|
||||
try:
|
||||
marker = capture_project_commit(self._data_root, root, commit)
|
||||
saved_protocol = marker.get('validation_required') == 'saved_netlist_v1'
|
||||
persisted = json.loads(commit['proof_json']) if commit.get('proof_json') else None
|
||||
commit_proof = dict(persisted or marker)
|
||||
if persisted:
|
||||
if ((saved_protocol or commit_proof.get('saved_sources'))
|
||||
and not validate_project_commit_source(commit_proof, root)
|
||||
or marker['files'][0]['sha256'] != commit_proof['files'][0]['sha256']):
|
||||
raise ValueError('saved project proof changed before receipt commit')
|
||||
elif saved_protocol and not bind_source:
|
||||
commit_proof, _ = self._closed_project_commit_proof(root, commit)
|
||||
if (not validate_project_commit_source(commit_proof, root)
|
||||
or marker['files'][0]['sha256'] != commit_proof['files'][0]['sha256']
|
||||
or commit_proof.get('native_outcomes') != marker['native_outcomes']):
|
||||
raise ValueError('saved project proof changed after native closure')
|
||||
elif bind_source:
|
||||
if _io_path(root/'uncertain.flag').exists():
|
||||
raise TaskCenterError('quick_run_uncertain', '工程已保留待核验保护,不能重新绑定保存源。')
|
||||
from .execution_proof import bind_project_commit_source
|
||||
commit_proof = bind_project_commit_source(commit_proof, self._data_root, root, commit['project_path'])
|
||||
attempts = journal.project_commit_attempts(task_id, commit['session_token'])
|
||||
if commit_proof.get('operation_attempts', attempts) != attempts:
|
||||
raise ValueError('original project attempts changed')
|
||||
commit_proof['operation_attempts'] = attempts
|
||||
staged = journal.staged_project_rows(task_id, commit['session_token'])
|
||||
if saved_protocol:
|
||||
from .saved_netlist_verification import verify_saved_project
|
||||
for receipt in staged:
|
||||
for item in receipt['proof'].get('files', ()):
|
||||
name = Path(item['path']).name
|
||||
if hashlib.sha256(_io_path(root/name).read_bytes()).hexdigest() != item['sha256']:
|
||||
raise ValueError('staged native target evidence changed before saved export')
|
||||
if bind_source and not persisted:
|
||||
names = [name for job, state in commit_proof['native_outcomes'].items() if state == 'pending_save'
|
||||
for name in (f'snapshot-{job}.tsv', f'snapshot-{job}.tsv.native.json', f'writer-expectations-{job}.tsv')]
|
||||
commit_proof['readback_files'] = {name:hashlib.sha256(_io_path(root/name).read_bytes()).hexdigest() for name in names}
|
||||
journal.bind_project_commit_proof(task_id, commit['session_token'], commit_proof)
|
||||
before_validation = persisted or commit_proof
|
||||
commit_proof = verify_saved_project(self._data_root, root, commit_proof, [receipt['row'] for receipt in staged])
|
||||
journal.promote_project_commit_proof(task_id, commit['session_token'], before_validation, commit_proof)
|
||||
proofs = {}
|
||||
for receipt in staged:
|
||||
row = receipt['row']
|
||||
outcome = (commit_proof['outcomes'][row['job_id']], 'project_committed',
|
||||
'工程统一保存,原生数据核验通过。' if saved_protocol else '工程保存回读校验通过。')
|
||||
proof = self.result_proof(root, row, outcome, project_commit=commit_proof) if saved_protocol else self.result_proof(root,row,outcome)
|
||||
if outcome[0] == 'saved_ok':
|
||||
from .value_readback import capture_value_readback
|
||||
proof['value_readback'] = capture_value_readback(root, row, outcome, project_commit=commit_proof)
|
||||
proofs[row['job_id']] = proof
|
||||
if bind_source and not commit.get('proof_json') and not saved_protocol:
|
||||
commit_proof['readback_files'] = {Path(item['path']).name: item['sha256']
|
||||
for proof in proofs.values() for item in proof.get('files', ())}
|
||||
for job, outcome in commit_proof['outcomes'].items():
|
||||
if outcome == 'saved_ok':
|
||||
value_path = _io_path(root / f'value-readback-{job}.tsv')
|
||||
if value_path.is_file():commit_proof['readback_files'][value_path.name] = hashlib.sha256(value_path.read_bytes()).hexdigest()
|
||||
journal.bind_project_commit_proof(task_id, commit['session_token'], commit_proof)
|
||||
statuses = journal.commit_project_results(task_id, commit['session_token'], proofs, commit_proof)
|
||||
if saved_protocol and statuses:
|
||||
_append_status_details(root/'status.tsv', statuses)
|
||||
record_operation(root, 'readback_proof', 'return', item_id=commit['session_token'])
|
||||
for job in commit_proof['jobs']:
|
||||
record_operation(root, 'item', 'return', item_id=job)
|
||||
except Exception as error:
|
||||
record_operation(root, 'readback_proof', 'error', item_id=commit['session_token'], error=str(error))
|
||||
raise
|
||||
|
||||
def _closed_project_commit_proof(self, root, commit):
|
||||
from .batch_resources import _valid_record
|
||||
|
||||
checkpoint = json.loads(_io_path(root / 'project-checkpoint.json').read_text(encoding='utf-8'))
|
||||
owner_path = Path(checkpoint['resource_record'])
|
||||
owner = json.loads(_io_path(owner_path).read_text(encoding='utf-8'))
|
||||
proof = checkpoint['project_commit_source']
|
||||
native = {key: checkpoint[key] for key in ('session_id', 'generation', 'attempt_id')}
|
||||
if (not _valid_record(owner, owner_path) or checkpoint.get('worker_closed') is not True
|
||||
or checkpoint.get('project_closed') is not True
|
||||
or Path(checkpoint['evidence_root']).resolve() != root.resolve()
|
||||
or _path_key(checkpoint['project_path']) != _path_key(commit['project_path'])
|
||||
or owner['session_id'] != native['session_id'] or owner['generation'] != native['generation']
|
||||
or owner['checkpoints'].get(str(root.resolve()).casefold()) != native['attempt_id']
|
||||
or proof.get('native_attempt') != native or proof.get('session_token') != commit['session_token']
|
||||
or proof.get('jobs') != json.loads(commit['jobs_json'])):
|
||||
raise ValueError('native close checkpoint is not bound to this project commit')
|
||||
return proof, owner_path
|
||||
|
||||
def recover_project_commit(self, task_id, *, confirmed=False):
|
||||
"""Reconcile an already closed/saved project without acquiring CAD."""
|
||||
journal = getattr(self, 'journal', None)
|
||||
if journal is None:return None
|
||||
commits = journal.project_commit_records(task_id)
|
||||
root = self._data_root / 'quick-run'
|
||||
flag = _io_path(root / 'uncertain.flag')
|
||||
audit = root / 'project-commit-recovery.json'
|
||||
if not commits:return None
|
||||
protection = flag.read_bytes() if flag.is_file() else None
|
||||
allowed = {f'reason={reason}; manual reconciliation required\n'.encode() for reason in
|
||||
('receipt_commit_failed', 'project_commit_unverified', 'uncommitted_receipt_requires_verification')}
|
||||
interrupted = None
|
||||
if protection is not None:
|
||||
import re
|
||||
match = re.fullmatch(rb'reason=supervisor_interrupted; batch=([0-9a-f]{32}); manual reconciliation required\r?\n', protection)
|
||||
if match:interrupted = match[1].decode('ascii')
|
||||
if protection is None:
|
||||
try:
|
||||
completed = json.loads(_io_path(audit).read_text(encoding='utf-8'))
|
||||
if (completed.get('task_id') != task_id or completed.get('native_replacement_called') is not False
|
||||
or completed.get('session_tokens') != [commit['session_token'] for commit in commits]):return None
|
||||
except (OSError, ValueError, TypeError):return None
|
||||
elif protection not in allowed and interrupted is None:return None
|
||||
if not confirmed:
|
||||
raise TaskCenterError('quick_run_recovery_confirmation', '工程提交回执核验需要明确确认。')
|
||||
from .execution_proof import capture_project_commit, validate_project_commit_source, proof_intact
|
||||
from .execution_journal import read_command_rows, command_identity, _hash
|
||||
from .batch_resources import write_record
|
||||
try:
|
||||
rows = read_command_rows(root / 'command.tsv')
|
||||
if not rows or any(row['task_id'] != task_id for row in rows):raise ValueError('foreign command plan')
|
||||
current = journal._round(task_id)
|
||||
if json.loads(current['plan_json']) != [_hash(command_identity(row)) for row in rows]:
|
||||
raise ValueError('frozen command plan changed')
|
||||
positions = {row['job_id']:index for index,row in enumerate(rows)}
|
||||
markers = {commit['session_token']:capture_project_commit(self._data_root,root,commit) for commit in commits}
|
||||
ends = {token:max(positions[job] for job in marker['jobs']) for token,marker in markers.items()}
|
||||
latest = {}
|
||||
for commit in commits:
|
||||
token = commit['session_token']
|
||||
if markers[token].get('validation_required') == 'saved_netlist_v1':
|
||||
latest[commit['project_path']] = max(latest.get(commit['project_path'],-1),ends[token])
|
||||
historical = set()
|
||||
def historical_intact(proof):
|
||||
saved = proof['saved_netlist']
|
||||
verified = json.loads(_io_path(saved['verification_path']).read_text(encoding='utf-8'))
|
||||
context = {key:proof[key] for key in ('session_token','project_path','saved_sources','command_sha256')}
|
||||
context['marker_sha256'] = proof['files'][0]['sha256']
|
||||
return (proof_intact(proof) and saved.get('status') == 'verified'
|
||||
and proof['command_sha256'] == hashlib.sha256(_io_path(root/'command.tsv').read_bytes()).hexdigest()
|
||||
and hashlib.sha256(_io_path(saved['verification_path']).read_bytes()).hexdigest() == saved['verification_sha256']
|
||||
and hashlib.sha256(_io_path(saved['export_path']).read_bytes()).hexdigest() == saved['export_sha256']
|
||||
and verified.get('status') == 'verified' and verified.get('context') == context
|
||||
and verified.get('export_path') == saved['export_path'] and verified.get('export_sha256') == saved['export_sha256']
|
||||
and verified.get('jobs') == saved.get('jobs') and set(saved['jobs']) == set(proof['jobs'])
|
||||
and proof['outcomes'] == {job:'saved_ok' if state == 'pending_save' else state
|
||||
for job,state in proof['native_outcomes'].items()})
|
||||
covered = set()
|
||||
covered_attempts = []
|
||||
for commit in commits:
|
||||
if Path(commit['evidence_root']).resolve() != root.resolve():raise ValueError('foreign commit root')
|
||||
proof = json.loads(commit['proof_json'] or '{}')
|
||||
prior = (commit['state'] == 'verified' and proof.get('validation_required') == 'saved_netlist_v1'
|
||||
and ends[commit['session_token']] < latest.get(commit['project_path'],-1))
|
||||
if prior:historical.add(commit['session_token'])
|
||||
if proof:
|
||||
attempts = journal.project_commit_attempts(task_id, commit['session_token'], require_recorded=True)
|
||||
if (proof.get('operation_attempts', attempts) != attempts
|
||||
or set(attempts) != set(proof['jobs'])):
|
||||
raise ValueError('original project attempts changed')
|
||||
proof.setdefault('operation_attempts', attempts)
|
||||
if not proof or interrupted is not None and not prior:
|
||||
proof, owner_path = self._closed_project_commit_proof(root, commit)
|
||||
if interrupted is not None:
|
||||
supervisor = json.loads(_io_path(owner_path.parent / 'supervisor.json').read_text(encoding='utf-8'))
|
||||
if (proof['native_attempt']['session_id'] != interrupted or supervisor.get('batch_id') != interrupted
|
||||
or supervisor.get('state') != 'blocked' or supervisor.get('business_uncertain') is not True
|
||||
or task_id not in supervisor.get('protected_task_ids', ())):
|
||||
raise ValueError('supervisor interruption is not bound to the closed native worker')
|
||||
proof = dict(proof, operation_attempts=journal.project_commit_attempts(task_id, commit['session_token']))
|
||||
if (not (historical_intact(proof) if prior else validate_project_commit_source(proof, root))
|
||||
or proof.get('operation_attempts') != journal.project_commit_attempts(task_id, commit['session_token'])):
|
||||
raise ValueError('saved source or original attempts changed')
|
||||
marker = markers[commit['session_token']]
|
||||
if (marker['jobs'] != proof.get('jobs') or marker['outcomes'] != proof.get('native_outcomes',proof.get('outcomes'))
|
||||
or marker['files'][0]['sha256'] != proof['files'][0]['sha256']):
|
||||
raise ValueError('native project marker changed')
|
||||
covered.update(proof['jobs'])
|
||||
covered_attempts.extend(proof['operation_attempts'].values())
|
||||
for row in rows:
|
||||
if row['job_id'] not in proof['jobs']:continue
|
||||
saved_protocol = proof.get('validation_required') == 'saved_netlist_v1'
|
||||
state = proof.get('native_outcomes',proof['outcomes'])[row['job_id']] if saved_protocol else proof['outcomes'][row['job_id']]
|
||||
if saved_protocol and state == 'no_match':continue
|
||||
outcome = (state, 'session_staged' if saved_protocol else 'project_committed', '工程保存回读校验通过。')
|
||||
result = self.result_proof(root, row, outcome)
|
||||
if proof.get('native_attempt') and outcome[0] in {'saved_ok','pending_save'} and any(
|
||||
result.get('native_attempt', {}).get(key) != value for key, value in proof['native_attempt'].items()):
|
||||
raise ValueError('native result differs from the exited project worker')
|
||||
if any(proof.get('readback_files', {}).get(Path(item['path']).name) != item['sha256'] for item in result.get('files', ())):
|
||||
raise ValueError('saved native readback evidence changed')
|
||||
if outcome[0] == 'saved_ok' and not saved_protocol:
|
||||
from .value_readback import capture_value_readback
|
||||
if capture_value_readback(root, row, outcome, project_commit=proof).get('value_readback_status') != 'verified':
|
||||
raise ValueError('saved target Value readback is incomplete')
|
||||
if any(Path(name).name != name or hashlib.sha256(_io_path(root / name).read_bytes()).hexdigest() != digest
|
||||
for name, digest in proof.get('readback_files', {}).items()):
|
||||
raise ValueError('native/readback files changed since project closure')
|
||||
if not commit.get('proof_json'):
|
||||
journal.bind_project_commit_proof(task_id, commit['session_token'], proof)
|
||||
commit['proof_json'] = json.dumps(proof)
|
||||
if (not set(journal.unresolved_jobs(task_id, rows)) <= covered
|
||||
or any(attempt not in covered_attempts for attempt in journal.project_commit_recovery_attempts(task_id))):
|
||||
raise ValueError('another native attempt is unresolved')
|
||||
# SQL/history may already have committed before a publication failure.
|
||||
self._commit_staged_project(task_id)
|
||||
for commit in commits:
|
||||
proof = json.loads(commit['proof_json'])
|
||||
if not (historical_intact(proof) if commit['session_token'] in historical else validate_project_commit_source(proof, root)):
|
||||
raise ValueError('saved source or historical proof changed during recovery')
|
||||
handoff = QuickRunResult(root/'QuickPartReplace.vbs', root/'command.tsv', root/'status.tsv', root/'quick-part-replace.log', self.vbs_root_compatible)
|
||||
self._bind_journal_handoff(task_id, handoff)
|
||||
if protection is not None:
|
||||
if flag.read_bytes() != protection:raise ValueError('protection reason changed')
|
||||
write_record(audit, {'task_id': task_id, 'session_tokens': [commit['session_token'] for commit in commits],
|
||||
'native_replacement_called': False, 'original_protection_sha256': hashlib.sha256(protection).hexdigest()})
|
||||
flag.replace(root / 'project-commit-resolved-uncertain.flag')
|
||||
details = _read_status_details(root / 'status.tsv')
|
||||
projects = tuple(project for project in journal.store.list_projects(task_id) if project.selected and project.available)
|
||||
progress = _progress_from_queue(projects, _read_commands(root/'command.tsv'), details, _read_command_sheet_targets(root/'command.tsv'))
|
||||
return QuickRunStepResult(handoff.script_path, handoff.command_path, handoff.status_path, audit,
|
||||
'saved_ok', '完整工程保存回读证明已恢复原回执;未启动 CAD 或重复替换。', progress.next_project_path,
|
||||
progress.next_old_part_number, progress.next_new_part_number, progress.next_schematic_name, progress.next_sheet_number, progress)
|
||||
except (OSError, ValueError, KeyError, TypeError) as error:
|
||||
raise TaskCenterError('quick_run_uncertain', '工程提交证明、保存源或原尝试无法核验,保护记录保持。') from error
|
||||
|
||||
def complete_project_session(self, task_id, scope, session, projects):
|
||||
"""Close the retained owner before exposing saved progress to the queue."""
|
||||
from .operation_diagnostics import record_operation
|
||||
worker = scope.worker
|
||||
if worker is not None:
|
||||
record_operation(worker.root, 'save', 'enter', item_id=worker.token)
|
||||
try:
|
||||
scope.close()
|
||||
record_operation(worker.root, 'save', 'return', item_id=worker.token)
|
||||
except Exception as error:
|
||||
record_operation(worker.root, 'save', 'error', item_id=worker.token, error=str(error))
|
||||
_write_uncertain_flag(worker.root / 'uncertain.flag', 'project_commit_unverified')
|
||||
raise
|
||||
if session is not None:
|
||||
appended, offset, identity, stamp = _read_appended_status_details(session)
|
||||
session.statuses += appended
|
||||
session.status_offset, session.status_identity, session.status_stamp = offset, identity, stamp
|
||||
session.latest = {job: (state, code) for job, state, code, _ in session.statuses}
|
||||
session.next_index = _next_unfinished_index(session.commands, session.latest)
|
||||
session.progress = _progress_from_queue(tuple(project for project in projects if project.selected and project.available),
|
||||
session.commands, session.statuses, session.sheets)
|
||||
session.completed_project_keys = {_path_key(path) for path in session.progress.completed_project_paths}
|
||||
return session.progress
|
||||
|
||||
def write_expected_baselines(self, task_id, root, project_path):
|
||||
journal=getattr(self,'journal',None)
|
||||
if journal is None:return
|
||||
rows=journal.expected_baselines(task_id,project_path)
|
||||
rows=tuple(row for row in journal.expected_baselines(task_id,project_path)
|
||||
if row.get('validation_mode') != 'saved_netlist_v1')
|
||||
text='project_path\tschematic_name\tsheet_number\tsemantic_state\r\n'
|
||||
text+=''.join('\t'.join(_encode_vbs_field(row[key]) for key in
|
||||
('project_path','schematic_name','sheet_number','state'))+'\r\n' for row in rows)
|
||||
@@ -403,7 +668,8 @@ class QuickRunPreparer:
|
||||
from .worker_monitor import WorkerMonitorError
|
||||
try:
|
||||
with ProjectSessionScope(self._process_runner,
|
||||
enabled=enabled and self._process_runner is _run_process, private_desktop=True, runtime=runtime) as scope:
|
||||
enabled=enabled and self._process_runner is _run_process and getattr(self, 'journal', None) is not None,
|
||||
private_desktop=True, runtime=runtime) as scope:
|
||||
yield scope
|
||||
except (WorkerMonitorError, subprocess.TimeoutExpired) as error:
|
||||
_write_uncertain_flag(self._data_root / 'quick-run/uncertain.flag', 'project_session_cleanup')
|
||||
@@ -428,6 +694,31 @@ class QuickRunPreparer:
|
||||
uncertain_path = self._data_root / "quick-run" / "uncertain.flag"
|
||||
if _io_path(uncertain_path).exists():
|
||||
raise TaskCenterError("quick_run_uncertain", "a previous quick-run worker timed out or returned an uncorrelated status; reconcile the project before retrying")
|
||||
if (self._process_runner is _run_process and getattr(self,'journal',None) is not None
|
||||
and (project_session is None or not project_session.enabled)):
|
||||
if not batch_mode:
|
||||
from .single_step import prepare_private_single_step
|
||||
prepare_private_single_step(self,task_id,projects,rules,copy_part_number_to_value,
|
||||
**({'runtime':runtime} if runtime is not None else {}))
|
||||
session = batch_session or self.begin_batch_session(task_id,projects,rules,
|
||||
copy_part_number_to_value=copy_part_number_to_value,runtime=runtime)
|
||||
if session.next_index >= len(session.commands):
|
||||
raise TaskCenterError('quick_run_complete','all replacement rules in this queue are complete')
|
||||
job = session.commands[session.next_index][1]
|
||||
# A standalone or explicitly nonretained call still needs one closed,
|
||||
# saved native prefix. The scope is held for this invocation only.
|
||||
with self.project_session_scope(enabled=True,runtime=runtime) as scope:
|
||||
result = self.execute_step(task_id,projects,rules,copy_part_number_to_value=copy_part_number_to_value,
|
||||
expected_item=expected_item,batch_mode=True,batch_session=session,project_session=scope,
|
||||
runtime=runtime,stop_event=stop_event)
|
||||
if result.state == 'pending_save':
|
||||
self.complete_project_session(task_id,scope,session,projects)
|
||||
if result.state not in DONE_STATES | {'pending_save'}:return result
|
||||
terminal = next(entry for entry in reversed(session.statuses) if entry[0] == job)
|
||||
progress = session.progress
|
||||
return QuickRunStepResult(result.script_path,result.command_path,result.status_path,result.log_path,
|
||||
terminal[1],terminal[3],progress.next_project_path,progress.next_old_part_number,
|
||||
progress.next_new_part_number,progress.next_schematic_name,progress.next_sheet_number,progress)
|
||||
if self._process_runner is _run_process and not batch_mode:
|
||||
from .single_step import prepare_private_single_step
|
||||
prepare_private_single_step(self, task_id, projects, rules, copy_part_number_to_value,
|
||||
@@ -636,6 +927,56 @@ class QuickRunPreparer:
|
||||
_write_uncertain_flag(uncertain_path, "malformed_status_after_worker")
|
||||
_append_log(handoff.log_path, "quick_run_uncertain malformed_status_after_worker")
|
||||
raise TaskCenterError("quick_run_uncertain", "quick-run worker left an unreadable status; reconcile the project before retrying") from error
|
||||
if completed.returncode == 75:
|
||||
if project_session is not None:
|
||||
project_session.on_commit = None
|
||||
if _io_path(uncertain_path).exists():
|
||||
raise TaskCenterError('quick_run_uncertain', '工程已保留待核验保护,请先核验原执行证据。')
|
||||
prefix_commit_pending = False
|
||||
try:
|
||||
proof = getattr(completed, 'prewrite_pause', {})
|
||||
pause_token = proof.get('session_token', '')
|
||||
marker = _io_path(handoff.command_path.parent / f'project-pause-{pause_token}.tsv')
|
||||
fields = marker.read_text(encoding='utf-8-sig').strip().split('\t')
|
||||
status_bytes = _io_path(handoff.status_path).read_bytes()
|
||||
if (runtime is None or journal_attempt is None or project_session is None
|
||||
or project_session.worker is None or pause_token != project_session.worker.token
|
||||
or proof.get('phase') != 'before_native_dispatch'
|
||||
or proof.get('session_id') != runtime.session_id or proof.get('generation') != runtime.generation
|
||||
or proof.get('attempt_id') != runtime.record.get('active_attempt')
|
||||
or proof.get('resource_record') != str(runtime.path)
|
||||
or len(fields) != 4 or fields[0] != pause_token or fields[2:] != [row['job_id'], 'prewrite_closed']
|
||||
or _path_key(_decode_vbs_field(fields[1])) != _path_key(row['project_path'])
|
||||
or proof.get('request_sha256') != hashlib.sha256(
|
||||
_io_path(handoff.command_path.with_name('project-session.request')).read_bytes()).hexdigest()
|
||||
or proof.get('current_command_sha256') != hashlib.sha256(
|
||||
_io_path(handoff.command_path.with_name('current-command.tsv')).read_bytes()).hexdigest()
|
||||
or proof.get('status_sha256') != hashlib.sha256(status_bytes).hexdigest()
|
||||
or any(entry[0] == row['job_id'] and entry[1] in TERMINAL_STATES for entry in after)
|
||||
or not ((stop_event is not None and stop_event.is_set())
|
||||
or (runtime.stop_event is not None and runtime.stop_event.is_set()))):
|
||||
raise ValueError('prewrite pause is not bound to the retained owner')
|
||||
journal.pause_before_dispatch(journal_attempt, row, dict(proof, status_size=len(status_bytes)))
|
||||
journal.finalize_startup_pause(task_id, handoff.command_path.parent)
|
||||
_append_status_details(handoff.status_path, (
|
||||
(row['job_id'], 'pending', 'prewrite_paused', '工程安全关闭;当前项确认未替换,保留待执行。'),))
|
||||
if proof.get('committed_prefix'):
|
||||
# Confirmed recovery can publish the prefix if the process
|
||||
# stops after committing SQL but before appending status.
|
||||
_write_uncertain_flag(uncertain_path, 'project_commit_unverified')
|
||||
prefix_commit_pending = True
|
||||
self._commit_staged_project(task_id, session_token=pause_token)
|
||||
if not any(commit['session_token'] == pause_token and commit['state'] == 'verified'
|
||||
and set(json.loads(commit['jobs_json'])) == set(proof['committed_prefix'])
|
||||
for commit in journal.project_commit_records(task_id)):
|
||||
raise ValueError('certified prewrite prefix has no matching verified receipt')
|
||||
_io_path(uncertain_path).unlink()
|
||||
except Exception as error:
|
||||
if not prefix_commit_pending:
|
||||
_write_uncertain_flag(uncertain_path, 'unproven_prewrite_pause')
|
||||
raise TaskCenterError('quick_run_uncertain', '暂停缺少完整的未写入与工程关闭证明;保留保护并核验。') from error
|
||||
_append_log(handoff.log_path, 'quick_run_paused prewrite_closed')
|
||||
raise TaskCenterError('operation_cancelled', '工程已安全关闭,已暂存前缀完成保存回读;当前项保留待执行。')
|
||||
if completed.returncode == 74:
|
||||
try:
|
||||
proof = getattr(completed, 'startup_pause', {})
|
||||
@@ -771,7 +1112,8 @@ class QuickRunPreparer:
|
||||
_append_log(handoff.log_path, "quick_run_uncertain native_failure_after_snapshot")
|
||||
raise TaskCenterError("quick_run_uncertain", detail)
|
||||
raise TaskCenterError("quick_run_execution", detail)
|
||||
if state not in TERMINAL_STATES:
|
||||
if state not in TERMINAL_STATES and not (state == 'pending_save' and project_session is not None
|
||||
and project_session.enabled and code in {'session_staged', 'session_staged_no_match'}):
|
||||
raise TaskCenterError("quick_run_execution", f"unexpected quick-run state: {state}")
|
||||
if batch_mode and state == "saved_ok":
|
||||
try:
|
||||
@@ -804,6 +1146,23 @@ class QuickRunPreparer:
|
||||
progress = _progress_from_queue(tuple(project for project in projects if project.selected and project.available),
|
||||
commands, after, _read_command_sheet_targets(handoff.command_path))
|
||||
try:
|
||||
if state == 'pending_save':
|
||||
proof = self.result_proof(handoff.command_path.parent, row, (state, code, message))
|
||||
journal.stage_project_result(journal_attempt, row, session_token=project_session.worker.token,
|
||||
root=handoff.command_path.parent, proof=proof)
|
||||
project_session.on_commit = lambda worker: self._commit_staged_project(task_id, session_token=worker.token, bind_source=True)
|
||||
if (next_command is None or _path_key(next_command[2]) != _path_key(row['project_path'])
|
||||
or (stop_event is not None and stop_event.is_set())):
|
||||
progress = self.complete_project_session(task_id, project_session, batch_session, projects)
|
||||
state = 'no_match' if code == 'session_staged_no_match' else 'saved_ok'
|
||||
message = ('工程统一保存,原生数据核验通过。' if proof.get('validation_required') == 'saved_netlist_v1'
|
||||
else '工程统一保存、关闭重开及回读校验通过。')
|
||||
next_command = commands[batch_session.next_index] if batch_session is not None and batch_session.next_index < len(commands) else None
|
||||
next_sheet = batch_session.sheets.get(next_command[1], (None, None)) if batch_session is not None and next_command is not None and batch_session.sheets is not None else (None, None)
|
||||
return QuickRunStepResult(handoff.script_path, handoff.command_path, handoff.status_path, handoff.log_path,
|
||||
state, message, Path(next_command[2]) if next_command is not None else None,
|
||||
next_command[3] if next_command is not None else None, next_command[4] if next_command is not None else None,
|
||||
next_sheet[0], next_sheet[1], progress)
|
||||
verified_source_health = None
|
||||
if (state in {'saved_ok', 'restored_ok', 'no_match'} and self._process_runner is _run_process
|
||||
and journal.failure_health_references(task_id, row['project_path'])):
|
||||
@@ -1196,9 +1555,21 @@ def _advance_batch_session(
|
||||
current_index = session.next_index
|
||||
current = session.commands[current_index]
|
||||
job_id, state, code, _ = changed
|
||||
if job_id != current[1] or state not in TERMINAL_STATES:
|
||||
if job_id != current[1] or state not in TERMINAL_STATES | {'pending_save'}:
|
||||
raise TaskCenterError("quick_run_queue_progress", "batch result does not complete the expected queue item")
|
||||
session.latest[job_id] = (state, code)
|
||||
if state == 'pending_save':
|
||||
from dataclasses import replace
|
||||
next_index = current_index + 1
|
||||
while next_index < len(session.commands) and session.latest.get(session.commands[next_index][1], ('', ''))[0] in TERMINAL_STATES | {'pending_save'}:
|
||||
next_index += 1
|
||||
session.next_index = next_index
|
||||
next_command = session.commands[next_index] if next_index < len(session.commands) else None
|
||||
next_sheet = session.sheets.get(next_command[1], (None, None)) if session.sheets is not None and next_command is not None else (None, None)
|
||||
session.progress = replace(session.progress, next_project_path=Path(next_command[2]) if next_command else None,
|
||||
next_old_part_number=next_command[3] if next_command else None, next_new_part_number=next_command[4] if next_command else None,
|
||||
next_schematic_name=next_sheet[0], next_sheet_number=next_sheet[1], queue_complete=False)
|
||||
return
|
||||
current_key = session.project_keys_by_index[current_index]
|
||||
partial_failures = {
|
||||
_path_key(path): (path, summary)
|
||||
@@ -1512,6 +1883,22 @@ def _recover_pre_mutation_hard_failures(
|
||||
)
|
||||
|
||||
|
||||
def _append_status_details(path: Path, statuses: tuple[tuple[str, str, str, str], ...]) -> None:
|
||||
"""Publish closed-project receipts without replacing an active batch stream."""
|
||||
payload = ''.join('\t'.join(_encode_vbs_field(field) for field in entry)+'\r\n' for entry in statuses).encode('utf-8')
|
||||
try:
|
||||
with _io_path(path).open('r+b') as stream:
|
||||
stream.seek(0,os.SEEK_END)
|
||||
end = stream.tell()
|
||||
if end:
|
||||
stream.seek(-1,os.SEEK_END)
|
||||
if stream.read(1) != b'\n':raise ValueError('cannot append to an incomplete status stream')
|
||||
stream.seek(0,os.SEEK_END)
|
||||
stream.write(payload);stream.flush();os.fsync(stream.fileno())
|
||||
except (OSError,ValueError) as error:
|
||||
raise TaskCenterError('quick_run_status','cannot publish saved project status') from error
|
||||
|
||||
|
||||
def _write_status_details(path: Path, statuses: tuple[tuple[str, str, str, str], ...]) -> None:
|
||||
temporary = path.with_name(f".{path.name}.{uuid4().hex}.tmp")
|
||||
text = "job_id\tstate\tcode\tmessage\r\n"
|
||||
@@ -2074,10 +2461,12 @@ def _correlated_status_change(
|
||||
job_id = appended[0][0]
|
||||
prior_state = next((state for status_job_id, state, _, _ in reversed(before) if status_job_id == job_id), None)
|
||||
transitions = {
|
||||
None: {"awaiting_native_replace", "hard_failure", "batch_failed", "batch_saved_unverified", "no_match"},
|
||||
None: {"awaiting_native_replace", "hard_failure", "batch_failed", "batch_saved_unverified", "no_match", "pending_save"},
|
||||
"pending": {"awaiting_native_replace", "hard_failure", "batch_failed", "batch_saved_unverified", "no_match", "pending_save"},
|
||||
"awaiting_native_replace": {
|
||||
"restored_ok",
|
||||
"saved_ok",
|
||||
"pending_save",
|
||||
"hard_failure",
|
||||
"batch_failed",
|
||||
"batch_saved_unverified",
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
"""Validate a closed project's applied targets through one saved native export."""
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
from . import native_netlist
|
||||
from .batch_resources import write_record
|
||||
from .execution_proof import proof_intact, validate_project_commit_source
|
||||
from .path_access import canonical_path, internal_path
|
||||
from .quick_run_protocol import _decode_vbs_field
|
||||
|
||||
|
||||
def _hash(path):
|
||||
return hashlib.sha256(internal_path(path).read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def _fields(line):
|
||||
return [_decode_vbs_field(value) for value in line.split('\t')]
|
||||
|
||||
|
||||
def _context(proof):
|
||||
return {key: proof[key] for key in ('session_token', 'project_path', 'saved_sources', 'command_sha256')}
|
||||
|
||||
|
||||
def _intact(record):
|
||||
return (record.get('files') and proof_intact(record)
|
||||
and _saved_sources_intact(record['context']['saved_sources'])
|
||||
and _hash(record['export_path']) == record['export_sha256'])
|
||||
|
||||
|
||||
def _saved_sources_intact(sources):
|
||||
return bool(sources) and all(_hash(path) == digest for path, digest in sources.items())
|
||||
|
||||
|
||||
def _read_export_pointer(root):
|
||||
try:
|
||||
record = json.loads(internal_path(root / 'saved-netlist-export.json').read_text(encoding='utf-8'))
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
if not isinstance(record, dict) or record.get('status') not in {'exported', 'verified'}:
|
||||
raise ValueError('invalid saved native export lookup')
|
||||
return record
|
||||
|
||||
|
||||
def _authoritative_export(record):
|
||||
"""The mutable lookup record cannot supply graph data or captured inputs."""
|
||||
result = json.loads(internal_path(record['export_path']).read_text(encoding='utf-8'))
|
||||
capture = result['capture']
|
||||
source_hashes = {str(canonical_path(path)).casefold():digest for path,digest in capture['source_hashes'].items()}
|
||||
if (result.get('status') != 'success' or result.get('complete') is not True
|
||||
or canonical_path(capture['project']) != canonical_path(record['context']['project_path'])
|
||||
or any(source_hashes.get(str(canonical_path(path)).casefold()) != digest
|
||||
for path,digest in record['context']['saved_sources'].items())
|
||||
or set(result['outputs'].values()) != set(record['output_hashes'])
|
||||
or any(_hash(path) != record['output_hashes'][path] for path in result['outputs'].values())):
|
||||
raise ValueError('immutable saved native export differs from its original source binding')
|
||||
native_netlist._check_capture(capture)
|
||||
return result
|
||||
|
||||
|
||||
def load_saved_export(root, project):
|
||||
"""Return a verified intact shared export for the current saved source, or None."""
|
||||
root = Path(root)
|
||||
try:
|
||||
record = _read_export_pointer(root)
|
||||
if record is None or record['status'] == 'exported':
|
||||
return None
|
||||
context = record['context']
|
||||
if (canonical_path(context['project_path']) != canonical_path(project)
|
||||
or context['command_sha256'] != _hash(root / 'command.tsv')):
|
||||
return None
|
||||
if _hash(record['verification_path']) != record['verification_sha256']:
|
||||
raise ValueError('immutable saved target verification changed')
|
||||
verified = json.loads(internal_path(record['verification_path']).read_text(encoding='utf-8'))
|
||||
if any(verified[key] != record[key] for key in ('status', 'context', 'export_path', 'export_sha256')):
|
||||
raise ValueError('saved target lookup differs from its immutable verification')
|
||||
if not context['saved_sources']:
|
||||
raise ValueError('saved source hashes are missing')
|
||||
if not _saved_sources_intact(context['saved_sources']):
|
||||
return None
|
||||
marker = root / f'project-commit-{context["session_token"]}.tsv'
|
||||
if _hash(marker) != context['marker_sha256'] or not _intact(record):
|
||||
raise ValueError('saved marker or export artifacts changed')
|
||||
record['jobs'] = verified['jobs']
|
||||
record['result'] = _authoritative_export(record)
|
||||
record['capture'] = record['result']['capture']
|
||||
return record
|
||||
except (OSError, ValueError, KeyError, TypeError, AttributeError) as error:
|
||||
raise ValueError(f'saved native export integrity check failed: {error}') from error
|
||||
|
||||
|
||||
def compact_saved_netlist(record, *, include_jobs=True):
|
||||
result = {key: record[key] for key in ('status', 'export_path', 'export_sha256')}
|
||||
result.update(verification_path=record['verification_path'], verification_sha256=record['verification_sha256'])
|
||||
if include_jobs:
|
||||
result.update(context=record['context'], jobs=record['jobs'], files=record['files'])
|
||||
return result
|
||||
|
||||
|
||||
def _expectations(root, row):
|
||||
from .execution_proof import capture_result
|
||||
from .native_helper import read_snapshot, require_resumable_transaction, saved_netlist_snapshot
|
||||
snapshot = internal_path(root / f'snapshot-{row["job_id"]}.tsv')
|
||||
if not saved_netlist_snapshot(snapshot):raise ValueError('saved netlist target snapshot protocol required')
|
||||
# This validates the command, selected original target set and native attempt.
|
||||
capture_result(root.parent, root, row, ('pending_save', 'session_staged', ''), strict=True, persist=False)
|
||||
metadata, components = read_snapshot(snapshot)
|
||||
report_path = Path(str(snapshot) + '.native.json')
|
||||
report = json.loads(report_path.read_text(encoding='utf-8'))
|
||||
binding = require_resumable_transaction(snapshot, report, metadata, components)
|
||||
library = binding['library_row']
|
||||
if library[1].casefold() != row['new_part'].casefold():raise ValueError('native library target changed')
|
||||
writer = internal_path(root / f'writer-expectations-{row["job_id"]}.tsv')
|
||||
records = [_fields(line) for line in writer.read_text(encoding='utf-8-sig').splitlines() if line]
|
||||
header = ['W', '1', row['task_id'], row['job_id'], row['project_path'],
|
||||
row.get('schematic_name',''), row.get('sheet_number',''), snapshot.read_bytes().decode('utf-8-sig')]
|
||||
if not records or records[0] != header:raise ValueError('writer expectations are not bound to the original snapshot and command')
|
||||
original = {item[1]: item[2] for item in components}
|
||||
if row.get('target_scope') == 'targets':
|
||||
selectors = [value.split('\x1f') for value in row.get('target_selectors','').split('\x1e')]
|
||||
if (not selectors or any(len(value) != 3 or not value[1].strip() for value in selectors)
|
||||
or {value[1].strip().casefold() for value in selectors} != {ref.strip().casefold() for ref in original.values()}):
|
||||
raise ValueError('native snapshot reference set differs from the frozen target selectors')
|
||||
old_values = {}
|
||||
for line in snapshot.read_text(encoding='utf-8-sig').splitlines():
|
||||
fields = _fields(line)
|
||||
if fields[0] == 'A' and len(fields) == 12 and fields[2].casefold() == 'value':
|
||||
if fields[1] in old_values:raise ValueError('original Value is ambiguous')
|
||||
old_values[fields[1]] = fields[4] or fields[3]
|
||||
has_nc = lambda text: '/' in text and any(part.strip().casefold() == 'nc' for part in text.split('/'))
|
||||
expected = {}
|
||||
for fields in records[1:]:
|
||||
if len(fields) != 9 or fields[0] != 'C':raise ValueError('malformed writer target expectation')
|
||||
_, uid, ref, value, present, name, label, part, cell = fields
|
||||
if (uid not in original or uid in expected or ref != original[uid] or present not in {'0','1'}
|
||||
or name != library[4] or label != library[5] or part.casefold() != row['new_part'].casefold()
|
||||
or cell != row.get('target_cell','')):
|
||||
raise ValueError('writer target expectation differs from native library/command')
|
||||
mode = row['value_mode']
|
||||
if mode != 'preserve':
|
||||
policy_value = {'custom': row.get('custom_value',''), 'part_number': row['new_part'], 'part_label': library[5]}[mode]
|
||||
if row.get('sync_nc','1') == '1' and has_nc(old_values.get(uid,'')) and not has_nc(policy_value):
|
||||
policy_value = policy_value.rstrip(' ') + ('NC' if policy_value.rstrip(' ').endswith('/') else '/NC')
|
||||
if present != '1' or value != policy_value:raise ValueError('writer Value contradicts the frozen write policy')
|
||||
if row.get('sync_nc','1') == '1' and has_nc(old_values.get(uid,'')) and not has_nc(value):
|
||||
raise ValueError('required original NC marker was not retained')
|
||||
expected[uid] = dict(refdes=ref, value=value, present=present == '1', old_value=old_values.get(uid),
|
||||
attributes={'Part Number': part, 'Part Name': name, 'Part Label': label}, target_cell=cell)
|
||||
if set(expected) != set(original):raise ValueError('writer target set is incomplete')
|
||||
return expected, binding, [snapshot, report_path, writer]
|
||||
|
||||
|
||||
def _scope(component, row):
|
||||
return all(not row.get(key) or component.get(field) == row[key]
|
||||
for key, field in (('schematic_name','source_schematic'), ('sheet_number','source_page')))
|
||||
|
||||
|
||||
def _verify_no_match(graph, row):
|
||||
components = graph['components']
|
||||
if row.get('target_scope') == 'targets':
|
||||
selectors = [item.split('\x1f') for item in row.get('target_selectors', '').split('\x1e')]
|
||||
if any(len(item) != 3 or not item[1].strip() for item in selectors):
|
||||
raise ValueError('saved native no-match coverage lacks valid frozen target selectors')
|
||||
for _, ref, _ in selectors:
|
||||
keys = native_netlist.reference_targets(graph, ref, sheet=row.get('sheet_number', ''),
|
||||
schematic=row.get('schematic_name', ''))
|
||||
if not keys or any(not _scope(components[key], row) for key in keys):
|
||||
raise ValueError(f'saved native no-match coverage lacks selected reference {ref}')
|
||||
# A frozen selector can return no_match only through already_target in the writer.
|
||||
if any(components[key]['attributes'].get('Part Number', '').casefold() != row['new_part'].casefold()
|
||||
for key in keys):
|
||||
raise ValueError(f'saved native no-match Part Number differs from the target for {ref}')
|
||||
return
|
||||
candidates = [item for component in components.values()
|
||||
for item in component.get('annotation_instances', [component]) if _scope(item, row)]
|
||||
if not candidates and [row['schematic_name'], row['sheet_number']] not in graph.get('source_sheets', ()):
|
||||
raise ValueError('saved native no-match coverage lacks the frozen schematic or sheet')
|
||||
if any(item['attributes'].get('Part Number', '').casefold() == row['old_part'].casefold() for item in candidates):
|
||||
raise ValueError('native no-match claim still has original targets in the saved scope')
|
||||
|
||||
|
||||
def _observations(graph, row, expected, binding):
|
||||
observations = []
|
||||
counts = Counter(item['refdes'].strip().casefold() for item in expected.values())
|
||||
remaining = {}
|
||||
for uid, item in expected.items():
|
||||
ref = item['refdes'].strip().casefold()
|
||||
if ref not in remaining:
|
||||
keys = native_netlist.reference_targets(graph, item['refdes'],
|
||||
sheet=row.get('sheet_number',''), schematic=row.get('schematic_name',''))
|
||||
if len(keys) != counts[ref] or not keys:
|
||||
raise ValueError('saved native target units are missing or ambiguous')
|
||||
if any(not _scope(graph['components'][key],row) or graph['components'][key].get('ambiguous_attributes') for key in keys):
|
||||
raise ValueError('saved native target sheet or attributes are ambiguous')
|
||||
remaining[ref] = list(keys)
|
||||
# Match the declared multiset within one physical reference. Native UIDs
|
||||
# may change; no expected property is inferred from the exported graph.
|
||||
matches = []
|
||||
for key in remaining[ref]:
|
||||
attrs = graph['components'][key]['attributes']
|
||||
if (all(attrs.get(name) is not None and
|
||||
(attrs[name].casefold() == value.casefold() if name == 'Part Number' else attrs[name] == value)
|
||||
for name,value in item['attributes'].items())
|
||||
and (not item['target_cell'] or attrs.get('Cell Name') == item['target_cell'])
|
||||
and attrs.get('Value','') == item['value']):
|
||||
matches.append(key)
|
||||
if not matches:raise ValueError('saved native target library attributes, Cell or Value differ from the write policy')
|
||||
key = matches[0]
|
||||
remaining[ref].remove(key)
|
||||
symbol = graph['components'][key].get('symbol','')
|
||||
if symbol:
|
||||
target = re.fullmatch(r'(?:[^:\s]+:)?([^:\s]+?)(?:\.(\d+))?', binding['target_symbol'])
|
||||
actual = re.match(r'([^\s]+?)\.(\d+) type ', symbol)
|
||||
if (not target or not actual or target[1] != actual[1]
|
||||
or target[2] is not None and target[2] != actual[2]
|
||||
or not 1 <= int(actual[2]) <= binding['slot_count']):
|
||||
raise ValueError('saved native target symbol differs from the selected library')
|
||||
actual = graph['components'][key]['attributes']
|
||||
observations.append(dict(uid=uid, refdes=item['refdes'], old_part_number=row['old_part'],
|
||||
actual_part_number=actual['Part Number'], old_value=item['old_value'], actual_value=actual.get('Value',''),
|
||||
old_value_present=item['old_value'] is not None,
|
||||
actual_value_present=True if actual.get('Value') else None,
|
||||
value_presence_coverage='native_ascii_empty_presence_unknown', actual_cell=actual.get('Cell Name','')))
|
||||
return dict(value_readback_status='verified', value_readback=observations, value_readback_job_id=row['job_id'],
|
||||
project_path=row['project_path'], schematic_name=row.get('schematic_name',''), sheet_number=row.get('sheet_number',''),
|
||||
coverage_limits=['Native ASCII empty Value does not prove attribute presence; CAD display/layout is not checked.'])
|
||||
|
||||
|
||||
def verify_saved_project(data_root, root, proof, rows, *, stop_event=None):
|
||||
"""Export once, validate all applied targets, and return the verified commit proof."""
|
||||
from .netlist_verification import _export_saved_capture
|
||||
root, data_root = Path(root), Path(data_root)
|
||||
if proof.get('validation_required') != 'saved_netlist_v1' or not validate_project_commit_source(proof, root):
|
||||
raise ValueError('saved source-bound project proof is required')
|
||||
outcomes = proof.get('native_outcomes', proof['outcomes'])
|
||||
readbacks = proof.get('readback_files')
|
||||
required = {name for job, state in outcomes.items() if state == 'pending_save'
|
||||
for name in (f'snapshot-{job}.tsv', f'snapshot-{job}.tsv.native.json', f'writer-expectations-{job}.tsv')}
|
||||
|
||||
def readbacks_intact():
|
||||
return (isinstance(readbacks, dict) and required <= readbacks.keys()
|
||||
and all(Path(name).name == name and _hash(root / name) == digest
|
||||
for name, digest in readbacks.items()))
|
||||
|
||||
if not readbacks_intact():raise ValueError('native writer evidence changed before saved export')
|
||||
rows = {row['job_id']: row for row in rows}
|
||||
if not set(proof['jobs']) <= rows.keys():raise ValueError('saved prefix is outside the frozen command plan')
|
||||
context = _context(proof)
|
||||
context['marker_sha256'] = proof['files'][0]['sha256']
|
||||
evidence, inputs = {}, []
|
||||
for job in proof['jobs']:
|
||||
if outcomes[job] == 'pending_save':
|
||||
evidence[job] = _expectations(root, rows[job])
|
||||
inputs.extend(evidence[job][2])
|
||||
elif not rows[job].get('schematic_name') or not rows[job].get('sheet_number'):
|
||||
raise ValueError('saved native no-match coverage requires a frozen schematic and sheet')
|
||||
pointer = root / 'saved-netlist-export.json'
|
||||
record = None
|
||||
candidate = _read_export_pointer(root)
|
||||
if candidate is not None and candidate['context'] == context:
|
||||
if not _intact(candidate):
|
||||
raise ValueError('saved native export changed before target validation')
|
||||
record = candidate
|
||||
if record is None:
|
||||
directory = data_root / 'execution-proof' / ('saved-netlist-' + proof['session_token'])
|
||||
capture_path = directory / 'capture/capture.json'
|
||||
if capture_path.is_file():
|
||||
capture = json.loads(capture_path.read_text(encoding='utf-8'))
|
||||
native_netlist._check_capture(capture)
|
||||
source_hashes = {str(canonical_path(path)).casefold():digest for path,digest in capture['source_hashes'].items()}
|
||||
if (canonical_path(capture['project']) != canonical_path(proof['project_path'])
|
||||
or any(source_hashes.get(str(canonical_path(path)).casefold()) != digest for path,digest in proof['saved_sources'].items())):
|
||||
raise ValueError('saved native capture belongs to a different original source')
|
||||
else:
|
||||
capture = native_netlist.capture_project(Path(proof['project_path']), directory / 'capture')
|
||||
if not validate_project_commit_source(proof, root):raise ValueError('saved source changed while capturing the native database')
|
||||
result = _export_saved_capture(capture, directory / 'export', stop_event=stop_event)
|
||||
if result.get('status') != 'success' or result.get('complete') is not True:
|
||||
raise ValueError('complete native saved export unavailable: ' + str(result.get('reason','')))
|
||||
manifest = directory / 'export/netlist.json'
|
||||
paths = [Path(capture['manifest_path']), manifest, *map(Path,result['outputs'].values()),
|
||||
*(Path(capture['root']) / name for name in capture['capture_hashes'])]
|
||||
for path in inputs:
|
||||
target = internal_path(directory / 'inputs' / path.name)
|
||||
target.parent.mkdir(parents=True,exist_ok=True)
|
||||
data = internal_path(path).read_bytes()
|
||||
if hashlib.sha256(data).hexdigest() != readbacks[path.name]:
|
||||
raise ValueError('native target evidence changed during capture')
|
||||
target.write_bytes(data)
|
||||
if internal_path(path).read_bytes() != data:raise ValueError('native target evidence changed during capture')
|
||||
paths.append(target)
|
||||
files = [{'path':str(canonical_path(path)), 'sha256':_hash(path)} for path in dict.fromkeys(paths)]
|
||||
record = dict(status='exported', context=context, capture=capture, result=result,
|
||||
export_path=str(canonical_path(manifest)), export_sha256=_hash(manifest),
|
||||
output_hashes={str(canonical_path(path)):_hash(path) for path in result['outputs'].values()}, files=files)
|
||||
write_record(pointer, record)
|
||||
record['result'] = _authoritative_export(record)
|
||||
record['capture'] = record['result']['capture']
|
||||
graph, jobs = record['result']['graph'], {}
|
||||
for job in proof['jobs']:
|
||||
row = rows[job]
|
||||
if job in evidence:
|
||||
expected, binding, _ = evidence[job]
|
||||
jobs[job] = _observations(graph, row, expected, binding)
|
||||
else:
|
||||
_verify_no_match(graph, row)
|
||||
jobs[job] = dict(value_readback_status='verified',value_readback=[],value_readback_job_id=job,
|
||||
project_path=row['project_path'],schematic_name=row.get('schematic_name',''),sheet_number=row.get('sheet_number',''))
|
||||
if not validate_project_commit_source(proof,root) or not _intact(record) or not readbacks_intact():
|
||||
raise ValueError('saved source/export or writer evidence changed during target validation')
|
||||
record.update(status='verified',jobs=jobs)
|
||||
verification = Path(record['export_path']).parent / 'verification.json'
|
||||
verification_value = {key:record[key] for key in ('status','context','export_path','export_sha256','jobs')}
|
||||
if verification.is_file():
|
||||
if json.loads(verification.read_text(encoding='utf-8')) != verification_value:
|
||||
raise ValueError('saved native target verification changed on resume')
|
||||
else:write_record(verification,verification_value)
|
||||
record.update(verification_path=str(canonical_path(verification)),verification_sha256=_hash(verification))
|
||||
if not any(item['path'] == str(canonical_path(verification)) for item in record['files']):
|
||||
record['files'].append({'path':str(canonical_path(verification)),'sha256':_hash(verification)})
|
||||
write_record(pointer,record)
|
||||
result = dict(proof, saved_netlist=compact_saved_netlist(record))
|
||||
result['native_outcomes'] = outcomes
|
||||
result['outcomes'] = {job:'saved_ok' if state == 'pending_save' else state for job,state in result['native_outcomes'].items()}
|
||||
result['files'] = list({(item['path'],item['sha256']):item for item in [*proof['files'],*record['files']]}.values())
|
||||
return result
|
||||
@@ -0,0 +1,160 @@
|
||||
"""Read saved sources after native close, retrying only actual Windows locks."""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .file_lock_diagnostics import locking_processes
|
||||
from .path_access import internal_path
|
||||
|
||||
|
||||
LOCK_WAIT_SECONDS = 3.0
|
||||
|
||||
|
||||
class SavedSourceReadError(TaskCenterError):
|
||||
def __init__(self, error, path, *, interrupted=False):
|
||||
self.winerror = getattr(error, 'winerror', None)
|
||||
self.path = str(path)
|
||||
self.retryable = self.winerror in (32, 33)
|
||||
self.interrupted = interrupted
|
||||
reason = '核验已暂停' if interrupted else '数据库仍被占用' if self.retryable else '源文件读取失败'
|
||||
super().__init__('saved_source_pending',
|
||||
f'工程已返回保存,{reason},结果待核验;不会重复替换。文件:{path};Windows 错误码:{self.winerror};{error}')
|
||||
|
||||
|
||||
def source_hash(path):
|
||||
"""Retain CreateFile's error code; Python open can reduce sharing errors to EACCES."""
|
||||
path = internal_path(path)
|
||||
if os.name != 'nt':
|
||||
with path.open('rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
import msvcrt
|
||||
import pywintypes
|
||||
import win32file
|
||||
try:
|
||||
# Match the CRT reads used by subsequent source validation.
|
||||
handle = win32file.CreateFile(str(path), 0x80000000, 3, None, 3, 0, None)
|
||||
except pywintypes.error as error:
|
||||
failure = OSError(error.winerror, error.strerror, str(path))
|
||||
failure.winerror = error.winerror
|
||||
raise failure from error
|
||||
try:
|
||||
fd = msvcrt.open_osfhandle(int(handle), os.O_RDONLY | os.O_BINARY)
|
||||
handle.Detach() # The file descriptor now owns this handle.
|
||||
finally:
|
||||
handle.Close()
|
||||
with os.fdopen(fd, 'rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
|
||||
|
||||
def bind_sources(proof, data_root, root, project, *, stop_event=None, release_mentor=None):
|
||||
from .batch_resources import write_record
|
||||
from .operation_diagnostics import record_operation
|
||||
from .project_session_health import _source_parameters, _source_digest
|
||||
from .project_isolation import write_sources
|
||||
from .worker_monitor import _publish_progress
|
||||
root = Path(root)
|
||||
parameters = _source_parameters(Path(project), data_root)
|
||||
started = time.monotonic()
|
||||
deadline = started + LOCK_WAIT_SECONDS
|
||||
status = dict(session_token=proof.get('session_token'), project_path=str(project),
|
||||
replacement_replayed=False, mentor_released_for_read=False)
|
||||
try:
|
||||
prior = json.loads(internal_path(root / 'saved-source-status.json').read_text(encoding='utf-8'))
|
||||
if (proof.get('session_token') and prior.get('session_token') == proof['session_token']
|
||||
and prior.get('project_path', '').casefold() == str(project).casefold()):
|
||||
status.update({key: prior[key] for key in ('first_error', 'last_error', 'lock_observation') if key in prior})
|
||||
status['mentor_released_for_read'] = prior.get('mentor_released_for_read') is True
|
||||
except (OSError, ValueError, AttributeError, TypeError):
|
||||
pass # Historical diagnostics never authorize a saved result.
|
||||
last_lock = None
|
||||
observed = False
|
||||
released_this_call = False
|
||||
|
||||
def publish(state, error=None):
|
||||
status.update(state=state, elapsed_seconds=time.monotonic()-started)
|
||||
if error is not None:
|
||||
status['last_error'] = dict(path=getattr(error, 'path', str(project)),
|
||||
winerror=getattr(error, 'winerror', None), message=str(error))
|
||||
status.setdefault('first_error', status['last_error'])
|
||||
write_record(root / 'saved-source-status.json', status)
|
||||
try:
|
||||
progress = json.loads(internal_path(root / 'worker-progress.json').read_text(encoding='utf-8'))
|
||||
except (OSError, ValueError):
|
||||
progress = {}
|
||||
if not isinstance(progress, dict) or type(progress.get('sequence', 0)) is not int:
|
||||
progress = {}
|
||||
progress.update(stage='saved_source_wait' if state in {'waiting', 'releasing_mentor'} else
|
||||
'awaiting_reconciliation' if state == 'pending' else 'verify_saved_project',
|
||||
wait_reason='readback', waiting_for_safe_completion=False,
|
||||
sequence=progress.get('sequence', 0)+1, stage_seconds={})
|
||||
if state == 'pending': progress['primary_error'] = str(error)
|
||||
_publish_progress(internal_path(root / 'worker-progress.json'), progress)
|
||||
|
||||
def publish_read(state, error):
|
||||
try:
|
||||
publish(state, error)
|
||||
except Exception as diagnostic:
|
||||
error.add_note('保存源文件交接状态记录失败:' + str(diagnostic))
|
||||
raise error from diagnostic
|
||||
|
||||
def check_paused():
|
||||
if last_lock is not None and stop_event is not None and stop_event.is_set():
|
||||
raise SavedSourceReadError(last_lock[0], last_lock[1], interrupted=True) from last_lock[0]
|
||||
|
||||
record_operation(root, 'saved_source_bind', 'enter', item_id=proof.get('session_token', ''))
|
||||
try:
|
||||
while True:
|
||||
check_paused()
|
||||
path = project
|
||||
try:
|
||||
before = _source_digest(parameters)
|
||||
hashes = {}
|
||||
for path in sorted(write_sources(project)):
|
||||
hashes[path] = source_hash(path)
|
||||
if not before or before != _source_digest(parameters):
|
||||
raise TaskCenterError('saved_source_changed', '保存后的源文件仍在变化,结果待核验;不会重复替换。')
|
||||
except OSError as failure:
|
||||
error = SavedSourceReadError(failure, path)
|
||||
stopped = stop_event is not None and stop_event.is_set()
|
||||
if stopped:
|
||||
raise SavedSourceReadError(failure, path, interrupted=True) from failure
|
||||
if not error.retryable:
|
||||
raise error from failure
|
||||
last_lock = failure, path
|
||||
if not observed:
|
||||
publish_read('waiting', error)
|
||||
status['lock_observation'] = locking_processes(path)
|
||||
observed = True
|
||||
continue # The lock may have disappeared during this diagnostic.
|
||||
if time.monotonic() >= deadline:
|
||||
if release_mentor is None or released_this_call:
|
||||
raise error from failure
|
||||
publish_read('releasing_mentor', error)
|
||||
check_paused()
|
||||
release_mentor(error)
|
||||
released_this_call = True
|
||||
status['mentor_released_for_read'] = True
|
||||
deadline = time.monotonic() + LOCK_WAIT_SECONDS
|
||||
else:
|
||||
if status.get('state') != 'waiting': publish_read('waiting', error)
|
||||
time.sleep(.05)
|
||||
continue
|
||||
|
||||
command_hash = hashlib.sha256(internal_path(root / 'command.tsv').read_bytes()).hexdigest()
|
||||
check_paused()
|
||||
try:
|
||||
publish('bound')
|
||||
except OSError as diagnostic:
|
||||
record_operation(root, 'saved_source_diagnostic', 'error', error=str(diagnostic))
|
||||
record_operation(root, 'saved_source_bind', 'return', item_id=proof.get('session_token', ''))
|
||||
return dict(proof, **parameters, source_sha256=before, saved_sources=hashes, command_sha256=command_hash)
|
||||
except Exception as error:
|
||||
record_operation(root, 'saved_source_bind', 'error', item_id=proof.get('session_token', ''), error=str(error))
|
||||
try:
|
||||
publish('pending', error)
|
||||
except Exception as diagnostic:
|
||||
error.add_note('保存源文件交接状态记录失败:' + str(diagnostic))
|
||||
raise
|
||||
@@ -1117,8 +1117,9 @@ class TaskCenterService:
|
||||
summary='quick_run_result:failed | '+message)
|
||||
if task.status in {TaskStatus.DRAFT,TaskStatus.READY,TaskStatus.PAUSED}:
|
||||
self._store.set_task_status(task_id,TaskStatus.RUNNING,summary=summary)
|
||||
self._store.set_task_status(task_id,TaskStatus.FAILED,summary=summary)
|
||||
elif task.status is TaskStatus.RUNNING:
|
||||
self._store.set_task_status(task_id,TaskStatus.RUNNING,summary=summary)
|
||||
self._store.set_task_status(task_id,TaskStatus.FAILED,summary=summary)
|
||||
HistoryRepository(self._store).append(task_id=task_id,task_name=task.name,
|
||||
operation='工程执行收尾',state='failed',summary='本工程安全失败,剩余项暂缓;继续后续工程。',
|
||||
details={'reason':message})
|
||||
@@ -1147,7 +1148,15 @@ class TaskCenterService:
|
||||
child._quick_run_preparer.data_root, started_after)
|
||||
if getattr(self, '_supervisor_waiting', False):
|
||||
from dataclasses import replace
|
||||
value = replace(value, stage='awaiting_reconciliation', waiting_for_safe_completion=True)
|
||||
from .execution_feedback import wait_reason_for_stage
|
||||
state = getattr(self, '_supervisor_state', None) or {}
|
||||
wait_reason = state.get('wait_reason') or value.wait_reason or wait_reason_for_stage(value.stage)
|
||||
stopping_reason = state.get('stopping_reason') or value.stopping_reason
|
||||
if not stopping_reason and getattr(self, '_supervisor_business_stalled', False):
|
||||
stopping_reason = 'inactivity'
|
||||
value = replace(value, waiting_for_safe_completion=True,
|
||||
wait_reason=wait_reason, stopping_reason=stopping_reason,
|
||||
primary_error=state.get('primary_error') or value.primary_error)
|
||||
if read_control is not None:
|
||||
read_control.checkpoint()
|
||||
return value
|
||||
@@ -1175,6 +1184,38 @@ class TaskCenterService:
|
||||
return self._quick_run_preparer.has_interrupted(task_id)
|
||||
|
||||
def recover_quick_run(self, task_id: str, *, confirmed: bool = False, runtime=None, stop_event=None) -> QuickRunStepResult:
|
||||
if self._store.get_task(task_id).status is TaskStatus.DELETED:
|
||||
raise TaskCenterError("quick_run_recovery", "请先恢复已删除的任务记录,再核验其中断步骤。")
|
||||
from .project_isolation import dependency_block, recover_dependency
|
||||
dependency = dependency_block(self._logging.root, task_id)
|
||||
dispatched = None
|
||||
if dependency is not None and dependency['state'] in {'readmission_prepared', 'readmitted'}:
|
||||
with self._store._lock:
|
||||
dispatched = self._store._connection.execute('''SELECT 1 FROM execution_operations o
|
||||
JOIN execution_rounds r ON r.task_id=o.task_id AND r.round_id=o.round_id
|
||||
WHERE o.task_id=? AND (o.attempt_id IS NOT NULL OR o.generation>0) LIMIT 1''', (task_id,)).fetchone()
|
||||
if dependency is not None and not dispatched:
|
||||
if stop_event is not None and stop_event.is_set():
|
||||
raise TaskCenterError('operation_cancelled', '共享数据源依赖核验已取消。')
|
||||
from .task_queue import assert_task_queue_idle
|
||||
assert_task_queue_idle(self._logging.root)
|
||||
with _quick_run_lease(self._logging.root):
|
||||
return self._operation('recover_quick_run', task_id,
|
||||
lambda: recover_dependency(self, task_id, confirmed=confirmed))
|
||||
commit_recovery = getattr(self._project_task_service(task_id)._quick_run_preparer, 'recover_project_commit', None)
|
||||
if commit_recovery is not None:
|
||||
from .batch_resources import guard_pending_resources
|
||||
from .task_queue import assert_task_queue_idle
|
||||
assert_task_queue_idle(self._logging.root)
|
||||
with _quick_run_lease(self._logging.root):
|
||||
guard_pending_resources(self._logging.root)
|
||||
result = self._operation('recover_project_commit', task_id,
|
||||
lambda: commit_recovery(task_id, confirmed=confirmed))
|
||||
if result is not None:
|
||||
if self._store.get_task(task_id).status in {TaskStatus.BLOCKED, TaskStatus.FAILED, TaskStatus.PAUSED}:
|
||||
self._store.set_task_status(task_id, TaskStatus.READY, summary='原工程提交已通过持久化保存回读证据核验。')
|
||||
self._synchronize_quick_run_progress(task_id)
|
||||
return result
|
||||
if runtime is None and self._supervised:
|
||||
return self._supervise('recover_quick_run', {'task_id': task_id, 'confirmed': confirmed}, stop_event=stop_event)
|
||||
if runtime is None:
|
||||
@@ -1221,7 +1262,7 @@ class TaskCenterService:
|
||||
self._quick_run_preparer._bind_journal_handoff(task_id,
|
||||
QuickRunResult(result.script_path, result.command_path, result.status_path,
|
||||
handoff_root/'quick-part-replace.log', True))
|
||||
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED}:
|
||||
if task.status in {TaskStatus.PAUSED, TaskStatus.FAILED, TaskStatus.BLOCKED}:
|
||||
self._store.set_task_status(task_id, TaskStatus.READY, summary="interrupted quick-run reconciled")
|
||||
from .recovery_commit import marker_path
|
||||
marker_path(self._quick_run_preparer).unlink(missing_ok=True)
|
||||
@@ -1409,8 +1450,8 @@ class TaskCenterService:
|
||||
rules,
|
||||
copy_part_number_to_value=task.copy_part_number_to_value,
|
||||
)
|
||||
if self._supervised:
|
||||
return progress # Only the batch process persists live execution state.
|
||||
if self._supervised or task.status is TaskStatus.BLOCKED:
|
||||
return progress # BLOCKED classification changes only through explicit reconciliation.
|
||||
by_path = {_path_key(project.path): project.project_id for project in projects}
|
||||
project_by_id = {project.project_id: project for project in projects}
|
||||
summary_by_id = {
|
||||
@@ -1457,7 +1498,7 @@ class TaskCenterService:
|
||||
) or (
|
||||
progress.queue_complete
|
||||
and (
|
||||
(bool(failed_project_ids) and (task.status is not TaskStatus.RUNNING or task.summary != retry_summary))
|
||||
(bool(failed_project_ids) and (task.status is not TaskStatus.FAILED or task.summary != retry_summary))
|
||||
or (not failed_project_ids and task.status is not TaskStatus.COMPLETED)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -80,7 +80,7 @@ def read_target_plan(store, task_id: str, path: Path) -> tuple[TargetPlanRow, ..
|
||||
raise ValueError("invalid or formula plan row")
|
||||
identity = tuple("" if value is None else str(value) for value in values[:8])
|
||||
key = (str(Path(identity[0]).resolve(strict=False)).casefold(),
|
||||
identity[2], identity[3], identity[4])
|
||||
identity[2], identity[3], identity[5].strip().casefold())
|
||||
original = expected.get(key)
|
||||
if original is None or key in by_key:
|
||||
raise ValueError("forged or duplicate target")
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Separate ordinary stop intent from an explicitly confirmed forced interruption."""
|
||||
from threading import Event
|
||||
import time
|
||||
import json
|
||||
import os
|
||||
|
||||
@@ -23,6 +24,13 @@ class StopControl(Event):
|
||||
self.resolution_error = ''
|
||||
self.cleanup_mode = None
|
||||
self.cleanup_reason = ''
|
||||
self.stopping_reason = ''
|
||||
self.wait_reason = ''
|
||||
self.primary_error = ''
|
||||
|
||||
def set(self):
|
||||
self.stopping_reason = self.stopping_reason or 'user_pause'
|
||||
super().set()
|
||||
|
||||
def request_force(self):
|
||||
self.set()
|
||||
@@ -33,10 +41,64 @@ class StopControl(Event):
|
||||
protect_interruption(root, MENTOR_RELEASE_REQUIRED + ': ' + str(error))
|
||||
self.cleanup_mode = MENTOR_RELEASE_REQUIRED
|
||||
self.cleanup_reason = self.cleanup_reason or str(error)
|
||||
self.primary_error = self.primary_error or str(error)
|
||||
self.stopping_reason = 'failure_cleanup'
|
||||
self.set()
|
||||
self.force_event.set()
|
||||
|
||||
|
||||
class _ProjectForce:
|
||||
def __init__(self, parent):
|
||||
self.local = Event()
|
||||
self.parent = getattr(parent, 'force_event', None)
|
||||
|
||||
def is_set(self):
|
||||
return self.local.is_set() or self.parent is not None and self.parent.is_set()
|
||||
|
||||
def set(self):
|
||||
self.local.set()
|
||||
|
||||
def clear(self):
|
||||
self.local.clear()
|
||||
if self.parent is not None:
|
||||
self.parent.clear()
|
||||
|
||||
def wait(self, timeout=None):
|
||||
deadline = None if timeout is None else time.monotonic() + timeout
|
||||
while not self.is_set():
|
||||
remaining = .1 if deadline is None else min(.1, deadline - time.monotonic())
|
||||
if remaining <= 0:
|
||||
return False
|
||||
self.local.wait(remaining)
|
||||
return True
|
||||
|
||||
|
||||
class ProjectStopControl(StopControl):
|
||||
"""An engineering failure stops its owner; user pause still stops the batch."""
|
||||
def __init__(self, parent):
|
||||
super().__init__()
|
||||
self.parent = parent
|
||||
self.force_event = _ProjectForce(parent)
|
||||
|
||||
def is_set(self):
|
||||
return self.parent.is_set() or super().is_set()
|
||||
|
||||
def set(self):
|
||||
if self.stopping_reason in {'failure_cleanup','inactivity'}:
|
||||
Event.set(self)
|
||||
else:
|
||||
self.parent.set()
|
||||
|
||||
def request_failure_cleanup(self, root, error):
|
||||
protect_interruption(root, MENTOR_RELEASE_REQUIRED + ': ' + str(error))
|
||||
self.stopping_reason = 'failure_cleanup'
|
||||
self.cleanup_mode = MENTOR_RELEASE_REQUIRED
|
||||
self.cleanup_reason = self.cleanup_reason or str(error)
|
||||
self.primary_error = self.primary_error or str(error)
|
||||
Event.set(self)
|
||||
self.force_event.set()
|
||||
|
||||
|
||||
def protect_interruption(root, reason):
|
||||
"""Create or re-sync a marker without overwriting prior protection evidence."""
|
||||
root = internal_path(root)
|
||||
@@ -69,9 +131,12 @@ def wait_for_resolution(control, root, error, *, resolved=None, release_on_failu
|
||||
return
|
||||
from .worker_monitor import WorkerMonitorError, _publish_progress
|
||||
root = internal_path(root)
|
||||
control.stopping_reason = 'failure_cleanup' if release_on_failure else (control.stopping_reason or 'inactivity')
|
||||
control.wait_reason = 'reconciliation'
|
||||
control.primary_error = control.primary_error or str(error)
|
||||
control.set()
|
||||
control.waiting_for_resolution = True
|
||||
control.resolution_error = str(error)
|
||||
control.resolution_error = control.resolution_error or str(error)
|
||||
def protect():
|
||||
protect_interruption(root, 'operation_interrupted: ' + str(error))
|
||||
try:
|
||||
@@ -91,7 +156,9 @@ def wait_for_resolution(control, root, error, *, resolved=None, release_on_failu
|
||||
record = {}
|
||||
record.update(stage='awaiting_reconciliation', waiting_for_safe_completion=True,
|
||||
resolution_error=str(error), stop_requested=True, safe_stop=False,
|
||||
cleanup_mode=control.cleanup_mode)
|
||||
cleanup_mode=control.cleanup_mode, stopping_reason=control.stopping_reason,
|
||||
wait_reason=control.wait_reason,
|
||||
primary_error=record.get('primary_error') or control.primary_error)
|
||||
_publish_progress(root / 'worker-progress.json', record)
|
||||
while True:
|
||||
if resolved is not None and resolved():
|
||||
|
||||
@@ -40,7 +40,7 @@ _TASK_TRANSITIONS = {
|
||||
TaskStatus.RUNNING: {TaskStatus.PAUSED, TaskStatus.COMPLETED, TaskStatus.FAILED},
|
||||
TaskStatus.PAUSED: {TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.FAILED, TaskStatus.DELETED},
|
||||
TaskStatus.COMPLETED: {TaskStatus.DELETED},
|
||||
TaskStatus.FAILED: {TaskStatus.PAUSED, TaskStatus.READY, TaskStatus.DELETED},
|
||||
TaskStatus.FAILED: {TaskStatus.PAUSED, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.DELETED},
|
||||
TaskStatus.DELETED: {
|
||||
TaskStatus.DRAFT,
|
||||
TaskStatus.READY,
|
||||
@@ -49,6 +49,13 @@ _TASK_TRANSITIONS = {
|
||||
TaskStatus.FAILED,
|
||||
},
|
||||
}
|
||||
for _state in (TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED, TaskStatus.FAILED):
|
||||
_TASK_TRANSITIONS[_state].add(TaskStatus.BLOCKED)
|
||||
_TASK_TRANSITIONS[TaskStatus.DRAFT].update({TaskStatus.PAUSED, TaskStatus.FAILED})
|
||||
_TASK_TRANSITIONS[TaskStatus.READY].add(TaskStatus.PAUSED)
|
||||
_TASK_TRANSITIONS[TaskStatus.FAILED].add(TaskStatus.FAILED)
|
||||
_TASK_TRANSITIONS[TaskStatus.BLOCKED] = {TaskStatus.BLOCKED, TaskStatus.PAUSED, TaskStatus.READY, TaskStatus.DELETED}
|
||||
_TASK_TRANSITIONS[TaskStatus.DELETED].add(TaskStatus.BLOCKED)
|
||||
_PROJECT_TRANSITIONS = {
|
||||
ProjectStatus.PENDING: {ProjectStatus.CAPTURING_BEFORE, ProjectStatus.FAILED, ProjectStatus.SKIPPED},
|
||||
ProjectStatus.CAPTURING_BEFORE: {ProjectStatus.AWAITING_OPERATOR, ProjectStatus.FAILED},
|
||||
@@ -467,7 +474,8 @@ class TaskStore:
|
||||
"""Persist current-sheet quick-run progress independently of manual attempts."""
|
||||
with self.transaction():
|
||||
task = self.get_task(task_id)
|
||||
allowed_task_states = {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED, TaskStatus.COMPLETED}
|
||||
allowed_task_states = {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED,
|
||||
TaskStatus.COMPLETED, TaskStatus.FAILED}
|
||||
if task.status not in allowed_task_states:
|
||||
raise TaskCenterError("quick_run_task_state", "task state does not allow quick-run progress")
|
||||
for project_id in completed_project_ids:
|
||||
@@ -512,7 +520,7 @@ class TaskStore:
|
||||
"quick-run queue completed before project states were synchronized",
|
||||
)
|
||||
has_failures = any(project.status is ProjectStatus.FAILED for project in selected)
|
||||
if task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.PAUSED}:
|
||||
if task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.PAUSED, TaskStatus.FAILED}:
|
||||
task = self.set_task_status(
|
||||
task_id,
|
||||
TaskStatus.RUNNING,
|
||||
@@ -520,10 +528,10 @@ class TaskStore:
|
||||
if has_failures else "current-sheet quick-run queue completed",
|
||||
)
|
||||
if has_failures:
|
||||
if task.status is TaskStatus.RUNNING and task.summary != "current-sheet quick-run queue completed with retryable failures":
|
||||
if task.status is TaskStatus.RUNNING:
|
||||
task = self.set_task_status(
|
||||
task_id,
|
||||
TaskStatus.RUNNING,
|
||||
TaskStatus.FAILED,
|
||||
summary="current-sheet quick-run queue completed with retryable failures",
|
||||
)
|
||||
else:
|
||||
|
||||
@@ -20,7 +20,8 @@ class ProjectIdentityResolver:
|
||||
return self._projects[spelling]
|
||||
|
||||
def target(self, target):
|
||||
return (self.project(target.project_path), target.schematic_name, target.sheet_number, target.uid)
|
||||
return (self.project(target.project_path), target.schematic_name, target.sheet_number,
|
||||
target.refdes.strip().casefold())
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
@@ -36,6 +37,8 @@ class ComponentTarget:
|
||||
actual_cell: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if not self.uid and self.refdes:
|
||||
object.__setattr__(self, 'uid', '@REFDES|' + self.refdes.strip())
|
||||
if any(value is not None and not isinstance(value, str)
|
||||
for value in (self.actual_part_label, self.actual_cell)):
|
||||
raise TaskCenterError('target_inventory', 'effective Part Label and Cell must be text or unknown')
|
||||
@@ -118,7 +121,7 @@ def compile_target_rules(rows: tuple[TargetPlanRow, ...], *, default_value_mode:
|
||||
raise TaskCenterError("target_identity", "component identity is incomplete")
|
||||
target_key = identities.target(target)
|
||||
if target_key in seen:
|
||||
raise TaskCenterError("target_overlap", "target UID appears more than once")
|
||||
raise TaskCenterError("target_overlap", "target reference appears more than once")
|
||||
seen.add(target_key)
|
||||
if not row.enabled:
|
||||
continue
|
||||
|
||||
@@ -23,7 +23,8 @@ def archive_completed_session(service):
|
||||
|
||||
def _archive(service):
|
||||
store = service._store
|
||||
pending = service.pending_batch_resources()
|
||||
from .batch_resources import resource_blockers
|
||||
pending = resource_blockers(service.pending_batch_resources())
|
||||
owned = set()
|
||||
unknown = False
|
||||
for record in pending:
|
||||
@@ -39,7 +40,7 @@ def _archive(service):
|
||||
reason = None
|
||||
if unknown or task.task_id in owned:
|
||||
reason = '后台资源尚未确认释放;重新打开后使用“重新清理资源”。'
|
||||
elif task.status is TaskStatus.RUNNING:
|
||||
elif task.status in {TaskStatus.RUNNING, TaskStatus.BLOCKED}:
|
||||
reason = '任务仍在执行状态;重新打开后暂停或核验恢复。'
|
||||
elif any(project.status in {ProjectStatus.CAPTURING_BEFORE, ProjectStatus.AWAITING_OPERATOR,
|
||||
ProjectStatus.CAPTURING_AFTER, ProjectStatus.VERIFYING}
|
||||
|
||||
@@ -12,6 +12,8 @@ from uuid import uuid4
|
||||
|
||||
from .errors import TaskCenterError
|
||||
from .quick_run import _write_atomic_queue_text
|
||||
from .project_isolation import (write_sources, record_task_outcome, isolate_project,
|
||||
block_dependency, dependency_block, dependency_readmitted)
|
||||
|
||||
|
||||
_QUEUE_LOCK = Lock()
|
||||
@@ -212,12 +214,14 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
with _queue_lease(root):
|
||||
children = {task_id: service.for_project_task(task_id) for task_id in ids}
|
||||
project_paths = []
|
||||
sources = {}
|
||||
for task_id, child in children.items():
|
||||
task = child.get_task(task_id)
|
||||
projects = child.projects(task_id)
|
||||
if not getattr(task, 'primary_project_id', None) or len(projects) != 1:
|
||||
raise TaskCenterError('project_binding', '顺序执行仅支持绑定单个工程的任务。')
|
||||
project_paths.append(str(projects[0].path.resolve(strict=False)).casefold())
|
||||
sources[task_id] = write_sources(projects[0].path)
|
||||
if len(set(project_paths)) != len(ids):
|
||||
raise TaskCenterError('task_queue_selection', '不能在同一队列重复处理同一个工程。')
|
||||
runtime.bind_logs(service._logging, ids)
|
||||
@@ -251,10 +255,38 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
_save(path, state)
|
||||
paused = False
|
||||
current = None
|
||||
from .stop_control import ProjectStopControl
|
||||
parent_control = getattr(runtime, 'stop_event', None)
|
||||
isolated_tasks = state.setdefault('isolated_tasks', {})
|
||||
isolated_sources = state.setdefault('isolated_sources', {})
|
||||
dependency_tasks = state.setdefault('dependency_tasks', {})
|
||||
from .netlist_verification import NetlistVerification
|
||||
checks = NetlistVerification(root, token, stop_event)
|
||||
try:
|
||||
for task_id in ids:
|
||||
readmitted = False
|
||||
if task_id in dependency_tasks:
|
||||
try:
|
||||
readmitted = dependency_readmitted(children[task_id], task_id, hashes[task_id])
|
||||
except TaskCenterError:
|
||||
pass
|
||||
if readmitted:
|
||||
isolated_tasks.pop(task_id, None)
|
||||
dependency_tasks.pop(task_id)
|
||||
state['entries'][task_id] = {'state': 'pending', 'message': '共享依赖已核验,恢复未派发原计划。'}
|
||||
if task_id in isolated_tasks:
|
||||
continue
|
||||
conflict = next((isolated_sources[source] for source in sources[task_id]
|
||||
if source in isolated_sources), None)
|
||||
if conflict is not None and not readmitted and state['entries'][task_id]['state'] not in _TERMINAL:
|
||||
message = f'与待核验工程 {conflict} 共享写入源,已隔离;未派发替换。'
|
||||
state['entries'][task_id] = {'state': 'blocked', 'message': message}
|
||||
isolated_tasks[task_id] = message
|
||||
dependency_tasks[task_id] = conflict
|
||||
block_dependency(root, children[task_id], task_id, conflict,
|
||||
sources[task_id] & sources[conflict], hashes[task_id], token)
|
||||
_save(path, state)
|
||||
continue
|
||||
if state['entries'][task_id]['state'] in _TERMINAL:
|
||||
if state['entries'][task_id]['state'] == 'completed':
|
||||
checks.before_task(children[task_id], task_id, existing_only=True)
|
||||
@@ -264,6 +296,10 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
break
|
||||
current = task_id
|
||||
child = children[task_id]
|
||||
project_control = ProjectStopControl(stop_event)
|
||||
runtime.stop_event = project_control
|
||||
if getattr(runtime, 'cad', None) is not None:
|
||||
runtime.cad.stop_control = project_control
|
||||
service._task_queue_feedback = (token, task_id, ids.index(task_id) + 1, len(ids))
|
||||
resolving_bom = False
|
||||
execution_dispatched = False
|
||||
@@ -277,7 +313,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
if _plan_hash(child, task_id) != hashes[task_id]:
|
||||
raise TaskCenterError('assistant_stale', '待执行 BOM 已变化,请重新确认。')
|
||||
resolving_bom = True
|
||||
resolved = child.resolve_pending_bom(task_id, stop_event=stop_event)
|
||||
resolved = child.resolve_pending_bom(task_id, stop_event=project_control)
|
||||
resolving_bom = False
|
||||
if not any(row.enabled for row in resolved):
|
||||
state['entries'][task_id] = {'state': 'skipped',
|
||||
@@ -304,7 +340,7 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
for parameter in parameters) else {})
|
||||
checks.before_task(child, task_id)
|
||||
execution_dispatched = True
|
||||
result = execute(task_id, expected_item=expected, stop_event=stop_event,
|
||||
result = execute(task_id, expected_item=expected, stop_event=project_control,
|
||||
**runtime_options)
|
||||
if child.has_interrupted_quick_run(task_id) or (
|
||||
result.last_result is not None and result.last_result.state == 'batch_saved_unverified'
|
||||
@@ -340,10 +376,31 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
entry = {'state': 'failed' if safe else 'paused' if error.code == 'operation_cancelled' else 'blocked',
|
||||
'message': f'{error.code}: {error.message}' if resolving_bom else error.message}
|
||||
paused = not safe
|
||||
if not safe and error.code != 'operation_cancelled' and not stop_event.is_set():
|
||||
try:
|
||||
isolated = isolate_project(runtime, child, task_id, error)
|
||||
except Exception as cleanup_error:
|
||||
entry['message'] += '; 资源收尾未确认:' + str(cleanup_error)
|
||||
isolated = False
|
||||
if isolated:
|
||||
isolated_tasks[task_id] = entry['message']
|
||||
isolated_sources.update((source,task_id) for source in sources[task_id])
|
||||
paused = False
|
||||
except Exception as error:
|
||||
entry = {'state': 'blocked', 'message': str(error)}
|
||||
paused = True
|
||||
if not stop_event.is_set():
|
||||
try:
|
||||
isolated = isolate_project(runtime, child, task_id, error)
|
||||
except Exception as cleanup_error:
|
||||
entry['message'] += '; 资源收尾未确认:' + str(cleanup_error)
|
||||
isolated = False
|
||||
if isolated:
|
||||
isolated_tasks[task_id] = entry['message']
|
||||
isolated_sources.update((source,task_id) for source in sources[task_id])
|
||||
paused = False
|
||||
state['entries'][task_id] = entry
|
||||
record_task_outcome(child, task_id, entry['state'], entry['message'])
|
||||
_save(path, state)
|
||||
if (entry['state'] in {'completed', 'failed'} and task_id in checks.tasks
|
||||
and not child.has_interrupted_quick_run(task_id)):
|
||||
@@ -369,7 +426,8 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
checks.after_task(task_id, execution_state='failed' if result.progress.failed_items else 'completed',
|
||||
refresh=True, execution_summary={key: getattr(result.progress, key, None)
|
||||
for key in ('saved_items', 'failed_items', 'no_match_items')})
|
||||
retried = run_problem_pass(service, ids, token, stop_event=stop_event, runtime=runtime,
|
||||
admitted_problem_tasks = tuple(item for item in ids if item not in isolated_tasks)
|
||||
retried = run_problem_pass(service, admitted_problem_tasks, token, stop_event=stop_event, runtime=runtime,
|
||||
on_result=capture_retry)
|
||||
for task_id, result in retried.results.items():
|
||||
state['entries'][task_id] = {'state': 'paused' if result.paused else 'failed' if result.progress.failed_items else 'completed',
|
||||
@@ -416,3 +474,6 @@ def execute_task_queue(service, task_ids: tuple[str, ...], *, stop_event: Event,
|
||||
checks.__exit__(None, None, None)
|
||||
finally:
|
||||
service._task_queue_feedback = None
|
||||
runtime.stop_event = parent_control
|
||||
if getattr(runtime, 'cad', None) is not None:
|
||||
runtime.cad.stop_control = parent_control
|
||||
@@ -9,12 +9,14 @@ STAGES = {
|
||||
'snapshot_attributes': '记录原始属性', 'capture': '采集替换前快照',
|
||||
'capture_untouched': '记录未选中器件状态', 'verify_untouched': '核验未选中器件',
|
||||
'native_snapshot_read': '采集并核验器件完整状态',
|
||||
'capture_semantic_state': '采集完整图页状态',
|
||||
'before_native': '准备原生替换', 'native_replace': '执行原生替换',
|
||||
'restore_preflight': '检查属性恢复条件',
|
||||
'before_restore_transaction': '准备恢复属性',
|
||||
'restore_attribute_fields': '恢复属性字段', 'restore_attributes': '恢复属性',
|
||||
'verify_restoration': '核验恢复结果', 'verify_attribute_fields': '核验属性字段',
|
||||
'save_and_close': '保存并关闭工程', 'verify_saved_project': '回读已保存工程',
|
||||
'save_and_close': '保存并关闭工程', 'verify_saved_project': '回读工程并核对',
|
||||
'saved_source_wait': '工程已保存,等待数据库释放后核验',
|
||||
'awaiting_reconciliation': '当前结果待核验,已保留工程现场',
|
||||
}
|
||||
|
||||
@@ -22,7 +24,20 @@ STAGES = {
|
||||
def format_feedback(value, task_name, elapsed, stopping):
|
||||
stage = STAGES.get(value.stage, '')
|
||||
details = f'{stage}({value.stage_seconds:.1f} 秒)' if stage else value.message
|
||||
prefix = '等待当前步骤保存核验后暂停 · ' if stopping else ''
|
||||
reason = getattr(value, 'stopping_reason', '')
|
||||
wait = getattr(value, 'wait_reason', '')
|
||||
prefix = ({'user_pause': '已请求暂停,等待当前步骤完成',
|
||||
'failure_cleanup': '执行失败,正在回收资源',
|
||||
'inactivity': '当前步骤长时间无进展'}
|
||||
.get(reason, '已请求暂停,等待当前步骤完成' if stopping else ''))
|
||||
waits = {'native_call': '等待原生调用返回', 'save': '等待工程保存完成',
|
||||
'readback': '等待工程重读核对', 'mentor_exit': '等待 Mentor 写进程退出',
|
||||
'desktop_release': '等待后台桌面释放', 'reconciliation': '等待结果核对'}
|
||||
if wait and (stopping or value.waiting_for_safe_completion):
|
||||
prefix = ','.join(filter(None, (prefix, waits.get(wait, ''))))
|
||||
if value.waiting_for_safe_completion:
|
||||
prefix = '当前保存或核验尚未确认,未自动结束应用;可继续等待、导出诊断,或确认强制结束后核验 · '
|
||||
prefix += ';最终结果尚未确认,可继续等待、导出诊断,或确认强制结束后核对'
|
||||
if getattr(value, 'primary_error', ''):
|
||||
details += ';原始错误:' + value.primary_error
|
||||
prefix = prefix + ' · ' if prefix else ''
|
||||
return f'{prefix}任务 {value.position}/{value.total}:{task_name} · {details} · 总计 {elapsed} 秒'
|
||||
@@ -3,12 +3,20 @@ from dataclasses import dataclass
|
||||
from ..models import TaskStatus
|
||||
|
||||
|
||||
def task_display_status(task):
|
||||
"""Legacy retryable failures retain running admission, but are not active work."""
|
||||
if (task.status is TaskStatus.RUNNING
|
||||
and task.summary == 'current-sheet quick-run queue completed with retryable failures'):
|
||||
return TaskStatus.FAILED
|
||||
return task.status
|
||||
|
||||
|
||||
def pending_batch_tasks(tasks, group_id):
|
||||
"""Use the persisted batch order for both execution and its UI controls."""
|
||||
if not group_id:
|
||||
return ()
|
||||
return tuple(sorted((task for task in tasks if task.group_id == group_id
|
||||
and task.status not in {TaskStatus.DELETED, TaskStatus.COMPLETED}
|
||||
and task.status not in {TaskStatus.DELETED, TaskStatus.COMPLETED, TaskStatus.BLOCKED}
|
||||
and getattr(task, 'summary', '') != 'current-sheet quick-run queue completed with retryable failures'),
|
||||
key=lambda task: task.group_position))
|
||||
|
||||
@@ -34,7 +42,8 @@ def execution_state(context, *, busy=False, dirty=False):
|
||||
if not project or task.status is TaskStatus.DELETED:
|
||||
return ExecutionState(message='任务已删除,可从已删除列表恢复。')
|
||||
editable = task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.FAILED} and not context.has_evidence and not busy
|
||||
retry = context.phase == 'retry' and not context.interrupted and not busy and not dirty
|
||||
retry = (context.phase == 'retry' and task.status is not TaskStatus.BLOCKED
|
||||
and not context.interrupted and not busy and not dirty)
|
||||
requests = getattr(context, 'pending_bom', ())
|
||||
deferred = bool(requests)
|
||||
from ..bom_pending import retained_bom_rows
|
||||
@@ -44,7 +53,7 @@ def execution_state(context, *, busy=False, dirty=False):
|
||||
execute = (pending and task.status in {TaskStatus.DRAFT, TaskStatus.READY, TaskStatus.RUNNING, TaskStatus.PAUSED}
|
||||
and project.available and not context.interrupted and not busy and not dirty)
|
||||
text = '核验并执行' if deferred else '继续当前任务' if task.status is TaskStatus.PAUSED else '执行当前任务'
|
||||
message = ('结果待核验,请先恢复中断步骤。' if context.interrupted else
|
||||
message = ('结果待核验,请先恢复中断步骤。' if context.interrupted or task.status is TaskStatus.BLOCKED else
|
||||
'位号计划有未保存编辑,请先保存。' if dirty else
|
||||
'正在处理当前步骤,暂停会等待安全收尾。' if busy else
|
||||
'任务已完成。可查看原计划和日志;点击“开始新一轮”归档本批已完成任务,再采集当前料号。' if task.status is TaskStatus.COMPLETED else
|
||||
@@ -54,5 +63,6 @@ def execution_state(context, *, busy=False, dirty=False):
|
||||
'BOM 表格已保存;确认后仅核验当前工程位号,再执行替换。' if deferred else
|
||||
'先填写旧、新料号,再采集并勾选位号。' if not pending else
|
||||
'配置就绪,可执行当前任务或按顺序执行本批任务。')
|
||||
return ExecutionState(editable, execute, retry, context.interrupted and not busy and not dirty, text, message,
|
||||
return ExecutionState(editable, execute, retry,
|
||||
(context.interrupted or task.status is TaskStatus.BLOCKED) and not busy and not dirty, text, message,
|
||||
task.status is TaskStatus.COMPLETED and not context.interrupted and not busy and not dirty)
|
||||
@@ -423,9 +423,10 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
def _refresh_task_list(self):
|
||||
deleted = self.task_list.recycle_button.isChecked()
|
||||
def received(tasks):
|
||||
self._known_task_groups.update((task.task_id, task.group_id) for task in tasks)
|
||||
self._tasks = tasks
|
||||
self.task_list.populate(tasks, selected_task_id=self._task_id)
|
||||
if tasks != self._tasks or not self.task_list.count.text():
|
||||
self._known_task_groups.update((task.task_id, task.group_id) for task in tasks)
|
||||
self._tasks = tasks
|
||||
self.task_list.populate(tasks, selected_task_id=self._task_id)
|
||||
self._update_gates()
|
||||
self._task_query.request(lambda control: self.service.tasks(status=TaskStatus.DELETED, read_control=control)
|
||||
if deleted else self.service.tasks(read_control=control), received, self._show_error, cancellable=True)
|
||||
@@ -529,7 +530,7 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
self.run_panel.retry_button.setVisible(bool(context and context.phase == 'retry'))
|
||||
self.run_panel.retry_button.setEnabled(state.retry and not resources_pending)
|
||||
self.run_panel.problems_button.setEnabled(bool(context and not busy))
|
||||
self.run_panel.recover_button.setVisible(bool(context and context.interrupted))
|
||||
self.run_panel.recover_button.setVisible(bool(context and (context.interrupted or context.task.status is TaskStatus.BLOCKED)))
|
||||
self.run_panel.recover_button.setEnabled(state.recover and not resources_pending)
|
||||
self.run_panel.new_round_button.setVisible(bool(context and context.task.status is TaskStatus.COMPLETED))
|
||||
self.run_panel.new_round_button.setEnabled(state.new_round and not group_unsaved)
|
||||
@@ -597,7 +598,8 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
self.run_panel.execution_status.setText('正在检查后台资源' if not self._resource_status.known else
|
||||
'资源状态读取失败' if self._resource_status.error is not None else
|
||||
'资源待清理' if resources_pending else '任务读取失败' if read_failed else
|
||||
'结果待核验' if context and context.interrupted else '空闲')
|
||||
'结果待核验' if context and (context.interrupted or context.task.status is TaskStatus.BLOCKED) else
|
||||
'旧桌面已隔离,可继续执行' if self._resource_status.records else '空闲')
|
||||
|
||||
def _group_has_unsaved_plans(self):
|
||||
if not self._context:
|
||||
@@ -1037,16 +1039,22 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
cleanup_finished = True
|
||||
start_force_if_ready()
|
||||
def forced(remaining):
|
||||
from ..batch_resources import resource_blockers
|
||||
self._resource_status.refresh()
|
||||
self._set_operator_message('仍有资源未能确认释放,请保留诊断。' if remaining else
|
||||
self._set_operator_message('仍有资源未能确认释放,请保留诊断。' if resource_blockers(remaining) else
|
||||
'自有写入进程已退出,残留桌面已隔离(未确认释放);相关工程仍须核验。' if remaining else
|
||||
'自有残留资源已释放;相关工程仍须核验后恢复。', 'warning')
|
||||
self._update_gates()
|
||||
def done(pending):
|
||||
nonlocal force_confirmed
|
||||
from ..batch_resources import resource_blockers
|
||||
warnings = tuple(row for row in pending if row.get('blocking') is False)
|
||||
pending = resource_blockers(pending)
|
||||
hints = tuple(dict.fromkeys(row['resolution_hint'] for row in pending if row.get('resolution_hint')))
|
||||
self._set_operator_message(
|
||||
'\n'.join(hints) if hints else '后台资源仍待清理,请保留诊断记录后重试。' if pending else '后台资源已释放,可以继续。',
|
||||
'warning' if pending else 'success')
|
||||
'\n'.join(hints) if hints else '后台资源仍待清理,请保留诊断记录后重试。' if pending else
|
||||
'旧桌面仍存在,已隔离;自有写进程已退出,可以继续执行。' if warnings else '后台资源已释放,可以继续。',
|
||||
'warning' if pending or warnings else 'success')
|
||||
if pending:
|
||||
self._dsh_blockers.report('resource-release', '', 'batch_resources_pending',
|
||||
'后台资源清理完成后仍有资源未确认释放。', {'resources': list(pending)[:20]}, cleanup_allowed=True)
|
||||
@@ -1187,7 +1195,7 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
if not self.run_panel.recover_button.isEnabled():
|
||||
return
|
||||
if QMessageBox.question(self, '核验并恢复',
|
||||
'先备份工程和证据,再打开工程核验;如需补齐属性,会保存并重开回读,不会重复调用原生替换。请确保待处理工程未在其他 CAD 会话中打开。',
|
||||
'共享数据源依赖导致隔离、且从未派发的任务,将核验依赖与原计划后解除隔离,不打开 CAD。其他中断任务先备份工程和证据,再打开核验;如需补齐属性,会保存并重开回读,不会重复调用原生替换。请确保待处理工程未在其他 CAD 会话中打开。',
|
||||
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No,
|
||||
QMessageBox.StandardButton.No) != QMessageBox.StandardButton.Yes:
|
||||
return
|
||||
@@ -1223,6 +1231,8 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
|
||||
def _update_quick_run_elapsed(self):
|
||||
self._update_gates()
|
||||
if self._batch_stop is not None and not self._task_query.busy:
|
||||
self._refresh_task_list()
|
||||
if self._batch_stop is not None and not self._feedback_query.busy:
|
||||
task_id, started = self._active_quick_run_task_id, self._run_started_at
|
||||
run_id = self._dsh_run_id
|
||||
@@ -1459,6 +1469,7 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
# If it fails, release the execution controls just as
|
||||
# the service failure path does.
|
||||
self._finish_run()
|
||||
self._refresh_task_list()
|
||||
self.operation_state = self.operation_state.finish(error)
|
||||
self._show_error(error, owner_task_id=owner, operation_area=operation_area)
|
||||
def failed(error):
|
||||
@@ -1470,6 +1481,8 @@ class TrialTaskCenterWindow(MaterialMainWindow):
|
||||
if self._close_saving or not (isinstance(error, TaskCenterError) and error.code == 'operation_cancelled'):
|
||||
self._close_pending = False
|
||||
self._finish_run()
|
||||
if execution:
|
||||
self._refresh_task_list()
|
||||
if isinstance(error, TaskCenterError) and error.code == 'operation_cancelled':
|
||||
self._set_operator_message(self.operation_state.message)
|
||||
return
|
||||
|
||||
@@ -29,7 +29,8 @@ class ResourceStatus(QObject):
|
||||
|
||||
@property
|
||||
def blocked(self):
|
||||
return not self.known or self.error is not None or bool(self.records)
|
||||
from ..batch_resources import resource_blockers
|
||||
return not self.known or self.error is not None or bool(resource_blockers(self.records))
|
||||
|
||||
def refresh(self):
|
||||
if self._closed:
|
||||
|
||||
@@ -12,6 +12,7 @@ from PySide6.QtWidgets import (QAbstractItemView,QComboBox,QHBoxLayout,QLabel,QL
|
||||
QListWidgetItem,QPushButton,QVBoxLayout,QWidget,QToolButton,QMenu,
|
||||
QStyledItemDelegate,QStyleOptionViewItem,QStyle,QApplication)
|
||||
from .command_menu import CommandMenu
|
||||
from .execution_state import task_display_status
|
||||
|
||||
|
||||
class TaskItemDelegate(QStyledItemDelegate):
|
||||
@@ -80,10 +81,13 @@ class TaskList(QWidget):
|
||||
self.count = QLabel(objectName='taskCount')
|
||||
self.filter = QComboBox(objectName="taskFilter")
|
||||
for label, key, picture in (('全部任务', 'all', 'grid'), ('未完成', 'unfinished', 'clock'),
|
||||
('已完成', 'completed', 'check-circle'), ('失败', 'failed', 'warning')):
|
||||
('待开始', 'pending', 'clock'), ('执行中', 'running', 'clock'),
|
||||
('已暂停', 'paused', 'clock'), ('已完成', 'completed', 'check-circle'),
|
||||
('待核验', 'blocked', 'warning'),
|
||||
('失败 / 待重试', 'failed', 'warning')):
|
||||
self.filter.addItem(icon(picture), label, key)
|
||||
self.filter.setAccessibleName('任务状态分类')
|
||||
self.filter.setToolTip('未完成包括待配置、待执行、执行中、已暂停和失败的任务;分类只筛选显示。')
|
||||
self.filter.setToolTip('待开始包括待配置和待执行;失败包括已收尾的待重试任务。未完成包括所有尚未完成的任务;分类只筛选显示。')
|
||||
self.list = QListWidget(objectName="taskList")
|
||||
self.list.setItemDelegate(TaskItemDelegate(self.list))
|
||||
self.list.setSelectionMode(QAbstractItemView.SelectionMode.ExtendedSelection)
|
||||
@@ -163,9 +167,17 @@ class TaskList(QWidget):
|
||||
return str(item.data(Qt.ItemDataRole.UserRole)) if item is not None else None
|
||||
|
||||
def populate(self, tasks: Iterable[object], *, selected_task_id: str | None = None) -> None:
|
||||
preserve_selection = selected_task_id in (None, self.current_task_id())
|
||||
selected_ids = {item.data(Qt.ItemDataRole.UserRole) for item in self.list.selectedItems()}
|
||||
self._selected_task_id = selected_task_id or self.current_task_id() or self._selected_task_id
|
||||
self._tasks = tuple(tasks)
|
||||
self._render()
|
||||
if preserve_selection:
|
||||
with QSignalBlocker(self.list):
|
||||
for row in range(self.list.count()):
|
||||
item = self.list.item(row)
|
||||
item.setSelected(item.data(Qt.ItemDataRole.UserRole) in selected_ids)
|
||||
self._update_selection_state()
|
||||
|
||||
def select_task(self, task_id: str) -> None:
|
||||
if task_id != self._selected_task_id:
|
||||
@@ -209,8 +221,9 @@ class TaskList(QWidget):
|
||||
category = self.filter.currentData()
|
||||
if not self.recycle_button.isChecked() and category != 'all':
|
||||
tasks = [task for task in tasks if
|
||||
(task.status.value not in {'completed', 'deleted'} if category == 'unfinished'
|
||||
else task.status.value == category)]
|
||||
(task_display_status(task).value not in {'completed', 'deleted'} if category == 'unfinished'
|
||||
else task_display_status(task).value in {'draft', 'ready'} if category == 'pending'
|
||||
else task_display_status(task).value == category)]
|
||||
mode = self.sort.currentData()
|
||||
if mode.startswith('name_'):
|
||||
tasks.sort(key=lambda task: tuple(int(part) if part.isdigit() else part
|
||||
@@ -230,9 +243,9 @@ class TaskList(QWidget):
|
||||
for task in tasks:
|
||||
group_id = getattr(task, "group_id", None)
|
||||
prefix = f"{getattr(task, 'group_position', 0) + 1:02d}. " if group_id else ""
|
||||
status = getattr(getattr(task, 'status', None), 'value', '')
|
||||
status = task_display_status(task).value
|
||||
labels = {'draft': '待配置', 'ready': '待执行', 'running': '执行中', 'paused': '已暂停',
|
||||
'completed': '已完成', 'failed': '失败', 'deleted': '已删除'}
|
||||
'completed': '已完成', 'failed': '失败', 'blocked': '待核验', 'deleted': '已删除'}
|
||||
item = QListWidgetItem(prefix + str(getattr(task, "name")) + f"\n{labels.get(status, status)}")
|
||||
item.setData(Qt.ItemDataRole.UserRole, str(getattr(task, "task_id")))
|
||||
item.setIcon(self._project_icon)
|
||||
|
||||
@@ -37,14 +37,28 @@ def _fields(line):
|
||||
return tuple(_decode_vbs_field(field) for field in line.split('\t'))
|
||||
|
||||
|
||||
def _validated(root, row, outcome):
|
||||
def _validated(root, row, outcome, *, project_commit=None):
|
||||
raw, _, snapshot_path = _paths(root, row)
|
||||
state, code, message = outcome
|
||||
if state not in {'saved_ok', 'restored_ok'}:
|
||||
raise ValueError('Value report requires a verified replacement terminal state')
|
||||
status_bytes = _read(internal_path(root) / 'status.tsv')
|
||||
terminal = _fields(status_bytes.decode('utf-8-sig').splitlines()[-1])
|
||||
if terminal != (row['job_id'], state, code, message):
|
||||
if project_commit is not None:
|
||||
from .execution_proof import proof_intact
|
||||
from .path_access import canonical_path
|
||||
if (code != 'project_committed' or not proof_intact(project_commit)
|
||||
or canonical_path(project_commit['project_path']) != canonical_path(row['project_path'])
|
||||
or project_commit['outcomes'].get(row['job_id']) != state):
|
||||
raise ValueError('Value report has no matching intact project commit')
|
||||
marker = internal_path(project_commit['files'][0]['path'])
|
||||
lines = [_fields(line) for line in _read(marker).decode('utf-8-sig').splitlines() if line]
|
||||
if (lines[0] != (project_commit['session_token'],row['project_path'],'project_verified')
|
||||
or [fields[0] for fields in lines[1:]] != project_commit['jobs']
|
||||
or dict(lines[1:]) != project_commit['outcomes']):
|
||||
raise ValueError('Value report project commit marker differs')
|
||||
terminal = (row['job_id'],state,code,message)
|
||||
elif terminal != (row['job_id'], state, code, message):
|
||||
raise ValueError('Value report does not belong to the current terminal job')
|
||||
raw_bytes, snapshot_bytes = _read(raw), _read(snapshot_path)
|
||||
snapshot = snapshot_bytes.decode('utf-8-sig')
|
||||
@@ -74,8 +88,10 @@ def _validated(root, row, outcome):
|
||||
if row.get('target_scope') == 'targets':
|
||||
selected = {parts[0]: parts[1] for selector in selectors.split('\x1e')
|
||||
if len(parts := selector.split('\x1f')) == 3}
|
||||
if components != selected:
|
||||
raise ValueError('Value snapshot UID set differs from frozen command')
|
||||
expected_refs = {ref.strip().casefold() for ref in selected.values()}
|
||||
if (not selected or len(expected_refs) != len(selected)
|
||||
or {ref.strip().casefold() for ref in components.values()} != expected_refs):
|
||||
raise ValueError('Value snapshot physical reference set differs from frozen command')
|
||||
values, observations, seen, ordinals = [], [], set(), set()
|
||||
names = {}
|
||||
for line in lines[1:]:
|
||||
@@ -117,14 +133,28 @@ def _validated(root, row, outcome):
|
||||
value_readback_job_id=row['job_id'], project_path=row['project_path'],
|
||||
schematic_name=row.get('schematic_name', ''), sheet_number=row.get('sheet_number', ''),
|
||||
attribute_observations=observations, attribute_ambiguities=ambiguities)
|
||||
binding = dict(command=dict(row), terminal=list(terminal), status_sha256=_digest(status_bytes),
|
||||
binding = dict(command=dict(row), terminal=list(terminal), status_sha256=None if project_commit else _digest(status_bytes),
|
||||
snapshot_sha256=_digest(snapshot_bytes), report_sha256=_digest(raw_bytes))
|
||||
if project_commit is not None:
|
||||
binding['project_commit'] = project_commit
|
||||
return details, binding
|
||||
|
||||
|
||||
def capture_value_readback(root, row, outcome):
|
||||
def capture_value_readback(root, row, outcome, *, project_commit=None):
|
||||
try:
|
||||
details, binding = _validated(root, row, outcome)
|
||||
if project_commit and project_commit.get('validation_required') == 'saved_netlist_v1':
|
||||
saved = project_commit['saved_netlist']
|
||||
details = saved['jobs'][row['job_id']]
|
||||
binding = dict(command=dict(row),terminal=[row['job_id'],*outcome],
|
||||
verification_path=saved['verification_path'], verification_sha256=saved['verification_sha256'],
|
||||
export_path=saved['export_path'], export_sha256=saved['export_sha256'])
|
||||
if (saved.get('status') != 'verified' or details.get('value_readback_status') != 'verified'
|
||||
or details.get('value_readback_job_id') != row['job_id'] or outcome[0] != 'saved_ok'):
|
||||
raise ValueError('saved native target Value proof differs')
|
||||
_, path, _ = _paths(root,row)
|
||||
write_record(path,dict(schema=3,binding=binding,details=details))
|
||||
return details
|
||||
details, binding = _validated(root, row, outcome, project_commit=project_commit)
|
||||
_, saved, _ = _paths(root, row)
|
||||
write_record(saved, dict(schema=2, binding=binding, details=details))
|
||||
return details
|
||||
@@ -134,11 +164,28 @@ def capture_value_readback(root, row, outcome):
|
||||
return {'value_readback_status': 'rejected', 'value_readback_reason': str(error)}
|
||||
|
||||
|
||||
def _load_saved_netlist_value(binding,row,outcome):
|
||||
if (binding['command'] != dict(row) or binding['terminal'] != [row['job_id'],*outcome]
|
||||
or _digest(_read(internal_path(binding['verification_path']))) != binding['verification_sha256']
|
||||
or _digest(_read(internal_path(binding['export_path']))) != binding['export_sha256']):
|
||||
raise ValueError('saved native Value evidence binding changed')
|
||||
verified = json.loads(_read(internal_path(binding['verification_path'])))
|
||||
if (verified['status'] != 'verified' or verified['export_path'] != binding['export_path']
|
||||
or verified['export_sha256'] != binding['export_sha256']):
|
||||
raise ValueError('saved native target verification is incomplete')
|
||||
return verified['jobs'][row['job_id']]
|
||||
|
||||
|
||||
def load_value_readback(root, row, outcome):
|
||||
try:
|
||||
details, binding = _validated(root, row, outcome)
|
||||
_, saved, _ = _paths(root, row)
|
||||
persisted = json.loads(_read(saved))
|
||||
if persisted.get('schema') == 3:
|
||||
details = _load_saved_netlist_value(persisted['binding'],row,outcome)
|
||||
if details != persisted['details']:raise ValueError('persisted native Value observation changed')
|
||||
return details
|
||||
details, binding = _validated(root, row, outcome,
|
||||
project_commit=persisted.get('binding',{}).get('project_commit'))
|
||||
if persisted != dict(schema=2, binding=binding, details=details):
|
||||
raise ValueError('persisted Value evidence binding changed')
|
||||
return details
|
||||
|
||||
@@ -362,6 +362,7 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
|
||||
stop_readers = threading.Event()
|
||||
output = [bytearray(), bytearray()]
|
||||
startup_pause = None
|
||||
prewrite_pause = None
|
||||
|
||||
def read_stream(stream, channel):
|
||||
try:
|
||||
@@ -379,7 +380,7 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
|
||||
stream.close()
|
||||
|
||||
def receive(channel, line):
|
||||
nonlocal startup_pause
|
||||
nonlocal startup_pause, prewrite_pause
|
||||
if channel == 0 and watch.accept(line, time.monotonic()):
|
||||
_publish_progress(progress_path, watch.snapshot())
|
||||
elif channel == 0 and line.startswith(b'DX_STARTUP_PAUSED\t'):
|
||||
@@ -399,6 +400,29 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
|
||||
'project_path': request[1], 'sequence': watch.sequence}
|
||||
except (OSError, UnicodeError, ValueError):
|
||||
pass
|
||||
elif channel == 0 and line.startswith(b'DX_PREWRITE_PAUSED\t'):
|
||||
try:
|
||||
from .quick_run import _decode_vbs_field
|
||||
from .path_access import canonical_path
|
||||
fields = line.decode('ascii').strip().split('\t')
|
||||
request_data = (root / 'project-session.request').read_bytes()
|
||||
request = request_data.decode('utf-8-sig').strip().split('\t')
|
||||
original = session_request.decode('utf-8-sig').strip().split('\t') if session_request else []
|
||||
if (len(fields) == 6 and fields[:3] == ['DX_PREWRITE_PAUSED', token, job_id]
|
||||
and len(request) == 3 and len(original) == 3 and request[:2] == original[:2]
|
||||
and fields[3:5] == [request[0], request[2]] and re.fullmatch('[0-9a-f]{32}', request[0])
|
||||
and re.fullmatch(r'[A-Za-z0-9_.-]{1,128}', request[2])
|
||||
and int(fields[5]) == watch.sequence and watch.sequence > 0
|
||||
and abort_event is not None and abort_event.is_set() and watch.stop_requested):
|
||||
marker = (root / ('project-pause-' + request[0] + '.tsv')).read_bytes()
|
||||
values = marker.decode('utf-8-sig').strip().split('\t')
|
||||
if (len(values) == 4 and values[0] == request[0] and values[2:] == [request[2], 'prewrite_closed']
|
||||
and canonical_path(_decode_vbs_field(values[1])) == canonical_path(request[1])):
|
||||
prewrite_pause = {'monitor_token': token, 'session_token': request[0], 'job_id': request[2],
|
||||
'project_path': request[1], 'sequence': watch.sequence,
|
||||
'request_bytes': request_data, 'marker_bytes': marker}
|
||||
except (OSError, UnicodeError, ValueError):
|
||||
pass
|
||||
elif len(output[channel]) < 1024 * 1024:
|
||||
output[channel].extend(line[:1024 * 1024 - len(output[channel])])
|
||||
|
||||
@@ -545,6 +569,15 @@ def run_monitored_process(command: tuple[str, ...], idle_timeout: float, *, abor
|
||||
or not _worker_tree_exited(process, owned)):
|
||||
raise WorkerMonitorError('startup pause lacks correlated no-project acknowledgement or worker exit')
|
||||
result.startup_pause = dict(startup_pause, worker_tree_exited=True)
|
||||
if process.returncode == 75:
|
||||
if (prewrite_pause is None or prewrite_pause['sequence'] != watch.sequence
|
||||
or result.stderr.strip() or not _worker_tree_exited(process, owned)
|
||||
or (root / 'project-session.request').read_bytes() != prewrite_pause['request_bytes']
|
||||
or (root / ('project-pause-' + prewrite_pause['session_token'] + '.tsv')).read_bytes()
|
||||
!= prewrite_pause['marker_bytes']):
|
||||
raise WorkerMonitorError('prewrite pause lacks correlated closed-project acknowledgement or worker exit')
|
||||
result.prewrite_pause = {key: value for key, value in prewrite_pause.items()
|
||||
if key not in {'request_bytes', 'marker_bytes'}} | {'worker_tree_exited': True}
|
||||
return result
|
||||
except Exception as error:
|
||||
if process is not None:
|
||||
|
||||
@@ -9,21 +9,14 @@ from uuid import uuid4
|
||||
from .batch_resources import write_record
|
||||
from .errors import TaskCenterError
|
||||
from .path_access import canonical_path, internal_path
|
||||
from .stop_control import StopControl, protect_interruption
|
||||
from .stop_control import ProjectStopControl, protect_interruption
|
||||
from .project_isolation import write_sources, record_task_outcome, block_dependency, dependency_readmitted
|
||||
|
||||
_LOCK = Lock()
|
||||
_TERMINAL = {'completed', 'failed', 'skipped', 'blocked'}
|
||||
|
||||
|
||||
class _ProjectControl(StopControl):
|
||||
def __init__(self, parent):
|
||||
super().__init__()
|
||||
self.parent = parent
|
||||
if hasattr(parent, 'force_event'):
|
||||
self.force_event = parent.force_event
|
||||
|
||||
def is_set(self):
|
||||
return super().is_set() or self.parent.is_set()
|
||||
_ProjectControl = ProjectStopControl
|
||||
|
||||
|
||||
@contextmanager
|
||||
@@ -68,7 +61,8 @@ def _released_after_recheck(root, task_id, row):
|
||||
journal = json.loads(internal_path(root / 'batch-resources' / batch_id / 'supervisor.json').read_text(encoding='utf-8'))
|
||||
return (valid_journal(journal, batch_id) and journal['task_ids'] == [task_id]
|
||||
and journal['state'] == 'finished' and journal.get('process')
|
||||
and _process_gone(journal['process']) and not pending_resources(root) and not pending_supervisors(root))
|
||||
and _process_gone(journal['process']) and not any(row.get('blocking', True) for row in pending_resources(root))
|
||||
and not any(row.get('blocking', True) for row in pending_supervisors(root)))
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
return False
|
||||
|
||||
@@ -88,6 +82,7 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
|
||||
for rows in projects}) != len(ids):
|
||||
raise TaskCenterError('project_binding', '每个任务必须对应不同的单个工程。')
|
||||
hashes = {item: _plan_hash(children[item], item) for item in ids}
|
||||
sources = {item: write_sources(rows[0].path) for item, rows in zip(ids, projects)}
|
||||
if expected_plan_hashes is not None and hashes != expected_plan_hashes:
|
||||
raise TaskCenterError('assistant_stale', '执行计划已变化,请重新预检。')
|
||||
token = queue_id or checkpoint_id or uuid4().hex
|
||||
@@ -115,6 +110,9 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
|
||||
for item in ids}}
|
||||
write_record(path, state)
|
||||
|
||||
isolated_sources = state.setdefault('isolated_sources', {})
|
||||
dependency_tasks = state.setdefault('dependency_tasks', {})
|
||||
|
||||
def result(paused=False, current=None):
|
||||
verification = {item: state['entries'][item]['netlist_verification'] for item in ids
|
||||
if state['entries'][item].get('netlist_verification')}
|
||||
@@ -140,17 +138,25 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
|
||||
or outcome.entries[0].state not in {'pending', 'running', 'paused', *_TERMINAL}):
|
||||
raise TaskCenterError('task_queue_state', '工程监督进程返回了不同的队列身份。')
|
||||
row.update(state=outcome.entries[0].state, message=outcome.entries[0].message, released=True)
|
||||
if row['state'] == 'blocked':
|
||||
record_task_outcome(children[item], item, 'blocked', row['message'])
|
||||
isolated_sources.update((source, item) for source in sources[item])
|
||||
verification = (outcome.netlist_verification or {}).get(item)
|
||||
if verification:
|
||||
row['netlist_verification'] = verification
|
||||
write_record(path, state)
|
||||
return outcome.paused and row['state'] != 'blocked'
|
||||
except TaskCenterError as error:
|
||||
released = getattr(error, 'resources_released', False) is True
|
||||
from .batch_resources import resource_blockers
|
||||
released = ((getattr(error, 'resources_released', False) is True
|
||||
or getattr(error, 'owned_writers_exited', False) is True)
|
||||
and not resource_blockers(service.pending_batch_resources() if hasattr(service, 'pending_batch_resources') else ()))
|
||||
row.update(state='blocked', message=error.message, released=released)
|
||||
row['supervisor_batch_id'] = getattr(service, '_last_supervisor_batch_id', None)
|
||||
evidence = getattr(getattr(children[item], '_quick_run_preparer', None), 'data_root', root / 'tasks' / item)
|
||||
protect_interruption(Path(evidence) / 'quick-run', 'project_supervisor_failed: ' + error.message)
|
||||
record_task_outcome(children[item], item, 'blocked', error.message)
|
||||
isolated_sources.update((source, item) for source in sources[item])
|
||||
if released:
|
||||
from .batch_supervisor import reconcile_supervisors
|
||||
try:
|
||||
@@ -165,9 +171,31 @@ def execute_workspace_projects(service, task_ids, *, stop_event, queue_id=None,
|
||||
raise
|
||||
|
||||
try:
|
||||
if state['phase'] == 'finished' and any(dependency_readmitted(children[item], item, hashes[item])
|
||||
for item in tuple(dependency_tasks)):
|
||||
state['phase'] = 'primary'
|
||||
if state['phase'] == 'primary':
|
||||
for item in ids:
|
||||
row = state['entries'][item]
|
||||
readmitted = False
|
||||
if item in dependency_tasks:
|
||||
try:
|
||||
readmitted = dependency_readmitted(children[item], item, hashes[item])
|
||||
except TaskCenterError:
|
||||
pass
|
||||
if readmitted:
|
||||
row.update(state='pending', message='共享依赖已核验,恢复未派发原计划。')
|
||||
dependency_tasks.pop(item)
|
||||
conflict = next((isolated_sources[source] for source in sources[item]
|
||||
if source in isolated_sources and isolated_sources[source] != item), None)
|
||||
if conflict is not None and not readmitted and row['state'] not in {'completed', 'failed', 'skipped'}:
|
||||
message = f'与待核验工程 {conflict} 共享写入源,已隔离;未派发替换。'
|
||||
block_dependency(root, children[item], item, conflict,
|
||||
sources[item] & sources[conflict], hashes[item], token)
|
||||
dependency_tasks[item] = conflict
|
||||
row.update(state='blocked', message=message, released=True, problems_done=True)
|
||||
write_record(path, state)
|
||||
continue
|
||||
if row['state'] in _TERMINAL:
|
||||
if row['state'] == 'blocked' and not row['released']:
|
||||
if not _released_after_recheck(root, item, row):
|
||||
|
||||
@@ -1,18 +1,14 @@
|
||||
import importlib.util
|
||||
import inspect
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import call, patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
RUNTIME = ROOT / "integration" / "dxdesigner795"
|
||||
SPEC = importlib.util.spec_from_file_location(
|
||||
"native", ROOT / "src" / "dx_productivity" / "trial_task_center" / "native_helper.py"
|
||||
)
|
||||
native = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(native)
|
||||
from dx_productivity.trial_task_center import native_helper as native
|
||||
|
||||
FAKES = r"""
|
||||
Class TestPoint
|
||||
@@ -107,7 +103,7 @@ End Sub
|
||||
|
||||
class NativeTests(unittest.TestCase):
|
||||
def test_native_command_does_not_use_physical_mouse_fallback(self):
|
||||
source = inspect.getsource(native.run_native)
|
||||
source = inspect.getsource(inspect.unwrap(native.run_native))
|
||||
|
||||
self.assertIn("replacement_command_from_resources(executable)", source)
|
||||
self.assertNotIn("win32api.SetCursorPos", source)
|
||||
@@ -115,20 +111,24 @@ class NativeTests(unittest.TestCase):
|
||||
self.assertNotIn("win32gui.SetCursorPos(screen_point)", source)
|
||||
|
||||
def test_snapshot_batch_is_reselected_before_the_strict_selection_gate(self):
|
||||
app = object()
|
||||
api = object()
|
||||
metadata = ["M"]
|
||||
first = type("Component", (), {"UID": "wanted"})()
|
||||
other = type("Component", (), {"UID": "other"})()
|
||||
first = SimpleNamespace(UID='wanted', Refdes='R1', Selected=False)
|
||||
other = SimpleNamespace(UID='other', Refdes='R2', Selected=True)
|
||||
first._oleobj_, other._oleobj_ = first, other
|
||||
app = SimpleNamespace(ActiveView=SimpleNamespace(Query=lambda kind, scope:
|
||||
[component for component in (first, other) if component.Selected]))
|
||||
api = SimpleNamespace(IVdComp=lambda value:value)
|
||||
records = [["C", "wanted"]]
|
||||
with patch.object(native, "check_context", side_effect=([first, other], [first, other])) as check:
|
||||
with patch.object(native, "selection") as select:
|
||||
with patch.object(native, "check_context", return_value=[first, other]) as check:
|
||||
with patch.object(native, "selection", side_effect=lambda component, selected:setattr(component,'Selected',selected)) as select, \
|
||||
patch.object(native, "check_active_context"):
|
||||
result = native.select_snapshot_batch(app, api, metadata, records)
|
||||
|
||||
self.assertEqual(result, [first, other])
|
||||
self.assertEqual(check.call_args_list[0], call(app, api, metadata, records, require_selection=False))
|
||||
self.assertEqual(check.call_args_list[1], call(app, api, metadata, records))
|
||||
self.assertEqual(select.call_args_list, [call(first, True), call(other, False)])
|
||||
self.assertTrue(first.Selected)
|
||||
self.assertFalse(other.Selected)
|
||||
|
||||
def test_snapshot_escape_format(self):
|
||||
self.assertEqual(native.decode("a%0009b%000Ac%0025"), "a\tb\nc%")
|
||||
|
||||
@@ -73,10 +73,11 @@ Class SessionApp
|
||||
End Function
|
||||
End Class
|
||||
Class SessionDocuments
|
||||
Public Owner, OpenCount, SheetCount
|
||||
Public Owner, OpenCount, SheetCount, DocumentCount
|
||||
Private Sub Class_Initialize()
|
||||
OpenCount = 0
|
||||
SheetCount = 1
|
||||
DocumentCount = 1
|
||||
End Sub
|
||||
Public Function GetAvailableSchematics()
|
||||
Dim Names
|
||||
@@ -94,13 +95,14 @@ Class SessionDocuments
|
||||
End Function
|
||||
Public Function Open(ByVal Schematic, ByVal Sheet)
|
||||
OpenCount = OpenCount + 1
|
||||
DocumentCount = 1
|
||||
Set Owner.ActiveView = Owner
|
||||
Set Open = Me
|
||||
End Function
|
||||
Public Sub Activate()
|
||||
End Sub
|
||||
Public Property Get Count
|
||||
Count = 1
|
||||
Count = DocumentCount
|
||||
End Property
|
||||
Public Function Item(ByVal Index)
|
||||
If Index <> 1 Then Err.Raise 51000,"Test","bad document index"
|
||||
@@ -108,6 +110,7 @@ Class SessionDocuments
|
||||
End Function
|
||||
Public Sub Close(ByVal SaveChanges, ByVal FileName)
|
||||
If SaveChanges Then Owner.SaveCount = Owner.SaveCount + 1
|
||||
DocumentCount = DocumentCount - 1
|
||||
Set Owner.ActiveView = Nothing
|
||||
End Sub
|
||||
End Class
|
||||
@@ -141,6 +144,9 @@ def run_session(tmp_path, body):
|
||||
target.touch()
|
||||
setup = f'''
|
||||
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
|
||||
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionSheets=CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary")
|
||||
QP_Root = "{tmp_path}"
|
||||
Dim Target, Attached
|
||||
Target = "{target}"
|
||||
|
||||
@@ -90,6 +90,7 @@ Class ValueApp
|
||||
End Function
|
||||
Public Sub Close(ByVal SaveChanges,ByVal FileName)
|
||||
CloseCount=CloseCount+1
|
||||
DocumentCount=DocumentCount-1
|
||||
If SaveChanges Then
|
||||
SaveCalled=True
|
||||
SaveCount=SaveCount+1
|
||||
@@ -149,6 +150,9 @@ End Function
|
||||
SETUP = r'''
|
||||
Dim Comp, Pin, Attr, Ignored, Command, Snapshot, SnapshotPath, Failure, Saved
|
||||
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
|
||||
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionSheets=CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary")
|
||||
QP_Root="__ROOT__"
|
||||
QP_CurrentJobId="value-job"
|
||||
Set QP_App=New ValueApp
|
||||
@@ -220,7 +224,7 @@ Require QP_App.UndoCount=0,"recovery undid prior CAD work"
|
||||
''')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('change', ['Comp.UID="imposter"', 'Comp.Refdes="R99"'])
|
||||
@pytest.mark.parametrize('change', ['Comp.Refdes="R99"'])
|
||||
def test_restore_rejects_changed_component_identity_before_any_write(tmp_path, change):
|
||||
run_vbs(tmp_path, 'part_number', '''
|
||||
On Error Resume Next
|
||||
@@ -233,6 +237,16 @@ Require QP_App.TransactionCount=0,"identity failure must precede CAD writes"
|
||||
''', after=change)
|
||||
|
||||
|
||||
def test_restore_accepts_uid_churn_without_changing_reference_or_replaying_replace(tmp_path):
|
||||
run_vbs(tmp_path, 'part_number', '''
|
||||
QPRestoreAfterOperatorReplace Command,SnapshotPath
|
||||
Require QP_App.TransactionCount=1,"restoration did not use exactly one transaction"
|
||||
Require Comp.Refdes="R1","reference identity changed"
|
||||
Require Comp.FindAttribute("Value").InstanceValue="NEW","Value policy was not verified"
|
||||
Require QP_App.UndoCount=0,"UID churn replayed or undid the native replacement"
|
||||
''', after='Comp.UID="changed-native-uid"')
|
||||
|
||||
|
||||
def _batch_readback_fixture(routines, *, fault=''):
|
||||
# Exercise real restore/finish/guards, replacing only the external open and
|
||||
# command transport with an in-memory CAD-shaped application.
|
||||
@@ -242,6 +256,7 @@ End Function''', routines)
|
||||
routines = re.sub(r'(?ms)^Sub QPEnsureProject\([^\n]*\n.*?^End Sub', '''Sub QPEnsureProject(ByRef App, ByVal TargetPath)
|
||||
App.ProjectPath=TargetPath
|
||||
App.CloseCount=0
|
||||
App.DocumentCount=1
|
||||
QP_OwnedProject=True
|
||||
QP_OwnedProjectPath=TargetPath
|
||||
''' + ('Set Attr=Other.FindAttribute("Value"): Attr.Value="DRIFT"' if fault == 'after_reopen' else '') + '''
|
||||
@@ -258,7 +273,7 @@ def test_save_then_project_close_refusal_retains_save_boundary(tmp_path):
|
||||
Dim FullChecks
|
||||
FullChecks=0
|
||||
Command.Add "project_path",QPProjectPath()
|
||||
QP_BatchMode=True: QP_SessionMode=True
|
||||
QP_BatchMode=True: QP_SessionMode=False
|
||||
QP_SessionToken=String(32,"b")
|
||||
QP_OwnedProject=True: QP_OwnedProjectPath=QPProjectPath()
|
||||
QPRestoreAfterOperatorReplace Command,SnapshotPath
|
||||
@@ -274,7 +289,7 @@ QP_App.RefuseClose=False
|
||||
QPDiscardFailedBatchProject CloseError
|
||||
Require Len(CloseError)>0,"uncertain save must not discard the open project"
|
||||
Require Len(QPProjectPath())>0,"failed saved project was silently discarded"
|
||||
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"project-session-result.tsv")),"failure published success"
|
||||
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"status.tsv")),"failure published success"
|
||||
''', routines_transform=_batch_readback_fixture)
|
||||
|
||||
|
||||
@@ -284,7 +299,7 @@ def test_owned_batch_checks_full_state_at_save_and_disk_boundaries(tmp_path, fau
|
||||
Dim FullChecks
|
||||
FullChecks=0
|
||||
Command.Add "project_path",QPProjectPath()
|
||||
QP_BatchMode=True: QP_SessionMode=True
|
||||
QP_BatchMode=True: QP_SessionMode=False
|
||||
QP_SessionToken=String(32,"b")
|
||||
QP_OwnedProject=True: QP_OwnedProjectPath=QPProjectPath()
|
||||
QPRestoreAfterOperatorReplace Command,SnapshotPath
|
||||
@@ -298,10 +313,10 @@ On Error GoTo 0
|
||||
Require Failure=0,"valid batch failed"
|
||||
Require FullChecks=2,"expected one full check before save and one after reopen"
|
||||
Require QP_App.SaveCalled,"valid batch did not save"
|
||||
Require InStr(QPReadUtf8(QP_Fso.BuildPath(QP_Root,"project-session-result.tsv")),"saved_ok")>0,"no verified result"
|
||||
Require InStr(QPReadUtf8(QP_Fso.BuildPath(QP_Root,"status.tsv")),"saved_ok")>0,"no verified result"
|
||||
''' if not fault else '''
|
||||
Require Failure<>0,"unselected drift passed the commit guard"
|
||||
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"project-session-result.tsv")),"failure published a completion"
|
||||
Require Not QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"status.tsv")),"failure published a completion"
|
||||
Require QP_App.SaveCalled=''' + ('True' if fault == 'after_reopen' else 'False') + ''',"wrong save boundary"
|
||||
Require QP_BatchSaveCommitted=''' + ('True' if fault == 'after_reopen' else 'False') + ''',"saved uncertainty marker is wrong"
|
||||
'''), after='''
|
||||
|
||||
@@ -104,11 +104,11 @@ def test_no_match_is_recorded_as_information_not_warning(tmp_path):
|
||||
assert executed[0]['details']['state'] == 'no_match'
|
||||
|
||||
|
||||
def test_discovery_inventory_completes_definite_misses_without_reopening_projects(tmp_path):
|
||||
def test_discovery_inventory_records_missing_targets_without_reopening_projects(tmp_path):
|
||||
service, task, calls = setup_queue(tmp_path, ('no_match', 'no_match'))
|
||||
preparer = service._quick_run_preparer
|
||||
preparer._sheet_discoverer = None
|
||||
preparer._discover_project_sheets = lambda _project, _parts: _ProjectDiscovery(
|
||||
preparer._discover_project_sheets = lambda _project, _parts, **_options: _ProjectDiscovery(
|
||||
(('Main', '1'),),
|
||||
{('Main', '1'): frozenset()},
|
||||
)
|
||||
@@ -116,16 +116,17 @@ def test_discovery_inventory_completes_definite_misses_without_reopening_project
|
||||
result = run_queue(service, task)
|
||||
|
||||
assert result.progress.queue_complete
|
||||
assert result.progress.no_match_items == 2
|
||||
assert result.progress.failed_items == 2
|
||||
assert result.progress.no_match_items == 0
|
||||
assert calls == []
|
||||
assert all(project.summary == 'quick_run_result:0:1:saved' for project in service.projects(task.task_id))
|
||||
assert all(project.summary.startswith('quick_run_result:0:0:failed') for project in service.projects(task.task_id))
|
||||
events = [
|
||||
json.loads(line)
|
||||
for line in (tmp_path / 'data/tasks' / task.task_id / 'events.jsonl').read_text(encoding='utf-8').splitlines()
|
||||
]
|
||||
prefiltered = [event for event in events if event['operation'] == 'quick_run_inventory_prefiltered']
|
||||
assert len(prefiltered) == 1
|
||||
assert prefiltered[0]['details']['prefiltered_items'] == 2
|
||||
assert not any(event['operation'] == 'quick_run_executed' for event in events)
|
||||
status = (tmp_path / 'data/quick-run/status.tsv').read_text(encoding='utf-8')
|
||||
assert status.count('\tbatch_skipped\tinventory_empty\t') == 2
|
||||
|
||||
|
||||
def test_discovery_inventory_executes_present_sources_and_skips_only_absent_sources(tmp_path):
|
||||
@@ -133,9 +134,11 @@ def test_discovery_inventory_executes_present_sources_and_skips_only_absent_sour
|
||||
service.add_rule(task.task_id, 'ABSENT', 'OTHER')
|
||||
preparer = service._quick_run_preparer
|
||||
preparer._sheet_discoverer = None
|
||||
preparer._discover_project_sheets = lambda _project, _parts: _ProjectDiscovery(
|
||||
from dx_productivity.trial_task_center.target_plan import ComponentTarget
|
||||
preparer._discover_project_sheets = lambda _project, _parts, **_options: _ProjectDiscovery(
|
||||
(('Main', '1'),),
|
||||
{('Main', '1'): frozenset({'old'})},
|
||||
(ComponentTarget(_project, 'Main', '1', 'u1', 'R1', 'OLD', '10K'),),
|
||||
)
|
||||
def worker(command, timeout):
|
||||
del timeout
|
||||
@@ -152,7 +155,8 @@ def test_discovery_inventory_executes_present_sources_and_skips_only_absent_sour
|
||||
|
||||
assert result.progress.queue_complete
|
||||
assert result.progress.saved_items == 1
|
||||
assert result.progress.no_match_items == 1
|
||||
assert result.progress.no_match_items == 0
|
||||
assert result.progress.failed_items == 1
|
||||
assert len(calls) == 1
|
||||
|
||||
|
||||
@@ -496,7 +500,7 @@ def test_real_failure_isolated_from_later_project(tmp_path):
|
||||
assert projects[0].status is ProjectStatus.COMPLETED
|
||||
assert projects[1].status is ProjectStatus.FAILED
|
||||
assert projects[2].status is ProjectStatus.COMPLETED
|
||||
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
|
||||
assert service.get_task(task.task_id).status is TaskStatus.FAILED
|
||||
assert service.quick_run_phase(task.task_id) == 'retry'
|
||||
|
||||
|
||||
@@ -512,7 +516,7 @@ def test_batch_failure_marks_project_failed_and_continues_to_next_project(tmp_pa
|
||||
assert projects[0].status is ProjectStatus.FAILED
|
||||
assert projects[0].summary == 'quick_run_result:0:0:failed|Main:1 result'
|
||||
assert projects[1].status is ProjectStatus.COMPLETED
|
||||
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
|
||||
assert service.get_task(task.task_id).status is TaskStatus.FAILED
|
||||
assert service.quick_run_phase(task.task_id) == 'retry'
|
||||
|
||||
|
||||
@@ -555,7 +559,7 @@ def test_missing_project_is_skipped_and_other_projects_continue(tmp_path):
|
||||
assert final_projects[0].status is ProjectStatus.FAILED
|
||||
assert '工程文件不存在,已跳过' in final_projects[0].summary
|
||||
assert final_projects[1].status is ProjectStatus.COMPLETED
|
||||
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
|
||||
assert service.get_task(task.task_id).status is TaskStatus.FAILED
|
||||
|
||||
|
||||
def test_batch_failure_continues_with_next_rule_in_same_project(tmp_path):
|
||||
@@ -602,7 +606,7 @@ def test_batch_failure_continues_with_next_rule_in_same_project(tmp_path):
|
||||
assert result.progress.saved_items == 1
|
||||
assert result.progress.failed_items == 1
|
||||
assert service.projects(task.task_id)[0].status is ProjectStatus.FAILED
|
||||
assert service.get_task(task.task_id).status is TaskStatus.RUNNING
|
||||
assert service.get_task(task.task_id).status is TaskStatus.FAILED
|
||||
assert service.quick_run_phase(task.task_id) == 'retry'
|
||||
status_text = (data / 'quick-run/status.tsv').read_text(encoding='utf-8')
|
||||
assert 'batch_skipped' not in status_text
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Task categories follow committed receipts before the whole batch releases."""
|
||||
from threading import Event
|
||||
from PySide6.QtCore import Qt
|
||||
|
||||
from dx_productivity.trial_task_center.models import TaskStatus
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.batch_resources import BatchResourceCleanupError
|
||||
|
||||
|
||||
def visible_ids(panel):
|
||||
return [panel.list.item(index).data(Qt.ItemDataRole.UserRole) for index in range(panel.list.count())]
|
||||
|
||||
|
||||
def test_completed_task_moves_category_while_next_task_is_still_executing(ui, qtbot, monkeypatch):
|
||||
window=ui.window
|
||||
first,second=ui.tasks
|
||||
waiting,release=Event(),Event()
|
||||
execute=TaskCenterService.execute_quick_run_queue
|
||||
def hold_second(service, task_id, **options):
|
||||
if task_id==second.task_id:
|
||||
waiting.set()
|
||||
assert release.wait(10), 'test did not release its controlled second task'
|
||||
return execute(service, task_id, **options)
|
||||
monkeypatch.setattr(TaskCenterService,'execute_quick_run_queue',hold_second)
|
||||
window._quick_run_confirmer=lambda *_args:True
|
||||
try:
|
||||
window._execute_project_tasks()
|
||||
qtbot.waitUntil(waiting.is_set, timeout=5000)
|
||||
assert ui.store.get_task(first.task_id).status is TaskStatus.COMPLETED
|
||||
ui.open(second)
|
||||
window.task_list.filter.setCurrentIndex(window.task_list.filter.findData('completed'))
|
||||
qtbot.waitUntil(lambda:visible_ids(window.task_list)==[first.task_id], timeout=3000)
|
||||
assert window._workers and window._batch_stop is not None
|
||||
assert window._task_id==second.task_id
|
||||
assert ui.store.get_task(second.task_id).status is not TaskStatus.COMPLETED
|
||||
assert len(ui.calls)==1
|
||||
finally:
|
||||
release.set()
|
||||
ui.settle()
|
||||
assert len(ui.calls)==2
|
||||
assert set(visible_ids(window.task_list))=={first.task_id,second.task_id}
|
||||
|
||||
|
||||
def test_resource_error_after_business_completion_refreshes_categories(ui):
|
||||
window=ui.window
|
||||
first=ui.tasks[0]
|
||||
window._begin_run(first.task_id)
|
||||
window._quick_run_timer.stop() # The final error path must refresh even without a timer tick.
|
||||
window.task_list.filter.setCurrentIndex(window.task_list.filter.findData('completed'))
|
||||
def execute_then_fail_release():
|
||||
progress=ui.service.quick_run_progress(first.task_id)
|
||||
expected=(str(progress.next_project_path),progress.next_old_part_number,
|
||||
progress.next_new_part_number,'preserve')
|
||||
ui.service.execute_quick_run_queue(first.task_id,expected_item=expected,
|
||||
stop_event=window._batch_stop)
|
||||
raise BatchResourceCleanupError('controlled desktop release failure after task completion')
|
||||
window._run_async(execute_then_fail_release,lambda _:None,expected_task_id=first.task_id)
|
||||
ui.settle()
|
||||
assert ui.store.get_task(first.task_id).status is TaskStatus.COMPLETED
|
||||
assert visible_ids(window.task_list)==[first.task_id]
|
||||
assert window._batch_stop is None
|
||||
assert len(ui.calls)==1
|
||||
assert 'controlled desktop release failure' in window.run_panel.operator_band.text()
|
||||
@@ -28,10 +28,14 @@ def visible_ids(panel):
|
||||
|
||||
|
||||
@pytest.mark.parametrize(('category','expected'),[
|
||||
('all',['draft','ready','running','paused','completed','failed']),
|
||||
('unfinished',['draft','ready','running','paused','failed']),
|
||||
('all',['draft','ready','running','paused','completed','failed','blocked']),
|
||||
('unfinished',['draft','ready','running','paused','failed','blocked']),
|
||||
('pending',['draft','ready']),
|
||||
('running',['running']),
|
||||
('paused',['paused']),
|
||||
('completed',['completed']),
|
||||
('failed',['failed']),
|
||||
('blocked',['blocked']),
|
||||
])
|
||||
def test_visible_status_category_filters_task_list(navigation,category,expected):
|
||||
panel,tasks,opened=navigation
|
||||
@@ -89,3 +93,36 @@ def test_recycle_bin_ignores_active_status_category(ui):
|
||||
assert panel.filter.isEnabled()
|
||||
assert panel.filter.currentData()=='completed'
|
||||
assert visible_ids(panel)==[]
|
||||
|
||||
|
||||
def test_retryable_failure_is_in_failed_category_and_never_displayed_as_running(navigation):
|
||||
panel,tasks,opened=navigation
|
||||
failed = replace(tasks[2], summary='current-sheet quick-run queue completed with retryable failures')
|
||||
panel.populate(tuple(failed if task.task_id == failed.task_id else task for task in tasks))
|
||||
panel.filter.setCurrentIndex(panel.filter.findData('failed'))
|
||||
assert visible_ids(panel)==['running','failed']
|
||||
assert panel.list.item(0).text().splitlines()[-1]=='失败'
|
||||
panel.filter.setCurrentIndex(panel.filter.findData('running'))
|
||||
assert visible_ids(panel)==[]
|
||||
assert failed.status is TaskStatus.RUNNING # Display classification does not authorize a retry.
|
||||
|
||||
|
||||
def test_status_refresh_preserves_multiple_selected_tasks(navigation):
|
||||
panel,tasks,opened=navigation
|
||||
panel.select_all_tasks()
|
||||
updated=tuple(replace(task,status=TaskStatus.COMPLETED) if task.task_id=='ready' else task for task in tasks)
|
||||
panel.populate(updated,selected_task_id='ready')
|
||||
assert {item.data(Qt.ItemDataRole.UserRole) for item in panel.list.selectedItems()} == {
|
||||
task.task_id for task in tasks}
|
||||
|
||||
|
||||
def test_status_refresh_does_not_reselect_a_current_item_the_user_deselected(navigation):
|
||||
panel,tasks,opened=navigation
|
||||
panel.select_all_tasks()
|
||||
panel.list.currentItem().setSelected(False)
|
||||
before={item.data(Qt.ItemDataRole.UserRole) for item in panel.list.selectedItems()}
|
||||
assert 'ready' not in before
|
||||
updated=tuple(replace(task,status=TaskStatus.COMPLETED) if task.task_id=='ready' else task for task in tasks)
|
||||
panel.populate(updated,selected_task_id='ready')
|
||||
assert {item.data(Qt.ItemDataRole.UserRole) for item in panel.list.selectedItems()}==before
|
||||
assert panel.current_task_id()=='ready'
|
||||
@@ -25,8 +25,12 @@ def run_vbs(tmp_path, names, body, prelude=''):
|
||||
names = tuple(names)
|
||||
support = tuple(name for name in ('QPReadOptionalAttributeValue', 'QPRequiredAttributeValue',
|
||||
'QPReadableAttributeValue', 'QPEffectiveAttributeValue',
|
||||
'QPComponentIdentityKey', 'QPComparableState', 'QPEncode', 'QPDecode')
|
||||
'QPComponentIdentityKey', 'QPComponentUnitKey', 'QPBindSheetReferences',
|
||||
'QPBindSnapshotReference', 'QPBindSnapshotFullState', 'QPCommandTargetKey',
|
||||
'QPComparableState', 'QPEncode', 'QPDecode')
|
||||
if name not in {n.lstrip('?') for n in names})
|
||||
if not re.search(r'(?mi)^Dim [^\n]*\bQP_ReferenceAliases\b', prelude + '\n' + body):
|
||||
prelude += '\nDim QP_ReferenceAliases\n'
|
||||
script.write_text(prelude + '\n' + '\n'.join(routine(name) for name in names + support) + '\n' + body,
|
||||
encoding='ascii')
|
||||
result = subprocess.run(['cscript.exe', '//nologo', str(script)], capture_output=True,
|
||||
@@ -112,6 +116,8 @@ End Sub
|
||||
Function QPUntouchedFullState(ByRef component)
|
||||
QPUntouchedFullState = component
|
||||
End Function
|
||||
Sub QPProgress(ByVal Stage,ByVal Done,ByVal Total)
|
||||
End Sub
|
||||
''')
|
||||
assert output=='stable semantic state'
|
||||
|
||||
|
||||
@@ -57,6 +57,36 @@ def test_cache_only_batch_does_not_create_desktop_or_cad(tmp_path):
|
||||
assert not Desktop.created
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', ['factory', 'first_write'])
|
||||
def test_desktop_startup_before_acquisition_does_not_block_later_batches(tmp_path, monkeypatch, failure):
|
||||
from dx_productivity.trial_task_center import batch_runtime
|
||||
|
||||
def fail_startup(*args, **options):
|
||||
raise OSError('desktop startup unavailable')
|
||||
|
||||
runtime = batch_runtime.BatchCadRuntime(
|
||||
tmp_path, 'execution', desktop_factory=fail_startup if failure == 'factory' else Desktop)
|
||||
if failure == 'first_write':
|
||||
write_record = batch_runtime.write_record
|
||||
|
||||
def fail_first_write(path, record):
|
||||
if record['state'] == 'starting':
|
||||
fail_startup()
|
||||
write_record(path, record)
|
||||
|
||||
monkeypatch.setattr(batch_runtime, 'write_record', fail_first_write)
|
||||
|
||||
with pytest.raises(OSError, match='desktop startup unavailable'):
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'startup'):
|
||||
pytest.fail('startup must fail before the desktop is acquired')
|
||||
|
||||
assert not Desktop.created
|
||||
assert batch_runtime.pending_resources(tmp_path) == ()
|
||||
with batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop):
|
||||
pass
|
||||
|
||||
|
||||
def test_batch_forwards_business_failure_to_owned_cad_close(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
runtime = BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
@@ -182,6 +212,28 @@ def command_for(tmp_path, name='one'):
|
||||
return ('cscript', '//nologo', 'worker.vbs', str(root), 'python', 'job'), project
|
||||
|
||||
|
||||
def test_unlaunched_worker_retry_leaves_no_ownership_record(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_runtime, worker_monitor
|
||||
|
||||
def fail_startup(*args, **options):
|
||||
raise OSError('desktop factory unavailable')
|
||||
|
||||
def unexpected_worker(*args, **options):
|
||||
pytest.fail('no worker may launch after the desktop factory fails')
|
||||
|
||||
monkeypatch.setattr(worker_monitor, 'run_monitored_process', unexpected_worker)
|
||||
command, _ = command_for(tmp_path)
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=fail_startup)
|
||||
with pytest.raises(OSError, match='desktop factory unavailable'):
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
|
||||
assert runtime.record['retries']
|
||||
assert batch_runtime.pending_resources(runtime.root) == ()
|
||||
with batch_runtime.BatchCadRuntime(runtime.root, 'execution', desktop_factory=Desktop):
|
||||
pass
|
||||
|
||||
|
||||
def worker_runtime(tmp_path, monkeypatch, monitor):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
from dx_productivity.trial_task_center import worker_monitor
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""An external Windows reference blocks a batch without replaying its results."""
|
||||
"""An external Windows reference is isolated after owned writers exit."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -21,26 +21,23 @@ def test_external_reference_preserves_completed_result_until_resource_only_clean
|
||||
runtime.bind_logs(configure_logging(tmp_path), ('windows-resource-test',))
|
||||
external = None
|
||||
try:
|
||||
with pytest.raises(BatchResourceCleanupError) as caught:
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'completed-operation') as scope:
|
||||
# The test owns this independent actor, outside runtime's
|
||||
# process/job list. The runtime must leave it alive.
|
||||
external = _Win32Api().create_process((sys.executable, '-u', '-c',
|
||||
external = _Win32Api().create_process((sys.executable, '-u', '-c',
|
||||
'import win32service,time,sys; '
|
||||
'd=win32service.OpenDesktop(sys.argv[1],0,False,0x10000000); '
|
||||
'd.SetThreadDesktop();print("ready",flush=True);time.sleep(30)', scope.name),
|
||||
build_startup_info(scope.name).lpDesktop, None, None,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
assert external.stdout.readline() == b'ready\r\n'
|
||||
runtime.result = 'already saved and independently verified'
|
||||
assert caught.value.completed_result == runtime.result
|
||||
assert external.stdout.readline() == b'ready\r\n'
|
||||
runtime.result = 'already saved and independently verified'
|
||||
assert external.poll() is None
|
||||
assert json.loads(runtime.path.read_text())['state'] == 'blocked'
|
||||
assert json.loads(runtime.path.read_text())['state'] == 'isolated'
|
||||
log = runtime.path.with_name('resource-release.log').read_text(encoding='utf-8')
|
||||
assert '资源释放失败' in log and '桌面消失:未确认' in log
|
||||
with pytest.raises(BatchResourceCleanupError):
|
||||
guard_pending_resources(tmp_path)
|
||||
assert 'isolated' in log
|
||||
guard_pending_resources(tmp_path)
|
||||
assert not list(Path(tmp_path).rglob('uncertain.flag'))
|
||||
assert current_desktop_name() == before
|
||||
finally:
|
||||
@@ -52,7 +49,7 @@ def test_external_reference_preserves_completed_result_until_resource_only_clean
|
||||
assert runtime.generation == 1
|
||||
rows = [json.loads(line) for line in (tmp_path / 'tasks/windows-resource-test/events.jsonl')
|
||||
.read_text(encoding='utf-8').splitlines()]
|
||||
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_failed',
|
||||
assert [row['operation'] for row in rows] == ['resource_release_started', 'resource_release_isolated',
|
||||
'resource_release_started', 'resource_release_succeeded']
|
||||
assert rows[-1]['details']['desktop_disappeared'] is True
|
||||
guard_pending_resources(tmp_path)
|
||||
@@ -419,3 +419,16 @@ def test_retained_scope_honors_optional_selectors_and_disabled_overrides(batch,
|
||||
retained = retained_bom_rows((request,), rows)
|
||||
assert len(retained) == 1 and retained[0].target.uid == retained_uid
|
||||
assert retained[0] is next(row for row in rows if row.target.uid == retained_uid)
|
||||
|
||||
|
||||
def test_unique_pending_reference_does_not_retain_old_row_due_to_stale_uid(batch):
|
||||
from dx_productivity.trial_task_center.bom_pending import PendingBomRequest, retained_bom_rows
|
||||
from dx_productivity.trial_task_center.bom_plan import BomRecord
|
||||
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
|
||||
service, store, tasks, targets, calls = batch
|
||||
first, second = targets[tasks[0].task_id]
|
||||
rows = (TargetPlanRow(first, True, 'KEEP-1'), TargetPlanRow(second, True, 'KEEP-2'))
|
||||
record = BomRecord(str(first.project_path), 'OLD', 'NEW', (first.refdes,),
|
||||
first.schematic_name, first.sheet_number, 'stale-uid', False, '', True)
|
||||
request = PendingBomRequest(record, 'source.xlsx', 8, 'hash')
|
||||
assert retained_bom_rows((request,), rows) == (rows[1],)
|
||||
@@ -20,6 +20,26 @@ def setup_store(tmp_path):
|
||||
return store, task, targets
|
||||
|
||||
|
||||
def test_unique_scoped_reference_ignores_stale_bom_uid(tmp_path):
|
||||
from dx_productivity.trial_task_center.bom_plan import BomMatcher, BomRecord
|
||||
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
|
||||
target = ComponentTarget(tmp_path / 'board.prj', 'MAIN', '1', 'current', 'R1', 'OLD', '10nF')
|
||||
matcher = BomMatcher((TargetPlanRow(target),))
|
||||
record = BomRecord('', 'OLD', 'NEW', ('R1',), 'MAIN', '1', 'stale', True, '', True)
|
||||
matched = matcher.match_record(record)
|
||||
assert len(matched) == 1 and next(iter(matched.values())).target is target
|
||||
|
||||
|
||||
def test_uid_can_only_disambiguate_multiple_reference_candidates(tmp_path):
|
||||
from dx_productivity.trial_task_center.bom_plan import BomMatcher, BomRecord
|
||||
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
|
||||
first = ComponentTarget(tmp_path / 'board.prj', 'MAIN', '1', 'u1', 'R1', 'OLD', '10nF')
|
||||
second = ComponentTarget(tmp_path / 'board.prj', 'MAIN', '2', 'u2', 'R1', 'OLD', '10nF')
|
||||
matcher = BomMatcher((TargetPlanRow(first), TargetPlanRow(second)))
|
||||
record = BomRecord('', 'OLD', 'NEW', ('R1',), '', '', 'u2', True, '', True)
|
||||
assert next(iter(matcher.match_record(record).values())).target is second
|
||||
|
||||
|
||||
def test_already_target_does_not_accept_duplicate_nc_from_leading_marker(tmp_path):
|
||||
from dx_productivity.trial_task_center.bom_plan import BomMatcher, BomRecord
|
||||
from dx_productivity.trial_task_center.target_plan import TargetPlanRow
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
|
||||
|
||||
CONTEXT = {'sid': 'S-1-5-21-123', 'logon': 'S-1-5-5-2-1234', 'station': 'WinSta0'}
|
||||
|
||||
|
||||
class Backend:
|
||||
def __init__(self):
|
||||
self.locked = False
|
||||
self.exists = False
|
||||
self.alive = set()
|
||||
self.jobs = set()
|
||||
self.owner = {'pid': 100, 'creation_time': 'owner-100'}
|
||||
|
||||
def lock(self, name):
|
||||
if self.locked:
|
||||
raise TaskCenterError('desktop_slot_busy', 'slot busy')
|
||||
self.locked = True
|
||||
return name
|
||||
|
||||
def unlock(self, handle): self.locked = False
|
||||
def desktop_exists(self, name): return self.exists
|
||||
def process_gone(self, identity): return identity['pid'] not in self.alive
|
||||
def job_empty(self, name): return name not in self.jobs
|
||||
def current_process(self): return dict(self.owner)
|
||||
|
||||
|
||||
def slot(tmp_path, backend, **options):
|
||||
from dx_productivity.trial_task_center.desktop_slot import DesktopSlot
|
||||
return DesktopSlot(root=tmp_path, context=CONTEXT, backend=backend, **options)
|
||||
|
||||
|
||||
def test_sequential_leases_reuse_name_but_never_reuse_attempt_identity(tmp_path):
|
||||
backend = Backend()
|
||||
first = slot(tmp_path, backend).acquire()
|
||||
name = first.name
|
||||
backend.exists = True # External reference keeps the object after our close.
|
||||
receipt = first.release(desktop_disappeared=False)
|
||||
second = slot(tmp_path, backend).acquire()
|
||||
assert second.name == name
|
||||
assert second.lease_id != first.lease_id
|
||||
assert receipt['desktop_disappeared'] is False and receipt['returned'] is True
|
||||
second.release(desktop_disappeared=False)
|
||||
|
||||
|
||||
def test_competing_owner_cannot_create_another_slot(tmp_path):
|
||||
backend = Backend()
|
||||
first = slot(tmp_path, backend).acquire()
|
||||
with pytest.raises(TaskCenterError): slot(tmp_path, backend).acquire()
|
||||
first.release(desktop_disappeared=True)
|
||||
assert slot(tmp_path, backend).acquire().name == first.name
|
||||
|
||||
|
||||
@pytest.mark.parametrize('damage', ['owner_alive', 'job_alive', 'launch_pending', 'unknown_desktop', 'bad_record'])
|
||||
def test_restart_admission_does_not_override_unresolved_writer(tmp_path, damage):
|
||||
backend = Backend()
|
||||
first = slot(tmp_path, backend).acquire()
|
||||
job = first.prepare_launch()
|
||||
first.launched(job, {'pid': 200, 'creation_time': 'worker-200'})
|
||||
backend.locked = False # Simulated abandoned mutex after owner loss.
|
||||
backend.exists = True
|
||||
if damage == 'owner_alive': backend.alive.add(100)
|
||||
elif damage == 'job_alive': backend.jobs.add(job)
|
||||
elif damage == 'launch_pending':
|
||||
record = json.loads(first.path.read_text())
|
||||
record['launches'][0]['state'] = 'launching'
|
||||
first.path.write_text(json.dumps(record))
|
||||
elif damage == 'unknown_desktop': first.path.unlink()
|
||||
else: first.path.write_text('{broken')
|
||||
with pytest.raises(TaskCenterError): slot(tmp_path, backend).acquire()
|
||||
assert not backend.locked
|
||||
|
||||
|
||||
def test_restart_reclaims_dead_owner_only_after_named_job_is_empty(tmp_path):
|
||||
backend = Backend()
|
||||
first = slot(tmp_path, backend).acquire()
|
||||
job = first.prepare_launch()
|
||||
first.launched(job, {'pid': 200, 'creation_time': 'worker-200'})
|
||||
backend.locked = False
|
||||
backend.exists = True
|
||||
second = slot(tmp_path, backend).acquire()
|
||||
assert second.name == first.name
|
||||
second.release(desktop_disappeared=False)
|
||||
|
||||
|
||||
def test_slot_cannot_be_returned_while_owned_job_is_alive(tmp_path):
|
||||
backend = Backend()
|
||||
current = slot(tmp_path, backend).acquire()
|
||||
job = current.prepare_launch()
|
||||
current.launched(job, {'pid': 200, 'creation_time': 'worker-200'})
|
||||
backend.jobs.add(job)
|
||||
with pytest.raises(TaskCenterError): current.release(desktop_disappeared=False)
|
||||
assert backend.locked
|
||||
backend.jobs.clear()
|
||||
assert current.release(desktop_disappeared=False)['returned'] is True
|
||||
|
||||
|
||||
def test_old_release_is_idempotent_and_cannot_change_new_lease(tmp_path):
|
||||
from dx_productivity.trial_task_center.desktop_slot import returned_lease_intact
|
||||
backend = Backend()
|
||||
first = slot(tmp_path, backend).acquire()
|
||||
receipt = first.release(desktop_disappeared=False)
|
||||
second = slot(tmp_path, backend).acquire()
|
||||
before = second.path.read_bytes()
|
||||
assert first.release(desktop_disappeared=True) == receipt
|
||||
assert second.path.read_bytes() == before and backend.locked
|
||||
assert returned_lease_intact(receipt, first.name)
|
||||
data = json.loads(first.return_path.read_text()); data['lease_id'] = 'wrong'
|
||||
first.return_path.write_text(json.dumps(data))
|
||||
assert not returned_lease_intact(receipt, first.name)
|
||||
second.release(desktop_disappeared=False)
|
||||
|
||||
|
||||
def test_launch_intent_must_be_durable_before_a_process_can_start(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import desktop_slot
|
||||
backend = Backend()
|
||||
current = slot(tmp_path, backend).acquire()
|
||||
def fail(*args): raise OSError('storage unavailable')
|
||||
monkeypatch.setattr(desktop_slot, 'write_record', fail)
|
||||
with pytest.raises(OSError): current.prepare_launch()
|
||||
assert backend.locked
|
||||
|
||||
|
||||
def test_user_and_logon_change_the_name_not_workspace_path(tmp_path):
|
||||
from dx_productivity.trial_task_center.desktop_slot import DesktopSlot
|
||||
backend = Backend()
|
||||
a = slot(tmp_path / 'workspace-one', backend)
|
||||
b = slot(tmp_path / 'workspace-two', backend)
|
||||
c = DesktopSlot(root=tmp_path, context=dict(CONTEXT, logon='different-logon'), backend=backend)
|
||||
assert a.name == b.name and a.name != c.name
|
||||
|
||||
|
||||
def test_atomic_job_launch_recovers_publication_gap_only_after_job_exit(tmp_path):
|
||||
backend = Backend()
|
||||
first = slot(tmp_path, backend).acquire()
|
||||
job = first.prepare_launch(atomic_job=True)
|
||||
backend.locked = False
|
||||
backend.jobs.add(job)
|
||||
with pytest.raises(TaskCenterError): slot(tmp_path, backend).acquire()
|
||||
backend.jobs.clear()
|
||||
recovered = slot(tmp_path, backend).acquire()
|
||||
recovered.release(desktop_disappeared=True)
|
||||
@@ -0,0 +1,44 @@
|
||||
import os
|
||||
from threading import Thread
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows mutex thread ownership')
|
||||
def test_slot_can_be_returned_by_the_worker_monitor_thread():
|
||||
from dx_productivity.trial_task_center.desktop_slot import _Windows
|
||||
backend = _Windows()
|
||||
name = 'Local\\DxPartReplace-Thread-Return-Test-' + uuid4().hex
|
||||
mutex = backend.lock(name)
|
||||
errors = []
|
||||
def release():
|
||||
try: backend.unlock(mutex)
|
||||
except Exception as error: errors.append(error)
|
||||
thread = Thread(target=release)
|
||||
thread.start(); thread.join(timeout=5)
|
||||
if errors: backend.unlock(mutex)
|
||||
assert not thread.is_alive() and not errors
|
||||
again = backend.lock(name)
|
||||
backend.unlock(again)
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != 'nt', reason='Windows mutex thread ownership')
|
||||
def test_release_error_closes_mutex_handle_and_allows_its_next_owner(monkeypatch):
|
||||
import win32event
|
||||
from dx_productivity.trial_task_center.desktop_slot import _Windows
|
||||
|
||||
def fail_release(handle):
|
||||
raise OSError('ReleaseMutex failed')
|
||||
|
||||
backend = _Windows()
|
||||
name = 'Local\\DxPartReplace-Release-Error-Test-' + uuid4().hex
|
||||
mutex = backend.lock(name)
|
||||
with monkeypatch.context() as patch:
|
||||
patch.setattr(win32event, 'ReleaseMutex', fail_release)
|
||||
with pytest.raises(OSError, match='ReleaseMutex failed'):
|
||||
backend.unlock(mutex)
|
||||
|
||||
assert not int(mutex[0])
|
||||
next_mutex = backend.lock(name)
|
||||
backend.unlock(next_mutex)
|
||||
@@ -0,0 +1,163 @@
|
||||
import json
|
||||
import os
|
||||
import zipfile
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import diagnostics_bundle as bundle
|
||||
from dx_productivity.trial_task_center.execution_feedback import read_feedback
|
||||
from dx_productivity.trial_task_center.ui.execution_feedback import format_feedback
|
||||
|
||||
|
||||
def write(root, name, text):
|
||||
path = root / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text, encoding='utf-8')
|
||||
return path
|
||||
|
||||
|
||||
def test_bundle_exports_execution_closure_and_inventory_evidence(tmp_path):
|
||||
names = {
|
||||
'tasks/t/quick-run/worker-progress.json': '{"stage":"save_and_close"}',
|
||||
'tasks/t/quick-run/quick-part-replace.log': 'native returned token=private-token',
|
||||
'tasks/t/quick-run/project-commit-' + 'a' * 32 + '.tsv': 'saved_ok\tjob1',
|
||||
'tasks/t/quick-run/project-pause-' + 'a' * 32 + '.tsv': 'paused\tjob2',
|
||||
'tasks/t/quick-run/post-state-job1.tsv': 'R1\tNEW',
|
||||
'tasks/t/quick-run/value-readback-job1.json': '{"verified":true}',
|
||||
'execution-proof/proof1/project-commit-' + 'a' * 32 + '.tsv': 'saved_ok\tjob1',
|
||||
'execution-proof/proof1/completion.json': '{"verified":true}',
|
||||
'tasks/t/quick-run-history/native-inventory-a/native-inventory.json': '{"state":"ready"}',
|
||||
'tasks/t/quick-run-history/native-inventory-a/output/scope.txt': 'R1\tUID1',
|
||||
'tasks/t/quick-run-history/native-inventory-a/output/effective.txt': 'R1\tNEW',
|
||||
'tasks/t/netlist-checks/run1/verification.json': '{"verified":true}',
|
||||
'tasks/t/netlist-checks/run1/after-export/netlist.json': '{"complete":true}',
|
||||
}
|
||||
for name, text in names.items():
|
||||
write(tmp_path, name, text)
|
||||
write(tmp_path, 'tasks/t/netlist-checks/run1/before/project.prj', 'private project')
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
assert set(names).issubset(archive.namelist())
|
||||
assert 'private-token' not in archive.read('tasks/t/quick-run/quick-part-replace.log').decode()
|
||||
assert all(not name.endswith('.prj') for name in archive.namelist())
|
||||
|
||||
|
||||
def test_bundle_tails_active_logs_and_keeps_latest_evidence_under_budget(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(bundle, 'MAX_FILE_BYTES', 160)
|
||||
monkeypatch.setattr(bundle, 'MAX_FILES', 2)
|
||||
for index in range(4):
|
||||
path = write(tmp_path, f'tasks/{index}/quick-run/worker-progress.json', json.dumps({'stage': str(index)}))
|
||||
os.utime(path, (100 + index, 100 + index))
|
||||
log = write(tmp_path, 'tasks/latest/quick-run/quick-part-replace.log',
|
||||
'old heartbeat\n' * 100 + 'last failure token=secret-value\n')
|
||||
os.utime(log, (500, 500))
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
text = archive.read('tasks/latest/quick-run/quick-part-replace.log').decode()
|
||||
assert 'last failure' in text and 'secret-value' not in text
|
||||
assert 'tasks/3/quick-run/worker-progress.json' in archive.namelist()
|
||||
manifest = json.loads(archive.read('manifest.json'))
|
||||
assert manifest['truncated'][0]['mode'] == 'tail'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('record,reason,wait', [
|
||||
({'stage': 'native_replace', 'stop_requested': True}, 'inactivity', 'native_call'),
|
||||
({'stage': 'save_and_close', 'stopping_reason': 'user_pause'}, 'user_pause', 'save'),
|
||||
({'stage': 'verify_saved_project'}, '', 'readback'),
|
||||
({'stage': 'private_shutdown', 'stopping_reason': 'failure_cleanup', 'wait_reason': 'mentor_exit'},
|
||||
'failure_cleanup', 'mentor_exit'),
|
||||
({'stage': 'awaiting_reconciliation', 'primary_error': 'native failed'}, '', 'reconciliation'),
|
||||
])
|
||||
def test_feedback_classifies_stop_and_actual_wait_without_generic_save(tmp_path, record, reason, wait):
|
||||
write(tmp_path, 'quick-run/worker-progress.json', json.dumps(record))
|
||||
value = read_feedback('t', 1, 1, 'running', tmp_path, 0)
|
||||
assert value.stopping_reason == reason
|
||||
assert value.wait_reason == wait
|
||||
text = format_feedback(value, 'board', 4, True)
|
||||
if wait != 'save':
|
||||
assert '保存' not in text
|
||||
if record.get('primary_error'):
|
||||
assert record['primary_error'] in text
|
||||
|
||||
|
||||
def test_operation_events_preserve_last_success_and_incomplete_call(tmp_path):
|
||||
from dx_productivity.trial_task_center.operation_diagnostics import record_operation
|
||||
assert record_operation(tmp_path / 'quick-run', 'item', 'return', item_id='job1')
|
||||
assert record_operation(tmp_path / 'quick-run', 'save', 'enter', item_id='job2')
|
||||
assert record_operation(tmp_path / 'quick-run', 'mentor_exit', 'enter')
|
||||
assert record_operation(tmp_path / 'quick-run', 'mentor_exit', 'error', error='cleanup failed')
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
summary = json.loads(archive.read('closure-summary.json'))['operations']['quick-run']
|
||||
assert summary['last_successful_item'] == 'job1'
|
||||
assert summary['incomplete_operations'][0]['operation'] == 'save'
|
||||
assert summary['errors'][0]['error'] == 'cleanup failed'
|
||||
|
||||
|
||||
def test_stop_control_keeps_failure_reason_and_first_error(tmp_path):
|
||||
from dx_productivity.trial_task_center.stop_control import StopControl, wait_for_resolution, MentorReleaseRequired
|
||||
control = StopControl()
|
||||
with pytest.raises(MentorReleaseRequired):
|
||||
wait_for_resolution(control, tmp_path, 'native failed', release_on_failure=True)
|
||||
assert control.stopping_reason == 'failure_cleanup'
|
||||
assert control.primary_error == 'native failed'
|
||||
control.request_failure_cleanup(tmp_path, 'later cleanup failure')
|
||||
assert control.primary_error == 'native failed'
|
||||
record = json.loads((tmp_path / 'worker-progress.json').read_text())
|
||||
assert record['primary_error'] == 'native failed'
|
||||
assert record['wait_reason'] == 'reconciliation'
|
||||
|
||||
|
||||
def test_closing_desktop_overrides_old_save_wait(tmp_path):
|
||||
write(tmp_path, 'quick-run/worker-progress.json', json.dumps({
|
||||
'stage': 'save_and_close', 'wait_reason': 'save', 'stopping_reason': 'failure_cleanup'}))
|
||||
write(tmp_path, 'quick-run/private-desktop.json', '{"state":"closing"}')
|
||||
value = read_feedback('t', 1, 1, 'running', tmp_path, 0)
|
||||
assert value.wait_reason == 'desktop_release'
|
||||
assert '保存' not in format_feedback(value, 'board', 4, True)
|
||||
|
||||
|
||||
def test_bundle_keeps_complete_event_tail_and_latest_successful_status(tmp_path):
|
||||
write(tmp_path, 'quick-run/operation-events.jsonl',
|
||||
'{"operation":"save","event":"enter","item_id":"job2"}\n{"broken":')
|
||||
write(tmp_path, 'quick-run/status.tsv',
|
||||
'job1\tsaved_ok\tsession_verified\tdone\njob2\tpending_save\tsession_staged\tstaged\n')
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
summary = json.loads(archive.read('closure-summary.json'))
|
||||
assert summary['items']['quick-run']['last_successful_item'] == 'job1'
|
||||
assert summary['items']['quick-run']['latest_item'] == {'job_id': 'job2', 'state': 'pending_save'}
|
||||
assert summary['operations']['quick-run']['incomplete_operations'][0]['operation'] == 'save'
|
||||
manifest = json.loads(archive.read('manifest.json'))
|
||||
assert any(row['reason'] == 'invalid_jsonl_lines' for row in manifest['omissions'])
|
||||
|
||||
|
||||
def test_bundle_keeps_valid_call_when_other_event_has_invalid_shape(tmp_path):
|
||||
write(tmp_path, 'quick-run/operation-events.jsonl',
|
||||
'{"operation":[],"event":"enter","item_id":{}}\n'
|
||||
'{"operation":"save","event":"enter","item_id":"job1"}\n')
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
summary = json.loads(archive.read('closure-summary.json'))
|
||||
assert summary['operations']['quick-run']['incomplete_operations'][0]['operation'] == 'save'
|
||||
|
||||
|
||||
def test_bundle_reads_native_ascii_report_using_local_windows_encoding(tmp_path):
|
||||
name = 'quick-run-history/native-inventory-a/output/scope.txt'
|
||||
path = write(tmp_path, name, '')
|
||||
path.write_bytes('R1 中文说明'.encode('mbcs' if os.name == 'nt' else 'cp1252', errors='replace'))
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
assert name in archive.namelist()
|
||||
assert 'R1 ' in archive.read(name).decode('utf-8')
|
||||
|
||||
|
||||
def test_redaction_growth_does_not_discard_latest_failure_tail(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(bundle, 'MAX_FILE_BYTES', 128)
|
||||
name = 'quick-run/quick-part-replace.log'
|
||||
write(tmp_path, name, 'token=x\n' * 100 + 'native failed token=y\n')
|
||||
output = bundle.export_diagnostic_bundle(tmp_path, tmp_path / 'support.zip')
|
||||
with zipfile.ZipFile(output) as archive:
|
||||
text = archive.read(name).decode()
|
||||
assert 'native failed' in text
|
||||
assert len(archive.read(name)) <= 128
|
||||
@@ -0,0 +1,161 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import batch_resources, batch_worker, operation_diagnostics
|
||||
from dx_productivity.trial_task_center.project_session import ProjectSessionWorker
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
from test_project_session import toy_worker
|
||||
|
||||
|
||||
def events(root):
|
||||
path = root / 'operation-events.jsonl'
|
||||
return [json.loads(line) for line in path.read_text(encoding='utf-8').splitlines()] if path.exists() else []
|
||||
|
||||
|
||||
def test_native_step_and_commit_enter_events_precede_real_worker_dispatch(tmp_path):
|
||||
command = toy_worker(tmp_path)
|
||||
script = Path(command[2])
|
||||
source = script.read_text(encoding='utf-8').replace('job=initial; sequence=0', '''job=initial; sequence=0
|
||||
events=[json.loads(line) for line in (root/'operation-events.jsonl').read_text().splitlines()]
|
||||
assert any(e['operation']=='native_step' and e['event']=='enter' and e['item_id']==job for e in events)''')
|
||||
source = source.replace("if job=='close':break", '''if job=='close':
|
||||
events=[json.loads(line) for line in (root/'operation-events.jsonl').read_text().splitlines()]
|
||||
assert any(e['operation']=='close_project' and e['event']=='enter' for e in events)
|
||||
break''')
|
||||
script.write_text(source, encoding='utf-8')
|
||||
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
|
||||
try:
|
||||
assert worker.run_step(command, 3).returncode == 0
|
||||
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [('native_step', 'enter'), ('native_step', 'return')]
|
||||
assert not (tmp_path / 'closed.txt').exists()
|
||||
finally:
|
||||
worker.close()
|
||||
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [
|
||||
('native_step', 'enter'), ('native_step', 'return'), ('close_project', 'enter'), ('close_project', 'return')]
|
||||
assert not any(e['operation'] in {'item', 'readback'} for e in events(tmp_path))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('diagnostic_failure', [False, True])
|
||||
def test_native_failure_keeps_primary_error_even_when_diagnostics_fail(tmp_path, monkeypatch, diagnostic_failure):
|
||||
command = toy_worker(tmp_path)
|
||||
script = Path(command[2])
|
||||
script.write_text(script.read_text(encoding='utf-8').replace('saved_ok', 'invalid_checkpoint'), encoding='utf-8')
|
||||
if diagnostic_failure:
|
||||
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
|
||||
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
|
||||
try:
|
||||
with pytest.raises(WorkerMonitorError, match='invalid checkpoint'):
|
||||
worker.run_step(command, 3)
|
||||
finally:
|
||||
worker.close()
|
||||
if not diagnostic_failure:
|
||||
records = [row for row in events(tmp_path) if row['operation'] == 'native_step']
|
||||
assert [row['event'] for row in records] == ['enter', 'error']
|
||||
assert 'invalid checkpoint' in records[-1]['error']
|
||||
|
||||
|
||||
def test_diagnostic_failure_does_not_block_worker_success_or_native_close(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
|
||||
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
|
||||
try:
|
||||
assert worker.run_step(toy_worker(tmp_path), 3).returncode == 0
|
||||
finally:
|
||||
worker.close()
|
||||
assert (tmp_path / 'closed.txt').exists()
|
||||
|
||||
|
||||
def test_native_close_failure_logs_error_and_never_a_successful_close(tmp_path):
|
||||
command = toy_worker(tmp_path)
|
||||
script = Path(command[2])
|
||||
script.write_text(script.read_text(encoding='utf-8') + '\nsys.exit(7)\n', encoding='utf-8')
|
||||
worker = ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
|
||||
assert worker.run_step(command, 3).returncode == 0
|
||||
with pytest.raises(WorkerMonitorError, match='did not close cleanly'):
|
||||
worker.close()
|
||||
closing = [row for row in events(tmp_path) if row['operation'] == 'close_project']
|
||||
assert [row['event'] for row in closing] == ['enter', 'error']
|
||||
worker.close()
|
||||
assert [row for row in events(tmp_path) if row['operation'] == 'close_project'] == closing
|
||||
|
||||
|
||||
def test_result_persistence_enter_is_durable_before_actual_result_write(tmp_path, monkeypatch):
|
||||
original = batch_resources.write_record
|
||||
def observe(path, value):
|
||||
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [('result_persistence', 'enter')]
|
||||
assert not path.exists()
|
||||
original(path, value)
|
||||
monkeypatch.setattr(batch_resources, 'write_record', observe)
|
||||
result = {'attempt_id': 'attempt', 'payload': {'status': 'ok'}}
|
||||
batch_worker._persist_result(tmp_path, result)
|
||||
assert json.loads((tmp_path / 'result.json').read_text(encoding='utf-8')) == result
|
||||
assert [(e['operation'], e['event']) for e in events(tmp_path)] == [('result_persistence', 'enter'), ('result_persistence', 'return')]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('diagnostic_failure', [False, True])
|
||||
def test_result_write_failure_remains_original_exception_and_never_logs_success(tmp_path, monkeypatch, diagnostic_failure):
|
||||
failure = OSError('result storage failed')
|
||||
monkeypatch.setattr(batch_resources, 'write_record', lambda *args: (_ for _ in ()).throw(failure))
|
||||
if diagnostic_failure:
|
||||
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
|
||||
with pytest.raises(OSError) as caught:
|
||||
batch_worker._persist_result(tmp_path, {'attempt_id': 'attempt'})
|
||||
assert caught.value is failure
|
||||
assert not (tmp_path / 'result.json').exists()
|
||||
if not diagnostic_failure:
|
||||
assert [e['event'] for e in events(tmp_path)] == ['enter', 'error']
|
||||
assert events(tmp_path)[-1]['error'] == 'result storage failed'
|
||||
|
||||
|
||||
def test_diagnostic_failure_does_not_block_actual_result_persistence(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(operation_diagnostics, 'record_operation', lambda *args, **kwargs: (_ for _ in ()).throw(OSError('diagnostic disk failure')))
|
||||
batch_worker._persist_result(tmp_path, {'attempt_id': 'attempt', 'payload': 'completed'})
|
||||
assert json.loads((tmp_path / 'result.json').read_text(encoding='utf-8'))['payload'] == 'completed'
|
||||
|
||||
|
||||
def test_real_supervisor_result_write_uses_structured_persistence_boundary(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_supervisor import run_supervised
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
store = TaskStore(tmp_path / 'tasks.db')
|
||||
try:
|
||||
service = TaskCenterService(store, configure_logging(tmp_path))
|
||||
result = run_supervised(service, 'probe', {})
|
||||
assert result['resource_state'] == 'not_acquired'
|
||||
directory = next((tmp_path / 'batch-resources').iterdir())
|
||||
assert [row['event'] for row in events(directory) if row['operation'] == 'result_persistence'] == ['enter', 'return']
|
||||
assert (directory / 'result.json').exists()
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
|
||||
def test_result_write_process_crash_leaves_durable_incomplete_boundary(tmp_path):
|
||||
source = Path(__file__).resolve().parents[2] / 'src'
|
||||
script = '''import os,sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0,sys.argv[1])
|
||||
from dx_productivity.trial_task_center import batch_resources,batch_worker
|
||||
batch_resources.write_record=lambda *args: os._exit(23)
|
||||
batch_worker._persist_result(Path(sys.argv[2]),{'attempt_id':'attempt'})
|
||||
'''
|
||||
result = subprocess.run((sys.executable, '-c', script, str(source), str(tmp_path)), capture_output=True, timeout=10)
|
||||
assert result.returncode == 23, result.stderr
|
||||
summary = operation_diagnostics.summarize_operations(events(tmp_path))
|
||||
assert [(e['operation'], e['item_id']) for e in summary['incomplete_operations']] == [('result_persistence', 'attempt')]
|
||||
assert not (tmp_path / 'result.json').exists()
|
||||
|
||||
|
||||
def test_incomplete_actual_operation_can_be_reported_before_return(tmp_path):
|
||||
with operation_diagnostics.operation_span(tmp_path, 'native_step', item_id='job'):
|
||||
summary = operation_diagnostics.summarize_operations(events(tmp_path))
|
||||
assert [(e['operation'], e['item_id']) for e in summary['incomplete_operations']] == [('native_step', 'job')]
|
||||
assert summary['last_successful_item'] is None
|
||||
assert operation_diagnostics.summarize_operations(events(tmp_path))['incomplete_operations'] == []
|
||||
|
||||
|
||||
def test_diagnostic_serialization_failure_is_nonblocking(tmp_path):
|
||||
assert operation_diagnostics.record_operation(tmp_path, 'native_step', 'enter', item_id=object()) is False
|
||||
@@ -116,17 +116,28 @@ def test_manual_failure_cannot_be_completed_by_native_progress(failed_native):
|
||||
service.quick_run_progress(task_id)
|
||||
|
||||
|
||||
def test_uncertain_batch_failed_blocks_next_project_in_task_queue(failed_native, tmp_path):
|
||||
def test_uncertain_batch_failed_isolated_before_next_project_in_task_queue(failed_native, tmp_path, monkeypatch):
|
||||
from threading import Event
|
||||
from dx_productivity.trial_task_center.task_queue import execute_task_queue
|
||||
service, store, task_id, _, _ = failed_native
|
||||
other = store.create_task('next must not run')
|
||||
other = store.create_task('independent next project')
|
||||
path = tmp_path / 'next.prj'
|
||||
path.touch()
|
||||
project = store.sync_projects(other.task_id, tmp_path, (path,))[0]
|
||||
store.set_project_selected(project.project_id, True)
|
||||
store.bind_project_task(other.task_id, project.project_id, 'batch', 1)
|
||||
result = execute_task_queue(service, (task_id, other.task_id), stop_event=Event())
|
||||
assert result.paused
|
||||
assert [entry.state for entry in result.entries] == ['blocked', 'pending']
|
||||
from test_project_isolation_queue import ReleasedRuntime
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import QuickRunProgress
|
||||
calls = []
|
||||
child = service.for_project_task(other.task_id)
|
||||
monkeypatch.setattr(child, 'quick_run_progress', lambda _: calls.append(other.task_id) or QuickRunProgress(
|
||||
0, 0, None, None, None, (path,), True))
|
||||
monkeypatch.setattr(child, 'has_interrupted_quick_run', lambda _: False)
|
||||
child._quick_run_preparer.reconcile_terminal_receipts = lambda _: None
|
||||
original_child = service.for_project_task
|
||||
monkeypatch.setattr(service, 'for_project_task', lambda identifier: child if identifier == other.task_id else original_child(identifier))
|
||||
result = execute_task_queue(service, (task_id, other.task_id), stop_event=Event(), runtime=ReleasedRuntime(service._logging.root))
|
||||
assert not result.paused
|
||||
assert [entry.state for entry in result.entries] == ['blocked', 'completed']
|
||||
assert calls == [other.task_id]
|
||||
assert not (service.for_project_task(other.task_id)._quick_run_preparer.data_root / 'quick-run').exists()
|
||||
@@ -42,6 +42,11 @@ def test_inventory_rejects_changed_or_removed_baseline_atomically(batch, change)
|
||||
incoming.append(incoming[0])
|
||||
else:
|
||||
incoming[0] = replace(incoming[0], **{change: 'CHANGED'})
|
||||
if change == 'uid':
|
||||
rows = store.record_target_inventory(task.task_id, incoming)
|
||||
assert len(rows) == 3
|
||||
assert {row.target.key: row for row in rows}[before[0].target.key] == before[0]
|
||||
return
|
||||
with pytest.raises(TaskCenterError):
|
||||
store.record_target_inventory(task.task_id, incoming)
|
||||
assert store.list_target_plan(task.task_id) == before
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from test_failed_native_recovery_guard import failed_native
|
||||
|
||||
from dx_productivity.trial_task_center import batch_resources, batch_runtime
|
||||
from dx_productivity.trial_task_center.stop_control import StopControl, ProjectStopControl
|
||||
|
||||
|
||||
def test_project_force_observes_user_force_without_leaking_failure_to_parent(tmp_path):
|
||||
from dx_productivity.trial_task_center.workspace_execution import _ProjectControl
|
||||
parent = StopControl()
|
||||
local = _ProjectControl(parent)
|
||||
local.request_failure_cleanup(tmp_path, 'A failure')
|
||||
assert local.force_event.is_set()
|
||||
assert not parent.force_event.is_set() and not parent.is_set()
|
||||
successor = _ProjectControl(parent)
|
||||
assert not successor.force_event.is_set() and not successor.is_set()
|
||||
parent.request_force()
|
||||
assert successor.is_set() and successor.force_event.wait(.01)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('residual', [False, True])
|
||||
def test_failed_project_retires_proven_exit_despite_shutdown_error(tmp_path, residual):
|
||||
from test_batch_runtime import Desktop
|
||||
class OwnedDesktop(Desktop):
|
||||
def close(self):
|
||||
self.cleanup_record = {'desktop_handle_closed': True, 'owned_cleanup_complete': True,
|
||||
'desktop_confirmation': {'released': not residual}, 'errors': []}
|
||||
if residual: raise OSError('foreign desktop reference')
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', ProjectStopControl(StopControl()), desktop_factory=OwnedDesktop)
|
||||
runtime._ensure_desktop()
|
||||
def cad_exit(*args): raise OSError('shutdown diagnostics failed after exit')
|
||||
runtime.cad = SimpleNamespace(cad=SimpleNamespace(poll=lambda: 0), record={}, __exit__=cad_exit)
|
||||
original = RuntimeError('original failed project')
|
||||
assert runtime.release_failed_project(original, tmp_path / 'failed')
|
||||
assert batch_resources.blocking_resources(tmp_path) == ()
|
||||
assert runtime.record['state'] == ('isolated' if residual else 'released')
|
||||
assert runtime.record['release_results']['desktop_released'] is (not residual)
|
||||
assert runtime.desktop is None and 'shutdown diagnostics failed' in original.shutdown_warning
|
||||
assert (tmp_path / 'failed' / 'uncertain.flag').exists()
|
||||
|
||||
|
||||
def test_runtime_cleanup_events_record_real_boundaries_and_preserve_error(tmp_path):
|
||||
import json
|
||||
from test_batch_runtime import Desktop
|
||||
runtime = batch_runtime.BatchCadRuntime(tmp_path, 'execution', desktop_factory=Desktop)
|
||||
runtime._ensure_desktop()
|
||||
def cad_exit(*args): raise OSError('actual exit error')
|
||||
runtime.cad = SimpleNamespace(cad=SimpleNamespace(poll=lambda: 0), record={}, __exit__=cad_exit)
|
||||
with pytest.raises(batch_resources.BatchResourceCleanupError, match='actual exit error'):
|
||||
runtime.close()
|
||||
rows = [json.loads(line) for line in (runtime.path.parent / 'operation-events.jsonl').read_text().splitlines()]
|
||||
assert [(row['operation'], row['event']) for row in rows] == [
|
||||
('mentor_exit', 'enter'), ('mentor_exit', 'error'), ('desktop_release', 'enter'), ('desktop_release', 'return')]
|
||||
|
||||
|
||||
def test_workspace_accepts_writer_exit_warning_and_blocks_shared_source_follower(tmp_path, monkeypatch):
|
||||
from test_workspace_execution import service_for
|
||||
from dx_productivity.trial_task_center.workspace_execution import execute_workspace_projects
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
failure = TaskCenterError('quick_run_uncertain', 'A business outcome unknown')
|
||||
failure.resources_released = False
|
||||
failure.owned_writers_exited = True
|
||||
service, calls = service_for(tmp_path, monkeypatch, {'A': failure, 'B': 'completed'})
|
||||
result = execute_workspace_projects(service, ('A', 'B'), stop_event=StopControl())
|
||||
assert not result.paused and result.entries[1].state == 'completed'
|
||||
calls.clear()
|
||||
for name in ('A', 'B'):
|
||||
path = tmp_path / name / 'board.prj'
|
||||
path.parent.mkdir(exist_ok=True)
|
||||
path.write_text('KEY iCDBDir "../shared"\n')
|
||||
result = execute_workspace_projects(service, ('A', 'B'), stop_event=StopControl())
|
||||
assert not result.paused and [row.state for row in result.entries] == ['blocked', 'blocked']
|
||||
assert [call[1]['task_ids'][0] for call in calls] == ['A']
|
||||
|
||||
|
||||
def test_blocked_progress_read_does_not_promote_or_throw(failed_native):
|
||||
from dx_productivity.trial_task_center.models import TaskStatus
|
||||
service, store, task_id, handoff, _ = failed_native
|
||||
store.set_task_status(task_id, TaskStatus.BLOCKED, summary='requires reconciliation')
|
||||
service.quick_run_progress(task_id)
|
||||
assert store.get_task(task_id).status is TaskStatus.BLOCKED
|
||||
|
||||
|
||||
def test_public_recovery_tries_verified_project_commit_before_cad(failed_native, monkeypatch):
|
||||
from dx_productivity.trial_task_center.models import TaskStatus
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import QuickRunStepResult
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
service, store, task_id, handoff, _ = failed_native
|
||||
store.set_task_status(task_id, TaskStatus.BLOCKED, summary='project commit publication failed')
|
||||
child = service.for_project_task(task_id)
|
||||
result = QuickRunStepResult(handoff.script_path, handoff.command_path, handoff.status_path,
|
||||
handoff.log_path, 'saved_ok', 'verified durable project commit')
|
||||
calls = []
|
||||
monkeypatch.setattr(child._quick_run_preparer, 'recover_project_commit',
|
||||
lambda identifier, *, confirmed: calls.append((identifier, confirmed)) or result, raising=False)
|
||||
monkeypatch.setattr(service, '_synchronize_quick_run_progress', lambda identifier: calls.append(('sync', identifier)))
|
||||
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda *args, **options: pytest.fail('project commit recovery acquired CAD'))
|
||||
assert service.recover_quick_run(task_id, confirmed=True) is result
|
||||
assert calls == [(task_id, True), ('sync', task_id)]
|
||||
assert store.get_task(task_id).status is TaskStatus.READY
|
||||
|
||||
|
||||
@pytest.mark.parametrize('publication_failure', [None, 'sql', 'marker'])
|
||||
@pytest.mark.parametrize('queue_directory', ['task-queues', 'workspace-queues'])
|
||||
@pytest.mark.parametrize('previous_round', [False, True])
|
||||
def test_dependency_recovery_needs_confirmation_clear_source_and_never_opens_cad(tmp_path, monkeypatch, publication_failure, queue_directory, previous_round):
|
||||
import json
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
from dx_productivity.trial_task_center.models import TaskStatus
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.task_queue import _plan_hash
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
store = TaskStore(tmp_path / 'tasks.db')
|
||||
service = TaskCenterService(store, configure_logging(tmp_path / 'data'))
|
||||
tasks = []
|
||||
for name in ('A', 'B'):
|
||||
task = store.create_task(name)
|
||||
path = tmp_path / name / 'board.prj'
|
||||
path.parent.mkdir()
|
||||
path.write_text(' key iCDBDir "../shared"\n')
|
||||
project = store.sync_projects(task.task_id, path.parent, (path,))[0]
|
||||
store.set_project_selected(project.project_id, True)
|
||||
store.bind_project_task(task.task_id, project.project_id, 'group', len(tasks))
|
||||
store.add_rule(task.task_id, 'OLD', 'NEW')
|
||||
tasks.append((task.task_id, path))
|
||||
for task_id, _ in tasks:
|
||||
store.set_task_status(task_id, TaskStatus.BLOCKED, summary='dependency')
|
||||
dependency, follower = tasks
|
||||
child = service.for_project_task(follower[0])
|
||||
if previous_round:
|
||||
with store.transaction():
|
||||
store._connection.execute('INSERT INTO execution_rounds(task_id, round_id) VALUES (?, ?)',
|
||||
(follower[0], 'current-round'))
|
||||
store._connection.execute('''INSERT INTO execution_operations
|
||||
(task_id, round_id, operation_key, command_json, state, attempt_id, generation)
|
||||
VALUES (?, 'old-round', 'old-operation', '{}', 'verified', 'old-attempt', 1)''', (follower[0],))
|
||||
marker = service._logging.root / 'dependency-blocks' / (follower[0] + '.json')
|
||||
marker.parent.mkdir()
|
||||
marker.write_text(json.dumps({'version': 1, 'task_id': follower[0], 'dependency_task_id': dependency[0],
|
||||
'project_path': str(follower[1].resolve()).casefold(), 'sources': [str((tmp_path / 'shared/icdb.dat').resolve()).casefold()],
|
||||
'plan_hash': _plan_hash(child, follower[0]), 'queue_id': 'a'*32, 'state': 'blocked', 'undispatched': True}))
|
||||
queue_path = service._logging.root / queue_directory / ('a'*32 + '.json')
|
||||
queue_path.parent.mkdir()
|
||||
queue_path.write_text(json.dumps({'plan_hashes': {follower[0]: _plan_hash(child, follower[0])},
|
||||
'dependency_tasks': {follower[0]: dependency[0]}, 'entries': {follower[0]: {'state': 'blocked'}}}))
|
||||
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda *args, **kw: pytest.fail('dependency recovery opened CAD'))
|
||||
with pytest.raises(TaskCenterError, match='quick_run_dependency'):
|
||||
service.recover_quick_run(follower[0], confirmed=False)
|
||||
with pytest.raises(TaskCenterError, match='quick_run_dependency'):
|
||||
service.recover_quick_run(follower[0], confirmed=True)
|
||||
store.set_task_status(dependency[0], TaskStatus.READY, summary='verified recovery fixture')
|
||||
if publication_failure:
|
||||
from dx_productivity.trial_task_center import batch_resources
|
||||
original_write, original_status = batch_resources.write_record, store.set_task_status
|
||||
failed = []
|
||||
def write(path, record):
|
||||
if publication_failure == 'marker' and Path(path) == marker and record.get('state') == 'readmitted' and not failed:
|
||||
failed.append(True)
|
||||
raise OSError('readmission marker publication failed')
|
||||
return original_write(path, record)
|
||||
def set_status(identifier, state, **options):
|
||||
if publication_failure == 'sql' and identifier == follower[0] and state is TaskStatus.READY and not failed:
|
||||
failed.append(True)
|
||||
raise OSError('SQL readmission failed')
|
||||
return original_status(identifier, state, **options)
|
||||
monkeypatch.setattr(batch_resources, 'write_record', write)
|
||||
monkeypatch.setattr(store, 'set_task_status', set_status)
|
||||
with pytest.raises(TaskCenterError): service.recover_quick_run(follower[0], confirmed=True)
|
||||
assert json.loads(marker.read_text())['state'] == 'readmission_prepared'
|
||||
assert store.get_task(follower[0]).status is (TaskStatus.READY if publication_failure == 'marker' else TaskStatus.BLOCKED)
|
||||
result = service.recover_quick_run(follower[0], confirmed=True)
|
||||
assert result.state == 'pending' and store.get_task(follower[0]).status is TaskStatus.READY
|
||||
assert json.loads(marker.read_text())['state'] == 'readmitted'
|
||||
assert service.recover_quick_run(follower[0], confirmed=True).state == 'pending'
|
||||
# A restart sees the same marker/SQL result; it does not need a snapshot.
|
||||
second_service = TaskCenterService(store, configure_logging(tmp_path / 'data'))
|
||||
assert second_service.recover_quick_run(follower[0], confirmed=True).state == 'pending'
|
||||
# The original isolated task stays skipped while its undispatched follower
|
||||
# can re-enter the same frozen queue after explicit readmission.
|
||||
from dx_productivity.trial_task_center import task_queue
|
||||
original_sources = set(json.loads(marker.read_text())['sources'])
|
||||
queue = {'version': 1, 'task_ids': [dependency[0], follower[0]],
|
||||
'plan_hashes': {key: _plan_hash(service.for_project_task(key), key) for key, _ in tasks},
|
||||
'phase': 'finished', 'entries': {dependency[0]: {'state': 'blocked', 'message': 'original uncertain'},
|
||||
follower[0]: {'state': 'blocked', 'message': 'dependency'}},
|
||||
'dependency_tasks': {follower[0]: dependency[0]},
|
||||
'isolated_tasks': {dependency[0]: 'original uncertain', follower[0]: 'dependency'},
|
||||
'isolated_sources': {source: dependency[0] for source in original_sources}}
|
||||
queue_path.write_text(json.dumps(queue))
|
||||
calls = []
|
||||
from test_project_isolation_queue import ReleasedRuntime
|
||||
original_child = service.for_project_task
|
||||
def selected_child(identifier):
|
||||
selected = original_child(identifier)
|
||||
if identifier == follower[0]:
|
||||
selected.has_interrupted_quick_run = lambda _: False
|
||||
selected._quick_run_preparer.reconcile_terminal_receipts = lambda _: None
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import QuickRunProgress
|
||||
selected.quick_run_progress = lambda _: calls.append(identifier) or QuickRunProgress(0, 0, None, None, None, (follower[1],), True)
|
||||
return selected
|
||||
monkeypatch.setattr(service, 'for_project_task', selected_child)
|
||||
def run_queue():
|
||||
if queue_directory == 'task-queues':
|
||||
return task_queue.execute_task_queue(service, tuple(key for key, _ in tasks), stop_event=StopControl(),
|
||||
queue_id='a'*32, runtime=ReleasedRuntime(service._logging.root))
|
||||
from dx_productivity.trial_task_center.workspace_execution import execute_workspace_projects
|
||||
return execute_workspace_projects(service, tuple(key for key, _ in tasks), stop_event=StopControl(), queue_id='a'*32)
|
||||
if queue_directory == 'workspace-queues':
|
||||
for index, (identifier, _) in enumerate(tasks):
|
||||
queue['entries'][identifier].update(released=True, problems_done=True, checkpoint_id=str(index+1)*32)
|
||||
queue_path.write_text(json.dumps(queue))
|
||||
def supervise(operation, arguments, **controls):
|
||||
identifier = arguments['task_ids'][0]
|
||||
calls.append(identifier)
|
||||
return task_queue.TaskQueueResult(arguments['checkpoint_id'] or arguments['queue_id'],
|
||||
(task_queue.TaskQueueEntry(identifier, 'completed'),), False)
|
||||
monkeypatch.setattr(service, '_supervise', supervise)
|
||||
resumed = run_queue()
|
||||
assert [row.state for row in resumed.entries] == ['blocked', 'completed'] and calls == [follower[0]]
|
||||
calls.clear()
|
||||
run_queue()
|
||||
assert not calls
|
||||
attempted_row = {'task_id': follower[0], 'job_id': 'actually-dispatched', 'project_path': str(follower[1]),
|
||||
'old_part': 'OLD', 'new_part': 'NEW', 'schematic_name': 'MAIN', 'sheet_number': '1'}
|
||||
child._quick_run_preparer.journal.bind(follower[0], [attempted_row], {})
|
||||
child._quick_run_preparer.journal.claim(follower[0], attempted_row)
|
||||
from dx_productivity.trial_task_center import recovery
|
||||
def native_recovery(*args, **options):
|
||||
raise TaskCenterError('native_recovery_reached', 'validate the actual follower attempt')
|
||||
monkeypatch.setattr(recovery, 'recover', native_recovery)
|
||||
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda self, name, action, *args, **options: action(SimpleNamespace()))
|
||||
with pytest.raises(TaskCenterError, match='native_recovery_reached'):
|
||||
service.recover_quick_run(follower[0], confirmed=True)
|
||||
store.close()
|
||||
@@ -0,0 +1,218 @@
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from test_desktop_slot import Backend, slot
|
||||
from dx_productivity.trial_task_center.isolated_desktop import PrivateDesktop
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
|
||||
|
||||
class Api:
|
||||
def __init__(self, backend): self.backend = backend; self.launches = []
|
||||
def create_desktop(self, name):
|
||||
self.backend.exists = True
|
||||
return SimpleNamespace(CloseDesktop=lambda: None)
|
||||
def create_process(self, command, desktop, cwd, env, **options):
|
||||
self.launches.append(options['job_name'])
|
||||
return SimpleNamespace(pid=200, handle=None, poll=lambda: 0,
|
||||
close=lambda: None, wait=lambda **options: 0, cleanup_record={})
|
||||
|
||||
|
||||
def test_managed_desktop_returns_external_residue_and_old_close_cannot_touch_new_owner(tmp_path):
|
||||
backend = Backend(); api = Api(backend)
|
||||
first = PrivateDesktop('ignored', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
first.__enter__(); first.close()
|
||||
assert first.cleanup_record['desktop_confirmation']['released'] is False
|
||||
assert first.cleanup_record['desktop_lease']['returned'] is True
|
||||
second = PrivateDesktop('another-task', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
second.__enter__()
|
||||
assert second.name == first.name
|
||||
first.close()
|
||||
assert backend.locked
|
||||
second.close()
|
||||
|
||||
|
||||
def test_managed_launch_uses_durable_named_job_and_keeps_failed_launch_unresolved(tmp_path):
|
||||
backend = Backend(); api = Api(backend)
|
||||
current = PrivateDesktop('ignored', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
current.__enter__()
|
||||
def failed(*args, **options):
|
||||
assert current._slot.record['launches'][0]['job'] == options['job_name']
|
||||
raise OSError('unknown create boundary')
|
||||
api.create_process = failed
|
||||
with pytest.raises(OSError): current.start_process(('worker',))
|
||||
with pytest.raises(TaskCenterError): current.close()
|
||||
assert backend.locked
|
||||
|
||||
|
||||
def test_managed_launch_cleanup_allows_return_only_after_job_exit(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources
|
||||
backend = Backend(); api = Api(backend)
|
||||
current = PrivateDesktop('ignored', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
monkeypatch.setattr(batch_resources, 'process_identity', lambda process: {'pid': process.pid, 'creation_time': 'worker-200'})
|
||||
current.__enter__()
|
||||
current.start_process(('worker',))
|
||||
assert api.launches == [current._slot.record['launches'][0]['job']]
|
||||
current.close()
|
||||
assert not backend.locked
|
||||
|
||||
|
||||
def test_runtime_publishes_actual_managed_name_before_starting_a_worker(tmp_path):
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
backend = Backend(); api = Api(backend)
|
||||
def factory(requested, **options):
|
||||
return PrivateDesktop(requested, **options, api=api, slot=slot(tmp_path / 'slot', backend))
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=factory)
|
||||
desktop = runtime._ensure_desktop()
|
||||
assert runtime.record['desktop'] == desktop.name
|
||||
runtime.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('confirmation', [{'released': False}, None])
|
||||
def test_catalog_accepts_intact_return_without_claiming_desktop_disappearance(tmp_path, confirmation):
|
||||
from dx_productivity.trial_task_center.cell_catalog import _require_cleanup
|
||||
backend = Backend(); current = slot(tmp_path, backend).acquire()
|
||||
receipt = current.release(desktop_disappeared=False)
|
||||
cleanup = dict(desktop=current.name, desktop_handle_closed=True, owned_cleanup_complete=True, errors=[],
|
||||
desktop_confirmation=confirmation, desktop_lease=receipt,
|
||||
processes=[dict(errors=[], job_active_processes=0, process_handle_closed=True, job_handle_closed=True)])
|
||||
_require_cleanup(cleanup)
|
||||
cleanup['desktop_lease'] = dict(receipt, lease_id='forged')
|
||||
with pytest.raises(TaskCenterError): _require_cleanup(cleanup)
|
||||
|
||||
|
||||
def test_mutex_release_error_preserves_the_durably_returned_lease(tmp_path):
|
||||
from dx_productivity.trial_task_center import batch_resources
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
|
||||
class ReleaseErrorBackend(Backend):
|
||||
def unlock(self, handle):
|
||||
super().unlock(handle)
|
||||
raise OSError('mutex release failed')
|
||||
|
||||
backend = ReleaseErrorBackend()
|
||||
api = Api(backend)
|
||||
|
||||
def factory(name, **options):
|
||||
return PrivateDesktop(name, **options, api=api, slot=slot(tmp_path / 'slot', backend))
|
||||
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=factory)
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
pass
|
||||
|
||||
record = batch_resources.pending_resources(runtime.root)[0]
|
||||
assert record['state'] == 'isolated' and record['blocking'] is False
|
||||
assert record['cleanup']['desktop_lease']['returned'] is True
|
||||
assert 'mutex release failed' in record['isolation_warning']
|
||||
|
||||
next_owner = slot(tmp_path / 'slot', backend).acquire()
|
||||
with pytest.raises(OSError, match='mutex release failed'):
|
||||
next_owner.release(desktop_disappeared=False)
|
||||
assert next_owner.handle is None
|
||||
|
||||
|
||||
def test_failed_lease_return_stays_blocking_until_the_slot_can_be_returned(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
|
||||
backend = Backend()
|
||||
api = Api(backend)
|
||||
|
||||
def factory(name, **options):
|
||||
return PrivateDesktop(name, **options, api=api, slot=slot(tmp_path / 'slot', backend))
|
||||
|
||||
def fail_return(*args):
|
||||
raise OSError('desktop observation unavailable')
|
||||
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=factory)
|
||||
with pytest.raises(TaskCenterError, match='desktop observation unavailable'):
|
||||
with runtime:
|
||||
with runtime.native_scope(tmp_path / 'operation'):
|
||||
monkeypatch.setattr(backend, 'desktop_exists', fail_return)
|
||||
|
||||
assert backend.locked and batch_resources.blocking_resources(runtime.root)
|
||||
monkeypatch.undo()
|
||||
batch_resources.cleanup_pending_resources(runtime.root, retry_failed=True)
|
||||
assert not backend.locked
|
||||
assert batch_resources.blocking_resources(runtime.root) == ()
|
||||
|
||||
|
||||
def test_failed_owned_handle_cleanup_keeps_slot_until_cleanup_succeeds(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources
|
||||
backend = Backend(); api = Api(backend)
|
||||
current = PrivateDesktop('ignored', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
monkeypatch.setattr(batch_resources, 'process_identity', lambda process: {'pid': process.pid, 'creation_time': 'worker-200'})
|
||||
current.__enter__(); process = current.start_process(('worker',))
|
||||
process.close = lambda: (_ for _ in ()).throw(OSError('Job handle not closed'))
|
||||
with pytest.raises(TaskCenterError): current.close()
|
||||
assert backend.locked
|
||||
process.close = lambda: None
|
||||
current.close()
|
||||
assert not backend.locked
|
||||
|
||||
|
||||
def test_resource_recheck_never_queries_a_reused_desktop_for_a_returned_lease(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources, isolated_desktop
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
backend = Backend(); api = Api(backend)
|
||||
def factory(name, **options): return PrivateDesktop(name, **options, api=api, slot=slot(tmp_path / 'slot', backend))
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=factory)
|
||||
runtime._ensure_desktop(); runtime.close()
|
||||
assert runtime.record['state'] == 'isolated'
|
||||
next_owner = slot(tmp_path / 'slot', backend).acquire()
|
||||
from dx_productivity.trial_task_center import desktop_slot
|
||||
monkeypatch.setattr(desktop_slot, 'managed_desktop_disappeared', lambda receipt: False)
|
||||
def forbidden(*args): pytest.fail('old batch queried the currently leased desktop')
|
||||
monkeypatch.setattr(isolated_desktop, '_Win32Api', forbidden)
|
||||
warnings = batch_resources.cleanup_pending_resources(runtime.root)
|
||||
assert warnings and warnings[0]['blocking'] is False
|
||||
next_owner.release(desktop_disappeared=False)
|
||||
|
||||
|
||||
def test_failed_acquisition_cleanup_does_not_release_another_owner(tmp_path):
|
||||
backend = Backend(); api = Api(backend)
|
||||
first = PrivateDesktop('ignored', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
first.__enter__()
|
||||
second = PrivateDesktop('ignored', enabled=True, api=api, slot=slot(tmp_path, backend))
|
||||
with pytest.raises(TaskCenterError): second.__enter__()
|
||||
second.close()
|
||||
assert backend.locked and second.cleanup_record['desktop_not_acquired'] is True
|
||||
first.close()
|
||||
|
||||
|
||||
def test_transient_return_proof_read_failure_can_be_rechecked_without_permanent_block(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import batch_resources, desktop_slot
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
backend = Backend(); api = Api(backend)
|
||||
def factory(name, **options): return PrivateDesktop(name, **options, api=api, slot=slot(tmp_path / 'slot', backend))
|
||||
runtime = BatchCadRuntime(tmp_path / 'data', 'execution', desktop_factory=factory)
|
||||
runtime._ensure_desktop(); runtime.close()
|
||||
original = desktop_slot.returned_lease_intact
|
||||
monkeypatch.setattr(desktop_slot, 'managed_desktop_disappeared', lambda receipt: False)
|
||||
monkeypatch.setattr(desktop_slot, 'returned_lease_intact', lambda *args: False)
|
||||
assert batch_resources.cleanup_pending_resources(runtime.root)[0]['blocking'] is True
|
||||
monkeypatch.setattr(desktop_slot, 'returned_lease_intact', original)
|
||||
assert batch_resources.cleanup_pending_resources(runtime.root)[0]['blocking'] is False
|
||||
assert not batch_resources.blocking_resources(runtime.root)
|
||||
|
||||
|
||||
def test_native_job_creation_failure_is_certified_before_process_creation():
|
||||
from dx_productivity.trial_task_center.isolated_desktop import _Win32Api
|
||||
api = _Win32Api.__new__(_Win32Api)
|
||||
api.process = SimpleNamespace(STARTUPINFO=SimpleNamespace)
|
||||
api.con = SimpleNamespace(CREATE_NO_WINDOW=1, CREATE_SUSPENDED=2)
|
||||
def denied(*args): raise OSError('temporary Job allocation failure')
|
||||
api.job = SimpleNamespace(CreateJobObject=denied)
|
||||
with pytest.raises(OSError) as caught:
|
||||
api.create_process(('worker',), 'winsta0\\probe', None, None)
|
||||
assert caught.value.desktop_launch_cleanup_proven is True
|
||||
|
||||
|
||||
def test_unsupported_platform_can_construct_evidence_without_win32_identity(monkeypatch):
|
||||
from dx_productivity.trial_task_center import desktop_slot, isolated_desktop
|
||||
monkeypatch.setattr(isolated_desktop, 'os', SimpleNamespace(name='posix'))
|
||||
def forbidden(): pytest.fail('non-Windows evidence construction queried a Windows logon')
|
||||
monkeypatch.setattr(desktop_slot, '_context', forbidden)
|
||||
desktop = PrivateDesktop('dx-evidence', enabled=True, managed=True)
|
||||
assert desktop.name == 'dx-evidence' and desktop._slot is None
|
||||
@@ -17,10 +17,13 @@ def test_explicit_no_button_closes_owned_unknown_popup_and_releases(tmp_path):
|
||||
process = desktop.start_process((sys.executable, '-c', code), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
guard = CadDialogGuard(desktop, process, tmp_path / 'cad-dialogs.json', 'manual-native-sample')
|
||||
deadline = time.monotonic() + 10
|
||||
while not guard.summary()['blocked'] and time.monotonic() < deadline:
|
||||
dialogs = []
|
||||
while time.monotonic() < deadline:
|
||||
guard.poll('manual_sample')
|
||||
dialogs = guard.summary()['current_dialogs']
|
||||
if len(dialogs) == 1 and any(button['id'] == 7 for button in dialogs[0]['buttons']):
|
||||
break
|
||||
time.sleep(.03)
|
||||
dialogs = guard.summary()['current_dialogs']
|
||||
assert len(dialogs) == 1 and process.poll() is None
|
||||
dialog = dialogs[0]
|
||||
button = next(button for button in dialog['buttons'] if button['id'] == 7)
|
||||
|
||||
@@ -10,7 +10,7 @@ import pytest
|
||||
@pytest.mark.parametrize('behavior, steps, changed, states', [
|
||||
('success', 24, 24, ['completed', 'completed', 'completed']),
|
||||
('failure_continue', 24, 23, ['failed', 'completed', 'completed']),
|
||||
('uncertain_stop', 1, 0, ['blocked', 'pending', 'pending']),
|
||||
('uncertain_stop', 17, 16, ['blocked', 'completed', 'completed']),
|
||||
('pause_resume', 24, 24, ['completed', 'completed', 'completed']),
|
||||
])
|
||||
def test_multiple_projects_preserve_task_order_scope_results_and_measurements(tmp_path, behavior, steps, changed, states):
|
||||
@@ -24,14 +24,14 @@ def test_multiple_projects_preserve_task_order_scope_results_and_measurements(tm
|
||||
assert report['executed_steps'] == steps
|
||||
assert report['changed_components'] == changed
|
||||
assert report['unselected_components_preserved'] == 12
|
||||
assert report['result']['paused'] == (behavior == 'uncertain_stop')
|
||||
assert report['result']['paused'] is False
|
||||
assert [entry['state'] for entry in report['result']['entries']] == states
|
||||
calls = report['calls']
|
||||
assert len(calls) == steps
|
||||
assert all(len(call['refdes']) == 1 for call in calls)
|
||||
assert len({(call['project'], call['sheet'], call['refdes'][0]) for call in calls}) == steps
|
||||
assert sum(call['state'] == 'saved_ok' for call in calls) == changed
|
||||
assert [call['project'] for call in calls] == (['board-000'] if behavior == 'uncertain_stop' else
|
||||
assert [call['project'] for call in calls] == (['board-000'] + ['board-001'] * 8 + ['board-002'] * 8 if behavior == 'uncertain_stop' else
|
||||
['board-000'] * 8 + ['board-001'] * 8 + ['board-002'] * 8)
|
||||
assert calls[0]['state'] == {'failure_continue': 'batch_failed', 'uncertain_stop': 'batch_saved_unverified'}.get(behavior, 'saved_ok')
|
||||
with sqlite3.connect(output / 'tasks.db') as connection:
|
||||
|
||||
@@ -16,7 +16,7 @@ def effective_report(tmp_path, *, part='NEW', value='0R/NC', path='', duplicate=
|
||||
|
||||
|
||||
@pytest.mark.parametrize('repeated_uid', [False, True])
|
||||
def test_hierarchical_project_can_select_only_globally_unique_root_target(tmp_path, repeated_uid):
|
||||
def test_hierarchical_project_selects_unambiguous_root_and_child_references(tmp_path, repeated_uid):
|
||||
hierarchy = report(tmp_path)
|
||||
text = hierarchy.read_text().replace('Block disconnected \tRoot', 'Block child ')
|
||||
text = text.replace('Block board \tRoot\n', 'Block board \tRoot\n'
|
||||
@@ -31,17 +31,13 @@ def test_hierarchical_project_can_select_only_globally_unique_root_target(tmp_pa
|
||||
text = text.replace('$28I200', '$27I100')
|
||||
hierarchy.write_text(hierarchy.read_text().replace('$28I200', '$27I100'))
|
||||
flat.write_text(text)
|
||||
if repeated_uid:
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported, match='唯一'):
|
||||
protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
|
||||
expected_root='board', effective_report_path=flat)
|
||||
else:
|
||||
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
|
||||
expected_root='board', effective_report_path=flat)
|
||||
assert [(row.uid, row.refdes) for row in result.components] == [('$27I100', 'R1')]
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported):
|
||||
protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('CHILD',),
|
||||
expected_root='board', effective_report_path=flat)
|
||||
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
|
||||
expected_root='board', effective_report_path=flat)
|
||||
assert [(row.uid, row.refdes, row.schematic_name, row.sheet_number) for row in result.components] == [
|
||||
('@REFDES|R1' if repeated_uid else '$27I100', 'R1', 'board', 'MAIN')]
|
||||
child = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('CHILD',),
|
||||
expected_root='board', effective_report_path=flat)
|
||||
assert [(row.refdes, row.schematic_name, row.sheet_number) for row in child.components] == [('R2', 'child', 'AUX')]
|
||||
|
||||
|
||||
def test_inventory_uses_effective_instance_part_and_value_before_filtering(tmp_path):
|
||||
@@ -129,6 +125,11 @@ def test_filtered_or_ambiguous_ascii_report_requires_com_fallback(tmp_path, chan
|
||||
old, new = substitutions[change]
|
||||
text = text.replace(old, new)
|
||||
path.write_text(text)
|
||||
if change == 'bad_uid':
|
||||
result = protocol.read_ascii_discovery(tmp_path / 'source.prj', path, ('OLD',),
|
||||
effective_report_path=effective_report(tmp_path, part='OLD', value='10K/NC'))
|
||||
assert {row.refdes for row in result.components} == {'R1', 'R2'}
|
||||
return
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported):
|
||||
protocol.read_ascii_discovery(tmp_path / 'source.prj', path, ('OLD',),
|
||||
effective_report_path=effective_report(tmp_path, part='OLD', value='10K/NC'))
|
||||
@@ -179,3 +180,41 @@ def test_unrequested_reference_override_does_not_block_certified_target_inventor
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported, match='位号与属性不一致'):
|
||||
protocol.read_ascii_discovery(tmp_path / 'board.prj', report(tmp_path), ('OLD',),
|
||||
effective_report_path=flat)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('uid', ['VCC', '$29I300', '$27I100'])
|
||||
@pytest.mark.parametrize('part', ['NEW', 'UNREQUESTED'])
|
||||
def test_explicit_non_pcb_symbols_are_not_replacement_identities(tmp_path, uid, part):
|
||||
hierarchy, flat = report(tmp_path), effective_report(tmp_path)
|
||||
# Electrical annotations may carry library properties and reuse names/UIDs.
|
||||
props = '\tSymbol name power.1 type Primitive\n\tProp Part Number ' + part + '\n'
|
||||
hierarchy.write_text(hierarchy.read_text() + (' Symbol ' + uid + '\tpage AUX\n' + props) * 2)
|
||||
flat.write_text(flat.read_text() + (' Symbol ' + uid + '\tref des \tpage \n\tPath \n' + props) * 2)
|
||||
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
|
||||
effective_report_path=flat)
|
||||
assert [(row.uid, row.refdes) for row in result.components] == [('$27I100', 'R1')]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('mode', ['hierarchy', 'effective'])
|
||||
@pytest.mark.parametrize('damage', ['uid', 'ref', 'classification'])
|
||||
def test_non_pcb_filter_keeps_real_component_identity_checks(tmp_path, mode, damage):
|
||||
hierarchy, flat = report(tmp_path), effective_report(tmp_path)
|
||||
flat.write_text(flat.read_text().replace('\tPath \n',
|
||||
'\tPath \n\tSymbol name res.1 type Primitive Frozen ForwardPCB\n'))
|
||||
path = hierarchy if mode == 'hierarchy' else flat
|
||||
text = path.read_text()
|
||||
if damage == 'uid':
|
||||
text = text.replace('$27I100', 'MISSING_UID')
|
||||
elif damage == 'ref':
|
||||
text = text.replace('\tProp Ref Designator R1\n', '')
|
||||
else:
|
||||
text = text.replace('res.1 type Primitive Frozen ForwardPCB', 'power.1 type Primitive', 1)
|
||||
path.write_text(text)
|
||||
if damage == 'uid':
|
||||
result = protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
|
||||
effective_report_path=flat)
|
||||
assert [(row.uid,row.refdes) for row in result.components] == [('@REFDES|R1','R1')]
|
||||
return
|
||||
with pytest.raises(protocol.NativeInventoryUnsupported):
|
||||
protocol.read_ascii_discovery(tmp_path / 'board.prj', hierarchy, ('NEW',),
|
||||
effective_report_path=flat)
|
||||
@@ -230,6 +230,7 @@ def test_hierarchy_expansion_repeated_blocks_requires_every_instance(tmp_path):
|
||||
path.write_text(text)
|
||||
graph = {'components': {uid: {} for uid in ('$9I2/$1I1', '$9I3/$1I1')}}
|
||||
native_netlist._check_hierarchy_graph(path, graph, 'board')
|
||||
assert graph['source_sheets'] == [['board', '1'], ['child', '1']]
|
||||
graph['components'].pop('$9I3/$1I1')
|
||||
with pytest.raises(ValueError, match='coverage'):
|
||||
native_netlist._check_hierarchy_graph(path, graph, 'board')
|
||||
@@ -254,6 +255,20 @@ def test_hierarchy_has_no_arbitrary_depth_limit(tmp_path):
|
||||
native_netlist._check_hierarchy_graph(path, {'components': {identity: {}}}, 'b0')
|
||||
|
||||
|
||||
def test_root_port_name_with_slash_is_not_a_hierarchy_instance(tmp_path):
|
||||
from dx_productivity.trial_task_center import native_netlist
|
||||
effective = tmp_path / 'effective.txt'
|
||||
effective.write_text(ascii_report() + ' Symbol LED2/RXDLY\tref des \tpage \n'
|
||||
'\tPath \n\tSymbol name out.1 type \n')
|
||||
components, nets = native_netlist._parse_ascii(effective)
|
||||
hierarchy = tmp_path / 'hier.txt'
|
||||
hierarchy.write_text(ascii_report().split('Config ')[0].replace('FLAT BLOCKS', 'HIER BLOCKS')
|
||||
+ 'Block board \tRoot\n Symbol $1I1\tpage 1\n\tProp Ref Designator R1\n'
|
||||
+ ' Symbol LED2/RXDLY\tpage 1\n\tSymbol name out.1 type \n')
|
||||
native_netlist._check_hierarchy_graph(hierarchy, {'components': components, 'nets': nets}, 'board')
|
||||
assert components['LED2/RXDLY']['source_page'] == '1'
|
||||
|
||||
|
||||
def test_duplicate_nonidentity_attributes_are_preserved_and_require_declared_resolution(tmp_path):
|
||||
from dx_productivity.trial_task_center import native_netlist
|
||||
graph(tmp_path)
|
||||
|
||||
@@ -23,9 +23,9 @@ def test_normal_replacement_requires_the_selected_library_identity(change):
|
||||
native.replacement_matches([current], RECORDS, 'NEW', True, library_row=LIBRARY)
|
||||
|
||||
|
||||
def test_recovery_never_substitutes_a_different_uid_with_the_same_refdes():
|
||||
with pytest.raises(ValueError, match='identity|UID'):
|
||||
native.existing_replacement_uids([component(uid='$2I999')], RECORDS, 'NEW', {'library_row': LIBRARY})
|
||||
def test_recovery_reassociates_uid_changes_by_the_same_unique_refdes():
|
||||
assert native.existing_replacement_uids([component(uid='$2I999')], RECORDS, 'NEW',
|
||||
{'library_row': LIBRARY}) == ['$2I100']
|
||||
|
||||
|
||||
def test_recovery_treats_part_label_case_as_significant():
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Full selected C/A frames share a correlated read with untouched baselines."""
|
||||
"""Selected writer snapshots and legacy correlated native capture frames."""
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
@@ -61,7 +62,7 @@ End If
|
||||
''')
|
||||
|
||||
|
||||
def test_generated_capture_uses_original_selected_snapshot_and_untouched_reader(tmp_path):
|
||||
def test_legacy_generated_capture_uses_original_selected_snapshot_and_untouched_reader(tmp_path):
|
||||
source=tmp_path/'source.vbs'
|
||||
source.write_text(reader.MAIN.read_text(encoding='ascii')+'\n'+reader.ADAPTER.read_text(encoding='ascii'),encoding='ascii')
|
||||
generated=tmp_path/'generated.vbs';result=tmp_path/'capture.tsv'
|
||||
@@ -87,43 +88,62 @@ QPWriteUtf8 ''' + reader.quote(generated) + ''',Text
|
||||
assert reader.PAIR in text and text.endswith('END\tnonce\t2\r\n')
|
||||
|
||||
|
||||
@pytest.mark.parametrize('changed', ['', 'part', 'value', 'refdes'])
|
||||
def test_capture_consumes_native_targets_without_external_pin_read_and_rechecks_scope(tmp_path, changed):
|
||||
state = STATE.replace('\tOLD\t', '\tSAME\t') + 'A\tu1\tPart Number\tOLD\t\t0\t0\t1\t2\t3\t0\t1\r\n'
|
||||
if changed == 'part':
|
||||
state = state.replace('Part Number\tOLD', 'Part Number\tOTHER')
|
||||
elif changed == 'value':
|
||||
state = state.replace('Value\tSAME', 'Value\tCHANGED')
|
||||
elif changed == 'refdes':
|
||||
state = state.replace('C\tu1\tR1', 'C\tu1\tR9')
|
||||
(tmp_path/'native-target.tsv').write_bytes(state.encode('ascii'))
|
||||
@pytest.mark.parametrize('changed', ['', 'part', 'value', 'refdes', 'missing'])
|
||||
def test_capture_reads_selected_com_attributes_without_native_or_fullpage_validation(tmp_path, changed):
|
||||
fake=reader.NATIVE_FAKES.replace('Dim App,Comp', 'Dim App,Comp,Attr').replace(
|
||||
'App.ActiveView.Components.Add Comp,True',
|
||||
'Set Attr=New FullAttribute:Comp.Attributes.Add Attr,True\n'
|
||||
' Set Attr=New FullAttribute:Attr.Name="Part Number":Attr.Value="OLD":Attr.InstanceValue="OLD":Comp.Attributes.Add Attr,True\n'
|
||||
' App.ActiveView.Components.Add Comp,True',1)
|
||||
def replace_native(code):
|
||||
import re
|
||||
code=re.sub(r'(?ms)^Function QPUseNativeSnapshot\([^\n]*\n.*?^End Function',
|
||||
'Function QPUseNativeSnapshot()\n QPUseNativeSnapshot=True\nEnd Function',code)
|
||||
return re.sub(r'(?ms)^Function QPNativeCapture\([^\n]*\n.*?^End Function', '''Function QPNativeCapture(ByRef Selected)
|
||||
Dim Result,Targets
|
||||
Set Result=CreateObject("Scripting.Dictionary"):Set Targets=CreateObject("Scripting.Dictionary")
|
||||
Targets.Add "u1",QPReadUtf8(QP_Fso.BuildPath(QP_Root,"native-target.tsv"))
|
||||
Result.Add "total",2:Result.Add "text","":Result.Add "selected",Targets
|
||||
Set QPNativeCapture=Result
|
||||
End Function''',code)
|
||||
fake=fake.replace('Comp.UID="u2": Comp.Refdes="R2"',
|
||||
'Comp.UID="u2": Comp.Refdes="R2":Comp.RaiseOnFind=True')
|
||||
fake=fake.replace(' If Kind=128 Then Set Query=Components Else Set Query=Pins',
|
||||
' If Kind<>128 Then Err.Raise 59000,"Forbidden","pin query"\n Set Query=Components')
|
||||
if changed == 'part':
|
||||
fake=fake.replace('Attr.Value="OLD":Attr.InstanceValue="OLD"',
|
||||
'Attr.Value="OTHER":Attr.InstanceValue="OTHER"')
|
||||
elif changed == 'value':
|
||||
fake=fake.replace('Set Attr=New FullAttribute:Comp.Attributes.Add Attr,True',
|
||||
'Set Attr=New FullAttribute:Attr.Value="CHANGED":Attr.InstanceValue="CHANGED":Comp.Attributes.Add Attr,True')
|
||||
elif changed == 'refdes':
|
||||
fake=fake.replace('Comp.UID="u1": Comp.Refdes="R1"','Comp.UID="u1": Comp.Refdes="R9"')
|
||||
elif changed == 'missing':
|
||||
fake=fake.replace('App.ActiveView.Components.Add Comp,True','',1)
|
||||
|
||||
def forbid_validation(code):
|
||||
for name in ('QPUseNativeSnapshot', 'QPNativeCapture', 'QPNativeReadUntouched',
|
||||
'QPCurrentSemanticState', 'QPVerifyUntouched', 'QPUntouchedComponent', 'QPComponentPins'):
|
||||
pattern=rf'(?m)^(Function|Sub) {name}\([^\n]*\n[\s\S]*?^End \1'
|
||||
def forbidden(match):
|
||||
return match[0].split('\n',1)[0] + '\n Err.Raise 59000,"Forbidden","' + name + '"\nEnd ' + match[1]
|
||||
code,count=re.subn(pattern,forbidden,code)
|
||||
assert count==1,name
|
||||
return code
|
||||
|
||||
output=reader.run_composed_fixture(tmp_path, '''
|
||||
Dim Command,Failure
|
||||
Dim Command,Failure,FailureSource
|
||||
Set Command=CreateObject("Scripting.Dictionary")
|
||||
Command.Add "project_path","c:\\board.prj":Command.Add "old_part_number","OLD":Command.Add "new_part_number","NEW"
|
||||
Command.Add "job_id","job":Command.Add "value_mode","preserve":Command.Add "custom_value","":Command.Add "sync_nc","1"
|
||||
Command.Add "target_scope","targets":Command.Add "target_selectors",QPEncode("u1" & Chr(31) & "R1" & Chr(31) & "SAME")
|
||||
On Error Resume Next
|
||||
QPCaptureAndSelect Command,QP_Fso.BuildPath(QP_Root,"snapshot.tsv")
|
||||
Failure=Err.Number
|
||||
Failure=Err.Number:FailureSource=Err.Source
|
||||
Err.Clear
|
||||
On Error GoTo 0
|
||||
WScript.Echo CStr(Failure<>0)
|
||||
WScript.Echo CStr(QP_Fso.FileExists(QP_Fso.BuildPath(QP_Root,"snapshot.tsv")))
|
||||
''',native_app=fake,routines_transform=replace_native)
|
||||
assert output==(['True','False'] if changed else ['False','True'])
|
||||
WScript.Echo CStr(FailureSource<>"Forbidden")
|
||||
If Failure<>0 Then WScript.Echo FailureSource
|
||||
''',native_app=fake,routines_transform=forbid_validation)
|
||||
if changed:
|
||||
assert output==['True','False','True','QuickPartReplace.Target']
|
||||
return
|
||||
assert output==['False','True','True']
|
||||
rows=[line.split('\t') for line in (tmp_path/'snapshot.tsv').read_text(encoding='utf-8-sig').splitlines() if line]
|
||||
assert [row for row in rows if row[0]=='X']==[['X','saved_netlist_v1']]
|
||||
assert [row for row in rows if row[0]=='C']==[['C','u1','R1','0','0','0','1','NETLIST']]
|
||||
assert not any(row[0] in {'U','G'} for row in rows)
|
||||
attributes=[row for row in rows if row[0]=='A']
|
||||
assert {row[1] for row in attributes}=={'u1'}
|
||||
assert {row[2]:row[3:5] for row in attributes}=={'Part Number':['OLD','OLD'],'Value':['SAME','SAME']}
|
||||
@@ -170,7 +170,7 @@ def test_resume_reuses_intact_before_export_but_rebuilds_changed_output(tmp_path
|
||||
result = {'status': 'success', 'complete': True, 'capture': saved,
|
||||
'outputs': {'effective': str(output)},
|
||||
'runs': [{'command': [sys.executable], 'tool_sha256': native_netlist._hash(Path(sys.executable))}],
|
||||
'graph': {'components': {'u1': {'attributes': {'Part Number': part, 'Value': 'V'}}},
|
||||
'graph': {'components': {'u1': {'refdes': 'R1', 'attributes': {'Part Number': part, 'Value': 'V'}}},
|
||||
'nets': [{'name': 'GND', 'endpoints': [['u1', '1']]}]}}
|
||||
write_record(directory / 'netlist.json', result)
|
||||
return result
|
||||
@@ -232,7 +232,7 @@ def test_graph_match_does_not_claim_failed_execution_verified(tmp_path, monkeypa
|
||||
destination.mkdir(parents=True)
|
||||
return {'project': str(source), 'root': str(destination)}
|
||||
|
||||
graph = {'components': {'u': {'attributes': {'Part Number': 'NEW', 'Value': 'V'}}},
|
||||
graph = {'components': {'u': {'refdes': 'R1', 'attributes': {'Part Number': 'NEW', 'Value': 'V'}}},
|
||||
'nets': [{'name': 'N', 'endpoints': [['u', '1']]}]}
|
||||
monkeypatch.setattr(native_netlist, 'capture_project', capture)
|
||||
monkeypatch.setattr(native_netlist, 'export_netlist', lambda *a, **k: {'status': 'success', 'complete': True, 'graph': graph})
|
||||
@@ -253,7 +253,8 @@ def test_expected_library_identity_comes_from_intact_bound_native_receipt(proof_
|
||||
receipt = {'native_state': 'saved_ok', 'proof': proof}
|
||||
library = library_from_receipt(command, receipt)
|
||||
assert library == {'uid': {'attributes': {'Part Number': 'NEW', 'Part Name': 'name', 'Part Label': 'label'},
|
||||
'target_symbol': 'sym', 'slot_count': 1}}
|
||||
'target_symbol': 'sym', 'slot_count': 1, 'refdes': 'R1',
|
||||
'schematic_name': 'MAIN', 'sheet_number': '1'}}
|
||||
native = next(Path(item['path']) for item in proof['files'] if item['path'].endswith('.native.json'))
|
||||
native.write_text('{}', encoding='utf-8')
|
||||
with pytest.raises(ValueError, match='证据'):
|
||||
@@ -339,7 +340,8 @@ def test_rule_netlist_accepts_bound_component_expansion_and_rejects_foreign_scop
|
||||
_rule_rows(child, 'task', plan, before)
|
||||
else:
|
||||
rows, scopes = _rule_rows(child, 'task', plan, before)
|
||||
assert rows[0]['target'] == {'uid': 'uid', 'old_value': 'ORIGINAL'}
|
||||
assert rows[0]['target'] == {'uid': 'uid', 'refdes': 'R1', 'old_value': 'ORIGINAL',
|
||||
'schematic_name': 'MAIN', 'sheet_number': '1'}
|
||||
assert rows[0]['new_part_number'] == 'NEW'
|
||||
assert scopes[0]['source_uids'] == ['uid']
|
||||
finally:
|
||||
|
||||
@@ -13,7 +13,8 @@ from dx_productivity.trial_task_center.target_plan import ComponentTarget, Targe
|
||||
from dx_productivity.trial_task_center.task_queue import execute_task_queue
|
||||
|
||||
|
||||
def test_hundred_uid_batch_keeps_93_verified_items_and_stops_at_unknown_write(tmp_path):
|
||||
def test_hundred_uid_batch_keeps_93_verified_items_and_isolates_unknown_writes(tmp_path):
|
||||
from test_project_isolation_queue import ReleasedRuntime
|
||||
calls = []
|
||||
outcomes = {}
|
||||
def worker(command, timeout):
|
||||
@@ -58,20 +59,22 @@ def test_hundred_uid_batch_keeps_93_verified_items_and_stops_at_unknown_write(tm
|
||||
# the production queue, receipts, backups, issue pass and runtime cleanup.
|
||||
def supervise(operation, arguments, *, stop_event):
|
||||
assert operation == 'execute_task_queue'
|
||||
return execute_task_queue(engine, stop_event=stop_event, **arguments)
|
||||
runtime = ReleasedRuntime(data)
|
||||
runtime.desktop = None
|
||||
return execute_task_queue(engine, stop_event=stop_event, runtime=runtime, **arguments)
|
||||
gui._supervise = supervise
|
||||
ids = tuple(task.task_id for task in tasks)
|
||||
try:
|
||||
result = execute_task_queue(gui, ids, stop_event=Event())
|
||||
assert result.paused
|
||||
assert [entry.state for entry in result.entries] == ['failed', 'completed', 'blocked', 'pending']
|
||||
assert len(calls) == 99 and set(Counter(calls).values()) == {1}
|
||||
assert (tasks[3].task_id, 'u1') not in calls
|
||||
assert not result.paused
|
||||
assert [entry.state for entry in result.entries] == ['failed', 'completed', 'blocked', 'blocked']
|
||||
assert len(calls) == 100 and set(Counter(calls).values()) == {1}
|
||||
assert (tasks[3].task_id, 'u1') in calls
|
||||
records = dict(store._connection.execute('SELECT state,count(*) FROM execution_operations GROUP BY state'))
|
||||
assert records == {'verified': 93, 'failed': 5, 'uncertain': 1}
|
||||
assert records == {'verified': 93, 'failed': 5, 'uncertain': 2}
|
||||
resumed = execute_task_queue(gui, ids, stop_event=Event(), queue_id=result.queue_id)
|
||||
assert resumed.paused and resumed.entries[2].state == 'blocked'
|
||||
assert len(calls) == 99
|
||||
assert not resumed.paused and resumed.entries[2].state == 'blocked'
|
||||
assert len(calls) == 100
|
||||
finally:
|
||||
store.close()
|
||||
reopened = TaskStore(database)
|
||||
|
||||
@@ -88,9 +88,13 @@ def test_uncertain_component_never_runs_same_project_remainder(component_case):
|
||||
service, _, tasks, calls, outcomes = component_case
|
||||
task_id = tasks[0].task_id
|
||||
outcomes[(task_id, ('R1',))] = 'batch_saved_unverified'
|
||||
result = execute_task_queue(service, (task_id,), stop_event=Event())
|
||||
assert result.paused
|
||||
assert calls == [(task_id, ('R1',))]
|
||||
ids = tuple(task.task_id for task in tasks)
|
||||
result = execute_task_queue(service, ids, stop_event=Event())
|
||||
assert not result.paused and [entry.state for entry in result.entries] == ['blocked', 'completed']
|
||||
assert calls == [(task_id, ('R1',)), (ids[1], ('R1',)), (ids[1], ('R2',))]
|
||||
assert service.has_interrupted_quick_run(task_id)
|
||||
execute_task_queue(service, ids, stop_event=Event(), queue_id=result.queue_id)
|
||||
assert len(calls) == 3
|
||||
|
||||
|
||||
def test_problem_retry_budget_survives_rebinding_and_is_shared(component_case):
|
||||
|
||||
@@ -2,6 +2,94 @@ import pytest
|
||||
import test_accuracy_vbs_guards as guards
|
||||
|
||||
|
||||
@pytest.mark.parametrize('fault', ['', 'snapshot', 'identity', 'transaction', 'native',
|
||||
'save_requested', 'save_committed', 'close'])
|
||||
def test_safe_pause_cannot_interrupt_verified_failure_cleanup(tmp_path, fault):
|
||||
output = guards.run_vbs(tmp_path, ['QPProgress', 'QPDiscardFailedBatchProject',
|
||||
'QPCloseBatchProject', 'QPReadUtf8'], '''
|
||||
Dim Failure,Stream,PausedBefore,PausedAfter
|
||||
Set QP_App=New FakeApp
|
||||
Set Stream=QP_Fso.CreateTextFile(QP_Fso.BuildPath(QP_Root,"monitor-token.stop"),True)
|
||||
Stream.Write "token":Stream.Close
|
||||
On Error Resume Next
|
||||
QPProgress "query_components",0,1
|
||||
PausedBefore=(Err.Source="QuickPartReplace.SafeStop"):Err.Clear
|
||||
QPDiscardFailedBatchProject Failure
|
||||
If Err.Number<>0 Then WScript.Quit 8
|
||||
QPProgress "query_components",0,1
|
||||
PausedAfter=(Err.Source="QuickPartReplace.SafeStop"):Err.Clear
|
||||
WScript.Echo "result:" & CStr(PausedBefore) & ":" & CStr(QP_App.Closed) & ":" & _
|
||||
CStr(Len(Failure)>0) & ":" & CStr(PausedAfter)
|
||||
''', prelude='''
|
||||
Option Explicit
|
||||
Dim QP_Fso,QP_Root,QP_App,QP_CurrentJobId,QP_AttemptToken,QP_ProgressSequence
|
||||
Dim QP_NativeMutationPossible,QP_RestoreTransactionOpen,QP_NativeCallActive,QP_ExpectedJobId
|
||||
Dim QP_BatchMode,QP_SessionMode,QP_MonitorJobId,QP_SessionToken,QP_OwnedProject,QP_OwnedProjectPath
|
||||
Dim QP_BatchSaveRequested,QP_BatchSaveCommitted,QP_SessionExternalChange,QP_FailureCleanupActive
|
||||
Dim QP_SessionItems,QP_SessionDocuments
|
||||
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary")
|
||||
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
|
||||
QP_Root="''' + str(tmp_path) + '''":QP_AttemptToken="token":QP_ProgressSequence=0
|
||||
QP_BatchMode=True:QP_OwnedProject=True:QP_OwnedProjectPath="board.prj"
|
||||
QP_NativeMutationPossible=''' + str(fault == 'native') + '''
|
||||
QP_BatchSaveRequested=''' + str(fault == 'save_requested') + '''
|
||||
QP_BatchSaveCommitted=''' + str(fault == 'save_committed') + '''
|
||||
Class FakeApp
|
||||
Public Closed,Path
|
||||
Private Documents
|
||||
Private Sub Class_Initialize()
|
||||
Closed=False:Path="board.prj"
|
||||
Set Documents=CreateObject("Scripting.Dictionary")
|
||||
Documents.Add 1,New FakeDocument
|
||||
End Sub
|
||||
Function SchematicSheetDocuments()
|
||||
Set SchematicSheetDocuments=Documents
|
||||
End Function
|
||||
Sub RemoveDocument()
|
||||
Documents.Remove 1
|
||||
End Sub
|
||||
Sub CloseProject()
|
||||
Closed=True:Path=""
|
||||
End Sub
|
||||
Function GetProjectData()
|
||||
Set GetProjectData=Me
|
||||
End Function
|
||||
Function GetProjectFilePath()
|
||||
GetProjectFilePath=Path
|
||||
End Function
|
||||
End Class
|
||||
Class FakeDocument
|
||||
Sub Close(ByVal Save,ByVal Name)
|
||||
If Save Then Err.Raise 1235,"Test","unexpected save"
|
||||
If "''' + fault + '''"="close" Then Err.Raise 1236,"FakeCOM","close failed"
|
||||
QP_App.RemoveDocument
|
||||
End Sub
|
||||
End Class
|
||||
Sub QPVerifyFailedOriginalSnapshot()
|
||||
If "''' + fault + '''"="snapshot" Then Err.Raise 1234,"Snapshot","source changed"
|
||||
If "''' + fault + '''"="identity" Then QP_App.Path="other.prj"
|
||||
If "''' + fault + '''"="transaction" Then QP_RestoreTransactionOpen=True
|
||||
QPProgress "verify_untouched",1,1
|
||||
End Sub
|
||||
Function QPCanonicalPath(ByVal Path)
|
||||
QPCanonicalPath=LCase(Path)
|
||||
End Function
|
||||
Function QPProjectPath()
|
||||
QPProjectPath=QP_App.Path
|
||||
End Function
|
||||
Sub QPInvalidatePinIndex()
|
||||
End Sub
|
||||
Sub QPInvalidateComponentIndex()
|
||||
End Sub
|
||||
Sub QPLog(ByVal Root,ByVal Level,ByVal Message)
|
||||
End Sub
|
||||
''')
|
||||
paused_before = fault != 'native'
|
||||
paused_after = fault not in {'native', 'transaction'}
|
||||
assert output.splitlines()[-1] == f'result:{paused_before}:{not fault}:{bool(fault)}:{paused_after}'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('snapshot_changed', [False, True])
|
||||
def test_failed_project_closes_only_after_unchanged_snapshot_is_certified(tmp_path, snapshot_changed):
|
||||
output = guards.run_vbs(tmp_path, ['QPDiscardFailedBatchProject'], '''
|
||||
|
||||
@@ -0,0 +1,383 @@
|
||||
from contextlib import contextmanager
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
from threading import Event
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer, _encode_vbs_field
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
from test_nonblocking_items import component_case
|
||||
from test_saved_netlist_verification import saved_case
|
||||
|
||||
|
||||
@pytest.mark.parametrize('pause_after_first', [False, True])
|
||||
def test_real_batch_preparer_and_journal_commit_staged_prefix_only_after_native_close(tmp_path, pause_after_first, monkeypatch):
|
||||
monkeypatch.setattr('dx_productivity.trial_task_center.scanner._is_schematic_project', lambda path: True)
|
||||
project = tmp_path / 'projects' / 'board.prj'
|
||||
project.parent.mkdir(); project.write_text('disposable project')
|
||||
root = tmp_path / 'data' / 'quick-run'
|
||||
stop = Event()
|
||||
preparer = QuickRunPreparer(tmp_path / 'data', process_runner=lambda *args: None,
|
||||
cscript_resolver=lambda: Path('cscript.exe'), sheet_discoverer=lambda project: (('Main', '1'),))
|
||||
service = TaskCenterService(TaskStore(tmp_path / 'tasks.db'), configure_logging(tmp_path / 'data'), quick_run_preparer=preparer)
|
||||
task = service.create_task('staged project')
|
||||
service.scan_projects(task.task_id, project.parent)
|
||||
for row in service.projects(task.task_id):service.set_project_selected(row.project_id, True)
|
||||
service.add_rule(task.task_id, 'OLD', 'NEW')
|
||||
service.add_rule(task.task_id, 'OTHER', 'NEW')
|
||||
|
||||
class NativeBoundary:
|
||||
enabled = True
|
||||
worker = None
|
||||
on_commit = None
|
||||
jobs = []
|
||||
closes = 0
|
||||
|
||||
def prepare(self, project):pass
|
||||
|
||||
def run_step(self, project, command, timeout):
|
||||
self.worker = SimpleNamespace(token='a' * 32, root=root)
|
||||
job = command[-1]
|
||||
if self.jobs:
|
||||
states = [row[0] for row in service._store._connection.execute('SELECT state FROM execution_operations')]
|
||||
assert states.count('pending_save') == 1
|
||||
assert 'verified' not in states
|
||||
assert self.closes == 0
|
||||
self.jobs.append(job)
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
stream.write(f'{job}\tpending_save\tsession_staged_no_match\tverified in memory\n')
|
||||
if pause_after_first:stop.set()
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
|
||||
def close(self):
|
||||
if self.worker is None:return
|
||||
self.closes += 1
|
||||
marker = f'{self.worker.token}\t{project}\tproject_verified\n'
|
||||
for job in self.jobs:
|
||||
state = '\t'.join(_encode_vbs_field(str(value)) for value in
|
||||
(job, project, 'Main', '1', 'S\tfull_sheet_v1\r\nunchanged'))
|
||||
(root / f'post-state-{job}.tsv').write_text(
|
||||
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' + state + '\n', encoding='utf-8')
|
||||
marker += f'{job}\tno_match\n'
|
||||
(root / f'project-commit-{self.worker.token}.tsv').write_text(marker, encoding='utf-8')
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
for job in self.jobs:stream.write(f'{job}\tno_match\tproject_committed\tclosed and read back\n')
|
||||
if self.on_commit:self.on_commit(self.worker)
|
||||
self.worker = None
|
||||
|
||||
boundary = NativeBoundary()
|
||||
@contextmanager
|
||||
def retained_scope(**kwargs):
|
||||
try:yield boundary
|
||||
finally:boundary.close()
|
||||
preparer.project_session_scope = retained_scope
|
||||
progress = service.quick_run_progress(task.task_id)
|
||||
result = service.execute_quick_run_queue(task.task_id,
|
||||
expected_item=(str(project), progress.next_old_part_number, progress.next_new_part_number, 'preserve'), stop_event=stop)
|
||||
assert boundary.closes == 1
|
||||
assert len(boundary.jobs) == (1 if pause_after_first else 2)
|
||||
assert result.progress.completed_items == len(boundary.jobs)
|
||||
assert result.progress.saved_items == 0
|
||||
assert result.progress.no_match_items == len(boundary.jobs)
|
||||
assert result.paused is pause_after_first
|
||||
assert result.progress.queue_complete is (not pause_after_first)
|
||||
assert service._store._connection.execute('SELECT state FROM execution_project_commits').fetchone()[0] == 'verified'
|
||||
states = [row[0] for row in service._store._connection.execute('SELECT state FROM execution_operations')]
|
||||
assert states.count('no_match') == len(boundary.jobs)
|
||||
if pause_after_first:assert states.count('pending') == 1
|
||||
service._store.close()
|
||||
|
||||
|
||||
def test_runtime_prewrite_pause_commits_prefix_and_retires_only_current_claim(tmp_path, monkeypatch):
|
||||
from test_batch_runtime import worker_runtime, Cad
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.stop_control import StopControl
|
||||
monkeypatch.setattr('dx_productivity.trial_task_center.scanner._is_schematic_project', lambda path: True)
|
||||
project = tmp_path / 'projects' / 'board.prj'
|
||||
project.parent.mkdir(); project.write_text('disposable project')
|
||||
root = tmp_path / 'data' / 'quick-run'
|
||||
stop = StopControl()
|
||||
preparer = QuickRunPreparer(tmp_path / 'data', process_runner=lambda *args: None,
|
||||
cscript_resolver=lambda: Path('cscript.exe'), sheet_discoverer=lambda project: (('Main', '1'),))
|
||||
service = TaskCenterService(TaskStore(tmp_path / 'tasks.db'), configure_logging(tmp_path / 'data'), quick_run_preparer=preparer)
|
||||
task = service.create_task('prewrite pause')
|
||||
service.scan_projects(task.task_id, project.parent)
|
||||
for row in service.projects(task.task_id):service.set_project_selected(row.project_id, True)
|
||||
service.add_rule(task.task_id, 'OLD', 'NEW'); service.add_rule(task.task_id, 'OTHER', 'NEW')
|
||||
token = 'a' * 32
|
||||
current = {}
|
||||
|
||||
def native_monitor(command, *args, **kwargs):
|
||||
stop.set()
|
||||
prefix, paused = current['jobs']
|
||||
fields = (prefix, str(project), 'Main', '1', 'S\tfull_sheet_v1\r\nverified final state')
|
||||
(root / f'post-state-{prefix}.tsv').write_text(
|
||||
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
|
||||
'\t'.join(_encode_vbs_field(value) for value in fields) + '\n', encoding='utf-8')
|
||||
(root / f'project-commit-{token}.tsv').write_text(f'{token}\t{project}\tproject_verified\n{prefix}\tno_match\n', encoding='utf-8')
|
||||
(root / f'project-pause-{token}.tsv').write_text(f'{token}\t{project}\t{paused}\tprewrite_closed', encoding='utf-8')
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
stream.write(f'{prefix}\tno_match\tproject_committed\tprefix closed\n')
|
||||
stream.write(f'{paused}\tawaiting_native_replace\t\tsnapshot before pause\n')
|
||||
result = subprocess.CompletedProcess(command, 75, '', '')
|
||||
result.prewrite_pause = dict(session_token=token, monitor_token='b' * 32, job_id=paused,
|
||||
project_path=str(project), sequence=7, worker_tree_exited=True)
|
||||
return result
|
||||
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, native_monitor)
|
||||
runtime.cad_factory = lambda *args, stop_control=None, **options: Cad(*args, **options)
|
||||
runtime.stop_event = stop
|
||||
|
||||
class RetainedNative:
|
||||
enabled = True
|
||||
worker = SimpleNamespace(token=token, root=root)
|
||||
on_commit = None
|
||||
def prepare(self, project):pass
|
||||
def run_step(self, project, command, timeout):
|
||||
current.setdefault('jobs', []).append(command[-1])
|
||||
(root / 'project-session.request').write_text(f'{token}\t{project}\t{command[-1]}', encoding='utf-8')
|
||||
if len(current['jobs']) == 1:
|
||||
current['initial_command'] = command
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
stream.write(f'{command[-1]}\tpending_save\tsession_staged_no_match\tstaged\n')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
return runtime.run_worker(current['initial_command'], timeout)
|
||||
def close(self):
|
||||
if self.on_commit:self.on_commit(self.worker)
|
||||
self.worker = None
|
||||
|
||||
boundary = RetainedNative()
|
||||
@contextmanager
|
||||
def retained_scope(**kwargs):
|
||||
try:yield boundary
|
||||
finally:boundary.close()
|
||||
preparer.project_session_scope = retained_scope
|
||||
progress = service.quick_run_progress(task.task_id)
|
||||
with runtime, pytest.raises(TaskCenterError) as raised:
|
||||
service.execute_quick_run_queue(task.task_id, expected_item=(str(project), progress.next_old_part_number,
|
||||
progress.next_new_part_number, 'preserve'), stop_event=stop, runtime=runtime)
|
||||
assert raised.value.code == 'operation_cancelled'
|
||||
assert not (root / 'uncertain.flag').exists()
|
||||
assert not (root / 'current-command.tsv').exists()
|
||||
states = [row[0] for row in service._store._connection.execute('SELECT state FROM execution_operations ORDER BY rowid')]
|
||||
assert states == ['no_match', 'pending']
|
||||
record = service._store._connection.execute("SELECT * FROM execution_operations WHERE state='pending'").fetchone()
|
||||
assert record['generation'] == 0
|
||||
assert preparer.journal.certified_startup_pause(task.task_id, record)
|
||||
assert service.quick_run_progress(task.task_id).completed_items == 1
|
||||
assert not preparer.has_interrupted(task.task_id)
|
||||
service._store.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('saved_case', ['execute'], indirect=True)
|
||||
@pytest.mark.parametrize('failure', [None, 'publication', 'marker_locked'])
|
||||
def test_saved_prewrite_pause_preserves_verified_prefix_and_retired_claim(
|
||||
saved_case, tmp_path, monkeypatch, failure):
|
||||
from dx_productivity.trial_task_center import quick_run
|
||||
from dx_productivity.trial_task_center.batch_attempt import _publish
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.quick_run import _file_identity, _read_status_details
|
||||
from dx_productivity.trial_task_center.stop_control import StopControl
|
||||
from test_batch_runtime import Cad, worker_runtime
|
||||
|
||||
service, store, task, preparer, project, rows, control = saved_case
|
||||
root = preparer.data_root / 'quick-run'
|
||||
prefix, paused = [row['job_id'] for row in rows]
|
||||
token = 'a' * 32
|
||||
stop = StopControl()
|
||||
(root / f'project-commit-{token}.tsv').unlink()
|
||||
monkeypatch.setattr('dx_productivity.trial_task_center.cell_catalog.preflight_command_target', lambda *args: None)
|
||||
marker_locked = failure == 'marker_locked'
|
||||
if marker_locked:
|
||||
unlink = Path.unlink
|
||||
|
||||
def lock_marker(path, *args, **kwargs):
|
||||
if marker_locked and path.name == 'current-command.tsv':
|
||||
raise PermissionError('current-command.tsv is locked')
|
||||
return unlink(path, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(Path, 'unlink', lock_marker)
|
||||
if failure == 'publication':
|
||||
append = quick_run._append_status_details
|
||||
|
||||
def fail_prefix_publication(path, statuses):
|
||||
if any(state == 'saved_ok' for _, state, _, _ in statuses):
|
||||
raise OSError('status append interrupted after SQL commit')
|
||||
return append(path, statuses)
|
||||
|
||||
monkeypatch.setattr(quick_run, '_append_status_details', fail_prefix_publication)
|
||||
|
||||
def native_monitor(command, *args, **kwargs):
|
||||
stop.set()
|
||||
path, expected = runtime._current_attempts[0]
|
||||
_publish(path, dict(expected, state='mutation_possible'))
|
||||
(root / f'project-commit-{token}.tsv').write_text(f'{token}\t{project}\tproject_saved\n{prefix}\tpending_save\n')
|
||||
(root / f'project-pause-{token}.tsv').write_text(f'{token}\t{project}\t{paused}\tprewrite_closed')
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
stream.write(f'{paused}\tawaiting_native_replace\t\tsnapshot before pause\n')
|
||||
result = subprocess.CompletedProcess(command, 75, '', '')
|
||||
result.prewrite_pause = dict(session_token=token, job_id=paused, project_path=str(project),
|
||||
sequence=7, worker_tree_exited=True)
|
||||
return result
|
||||
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, native_monitor)
|
||||
runtime.stop_event = stop
|
||||
runtime.cad_factory = lambda *args, stop_control=None, **options: Cad(*args, **options)
|
||||
runtime_attempt = 'c' * 32
|
||||
from dx_productivity.trial_task_center import batch_runtime
|
||||
monkeypatch.setattr(batch_runtime, 'uuid4', lambda: SimpleNamespace(hex=runtime_attempt))
|
||||
report_path = root / f'snapshot-{prefix}.tsv.native.json'
|
||||
report = json.loads(report_path.read_text())
|
||||
report['batch_attempt'].update(session_id=runtime.session_id, attempt_id=runtime_attempt)
|
||||
report_path.write_text(json.dumps(report))
|
||||
|
||||
class RetainedNative:
|
||||
enabled = True
|
||||
worker = SimpleNamespace(token=token, root=root)
|
||||
on_commit = None
|
||||
|
||||
def prepare(self, project): pass
|
||||
|
||||
def run_step(self, project, command, timeout):
|
||||
(root / 'project-session.request').write_text(f'{token}\t{project}\t{command[-1]}')
|
||||
if command[-1] == prefix:
|
||||
self.initial_command = command
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
stream.write(f'{prefix}\tpending_save\tsession_staged\tapplied\n')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
return runtime.run_worker(self.initial_command, timeout)
|
||||
|
||||
def close(self):
|
||||
if self.on_commit:
|
||||
self.on_commit(self.worker)
|
||||
self.worker = None
|
||||
|
||||
identity = _file_identity((root / 'status.tsv').stat())
|
||||
boundary = RetainedNative()
|
||||
def execute():
|
||||
return preparer.execute_step(task, service.projects(task), service.rules(task), batch_mode=True,
|
||||
batch_session=control['session'], project_session=boundary, stop_event=stop, runtime=runtime)
|
||||
with runtime:
|
||||
assert execute().state == 'pending_save'
|
||||
with pytest.raises(TaskCenterError) as raised:
|
||||
execute()
|
||||
if raised.value.code == 'operation_cancelled':
|
||||
boundary.close()
|
||||
assert _file_identity((root / 'status.tsv').stat()) == identity
|
||||
if marker_locked:
|
||||
assert (root / 'current-command.tsv').is_file()
|
||||
marker_locked = False
|
||||
assert not service.has_interrupted_quick_run(task)
|
||||
assert not (root / 'current-command.tsv').exists()
|
||||
record = store._connection.execute("SELECT * FROM execution_operations WHERE state='pending'").fetchone()
|
||||
assert preparer.journal.certified_startup_pause(task, record)
|
||||
proof = json.loads(preparer.journal.project_commit_records(task)[0]['proof_json'])
|
||||
assert proof['operation_attempts'] == preparer.journal.project_commit_attempts(task, token)
|
||||
assert proof['native_attempt']['attempt_id'] == runtime_attempt
|
||||
if failure == 'publication':
|
||||
assert raised.value.code == 'quick_run_uncertain'
|
||||
assert service.has_interrupted_quick_run(task)
|
||||
assert 'project_commit_unverified' in (root / 'uncertain.flag').read_text()
|
||||
# Older project proofs can recover from the independently bound per-job receipt.
|
||||
del proof['operation_attempts']
|
||||
store._connection.execute('UPDATE execution_project_commits SET proof_json=?', (json.dumps(proof),))
|
||||
assert service.recover_quick_run(task, confirmed=True).progress.saved_items == 1
|
||||
else:
|
||||
assert raised.value.code == 'operation_cancelled', raised.value.__cause__
|
||||
assert {job: (state, code) for job, state, code, _ in _read_status_details(root / 'status.tsv')} == {
|
||||
prefix: ('saved_ok', 'project_committed'), paused: ('pending', 'prewrite_paused')}
|
||||
assert not (root / 'uncertain.flag').exists()
|
||||
assert service.quick_run_progress(task).saved_items == 1
|
||||
assert not service.has_interrupted_quick_run(task)
|
||||
assert len(control['calls']) == 3
|
||||
|
||||
|
||||
def test_component_failure_closes_retained_prefix_with_commit_callback(component_case, monkeypatch):
|
||||
from dx_productivity.trial_task_center.project_session import ProjectSessionScope
|
||||
from dx_productivity.trial_task_center.target_plan import ComponentTarget, TargetPlanRow
|
||||
|
||||
service, store, tasks, calls, outcomes = component_case
|
||||
task = tasks[0].task_id
|
||||
child = service.for_project_task(task)
|
||||
preparer = child._quick_run_preparer
|
||||
project = child.projects(task)[0].path
|
||||
database = project.parent / 'database'
|
||||
database.mkdir()
|
||||
(database / 'icdb.dat').write_bytes(b'database')
|
||||
root = preparer.data_root / 'quick-run'
|
||||
targets = tuple(ComponentTarget(project, 'MAIN', '1', f'u{i}', f'R{i}', 'OLD', '10K') for i in (1, 2, 3))
|
||||
store.record_target_inventory(task, targets)
|
||||
store.replace_target_plan(task, tuple(TargetPlanRow(target, True, 'NEW') for target in targets))
|
||||
dispatched = []
|
||||
|
||||
class RetainedWorker:
|
||||
def __init__(self, project, root, **kwargs):
|
||||
self.project, self.root = project, root
|
||||
self.token = ('a' if not dispatched else 'b') * 32
|
||||
self._done = Event()
|
||||
self._abort = Event()
|
||||
self.staged = []
|
||||
|
||||
def run_step(self, command, timeout):
|
||||
job = command[-1]
|
||||
dispatched.append(job)
|
||||
if len(dispatched) == 2:
|
||||
outcomes[(task, ('R2',))] = 'batch_failed'
|
||||
return preparer._process_runner(command, timeout)
|
||||
if len(dispatched) == 3:
|
||||
assert preparer.journal.project_commit_records(task)[0]['state'] == 'verified'
|
||||
self.staged.append(job)
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
stream.write(f'{job}\tpending_save\tsession_staged_no_match\tstaged\n')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
|
||||
def close(self):
|
||||
if self._done.is_set():
|
||||
return
|
||||
marker = f'{self.token}\t{project}\tproject_verified\n'
|
||||
for job in self.staged:
|
||||
fields = (job, str(project), 'MAIN', '1', 'S\tfull_sheet_v1\r\nverified final state')
|
||||
(root / f'post-state-{job}.tsv').write_text(
|
||||
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
|
||||
'\t'.join(_encode_vbs_field(value) for value in fields) + '\n', encoding='utf-8')
|
||||
marker += f'{job}\tno_match\n'
|
||||
(root / f'project-commit-{self.token}.tsv').write_text(marker, encoding='utf-8')
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:
|
||||
for job in self.staged:
|
||||
stream.write(f'{job}\tno_match\tproject_committed\tclosed\n')
|
||||
self._done.set()
|
||||
|
||||
monkeypatch.setattr('dx_productivity.trial_task_center.project_session.ProjectSessionWorker', RetainedWorker)
|
||||
preparer.project_session_scope = lambda **kwargs: ProjectSessionScope(preparer._process_runner, enabled=True)
|
||||
progress = child.quick_run_progress(task)
|
||||
result = child.execute_quick_run_queue(task, expected_item=(str(project), progress.next_old_part_number,
|
||||
progress.next_new_part_number, 'preserve'), stop_event=Event())
|
||||
assert len(dispatched) == 3
|
||||
assert result.progress.queue_complete
|
||||
assert (result.progress.no_match_items, result.progress.failed_items) == (2, 1)
|
||||
assert [row[0] for row in store._connection.execute('SELECT state FROM execution_operations ORDER BY rowid')] == [
|
||||
'no_match', 'failed', 'no_match']
|
||||
|
||||
|
||||
@pytest.mark.parametrize('error_type', [ValueError, OSError])
|
||||
def test_reconciliation_marks_verification_errors_uncertain(saved_case, monkeypatch, error_type):
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
|
||||
_, _, task, preparer, _, _, _ = saved_case
|
||||
failure = error_type('saved database verification failed')
|
||||
|
||||
def fail(*args):
|
||||
raise failure
|
||||
|
||||
monkeypatch.setattr(preparer, '_commit_staged_project', fail)
|
||||
with pytest.raises(TaskCenterError) as raised:
|
||||
preparer.reconcile_terminal_receipts(task)
|
||||
assert raised.value.code == 'quick_run_uncertain'
|
||||
assert raised.value.__cause__ is failure
|
||||
assert (preparer.data_root / 'quick-run/uncertain.flag').read_text().startswith(
|
||||
'reason=uncommitted_receipt_requires_verification;')
|
||||
@@ -0,0 +1,116 @@
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.execution_journal import ExecutionJournal, create_execution_schema
|
||||
from dx_productivity.trial_task_center.history_store import HistoryRepository, create_history_schema
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def staged_case(tmp_path):
|
||||
store = TaskStore(tmp_path / 'tasks.db')
|
||||
with store.transaction():
|
||||
create_execution_schema(store._connection)
|
||||
create_history_schema(store._connection)
|
||||
task = store.create_task('project commit')
|
||||
journal = ExecutionJournal(store)
|
||||
rows = [dict(task_id=task.task_id, job_id=f'job{index}', project_path=str(tmp_path / 'board.prj'),
|
||||
schematic_name='MAIN', sheet_number='1', old_part='OLD', new_part='NEW',
|
||||
target_scope='targets', target_selectors=f'uid{index}\x1fR{index}\x1f10K') for index in (1, 2)]
|
||||
journal.bind(task.task_id, rows, {})
|
||||
evidence = tmp_path / 'readback'; evidence.write_text('persisted sheet state')
|
||||
proof = {'files': [{'path': str(evidence), 'sha256': hashlib.sha256(evidence.read_bytes()).hexdigest()}],
|
||||
'baseline': dict(project_path=rows[0]['project_path'], schematic_name='MAIN', sheet_number='1', state='persisted sheet state')}
|
||||
yield store, journal, task.task_id, rows, proof
|
||||
store.close()
|
||||
|
||||
|
||||
def test_staged_prefix_is_not_saved_and_cannot_replay_after_rebind(staged_case, tmp_path):
|
||||
store, journal, task, rows, proof = staged_case
|
||||
attempt = journal.claim(task, rows[0])
|
||||
journal.stage_project_result(attempt, rows[0], session_token='a' * 32, root=tmp_path, proof=proof)
|
||||
record = store._connection.execute('SELECT state FROM execution_operations WHERE attempt_id=?', (attempt.attempt_id,)).fetchone()
|
||||
assert record['state'] == 'pending_save'
|
||||
assert HistoryRepository(store).page().rows == ()
|
||||
with pytest.raises(TaskCenterError, match='quick_run_uncertain'):
|
||||
journal.bind(task, rows, {rows[0]['job_id']: ('saved_ok', 'session_verified', 'uncommitted')})
|
||||
assert journal.needs_reconciliation(task)
|
||||
assert journal.unresolved_jobs(task, rows) == ('job1',)
|
||||
# The active retained owner can stage another independent operation.
|
||||
assert journal.claim(task, rows[1]).generation == 1
|
||||
|
||||
|
||||
def test_project_commit_is_atomic_idempotent_and_preserves_operation_ids(staged_case, tmp_path, monkeypatch):
|
||||
store, journal, task, rows, proof = staged_case
|
||||
attempts = [journal.claim(task, row) for row in rows]
|
||||
for attempt, row in zip(attempts, rows):
|
||||
journal.stage_project_result(attempt, row, session_token='a' * 32, root=tmp_path, proof=proof)
|
||||
commit_proof = dict(proof, session_token='a' * 32, project_path=rows[0]['project_path'], jobs=['job1', 'job2'])
|
||||
original = HistoryRepository.append
|
||||
calls = []
|
||||
def fail_second(self, **values):
|
||||
calls.append(values)
|
||||
if len(calls) == 2:
|
||||
raise OSError('disk full')
|
||||
return original(self, **values)
|
||||
monkeypatch.setattr(HistoryRepository, 'append', fail_second)
|
||||
with pytest.raises(OSError):
|
||||
journal.commit_project_results(task, 'a' * 32, {'job1': proof, 'job2': proof}, commit_proof)
|
||||
assert [r[0] for r in store._connection.execute('SELECT state FROM execution_operations')] == ['pending_save', 'pending_save']
|
||||
assert HistoryRepository(store).page().rows == ()
|
||||
monkeypatch.setattr(HistoryRepository, 'append', original)
|
||||
journal.commit_project_results(task, 'a' * 32, {'job1': proof, 'job2': proof}, commit_proof)
|
||||
journal.commit_project_results(task, 'a' * 32, {'job1': proof, 'job2': proof}, commit_proof)
|
||||
assert len(HistoryRepository(store).page().rows) == 2
|
||||
records = store._connection.execute('SELECT state,attempt_id,generation FROM execution_operations').fetchall()
|
||||
assert [r['state'] for r in records] == ['verified', 'verified']
|
||||
assert [r['attempt_id'] for r in records] == [a.attempt_id for a in attempts]
|
||||
assert journal.bind(task, rows, {})['job1'][0] == 'saved_ok'
|
||||
|
||||
|
||||
def test_partial_pause_commits_only_staged_prefix_and_rejects_incomplete_readback(staged_case, tmp_path):
|
||||
store, journal, task, rows, proof = staged_case
|
||||
attempt = journal.claim(task, rows[0])
|
||||
journal.stage_project_result(attempt, rows[0], session_token='a' * 32, root=tmp_path, proof=proof)
|
||||
with pytest.raises(TaskCenterError):
|
||||
journal.commit_project_results(task, 'a' * 32, {}, dict(proof, jobs=[]))
|
||||
commit = dict(proof, session_token='a' * 32, project_path=rows[0]['project_path'], jobs=['job1'])
|
||||
journal.commit_project_results(task, 'a' * 32, {'job1': proof}, commit)
|
||||
assert journal.claim(task, rows[1]).generation == 1
|
||||
record = store._connection.execute('SELECT state FROM execution_project_commits').fetchone()
|
||||
assert record['state'] == 'verified'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('persist_legacy_marker', [False, True])
|
||||
def test_crash_reconciliation_commits_native_marker_without_replaying_any_worker(staged_case, tmp_path, persist_legacy_marker):
|
||||
from dx_productivity.trial_task_center.execution_proof import capture_project_commit
|
||||
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer, _encode_vbs_field
|
||||
store, journal, task, rows, proof = staged_case
|
||||
attempt = journal.claim(task, rows[0])
|
||||
journal.stage_project_result(attempt, rows[0], session_token='a' * 32, root=tmp_path, proof=proof)
|
||||
def forbidden_worker(*args):
|
||||
pytest.fail('recovery replayed native replacement')
|
||||
preparer = QuickRunPreparer(tmp_path, process_runner=forbidden_worker)
|
||||
preparer.journal = journal
|
||||
with pytest.raises(TaskCenterError):preparer._commit_staged_project(task)
|
||||
assert store._connection.execute('SELECT state FROM execution_operations WHERE attempt_id=?', (attempt.attempt_id,)).fetchone()[0] == 'pending_save'
|
||||
job = rows[0]['job_id']
|
||||
fields = (job, rows[0]['project_path'], 'MAIN', '1', 'S\tfull_sheet_v1\r\nfinal persisted state')
|
||||
(tmp_path / f'post-state-{job}.tsv').write_text(
|
||||
'job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
|
||||
'\t'.join(_encode_vbs_field(value) for value in fields) + '\n', encoding='utf-8')
|
||||
(tmp_path / ('project-commit-' + 'a' * 32 + '.tsv')).write_text(
|
||||
'a' * 32 + '\t' + rows[0]['project_path'] + '\tproject_verified\n' + job + '\tno_match\n', encoding='utf-8')
|
||||
if persist_legacy_marker:
|
||||
commit = journal.pending_project_commits(task)[0]
|
||||
marker_proof = capture_project_commit(tmp_path, tmp_path, commit)
|
||||
journal.bind_project_commit_proof(task, commit['session_token'], marker_proof)
|
||||
preparer._commit_staged_project(task)
|
||||
preparer._commit_staged_project(task)
|
||||
assert store._connection.execute('SELECT state FROM execution_operations WHERE attempt_id=?', (attempt.attempt_id,)).fetchone()[0] == 'no_match'
|
||||
assert len(HistoryRepository(store).page().rows) == 1
|
||||
commit_proof = json.loads(journal.project_commit_records(task)[0]['proof_json'])
|
||||
assert commit_proof['operation_attempts'] == {job: [attempt.operation_key, attempt.attempt_id, attempt.generation]}
|
||||
@@ -0,0 +1,27 @@
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
from dx_productivity.trial_task_center.quick_run import QuickRunProgress, _advance_batch_session, _correlated_status_change
|
||||
|
||||
|
||||
def test_staging_advances_live_cursor_without_counting_saved_or_completed_items(tmp_path):
|
||||
project = str(tmp_path / 'board.prj')
|
||||
commands = (('task', 'one', project, 'OLD', 'NEW'), ('task', 'two', project, 'OTHER', 'NEW'))
|
||||
progress = QuickRunProgress(0, 2, Path(project), 'OLD', 'NEW', (), False)
|
||||
session = SimpleNamespace(commands=commands, next_index=0, latest={}, progress=progress, sheets=None)
|
||||
_advance_batch_session(session, ('one', 'pending_save', 'session_staged', 'staged'))
|
||||
assert session.next_index == 1
|
||||
assert session.progress.completed_items == 0
|
||||
assert session.progress.saved_items == 0
|
||||
assert session.progress.completed_project_paths == ()
|
||||
assert session.progress.next_old_part_number == 'OTHER'
|
||||
assert not session.progress.queue_complete
|
||||
|
||||
|
||||
def test_native_staged_and_resumed_pause_statuses_correlate_to_current_operation():
|
||||
command = ('task', 'one', 'board.prj', 'OLD', 'NEW')
|
||||
waiting = ('one', 'awaiting_native_replace', '', 'snapshot')
|
||||
staged = ('one', 'pending_save', 'session_staged', 'verified in memory')
|
||||
assert _correlated_status_change((), (waiting, staged), (command,), expected_command=command) == staged
|
||||
paused = ('one', 'pending', 'prewrite_paused', 'not replaced')
|
||||
assert _correlated_status_change((paused,), (paused, waiting, staged), (command,), expected_command=command) == staged
|
||||
@@ -0,0 +1,29 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.execution_proof import capture_project_commit, proof_intact
|
||||
|
||||
|
||||
def test_commit_manifest_binds_ordered_prefix_project_and_immutable_copy(tmp_path):
|
||||
root = tmp_path / 'worker'; root.mkdir()
|
||||
project = tmp_path / 'board.prj'
|
||||
token = 'a' * 32
|
||||
commit = dict(session_token=token, project_path=str(project.resolve()).casefold(), jobs_json=json.dumps(['one', 'two']))
|
||||
marker = root / f'project-commit-{token}.tsv'
|
||||
marker.write_text(f'{token}\t{project}\tproject_verified\none\tsaved_ok\ntwo\tno_match\n', encoding='utf-8')
|
||||
proof = capture_project_commit(tmp_path, root, commit)
|
||||
assert proof['jobs'] == ['one', 'two']
|
||||
assert proof['outcomes'] == {'one': 'saved_ok', 'two': 'no_match'}
|
||||
marker.write_text('crashed next session', encoding='utf-8')
|
||||
assert proof_intact(proof)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('tail', ['one\tsaved_ok\n', 'two\tsaved_ok\none\tsaved_ok\n', 'one\tpending_save\ntwo\tsaved_ok\n'])
|
||||
def test_commit_manifest_rejects_partial_or_unverified_prefix(tmp_path, tail):
|
||||
project = tmp_path / 'board.prj'; token = 'a' * 32
|
||||
commit = dict(session_token=token, project_path=str(project.resolve()).casefold(), jobs_json=json.dumps(['one', 'two']))
|
||||
(tmp_path / f'project-commit-{token}.tsv').write_text(f'{token}\t{project}\tproject_verified\n' + tail, encoding='utf-8')
|
||||
with pytest.raises(TaskCenterError):
|
||||
capture_project_commit(tmp_path, tmp_path, commit)
|
||||
@@ -0,0 +1,216 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.history_store import HistoryRepository
|
||||
from dx_productivity.trial_task_center.logging_setup import configure_logging
|
||||
from dx_productivity.trial_task_center.models import TaskStatus
|
||||
from dx_productivity.trial_task_center.native_helper import read_snapshot, transaction_binding
|
||||
from dx_productivity.trial_task_center.quick_run import QuickRunPreparer, _encode_vbs_field as enc, _write_uncertain_flag, _run_process
|
||||
from dx_productivity.trial_task_center.service import TaskCenterService
|
||||
from dx_productivity.trial_task_center.store import TaskStore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def public_commit_case(tmp_path, monkeypatch, request):
|
||||
monkeypatch.setattr('dx_productivity.trial_task_center.scanner._is_schematic_project', lambda _: True)
|
||||
project = tmp_path / 'project' / 'board.prj'
|
||||
project.parent.mkdir(); project.write_text('saved both replacements')
|
||||
data = tmp_path / 'data'
|
||||
preparer = QuickRunPreparer(data, process_runner=lambda *args: pytest.fail('native replacement/recovery replayed'),
|
||||
cscript_resolver=lambda: Path('cscript.exe'), sheet_discoverer=lambda _: (('MAIN', '1'),))
|
||||
store = TaskStore(tmp_path / 'tasks.db')
|
||||
service = TaskCenterService(store, configure_logging(data), quick_run_preparer=preparer)
|
||||
task = service.create_task('saved engineering project')
|
||||
service.scan_projects(task.task_id, project.parent)
|
||||
for row in service.projects(task.task_id):service.set_project_selected(row.project_id, True)
|
||||
service.add_rule(task.task_id, 'OLD1', 'NEW1'); service.add_rule(task.task_id, 'OLD2', 'NEW2')
|
||||
root = data / 'quick-run'
|
||||
handoff = preparer.prepare(task.task_id, service.projects(task.task_id), service.rules(task.task_id),
|
||||
sheet_targets={str(project.resolve()).casefold(): (('MAIN', '1'),)})
|
||||
rows = list(preparer._bind_journal_handoff(task.task_id, handoff).values())
|
||||
preparer._process_runner = _run_process
|
||||
attempts = []
|
||||
for index, row in enumerate(rows, 1):
|
||||
job = row['job_id']; uid = f'original{index}'; ref = f'R{index}'
|
||||
snapshot = root / f'snapshot-{job}.tsv'
|
||||
snapshot.write_text('\t'.join(map(enc, ['M', str(project), row['old_part'], row['new_part'], 'MAIN:1', 'preserve', '', '1'])) + '\n' +
|
||||
f'C\t{uid}\t{ref}\t{index*100}\t0\t0\t1\tNO_PINS\nA\t{uid}\tValue\t1K\t1K\t0\t0\t0\t0\t1\t0\t1\n', encoding='utf-8')
|
||||
metadata, components = read_snapshot(snapshot)
|
||||
report = {'ok': True, 'stage': 'native_replace_verified',
|
||||
'transaction': transaction_binding(snapshot, metadata, components, ['lib', row['new_part'], 'symbol', '', 'name', 'label'], 'symbol'),
|
||||
'batch_attempt': dict(task_id=task.task_id, job_id=job, project_path=str(project), session_id='b'*32, attempt_id='c'*32, generation=1)}
|
||||
Path(str(snapshot) + '.native.json').write_text(json.dumps(report), encoding='utf-8')
|
||||
(root / f'post-state-{job}.tsv').write_text('job_id\tproject_path\tschematic_name\tsheet_number\tsemantic_state\n' +
|
||||
'\t'.join(map(enc, [job, str(project), 'MAIN', '1', 'S\tfull_sheet_v1\r\nR1=NEW1\r\nR2=NEW2\r\n'])) + '\n', encoding='utf-8')
|
||||
raw = [['V','2',task.task_id,job,'saved_ok',str(project),'MAIN','1',row['old_part'],row['new_part'],
|
||||
'preserve','','1','','',snapshot.read_bytes().decode('utf-8')], ['C',uid,ref,row['new_part'],'1K','1','']]
|
||||
(root / f'value-readback-{job}.tsv').write_text('\n'.join('\t'.join(map(enc, fields)) for fields in raw) + '\n', encoding='utf-8')
|
||||
attempt = preparer.journal.claim(task.task_id, row); attempts.append(attempt)
|
||||
proof = preparer.result_proof(root, row, ('pending_save', 'session_staged', 'staged'))
|
||||
preparer.journal.stage_project_result(attempt, row, session_token='a'*32, root=root, proof=proof)
|
||||
(root / ('project-commit-' + 'a'*32 + '.tsv')).write_text('a'*32 + '\t' + str(project) + '\tproject_verified\n' +
|
||||
''.join(row['job_id'] + '\tsaved_ok\n' for row in rows), encoding='utf-8')
|
||||
# Production native status append failed/truncated after the durable marker.
|
||||
with (root / 'status.tsv').open('a', encoding='utf-8') as stream:stream.write(rows[0]['job_id'] + '\tpending_save\tsession_staged\tstaged\n')
|
||||
if getattr(request, 'param', None) != 'checkpoint':
|
||||
original = HistoryRepository.append
|
||||
def disk_full(*args, **kwargs):raise OSError('history disk full')
|
||||
monkeypatch.setattr(HistoryRepository, 'append', disk_full)
|
||||
with pytest.raises(OSError, match='history disk full'):
|
||||
preparer._commit_staged_project(task.task_id, session_token='a'*32, bind_source=True)
|
||||
monkeypatch.setattr(HistoryRepository, 'append', original)
|
||||
_write_uncertain_flag(root / 'uncertain.flag', 'receipt_commit_failed')
|
||||
store.set_task_status(task.task_id, TaskStatus.BLOCKED, summary='saved project receipt commit failed')
|
||||
monkeypatch.setattr(TaskCenterService, '_owned_cad_operation', lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
TaskCenterError('native_recovery_required', 'ordinary recovery path requires a separately verified native boundary')))
|
||||
yield service, store, task.task_id, preparer, project, rows, attempts
|
||||
store.close()
|
||||
|
||||
|
||||
def test_deleted_task_cannot_recover_bound_project_commit(public_commit_case):
|
||||
service, store, task, preparer, _, _, _ = public_commit_case
|
||||
flag = preparer.data_root / 'quick-run' / 'uncertain.flag'
|
||||
protection = flag.read_bytes()
|
||||
store.set_task_status(task, TaskStatus.DELETED, summary='deleted by operator')
|
||||
with pytest.raises(TaskCenterError) as caught:
|
||||
service.recover_quick_run(task, confirmed=True)
|
||||
assert caught.value.code == 'quick_run_recovery'
|
||||
assert caught.value.message == '请先恢复已删除的任务记录,再核验其中断步骤。'
|
||||
assert flag.read_bytes() == protection
|
||||
assert store.get_task(task).status is TaskStatus.DELETED
|
||||
assert [row[0] for row in store._connection.execute('SELECT state FROM execution_operations')] == ['pending_save', 'pending_save']
|
||||
|
||||
|
||||
def test_public_confirmed_project_commit_recovery_is_atomic_without_native_replay(public_commit_case):
|
||||
service, store, task, preparer, project, rows, attempts = public_commit_case
|
||||
root = preparer.data_root / 'quick-run'
|
||||
assert [r[0] for r in store._connection.execute('SELECT state FROM execution_operations')] == ['pending_save', 'pending_save']
|
||||
with pytest.raises(TaskCenterError):service.recover_quick_run(task, confirmed=False)
|
||||
assert store.get_task(task).status is TaskStatus.BLOCKED
|
||||
result = service.recover_quick_run(task, confirmed=True)
|
||||
assert result.state == 'saved_ok' and result.progress.saved_items == 2
|
||||
assert store.get_task(task).status is TaskStatus.COMPLETED
|
||||
assert not (root / 'uncertain.flag').exists()
|
||||
actual = store._connection.execute('SELECT state,attempt_id,generation FROM execution_operations ORDER BY rowid').fetchall()
|
||||
assert [r['state'] for r in actual] == ['verified', 'verified']
|
||||
assert [(r['attempt_id'], r['generation']) for r in actual] == [(a.attempt_id, a.generation) for a in attempts]
|
||||
assert len([r for r in HistoryRepository(store).page().rows if r['operation'] == '器件替换']) == 2
|
||||
assert service.recover_quick_run(task, confirmed=True).state == 'saved_ok'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('damage', ['manifest', 'source', 'later_attempt', 'later_job', 'unrelated_reason', 'native_report'])
|
||||
def test_public_recovery_refuses_unproven_or_later_write_uncertainty(public_commit_case, damage):
|
||||
service, store, task, preparer, project, rows, _ = public_commit_case
|
||||
root = preparer.data_root / 'quick-run'
|
||||
if damage == 'manifest':(root / ('project-commit-' + 'a'*32 + '.tsv')).write_text('partial')
|
||||
elif damage == 'source':project.write_text('external or later native modification')
|
||||
elif damage == 'later_attempt':
|
||||
store._connection.execute("UPDATE execution_operations SET generation=generation+1 WHERE operation_key=(SELECT operation_key FROM execution_operations LIMIT 1)")
|
||||
elif damage == 'later_job':
|
||||
record = dict(store._connection.execute('SELECT * FROM execution_operations LIMIT 1').fetchone())
|
||||
command = json.loads(record['command_json']); command['old_part'] = 'LATER'
|
||||
store._connection.execute('''INSERT INTO execution_operations(task_id,round_id,operation_key,command_json,state,attempt_id,generation)
|
||||
VALUES (?,?,?,?, 'prepared',?,1)''', (task, record['round_id'], 'later-operation', json.dumps(command), 'd'*32))
|
||||
elif damage == 'native_report':(root / f'snapshot-{rows[0]["job_id"]}.tsv.native.json').write_text('{}')
|
||||
else:_write_uncertain_flag(root / 'uncertain.flag', 'native_failure_after_snapshot')
|
||||
with pytest.raises((TaskCenterError, AssertionError)):service.recover_quick_run(task, confirmed=True)
|
||||
assert (root / 'uncertain.flag').exists()
|
||||
assert store.get_task(task).status is TaskStatus.BLOCKED
|
||||
|
||||
|
||||
def test_public_recovery_after_real_store_and_preparer_restart(public_commit_case):
|
||||
service, store, task, preparer, project, rows, attempts = public_commit_case
|
||||
path, data = store.path, preparer.data_root
|
||||
store.close()
|
||||
reopened = TaskStore(path)
|
||||
try:
|
||||
worker = QuickRunPreparer(data, process_runner=_run_process)
|
||||
resumed = TaskCenterService(reopened, configure_logging(data), quick_run_preparer=worker)
|
||||
result = resumed.recover_quick_run(task, confirmed=True)
|
||||
assert result.progress.saved_items == 2 and reopened.get_task(task).status is TaskStatus.COMPLETED
|
||||
finally:reopened.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('public_commit_case', ['checkpoint'], indirect=True)
|
||||
@pytest.mark.parametrize('damage', [None, 'missing_checkpoint', 'missing_owner', 'native_attempt', 'readback', 'supervisor', 'foreign_supervisor'])
|
||||
def test_public_recovery_uses_runtime_checkpoint_when_sql_callback_never_binds(public_commit_case, damage):
|
||||
from types import SimpleNamespace
|
||||
from dx_productivity.trial_task_center.batch_runtime import BatchCadRuntime
|
||||
from test_batch_runtime import Desktop
|
||||
service, store, task, preparer, project, rows, attempts = public_commit_case
|
||||
root = preparer.data_root / 'quick-run'
|
||||
assert preparer.journal.project_commit_records(task)[0]['proof_json'] is None
|
||||
Desktop.fail_release = False
|
||||
runtime = BatchCadRuntime(preparer.data_root, 'execution', desktop_factory=Desktop, session_id='b'*32)
|
||||
runtime._ensure_desktop()
|
||||
runtime.cad = SimpleNamespace(checkpoint=lambda _: None)
|
||||
runtime.record['active_job'] = rows[0]['job_id']
|
||||
runtime._current_attempts = [(root / (row['job_id'] + '.json'), row) for row in rows]
|
||||
(root / 'project-session.request').write_text('a'*32 + '\t' + str(project) + '\t' + rows[0]['job_id'])
|
||||
runtime._checkpoint(root, project, 'c'*32)
|
||||
runtime.desktop.close()
|
||||
runtime.record['cleanup'] = runtime.desktop.cleanup_record
|
||||
runtime.desktop = runtime.cad = None
|
||||
runtime._record_release()
|
||||
checkpoint_path = root / 'project-checkpoint.json'
|
||||
if damage == 'missing_checkpoint':checkpoint_path.unlink()
|
||||
elif damage == 'missing_owner':runtime.path.unlink()
|
||||
elif damage == 'native_attempt':
|
||||
checkpoint = json.loads(checkpoint_path.read_text())
|
||||
checkpoint['project_commit_source']['native_attempt']['attempt_id'] = 'e'*32
|
||||
checkpoint_path.write_text(json.dumps(checkpoint))
|
||||
elif damage == 'readback':(root / ('value-readback-' + rows[0]['job_id'] + '.tsv')).write_text('changed')
|
||||
elif damage in {'supervisor', 'foreign_supervisor'}:
|
||||
from dx_productivity.trial_task_center.batch_resources import write_record
|
||||
from dx_productivity.trial_task_center.batch_supervisor import _mark_interrupted
|
||||
(root / 'uncertain.flag').unlink()
|
||||
_mark_interrupted(preparer.data_root, {'batch_id': runtime.session_id, 'operation': 'execute_task_queue',
|
||||
'task_ids': [task], 'uncertainty_roots': {task: str(preparer.data_root)}},
|
||||
'parent crashed after native checkpoint before SQL callback')
|
||||
if damage == 'foreign_supervisor':
|
||||
supervisor_path = runtime.path.parent / 'supervisor.json'
|
||||
supervisor = json.loads(supervisor_path.read_text()); supervisor['protected_task_ids'] = ['foreign-task']
|
||||
write_record(supervisor_path, supervisor)
|
||||
# The SQL callback was never entered; a new process consumes the native certificate.
|
||||
reopened = TaskStore(store.path)
|
||||
try:
|
||||
worker = QuickRunPreparer(preparer.data_root, process_runner=_run_process)
|
||||
resumed = TaskCenterService(reopened, configure_logging(preparer.data_root), quick_run_preparer=worker)
|
||||
if damage in {None, 'supervisor'}:
|
||||
assert resumed.recover_quick_run(task, confirmed=True).progress.saved_items == 2
|
||||
assert reopened.get_task(task).status is TaskStatus.COMPLETED
|
||||
else:
|
||||
with pytest.raises(TaskCenterError):resumed.recover_quick_run(task, confirmed=True)
|
||||
assert (root / 'uncertain.flag').is_file()
|
||||
assert reopened.get_task(task).status is TaskStatus.BLOCKED
|
||||
finally:reopened.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure', ['status', 'flag', 'sql_status'])
|
||||
def test_public_recovery_retries_publication_after_sql_commit_without_duplicate_history(public_commit_case, monkeypatch, failure):
|
||||
from dx_productivity.trial_task_center import quick_run
|
||||
service, store, task, preparer, project, rows, attempts = public_commit_case
|
||||
root = preparer.data_root / 'quick-run'
|
||||
original_status, original_replace = quick_run._write_status_details, Path.replace
|
||||
count = []
|
||||
def status(*args, **kwargs):
|
||||
if failure == 'status' and not count:count.append(1); raise OSError('status publication failed')
|
||||
return original_status(*args, **kwargs)
|
||||
def replace(path, target):
|
||||
if failure == 'flag' and path.name == 'uncertain.flag' and not count:count.append(1); raise OSError('flag retirement failed')
|
||||
return original_replace(path, target)
|
||||
monkeypatch.setattr(quick_run, '_write_status_details', status); monkeypatch.setattr(Path, 'replace', replace)
|
||||
original_task_status = store.set_task_status
|
||||
def task_status(task_id, state, **kwargs):
|
||||
if failure == 'sql_status' and state is TaskStatus.READY and not count:
|
||||
count.append(1); raise OSError('task status publication failed')
|
||||
return original_task_status(task_id, state, **kwargs)
|
||||
monkeypatch.setattr(store, 'set_task_status', task_status)
|
||||
with pytest.raises((TaskCenterError, OSError)):service.recover_quick_run(task, confirmed=True)
|
||||
assert (root / 'uncertain.flag').exists() is (failure != 'sql_status')
|
||||
reopened = TaskCenterService(store, configure_logging(service._logging.root), quick_run_preparer=preparer)
|
||||
assert reopened.recover_quick_run(task, confirmed=True).state == 'saved_ok'
|
||||
assert len([r for r in HistoryRepository(store).page().rows if r['operation'] == '器件替换']) == 2
|
||||
@@ -0,0 +1,97 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.execution_proof import validate_project_commit_source
|
||||
|
||||
|
||||
@pytest.mark.parametrize('resume_later_job', [False, True])
|
||||
def test_native_closed_checkpoint_binds_project_marker_and_saved_source(tmp_path, monkeypatch, resume_later_job):
|
||||
from test_batch_runtime import command_for, worker_runtime
|
||||
command, project = command_for(tmp_path)
|
||||
root = Path(command[3])
|
||||
if resume_later_job:
|
||||
with (root / 'command.tsv').open('a') as stream:
|
||||
stream.write(f'task\tnext\t{project}\told\tnew\n')
|
||||
command = (*command[:-1], 'next')
|
||||
job = command[-1]
|
||||
token = 'a' * 32
|
||||
(root / 'project-session.request').write_text(f'{token}\t{project}\t{job}')
|
||||
marker = root / ('project-commit-' + token + '.tsv')
|
||||
def monitor(*args, **kwargs):
|
||||
marker.write_text(f'{token}\t{project}\tproject_verified\n{job}\tno_match\n')
|
||||
(root / ('post-state-' + job + '.tsv')).write_text('native final semantic state')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
checkpoint = json.loads((root / 'project-checkpoint.json').read_text(encoding='utf-8'))
|
||||
proof = checkpoint['project_commit_source']
|
||||
assert proof['session_token'] == token and proof['jobs'] == [job]
|
||||
assert set(proof['readback_files']) == {'post-state-' + job + '.tsv'}
|
||||
assert proof['native_attempt'] == {'session_id': runtime.session_id, 'generation': runtime.generation,
|
||||
'attempt_id': runtime.record['active_attempt']}
|
||||
assert validate_project_commit_source(proof, root)
|
||||
marker.write_text('changed')
|
||||
assert not validate_project_commit_source(proof, root)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('damage', [None, 'foreign_attempt', 'missing_native', 'bad_manifest', 'missing_expectations', 'old_snapshot'])
|
||||
def test_saved_native_checkpoint_only_authorizes_export_validation(tmp_path, monkeypatch, damage):
|
||||
from test_batch_runtime import command_for, worker_runtime
|
||||
from dx_productivity.trial_task_center.worker_monitor import WorkerMonitorError
|
||||
command, project = command_for(tmp_path)
|
||||
root = Path(command[3])
|
||||
token = 'a' * 32
|
||||
(root / 'project-session.request').write_text(f'{token}\t{project}\tjob')
|
||||
marker = root / ('project-commit-' + token + '.tsv')
|
||||
def monitor(*args, **kwargs):
|
||||
marker.write_text(f'{token}\t{project}\tproject_saved\njob\t' + ('saved_ok' if damage == 'bad_manifest' else 'pending_save') + '\n')
|
||||
(root / 'snapshot-job.tsv').write_text('M\ttarget snapshot\n' + ('' if damage == 'old_snapshot' else 'X\tsaved_netlist_v1\n'))
|
||||
if damage != 'missing_expectations':
|
||||
(root / 'writer-expectations-job.tsv').write_text('bound target writer expectations')
|
||||
if damage != 'missing_native':
|
||||
expected = dict(runtime._current_attempts[0][1])
|
||||
from dx_productivity.trial_task_center import batch_attempt
|
||||
batch_attempt._publish(runtime._current_attempts[0][0], dict(expected, state='mutation_possible'))
|
||||
if damage == 'foreign_attempt': expected['attempt_id'] = 'f' * 32
|
||||
(root / 'snapshot-job.tsv.native.json').write_text(json.dumps({
|
||||
'ok': True, 'stage': 'native_replace_applied', 'batch_attempt': expected}))
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
monkeypatch.setattr(runtime, '_retain_uncertain', lambda *args: None)
|
||||
with runtime:
|
||||
if damage:
|
||||
with pytest.raises((WorkerMonitorError, OSError)):
|
||||
runtime.run_worker(command, 5)
|
||||
return
|
||||
runtime.run_worker(command, 5)
|
||||
proof = json.loads((root / 'project-checkpoint.json').read_text(encoding='utf-8'))['project_commit_source']
|
||||
assert proof['marker_state'] == 'project_saved' and proof['saved_only'] is True
|
||||
assert proof['validation_required'] == 'saved_netlist_v1'
|
||||
assert proof['outcomes'] == {'job': 'pending_save'}
|
||||
assert proof['native_outcomes'] == {'job': 'pending_save'}
|
||||
assert set(proof['readback_files']) == {'snapshot-job.tsv', 'snapshot-job.tsv.native.json', 'writer-expectations-job.tsv'}
|
||||
assert not list(root.glob('post-state-*')) and not list(root.glob('value-readback-*'))
|
||||
assert validate_project_commit_source(proof, root)
|
||||
project.write_text('unexplained change after saved closure')
|
||||
assert not validate_project_commit_source(proof, root)
|
||||
|
||||
|
||||
def test_saved_native_no_match_checkpoint_requires_unmutated_attempt(tmp_path, monkeypatch):
|
||||
from test_batch_runtime import command_for, worker_runtime
|
||||
command, project = command_for(tmp_path)
|
||||
root = Path(command[3])
|
||||
token = 'a' * 32
|
||||
(root / 'project-session.request').write_text(f'{token}\t{project}\tjob')
|
||||
def monitor(*args, **kwargs):
|
||||
(root / ('project-commit-' + token + '.tsv')).write_text(f'{token}\t{project}\tproject_saved\njob\tno_match\n')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
runtime = worker_runtime(tmp_path, monkeypatch, monitor)
|
||||
with runtime:
|
||||
runtime.run_worker(command, 5)
|
||||
proof = json.loads((root / 'project-checkpoint.json').read_text(encoding='utf-8'))['project_commit_source']
|
||||
assert proof['saved_only'] is True and proof['outcomes'] == {'job': 'no_match'}
|
||||
assert proof['validation_required'] == 'saved_netlist_v1' and proof['readback_files'] == {}
|
||||
assert validate_project_commit_source(proof, root)
|
||||
@@ -0,0 +1,44 @@
|
||||
from pathlib import Path
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
from dx_productivity.trial_task_center.project_session import ProjectSessionScope
|
||||
from test_project_session import toy_worker
|
||||
|
||||
|
||||
def test_staged_worker_stays_open_until_project_commit_and_callback_runs_once(tmp_path):
|
||||
command = toy_worker(tmp_path)
|
||||
script = Path(command[2])
|
||||
script.write_text(script.read_text(encoding='utf-8').replace('saved_ok', 'pending_save').replace('session_verified', 'session_staged'), encoding='utf-8')
|
||||
commits = []
|
||||
with ProjectSessionScope(None, enabled=True) as scope:
|
||||
result = scope.run_step(tmp_path / 'board.prj', command, 3)
|
||||
assert result.returncode == 0
|
||||
assert not (tmp_path / 'closed.txt').exists()
|
||||
scope.on_commit = lambda worker: commits.append(worker.token)
|
||||
assert scope.run_step(tmp_path / 'board.prj', (*command[:-1], 'job2'), 3).returncode == 0
|
||||
assert (tmp_path / 'project-session.request').read_text(encoding='utf-8').split('\t')[2] == 'job2'
|
||||
scope.close()
|
||||
assert (tmp_path / 'closed.txt').exists()
|
||||
scope.close()
|
||||
assert len(commits) == 1
|
||||
|
||||
|
||||
def test_project_close_extends_wait_while_native_commit_keeps_reporting_progress(tmp_path, monkeypatch):
|
||||
from dx_productivity.trial_task_center import project_session
|
||||
elapsed = [0.0]
|
||||
class FinishingCommit:
|
||||
def is_set(self):return elapsed[0] >= 12
|
||||
def wait(self, timeout):
|
||||
elapsed[0] += .1
|
||||
(tmp_path / 'worker-progress.json').write_text(json.dumps({'sequence': int(elapsed[0]), 'native_progress_frames': 0}))
|
||||
return self.is_set()
|
||||
monkeypatch.setattr(project_session, 'time', SimpleNamespace(monotonic=lambda: elapsed[0], sleep=lambda seconds: None))
|
||||
worker = project_session.ProjectSessionWorker(tmp_path / 'board.prj', tmp_path)
|
||||
worker._thread = object()
|
||||
worker._timeout = 1
|
||||
worker._done = FinishingCommit()
|
||||
worker._result = SimpleNamespace(returncode=0)
|
||||
worker.close()
|
||||
assert elapsed[0] >= 12
|
||||
assert not worker._abort.is_set()
|
||||
@@ -0,0 +1,27 @@
|
||||
import hashlib
|
||||
|
||||
import pytest
|
||||
|
||||
from test_value_readback import case
|
||||
from dx_productivity.trial_task_center.value_readback import capture_value_readback, load_value_readback
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field
|
||||
|
||||
|
||||
@pytest.mark.parametrize('damage',['','foreign_project','changed_marker','wrong_state'])
|
||||
def test_committed_prefix_readback_is_bound_to_manifest_not_last_status(tmp_path,damage):
|
||||
row,_=case(tmp_path)
|
||||
token='a'*32
|
||||
marker=tmp_path/f'project-commit-{token}.tsv'
|
||||
marker.write_text(f'{token}\t{_encode_vbs_field(row["project_path"])}\tproject_verified\njob\tsaved_ok\nnext\tno_match\n')
|
||||
proof={'session_token':token,'project_path':row['project_path'],'jobs':['job','next'],
|
||||
'outcomes':{'job':'saved_ok','next':'no_match'},
|
||||
'files':[{'path':str(marker),'sha256':hashlib.sha256(marker.read_bytes()).hexdigest()}]}
|
||||
with (tmp_path/'status.tsv').open('a') as stream:
|
||||
stream.write('next\tno_match\tproject_committed\tverified\n')
|
||||
if damage=='foreign_project':proof['project_path']=str(tmp_path/'other.prj')
|
||||
elif damage=='changed_marker':marker.write_text('changed')
|
||||
elif damage=='wrong_state':proof['outcomes']['job']='no_match'
|
||||
result=capture_value_readback(tmp_path,row,('saved_ok','project_committed','verified'),project_commit=proof)
|
||||
assert result['value_readback_status']==('rejected' if damage else 'verified')
|
||||
if not damage:
|
||||
assert load_value_readback(tmp_path,row,('saved_ok','project_committed','verified'))==result
|
||||
@@ -0,0 +1,403 @@
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
SOURCE = Path(__file__).resolve().parents[2] / 'integration/dxdesigner795/QuickPartReplace.vbs'
|
||||
|
||||
|
||||
def procedure(name):
|
||||
source = SOURCE.read_text(encoding='utf-8-sig')
|
||||
match = re.search(rf'(?ms)^Sub {name}\(.*?^End Sub', source)
|
||||
assert match, f'missing production procedure {name}'
|
||||
return match.group()
|
||||
|
||||
|
||||
def run_script(tmp_path, script):
|
||||
path = tmp_path / 'production-extract.vbs'
|
||||
path.write_text(script, encoding='ascii')
|
||||
return subprocess.run((r'C:\Windows\SysWOW64\cscript.exe', '//nologo', str(path)),
|
||||
capture_output=True, text=True, timeout=10)
|
||||
|
||||
|
||||
def test_production_batch_finish_stages_session_item_before_per_step_save(tmp_path):
|
||||
result = run_script(tmp_path, '''QP_SessionMode = True
|
||||
QP_BatchState = "restored_ok"
|
||||
QPFinishBatchItem
|
||||
Sub QPStageSessionItem()
|
||||
WScript.Echo "pending_save"
|
||||
End Sub
|
||||
''' + procedure('QPFinishBatchItem'))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip() == 'pending_save'
|
||||
|
||||
|
||||
def test_production_baseline_wrapper_preserves_safe_pause_source(tmp_path):
|
||||
result = run_script(tmp_path, '''On Error Resume Next
|
||||
QPVerifyExpectedBaselines "board"
|
||||
WScript.Echo Err.Source
|
||||
Sub QPCheckExpectedBaselines(ByVal TargetPath)
|
||||
Err.Raise vbObjectError + 860, "QuickPartReplace.SafeStop", "paused"
|
||||
End Sub
|
||||
''' + procedure('QPVerifyExpectedBaselines'))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip() == 'QuickPartReplace.SafeStop'
|
||||
|
||||
|
||||
def test_production_project_commit_closes_empty_owned_project_without_save(tmp_path):
|
||||
result = run_script(tmp_path, '''Set QP_SessionItems = CreateObject("Scripting.Dictionary")
|
||||
QP_SessionProject = "board.prj"
|
||||
QPCommitSessionProject
|
||||
Sub QPCloseBatchProject(ByVal Target, ByVal SaveChanges)
|
||||
WScript.Echo Target & ":" & CStr(SaveChanges)
|
||||
End Sub
|
||||
''' + procedure('QPCommitSessionProject'))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip() == 'board.prj:False'
|
||||
|
||||
|
||||
def test_production_pause_acknowledges_only_after_original_proof_and_prefix_close(tmp_path):
|
||||
script = '''Set QP_Fso = CreateObject("Scripting.FileSystemObject")
|
||||
Set QP_SessionItems = CreateObject("Scripting.Dictionary")
|
||||
QP_CurrentJobId = "current"
|
||||
QP_ExpectedJobId = "current"
|
||||
QP_MonitorJobId = "initial"
|
||||
QP_OwnedProject = True
|
||||
QP_Root = "."
|
||||
QP_SessionToken = String(32, "a")
|
||||
QP_SessionProject = "board.prj"
|
||||
QP_AttemptToken = "monitor"
|
||||
QP_ProgressSequence = 3
|
||||
QPAcknowledgePrewritePause
|
||||
WScript.Echo "unsafe-no-ack"
|
||||
Sub QPVerifyFailedOriginalSnapshot()
|
||||
WScript.Echo "original-verified"
|
||||
End Sub
|
||||
Function QPReadCommand(ByVal Path)
|
||||
Set QPReadCommand = CreateObject("Scripting.Dictionary")
|
||||
End Function
|
||||
Sub QPWriteVerifiedState(ByRef Command)
|
||||
WScript.Echo "unchanged-semantic-state"
|
||||
End Sub
|
||||
Sub QPCommitSessionProject()
|
||||
WScript.Echo "prefix-committed-and-closed"
|
||||
QP_OwnedProject = False
|
||||
End Sub
|
||||
Sub QPWriteUtf8(ByVal Path, ByVal Data)
|
||||
WScript.Echo "durable-pause-marker"
|
||||
End Sub
|
||||
Function QPEncode(ByVal Value)
|
||||
QPEncode = Value
|
||||
End Function
|
||||
'''
|
||||
result = run_script(tmp_path, script + procedure('QPAcknowledgePrewritePause'))
|
||||
assert result.returncode == 75
|
||||
assert result.stdout.splitlines()[:4] == ['original-verified', 'unchanged-semantic-state',
|
||||
'prefix-committed-and-closed', 'durable-pause-marker']
|
||||
assert result.stdout.splitlines()[4] == 'DX_PREWRITE_PAUSED\tmonitor\tinitial\t' + 'a' * 32 + '\tcurrent\t3'
|
||||
unsafe = run_script(tmp_path, script.replace('QPAcknowledgePrewritePause\n',
|
||||
'QP_NativeMutationPossible = True\nQPAcknowledgePrewritePause\n', 1) + procedure('QPAcknowledgePrewritePause'))
|
||||
assert unsafe.returncode == 0
|
||||
assert unsafe.stdout.strip() == 'unsafe-no-ack'
|
||||
|
||||
|
||||
def test_production_commit_saves_each_sheet_once_and_reopens_project_once(tmp_path):
|
||||
# COM is the test boundary; all project commit/close sequencing comes from VBS.
|
||||
script = '''On Error Resume Next
|
||||
Set QP_Fso = CreateObject("Scripting.FileSystemObject")
|
||||
QP_Root = "."
|
||||
QP_SessionMode = True
|
||||
QP_SessionProject = "board.prj"
|
||||
QP_OwnedProjectPath = "board.prj"
|
||||
QP_OwnedProject = True
|
||||
QP_ProjectOpen = True
|
||||
QP_DocCount = 2
|
||||
QP_Saves = 0
|
||||
QP_Closes = 0
|
||||
QP_Opens = 0
|
||||
QP_Reads = 0
|
||||
QP_TargetSchematic = ""
|
||||
QP_TargetSheet = ""
|
||||
QP_CurrentJobId = ""
|
||||
QP_SessionToken = String(32, "a")
|
||||
Set QP_App = New FakeApp
|
||||
Set QP_SessionItems = CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionSheets = CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionDocuments = CreateObject("Scripting.Dictionary")
|
||||
QP_SessionDocuments.Add "A", True
|
||||
QP_SessionDocuments.Add "B", True
|
||||
QP_SessionSheets.Add "A:1", "finalA"
|
||||
QP_SessionSheets.Add "B:1", "finalB"
|
||||
For I = 1 To 3
|
||||
Set Entry = CreateObject("Scripting.Dictionary")
|
||||
Set Command = CreateObject("Scripting.Dictionary")
|
||||
Command.Add "job_id", "job" & I
|
||||
Set Records = CreateObject("Scripting.Dictionary")
|
||||
Set Components = CreateObject("Scripting.Dictionary")
|
||||
Records.Add "components", Components
|
||||
Entry.Add "command", Command
|
||||
Entry.Add "snapshot_path", "snapshot-job" & I
|
||||
Set Values = CreateObject("Scripting.Dictionary")
|
||||
Entry.Add "value_states", Values
|
||||
Entry.Add "schematic", "A"
|
||||
If I = 3 Then Entry("schematic") = "B"
|
||||
Entry.Add "sheet", "1"
|
||||
Entry.Add "final_state", "restored_ok"
|
||||
QP_SessionItems.Add "job" & I, Entry
|
||||
Next
|
||||
QPCommitSessionProject
|
||||
If Err.Number <> 0 Then
|
||||
WScript.StdErr.WriteLine Err.Source & ":" & Err.Description
|
||||
WScript.Quit 1
|
||||
End If
|
||||
WScript.Echo "counts=" & QP_Saves & "," & QP_Closes & "," & QP_Opens & "," & QP_Reads
|
||||
Class FakeDocument
|
||||
Sub Close(ByVal SaveChanges, ByVal Ignored)
|
||||
If SaveChanges Then QP_Saves = QP_Saves + 1
|
||||
QP_DocCount = QP_DocCount - 1
|
||||
End Sub
|
||||
End Class
|
||||
Class FakeDocuments
|
||||
Property Get Count
|
||||
Count = QP_DocCount
|
||||
End Property
|
||||
Function Item(ByVal Index)
|
||||
Set Item = New FakeDocument
|
||||
End Function
|
||||
End Class
|
||||
Class FakeProjectData
|
||||
Function GetProjectFilePath()
|
||||
GetProjectFilePath = ""
|
||||
If QP_ProjectOpen Then GetProjectFilePath = "board.prj"
|
||||
End Function
|
||||
End Class
|
||||
Class FakeApp
|
||||
Function SchematicSheetDocuments()
|
||||
Set SchematicSheetDocuments = New FakeDocuments
|
||||
End Function
|
||||
Function GetProjectData()
|
||||
Set GetProjectData = New FakeProjectData
|
||||
End Function
|
||||
Sub CloseProject()
|
||||
QP_Closes = QP_Closes + 1
|
||||
QP_ProjectOpen = False
|
||||
End Sub
|
||||
End Class
|
||||
Sub QPEnsureProject(ByRef App, ByVal Project)
|
||||
If Not QP_ProjectOpen Then
|
||||
QP_ProjectOpen = True
|
||||
QP_Opens = QP_Opens + 1
|
||||
QP_DocCount = 2
|
||||
QP_OwnedProject = True
|
||||
QP_OwnedProjectPath = Project
|
||||
QP_SessionDocuments.Add "A", True
|
||||
QP_SessionDocuments.Add "B", True
|
||||
End If
|
||||
End Sub
|
||||
Function QPSheetKey(ByVal Schematic, ByVal Sheet)
|
||||
QPSheetKey = Schematic & ":" & Sheet
|
||||
End Function
|
||||
Function QPCurrentSemanticState()
|
||||
QP_Reads = QP_Reads + 1
|
||||
QPCurrentSemanticState = "final" & QP_TargetSchematic
|
||||
End Function
|
||||
Sub QPRequireExpectedState(ByVal Expected, ByVal Actual)
|
||||
If Expected <> Actual Then Err.Raise 1, "Readback", "changed sheet"
|
||||
End Sub
|
||||
Function QPProjectPath()
|
||||
QPProjectPath = "board.prj"
|
||||
End Function
|
||||
Function QPCanonicalPath(ByVal Value)
|
||||
QPCanonicalPath = Value
|
||||
End Function
|
||||
Sub QPProgress(ByVal Stage, ByVal Done, ByVal Total)
|
||||
End Sub
|
||||
Sub QPLog(ByVal Root, ByVal Level, ByVal Message)
|
||||
End Sub
|
||||
Sub QPInvalidatePinIndex()
|
||||
End Sub
|
||||
Sub QPInvalidateComponentIndex()
|
||||
End Sub
|
||||
Sub QPWriteSemanticState(ByRef Command, ByVal SemanticState)
|
||||
End Sub
|
||||
Function QPReadSnapshot(ByVal Path)
|
||||
Set Result = CreateObject("Scripting.Dictionary")
|
||||
Set Parts = CreateObject("Scripting.Dictionary")
|
||||
Result.Add "components", Parts
|
||||
Set QPReadSnapshot = Result
|
||||
End Function
|
||||
Sub QPVerifyRestoredPlans(ByRef Command, ByRef Records, ByRef Plans, ByRef Values)
|
||||
End Sub
|
||||
Sub QPWriteValueReadback(ByRef Command, ByRef Records, ByRef Plans, ByVal State)
|
||||
End Sub
|
||||
Function QPEncode(ByVal Value)
|
||||
QPEncode = Value
|
||||
End Function
|
||||
Sub QPWriteUtf8(ByVal Path, ByVal Data)
|
||||
WScript.Echo "marker"
|
||||
End Sub
|
||||
Sub QPWriteStatus(ByVal Root, ByVal Job, ByVal State, ByVal Code, ByVal Message)
|
||||
WScript.Echo Job & ":" & State
|
||||
End Sub
|
||||
'''
|
||||
result = run_script(tmp_path, script + procedure('QPCommitSessionProject') + '\n' + procedure('QPCloseBatchProject'))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.splitlines() == ['marker', 'job1:saved_ok', 'job2:saved_ok', 'job3:saved_ok', 'counts=2,2,1,4']
|
||||
# A failed reopened sheet readback must not publish the commit marker/status.
|
||||
changed = script.replace('QPCurrentSemanticState = "final" & QP_TargetSchematic',
|
||||
'QPCurrentSemanticState = "final" & QP_TargetSchematic\n If QP_Opens > 0 Then QPCurrentSemanticState = "changed"')
|
||||
failed = run_script(tmp_path, changed + procedure('QPCommitSessionProject') + '\n' + procedure('QPCloseBatchProject'))
|
||||
assert failed.returncode == 1
|
||||
assert 'Readback:changed sheet' in failed.stderr
|
||||
assert 'marker' not in failed.stdout
|
||||
assert 'saved_ok' not in failed.stdout
|
||||
|
||||
|
||||
@pytest.mark.parametrize('state', ['restored_ok', 'no_match'])
|
||||
def test_stage_rereads_real_current_command_before_publishing_pending_save(tmp_path, state):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
header='task_id\tjob_id\tproject_path\told_part\tnew_part\tvalue_mode\n'
|
||||
(tmp_path/'command.tsv').write_text(header+'task\tother\tboard.prj\tOLD\tNEW\tpreserve\n')
|
||||
(tmp_path/'current-command.tsv').write_text(header+'task\tjob\tboard.prj\tOLD\tNEW\tpreserve\n')
|
||||
(tmp_path/'current-state.tsv').write_text('job_id\tstate\tcode\tmessage\njob\tawaiting_native_replace\t\tdispatched\n')
|
||||
names=['QPStageSessionItem','QPReadCommand','QPActiveCommandPath','QPReadProjectRuleStates',
|
||||
'QPReadState','QPIsNextRuleState','QPReadUtf8','QPWriteUtf8','QPWriteStatus','QPAppendAscii','QPSheetKey']
|
||||
output=guards.run_vbs(tmp_path,names,'''
|
||||
QP_Root="'''+str(tmp_path)+'''"
|
||||
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
|
||||
Set QP_App=New FakeApp
|
||||
Set QP_SessionItems=CreateObject("Scripting.Dictionary")
|
||||
Set QP_SessionSheets=CreateObject("Scripting.Dictionary")
|
||||
QP_ExpectedJobId="job":QP_CurrentJobId="job":QP_BatchMode=True
|
||||
QP_BatchState="'''+state+'''":QP_SessionToken=String(32,"a")
|
||||
QPStageSessionItem
|
||||
WScript.Echo QP_SessionItems("job")("command")("job_id")
|
||||
WScript.Echo QP_SessionItems("job")("final_state")
|
||||
WScript.Echo CStr(InStr(QPReadUtf8(QP_Fso.BuildPath(QP_Root,"status.tsv")),"pending_save")>0)
|
||||
WScript.Echo QP_BatchState
|
||||
''',prelude='''
|
||||
Dim QP_Root,QP_Fso,QP_App,QP_SessionItems,QP_SessionSheets,QP_ExpectedJobId,QP_CurrentJobId
|
||||
Dim QP_BatchMode,QP_BatchState,QP_SessionToken
|
||||
Class FakeApp
|
||||
Property Get ActiveView()
|
||||
Set ActiveView=Me
|
||||
End Property
|
||||
Function GetName(ByVal Ignored)
|
||||
GetName="MAIN"
|
||||
End Function
|
||||
Property Get Block()
|
||||
Set Block=Me
|
||||
End Property
|
||||
Property Get SheetNum()
|
||||
SheetNum="1"
|
||||
End Property
|
||||
End Class
|
||||
Function QPProjectPath()
|
||||
QPProjectPath="board.prj"
|
||||
End Function
|
||||
Function QPCanonicalPath(ByVal Value)
|
||||
QPCanonicalPath=Value
|
||||
End Function
|
||||
Function QPValueMode(ByRef Command)
|
||||
QPValueMode=Command("value_mode")
|
||||
End Function
|
||||
Function QPReadSnapshot(ByVal Path)
|
||||
Dim R,C:Set R=CreateObject("Scripting.Dictionary"):Set C=CreateObject("Scripting.Dictionary")
|
||||
R.Add "components",C:Set QPReadSnapshot=R
|
||||
End Function
|
||||
Function QPJobSnapshotPath(ByVal Job)
|
||||
QPJobSnapshotPath="unused-snapshot"
|
||||
End Function
|
||||
Sub QPRequireCommandSnapshot(ByRef Command,ByVal Path)
|
||||
End Sub
|
||||
Sub QPVerifyUntouched(ByRef Records)
|
||||
End Sub
|
||||
Function QPCurrentSemanticState()
|
||||
QPCurrentSemanticState="verified-memory"
|
||||
End Function
|
||||
''')
|
||||
assert output.splitlines()==['job',state,'True',state]
|
||||
assert (tmp_path/'project-session-result.tsv').read_text(encoding='utf-8-sig').split('\t')[2]=='pending_save'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('continuation', ['close','next'])
|
||||
def test_pause_during_staged_checkpoint_finishes_prefix_or_pauses_next_prewrite(tmp_path, continuation):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
token='a'*32
|
||||
(tmp_path/'monitor-monitor.stop').write_text('monitor')
|
||||
(tmp_path/'project-session-next.tsv').write_text(token+'\t'+continuation)
|
||||
(tmp_path/'batch.request').write_text('next')
|
||||
output=guards.run_vbs(tmp_path,['QPSessionNext','QPRequireSessionCheckpoint','QPProgress','QPReadUtf8','QPSheetKey'],'''
|
||||
QP_Root="'''+str(tmp_path)+'''"
|
||||
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
|
||||
Set QP_App=New App
|
||||
Set QP_SessionItems=CreateObject("Scripting.Dictionary"):QP_SessionItems.Add "previous",True
|
||||
Set QP_SessionDocuments=CreateObject("Scripting.Dictionary"):QP_SessionDocuments.Add "MAIN:1",True
|
||||
Set QP_SessionSheets=CreateObject("Scripting.Dictionary"):QP_SessionSheets.Add QPSheetKey("MAIN","1"),"original"
|
||||
QP_SessionMode=True:QP_SessionWaiting=True:QP_SessionToken="'''+token+'''"
|
||||
QP_SessionProject="board":QP_OwnedProject=True:QP_OwnedProjectPath="board"
|
||||
QP_CheckpointSchematic="MAIN":QP_CheckpointSheet="1"
|
||||
QP_ExpectedJobId="previous":QP_CurrentJobId="previous":QP_MonitorJobId="initial"
|
||||
QP_AttemptToken="monitor":QP_ProgressSequence=0
|
||||
QPSessionNext
|
||||
If QP_SessionMode Then
|
||||
On Error Resume Next
|
||||
QPProgress "before_native",0,1
|
||||
WScript.Echo "next_prewrite_stop=" & CStr(Err.Source="QuickPartReplace.SafeStop")
|
||||
End If
|
||||
''',prelude='''
|
||||
Dim QP_Root,QP_Fso,QP_App,QP_SessionItems,QP_SessionDocuments,QP_SessionSheets
|
||||
Dim QP_SessionMode,QP_SessionWaiting,QP_SessionToken,QP_SessionProject,QP_OwnedProject,QP_OwnedProjectPath
|
||||
Dim QP_CheckpointSchematic,QP_CheckpointSheet,QP_ExpectedJobId,QP_CurrentJobId,QP_MonitorJobId
|
||||
Dim QP_AttemptToken,QP_ProgressSequence
|
||||
Class App
|
||||
Function SchematicSheetDocuments()
|
||||
Set SchematicSheetDocuments=Me
|
||||
End Function
|
||||
Property Get Count()
|
||||
Count=1
|
||||
End Property
|
||||
Property Get ActiveView()
|
||||
Set ActiveView=Me
|
||||
End Property
|
||||
Function GetName(ByVal Flags)
|
||||
GetName="MAIN"
|
||||
End Function
|
||||
Property Get Block()
|
||||
Set Block=Me
|
||||
End Property
|
||||
Property Get SheetNum()
|
||||
SheetNum="1"
|
||||
End Property
|
||||
End Class
|
||||
Function QPProjectPath()
|
||||
QPProjectPath="board"
|
||||
End Function
|
||||
Function QPCanonicalPath(ByVal Value)
|
||||
QPCanonicalPath=Value
|
||||
End Function
|
||||
Function QPReadContinuation(ByVal Path)
|
||||
QPReadContinuation=QPReadUtf8(Path)
|
||||
End Function
|
||||
Function QPCurrentSemanticState()
|
||||
QPProgress "capture_semantic_state",1,104
|
||||
QPCurrentSemanticState="original"
|
||||
End Function
|
||||
Sub QPRequireExpectedState(ByVal Expected,ByVal Actual)
|
||||
If Expected<>Actual Then Err.Raise 1,"checkpoint","changed"
|
||||
End Sub
|
||||
Sub QPCommitSessionProject()
|
||||
WScript.Echo "prefix_commit"
|
||||
End Sub
|
||||
Function QPConfirmedProjectPath()
|
||||
QPConfirmedProjectPath="board"
|
||||
End Function
|
||||
Sub QPEnsureProject(ByRef App,ByVal Path)
|
||||
End Sub
|
||||
''')
|
||||
lines=output.splitlines()
|
||||
semantic=[line for line in lines if '\tcapture_semantic_state\t' in line]
|
||||
assert len(semantic)==1 and semantic[0].split('\t')[-2:]==['0','previous']
|
||||
assert ('prefix_commit' in lines) if continuation=='close' else ('next_prewrite_stop=True' in lines)
|
||||
@@ -0,0 +1,65 @@
|
||||
from pathlib import Path
|
||||
from threading import Event
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center.errors import TaskCenterError
|
||||
from dx_productivity.trial_task_center.task_queue import execute_task_queue
|
||||
from test_task_queue import Service
|
||||
|
||||
|
||||
class ReleasedRuntime:
|
||||
def __init__(self,root,released=True):
|
||||
self.root=root
|
||||
self.stop_event=None
|
||||
self.cad=None
|
||||
self.released=released
|
||||
self.release_calls=[]
|
||||
def bind_logs(self,*args):
|
||||
pass
|
||||
def release_failed_project(self,error,evidence_root):
|
||||
self.release_calls.append(error.code)
|
||||
return self.released
|
||||
|
||||
|
||||
@pytest.mark.parametrize('released',[False,True])
|
||||
def test_uncertain_project_only_holds_independent_tasks_until_writers_exit(tmp_path,released):
|
||||
service=Service(tmp_path,{'A':TaskCenterError('quick_run_uncertain','unknown saved result')})
|
||||
runtime=ReleasedRuntime(tmp_path,released)
|
||||
result=execute_task_queue(service,('A','B'),stop_event=Event(),runtime=runtime)
|
||||
assert result.entries[0].state=='blocked'
|
||||
assert result.paused is (not released)
|
||||
assert service.calls==([('start','A'),('start','B'),('end','B')] if released else [('start','A')])
|
||||
assert runtime.release_calls==['quick_run_uncertain']
|
||||
|
||||
|
||||
def test_local_failure_cleanup_does_not_become_user_batch_pause(tmp_path):
|
||||
from dx_productivity.trial_task_center.stop_control import ProjectStopControl,StopControl
|
||||
parent=StopControl()
|
||||
local=ProjectStopControl(parent)
|
||||
local.request_failure_cleanup(tmp_path,'injected project failure')
|
||||
assert local.is_set() and local.force_event.is_set()
|
||||
assert not parent.is_set() and not parent.force_event.is_set()
|
||||
next_project=ProjectStopControl(parent)
|
||||
assert not next_project.is_set()
|
||||
next_project.set()
|
||||
assert parent.is_set()
|
||||
|
||||
|
||||
def test_resolution_failure_release_keeps_the_parent_user_intent_unset(tmp_path):
|
||||
from dx_productivity.trial_task_center.stop_control import ProjectStopControl,StopControl,wait_for_resolution
|
||||
parent=StopControl()
|
||||
local=ProjectStopControl(parent)
|
||||
wait_for_resolution(local,tmp_path,'injected failure',resolved=lambda:True,release_on_failure=True)
|
||||
assert local.is_set() and not parent.is_set()
|
||||
|
||||
|
||||
def test_isolated_project_is_not_replayed_on_queue_resume(tmp_path):
|
||||
service=Service(tmp_path,{'A':TaskCenterError('quick_run_uncertain','unknown')})
|
||||
runtime=ReleasedRuntime(tmp_path)
|
||||
first=execute_task_queue(service,('A','B'),stop_event=Event(),runtime=runtime)
|
||||
before=list(service.calls)
|
||||
resumed=execute_task_queue(service,('A','B'),stop_event=Event(),runtime=runtime,queue_id=first.queue_id)
|
||||
assert service.calls==before
|
||||
assert resumed.entries[0].state=='blocked'
|
||||
@@ -50,9 +50,13 @@ def test_changed_project_after_controlled_failure_blocks_remaining_uid(component
|
||||
project = service.projects(task_id)[0].path
|
||||
outcomes[(task_id, ('R1',))] = 'batch_failed'
|
||||
outcomes['after_call'] = lambda count: project.write_text('unexpected source mutation') if count == 1 else None
|
||||
result = execute_task_queue(service, (task_id,), stop_event=Event())
|
||||
assert result.paused
|
||||
assert calls == [(task_id, ('R1',))]
|
||||
ids = tuple(task.task_id for task in tasks)
|
||||
result = execute_task_queue(service, ids, stop_event=Event())
|
||||
assert not result.paused and [entry.state for entry in result.entries] == ['blocked', 'completed']
|
||||
assert calls == [(task_id, ('R1',)), (ids[1], ('R1',)), (ids[1], ('R2',))]
|
||||
assert service.has_interrupted_quick_run(task_id)
|
||||
execute_task_queue(service, ids, stop_event=Event(), queue_id=result.queue_id)
|
||||
assert len(calls) == 3
|
||||
|
||||
|
||||
def test_tampered_failure_health_denies_resume(component_case):
|
||||
|
||||
@@ -0,0 +1,590 @@
|
||||
"""Reference designators remain usable when native UID metadata is broken."""
|
||||
from copy import deepcopy
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from dx_productivity.trial_task_center import native_inventory_protocol as inventory, native_netlist
|
||||
from dx_productivity.trial_task_center.target_plan import ComponentTarget, TargetPlanRow, compile_target_rules
|
||||
from test_native_ascii_inventory import report, effective_report
|
||||
|
||||
|
||||
@pytest.mark.parametrize('damage', ['invalid', 'duplicate', 'changed_between_reports'])
|
||||
def test_inventory_joins_real_parts_by_refdes_when_uid_is_unusable(tmp_path, damage):
|
||||
hier,flat=report(tmp_path),effective_report(tmp_path,part='OLD')
|
||||
if damage=='invalid':
|
||||
hier.write_text(hier.read_text().replace('$27I100','broken'))
|
||||
flat.write_text(flat.read_text().replace('$27I100','broken'))
|
||||
elif damage=='duplicate':
|
||||
hier.write_text(hier.read_text().replace('$28I200','$27I100'))
|
||||
flat.write_text(flat.read_text().replace('$28I200','$27I100'))
|
||||
else:
|
||||
flat.write_text(flat.read_text().replace('$27I100','$99I999'))
|
||||
result=inventory.read_ascii_discovery(tmp_path/'board.prj',hier,('OLD',),effective_report_path=flat)
|
||||
assert {(row.refdes,row.old_part_number,row.sheet_number) for row in result.components}=={
|
||||
('R1','OLD','MAIN'),('R2','OLD','AUX')}
|
||||
assert len({row.key for row in result.components})==2
|
||||
|
||||
|
||||
def test_plan_target_identity_is_refdes_based_without_rewriting_legacy_uid(tmp_path):
|
||||
old=ComponentTarget(tmp_path/'board.prj','MAIN','1','legacy-u1','R1','OLD','1K')
|
||||
new=ComponentTarget(tmp_path/'board.prj','MAIN','1','changed-u1','R1','OLD','1K')
|
||||
assert old.key==new.key
|
||||
assert old.uid=='legacy-u1'
|
||||
with pytest.raises(Exception,match='more than once|重复'):
|
||||
compile_target_rules((TargetPlanRow(old,True,'NEW'),TargetPlanRow(new,True,'NEW')))
|
||||
|
||||
|
||||
def test_blank_uid_can_be_represented_by_a_reference_target(tmp_path):
|
||||
target=ComponentTarget(tmp_path/'board.prj','MAIN','1','','R1','OLD','1K')
|
||||
assert target.uid=='@REFDES|R1'
|
||||
|
||||
|
||||
def graph(uid):
|
||||
return {'status':'success','complete':True,'graph':{
|
||||
'components':{uid:{'refdes':'R1','attributes':{'Part Number':'OLD'},
|
||||
'symbol':'res.1 type Primitive ForwardPCB',
|
||||
'pins':{'p1':{'number':'1','nets':['GND']}}}},
|
||||
'nets':[{'name':'GND','endpoints':[[uid,'p1']]}]}}
|
||||
|
||||
|
||||
def test_uid_change_alone_does_not_change_netlist_acceptance():
|
||||
before,after=graph('$1I1'),graph('$2I2')
|
||||
result=native_netlist.compare_netlists(before,after)
|
||||
assert result['status']=='verified',result
|
||||
assert '$1I1' in before['graph']['components'] # comparisons do not mutate evidence
|
||||
|
||||
|
||||
def test_reference_mapping_does_not_hide_real_pin_net_changes():
|
||||
before,after=graph('$1I1'),graph('$2I2')
|
||||
after['graph']['components']['$2I2']['pins']['p1']['nets']=['VCC']
|
||||
after['graph']['nets']=[{'name':'VCC','endpoints':[['$2I2','p1']]}]
|
||||
result=native_netlist.compare_netlists(before,after)
|
||||
assert result['status']=='different'
|
||||
|
||||
|
||||
def test_native_readback_can_reassociate_changed_uid_by_unique_refdes(monkeypatch):
|
||||
from dx_productivity.trial_task_center import native_helper
|
||||
components=[SimpleNamespace(UID='new-uid',Refdes='R1')]
|
||||
values={'Part Number':'NEW','Part Name':'LIB:NEW','Part Label':'LABEL'}
|
||||
monkeypatch.setattr(native_helper,'attribute_value',lambda component,name,**options:values.get(name,''))
|
||||
records=[['C','old-uid','R1','0','0','0','1','']]
|
||||
assert native_helper.replacement_matches(components,records,'NEW',True,
|
||||
library_row=['','NEW','','','LIB:NEW','LABEL'])==['old-uid']
|
||||
|
||||
|
||||
def test_native_snapshot_scope_binds_legacy_keys_by_reference_and_resets(tmp_path):
|
||||
from dx_productivity.trial_task_center import native_helper
|
||||
snapshot=tmp_path/'snapshot.tsv'
|
||||
snapshot.write_text('M\tjob\tproject.prj\tOLD\tNEW\tMAIN:1\n'
|
||||
'C\tlegacy\tR1\t0\t0\t0\t1\t\n')
|
||||
component=SimpleNamespace(UID='changed',Refdes='R1')
|
||||
with native_helper.snapshot_identity_scope(snapshot):
|
||||
assert native_helper._component_uid(component)=='legacy'
|
||||
assert native_helper._component_uid(component)=='changed'
|
||||
|
||||
|
||||
def test_complete_ascii_graph_does_not_require_numeric_uid(tmp_path):
|
||||
from test_native_cli_netlist import ascii_report
|
||||
ascii_path,keyin=tmp_path/'flat.txt',tmp_path/'design.kyn'
|
||||
ascii_path.write_text(ascii_report().replace('$1I1','broken-uid'))
|
||||
keyin.write_text('%net\n%page=board\n\\N1\\ \\R1\\-\\1\\\n%Part\n\\OLD\\ \\R1\\\n')
|
||||
result=native_netlist.read_graph(ascii_path,keyin)
|
||||
assert {row['refdes'] for row in result['components'].values()}=={'R1'}
|
||||
|
||||
|
||||
def test_vbs_selector_can_resolve_a_changed_uid_by_reference(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
output=guards.run_vbs(tmp_path,['QPComponentIdentityKey'],'''
|
||||
Dim Comp
|
||||
Set Comp=New FakeComponent
|
||||
Set QP_ReferenceAliases=CreateObject("Scripting.Dictionary")
|
||||
QP_ReferenceAliases.Add "main:1|r1","legacy-id"
|
||||
WScript.Echo QPComponentIdentityKey(Comp)
|
||||
''',prelude='''
|
||||
Dim QP_ReferenceAliases
|
||||
Class FakeComponent
|
||||
Public UID,Refdes
|
||||
Private Sub Class_Initialize()
|
||||
UID="changed-id":Refdes="R1"
|
||||
End Sub
|
||||
End Class
|
||||
Function QPSheetIdentity()
|
||||
QPSheetIdentity="MAIN:1"
|
||||
End Function
|
||||
''')
|
||||
assert output=='legacy-id'
|
||||
|
||||
|
||||
def test_netlist_plan_can_bind_reference_selector_to_original_graph_uid():
|
||||
from dx_productivity.trial_task_center.netlist_verification import _reference_plan
|
||||
plan={'rows':[{'enabled':True,'target':{'uid':'@REFDES|R1','refdes':'R1',
|
||||
'schematic_name':'MAIN','sheet_number':'1','old_value':'1K'},'new_part_number':'NEW'}]}
|
||||
rebound,library=_reference_plan(plan,graph('$1I1'),{'@REFDES|R1':{
|
||||
'refdes':'R1','schematic_name':'MAIN','sheet_number':'1','attributes':{'Part Number':'NEW'}}})
|
||||
assert rebound['rows'][0]['target']['uid']=='$1I1'
|
||||
assert '$1I1' in library
|
||||
assert plan['rows'][0]['target']['uid']=='@REFDES|R1'
|
||||
|
||||
|
||||
def test_reused_uid_cannot_override_a_known_reference_mismatch():
|
||||
from dx_productivity.trial_task_center.netlist_verification import _reference_plan
|
||||
plan={'rows':[{'enabled':True,'target':{'uid':'$1I1','refdes':'R2','old_value':'1K'},'new_part_number':'NEW'}]}
|
||||
with pytest.raises(ValueError,match='位号'):
|
||||
_reference_plan(plan,graph('$1I1'),{})
|
||||
|
||||
|
||||
def test_multiunit_reference_plan_keeps_each_saved_value_with_its_source_unit():
|
||||
from dx_productivity.trial_task_center.netlist_verification import NetlistVerification, _reference_plan
|
||||
before = graph('$1I1')
|
||||
components = before['graph']['components']
|
||||
components['rebound-unit-2'] = deepcopy(components['$1I1'])
|
||||
components['rebound-unit-2'].update(source_uid='$1I2', symbol='res.2 type Primitive ForwardPCB')
|
||||
components['rebound-unit-2']['pins']['p1']['number'] = '2'
|
||||
before['graph']['nets'][0]['endpoints'].append(['rebound-unit-2', 'p1'])
|
||||
target = dict(uid='@REFDES|R1', refdes='R1', old_value='')
|
||||
plan = {'copy_value': False, 'rows': [dict(enabled=True, target=target,
|
||||
new_part_number='NEW', custom_value_enabled=False, sync_nc=False)]}
|
||||
library = {uid: dict(refdes='R1', attributes={'Part Number': 'NEW'}, preserved_value=value)
|
||||
for uid, value in [('$1I1', '10K'), ('$1I2', '20K')]}
|
||||
after = deepcopy(before)
|
||||
for uid, value in [('$1I1', '10K'), ('rebound-unit-2', '20K')]:
|
||||
after['graph']['components'][uid]['attributes'] = {'Part Number': 'NEW', 'Value': value}
|
||||
rows, rebound = _reference_plan(plan, before, library)
|
||||
expected = NetlistVerification._expected(rows, after, rebound)
|
||||
assert native_netlist.compare_netlists(before, after, expected_changes=expected)['verified'] is True
|
||||
|
||||
|
||||
def test_vbs_unselected_duplicate_uids_get_distinct_reference_keys(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
output=guards.run_vbs(tmp_path,['QPComponentIdentityKey','QPBindSheetReferences'],'''
|
||||
Dim Rows,A,B
|
||||
Set Rows=CreateObject("Scripting.Dictionary")
|
||||
Set A=New FakeComponent:A.UID="duplicate":A.Refdes="R2"
|
||||
Set B=New FakeComponent:B.UID="duplicate":B.Refdes="R3"
|
||||
Rows.Add A,True:Rows.Add B,True
|
||||
QPBindSheetReferences Rows
|
||||
WScript.Echo QPComponentIdentityKey(A)
|
||||
WScript.Echo QPComponentIdentityKey(B)
|
||||
''',prelude='''
|
||||
Dim QP_ReferenceAliases
|
||||
Class FakeComponent
|
||||
Public UID,Refdes
|
||||
End Class
|
||||
Function QPSheetIdentity()
|
||||
QPSheetIdentity="MAIN:1"
|
||||
End Function
|
||||
''')
|
||||
assert output.splitlines()==['@REFDES|R2','@REFDES|R3']
|
||||
|
||||
|
||||
def test_inventory_disambiguates_same_reference_by_sheet(tmp_path):
|
||||
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
|
||||
hier.write_text(hier.read_text().replace('Ref Designator R2', 'Ref Designator R1'))
|
||||
flat.write_text(flat.read_text().replace('ref des R1\tpage ', 'ref des R1\tpage MAIN')
|
||||
.replace('ref des R2\tpage ', 'ref des R1\tpage AUX')
|
||||
.replace('Ref Designator R2', 'Ref Designator R1'))
|
||||
result = inventory.read_ascii_discovery(tmp_path/'board.prj', hier, ('OLD',), effective_report_path=flat)
|
||||
assert {(row.refdes, row.sheet_number) for row in result.components} == {('R1','MAIN'),('R1','AUX')}
|
||||
|
||||
|
||||
def test_inventory_groups_legal_slots_as_one_physical_target(tmp_path):
|
||||
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
|
||||
text = hier.read_text()
|
||||
first = text[text.index(' Symbol $27I100'):text.index(' Symbol IN_PORT')]
|
||||
hier.write_text(text.replace(' Symbol IN_PORT', first.replace('$27I100','$27I101').replace('res.1','res.2')+' Symbol IN_PORT'))
|
||||
text = flat.read_text().replace('\tPath \n', '\tPath \n\tSymbol name res.1 type Primitive ForwardPCB\n')
|
||||
first = text[text.index(' Symbol $27I100'):text.index(' Symbol $28I200')]
|
||||
flat.write_text(text+first.replace('$27I100','$27I101').replace('res.1','res.2'))
|
||||
result = inventory.read_ascii_discovery(tmp_path/'board.prj', hier, ('OLD',), effective_report_path=flat)
|
||||
assert [(row.refdes,row.sheet_number) for row in result.components] == [('R1','MAIN'),('R2','AUX')]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('uid', ['broken', '$27I100'])
|
||||
def test_hierarchical_inventory_joins_child_by_reference_despite_uid_damage(tmp_path, uid):
|
||||
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
|
||||
text = hier.read_text().replace('Block disconnected \tRoot', 'Block child ')
|
||||
text = text.replace('Block board \tRoot\n', 'Block board \tRoot\n'
|
||||
'\tChild child.1\tinstance name $9I2\tview unrecognized\n Symbol $9I2\tpage MAIN\n')
|
||||
hier.write_text(text.replace('$28I200',uid))
|
||||
flat.write_text(flat.read_text().replace('$28I200',uid).replace(
|
||||
'ref des R2\tpage \n\tPath \n', 'ref des R2\tpage AUX\n\tPath $9I2\n')
|
||||
.replace('Prop Ref Designator R2','Prop Ref Designator 2R2')
|
||||
+' Symbol IN_PORT\tref des \tpage \n\tPath \n\tSymbol name input.1 type \n')
|
||||
result = inventory.read_ascii_discovery(tmp_path/'board.prj', hier, ('OLD',), expected_root='board', effective_report_path=flat)
|
||||
assert {(row.refdes,row.schematic_name,row.sheet_number) for row in result.components} == {('R1','board','MAIN'),('2R2','child','AUX')}
|
||||
|
||||
|
||||
def test_hierarchy_graph_coverage_uses_reference_when_raw_uid_repeats(tmp_path):
|
||||
from test_native_cli_netlist import ascii_report
|
||||
text = ascii_report()
|
||||
text = text.replace('\t\tPath root\n', '\t\tPath root\n\t\tConn $1I1 pin B page \n\t\tPath root\n')
|
||||
second = text[text.index(' Symbol '):].replace('ref des R1','ref des R2').replace('Ref Designator R1','Ref Designator R2').replace('Pin A','Pin B').replace('num 1','num 2')
|
||||
flat, kyn, hier = tmp_path/'flat.txt',tmp_path/'g.kyn',tmp_path/'hier.txt'
|
||||
flat.write_text(text+second)
|
||||
kyn.write_text('%net\n\\N1\\ \\R1\\-\\1\\ \\R2\\-\\2\\\n%Part\n\\OLD\\ \\R1\\ \\R2\\\n')
|
||||
header = ascii_report().split('Config ')[0].replace('FLAT BLOCKS','HIER BLOCKS')
|
||||
hier.write_text(header+'Block board \tRoot\n Symbol $1I1\tpage 1\n\tProp Ref Designator R1\n'
|
||||
+' Symbol $1I1\tpage 1\n\tProp Ref Designator R2\n')
|
||||
parsed = native_netlist.read_graph(flat,kyn,hierarchy_path=hier,root_block='board')
|
||||
assert {row['source_uid'] for row in parsed['components'].values()} == {'$1I1'}
|
||||
assert {row['refdes'] for row in parsed['components'].values()} == {'R1','R2'}
|
||||
|
||||
|
||||
def test_reference_graph_distinguishes_same_ref_on_different_sheets():
|
||||
before = graph('a')
|
||||
before['graph']['components']['a'].update(source_page='1',symbol='res.1 type Primitive ForwardPCB')
|
||||
second = deepcopy(before['graph']['components']['a'])
|
||||
second['source_page'] = '2'
|
||||
before['graph']['components']['b'] = second
|
||||
before['graph']['nets'][0]['endpoints'].append(['b','p1'])
|
||||
after = deepcopy(before)
|
||||
after['graph']['components']['new-a'] = after['graph']['components'].pop('a')
|
||||
after['graph']['components']['new-b'] = after['graph']['components'].pop('b')
|
||||
after['graph']['nets'][0]['endpoints'] = [['new-a','p1'],['new-b','p1']]
|
||||
assert native_netlist.compare_netlists(before,after)['verified']
|
||||
assert native_netlist.reference_targets(after['graph'],'R1',sheet='2') == ('new-b',)
|
||||
|
||||
|
||||
def test_declared_reference_changes_expand_to_all_legal_symbol_units():
|
||||
before = graph('a')
|
||||
before['graph']['components']['a']['symbol'] = 'old.1 type Primitive ForwardPCB'
|
||||
second = deepcopy(before['graph']['components']['a'])
|
||||
second['symbol'] = 'old.2 type Primitive ForwardPCB'
|
||||
second['pins']['p1']['number'] = '2'
|
||||
before['graph']['components']['b'] = second
|
||||
before['graph']['nets'][0]['endpoints'].append(['b','p1'])
|
||||
after = deepcopy(before)
|
||||
for row in after['graph']['components'].values():
|
||||
row['attributes']['Part Number'] = 'NEW'
|
||||
result = native_netlist.compare_netlists(before,after,expected_changes={'@REFDES|R1':{'Part Number':'NEW'}})
|
||||
assert result['verified'],result
|
||||
after['graph']['components']['b']['attributes']['Part Number'] = 'WRONG'
|
||||
assert not native_netlist.compare_netlists(before,after,expected_changes={'@REFDES|R1':{'Part Number':'NEW'}})['verified']
|
||||
|
||||
|
||||
def test_native_inventory_and_reciprocal_graph_accept_missing_uid(tmp_path):
|
||||
from test_native_cli_netlist import ascii_report
|
||||
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
|
||||
hier.write_text(hier.read_text().replace('$27I100',''))
|
||||
flat.write_text(flat.read_text().replace('$27I100',''))
|
||||
result = inventory.read_ascii_discovery(tmp_path/'board.prj',hier,('OLD',),effective_report_path=flat)
|
||||
assert {row.refdes for row in result.components} == {'R1','R2'}
|
||||
graph_path, keyin = tmp_path/'graph.txt',tmp_path/'graph.kyn'
|
||||
graph_path.write_text(ascii_report().replace('$1I1',''))
|
||||
keyin.write_text('%net\n\\N1\\ \\R1\\-\\1\\\n%Part\n\\OLD\\ \\R1\\\n')
|
||||
parsed = native_netlist.read_graph(graph_path,keyin)
|
||||
assert [row['source_uid'] for row in parsed['components'].values()] == ['']
|
||||
|
||||
|
||||
@pytest.mark.parametrize('damage', ['missing_value', 'missing_reference'])
|
||||
def test_hierarchical_inventory_keeps_required_observation_checks(tmp_path, damage):
|
||||
hier, flat = report(tmp_path), effective_report(tmp_path, part='OLD')
|
||||
text = hier.read_text().replace('Block disconnected \tRoot', 'Block child ')
|
||||
text = text.replace('Block board \tRoot\n', 'Block board \tRoot\n'
|
||||
'\tChild child.1\tinstance name $9I2\tview unrecognized\n Symbol $9I2\tpage MAIN\n')
|
||||
hier.write_text(text.replace('\tProp Ref Designator R2\n','') if damage == 'missing_reference' else text)
|
||||
text = flat.read_text()
|
||||
if damage == 'missing_value':
|
||||
text = text.replace('\tProp Value 20K\n','')
|
||||
flat.write_text(text.replace('ref des R2\tpage \n\tPath \n',
|
||||
'ref des R2\tpage \n\tPath $9I2\n')
|
||||
+' Symbol IN_PORT\tref des \tpage \n\tPath \n\tSymbol name input.1 type \n')
|
||||
with pytest.raises(inventory.NativeInventoryUnsupported):
|
||||
inventory.read_ascii_discovery(tmp_path/'board.prj',hier,('OLD',),expected_root='board',effective_report_path=flat)
|
||||
|
||||
|
||||
def test_six_heterogeneous_units_with_shared_slot_number_compare_by_physical_pins():
|
||||
from pathlib import Path
|
||||
fixtures = Path(__file__).parent/'fixtures/native-complex'
|
||||
parsed = native_netlist.read_graph(fixtures/'six-physical-gates.txt',fixtures/'six-physical-gates.kyn')
|
||||
result = {'status':'success','complete':True,'graph':parsed}
|
||||
assert native_netlist.compare_netlists(result,result)['verified']
|
||||
|
||||
|
||||
@pytest.mark.parametrize('field,actual,wanted', [('source_page','2','1'),('source_schematic','OTHER','MAIN')])
|
||||
def test_reused_uid_cannot_override_known_scope_mismatch(field,actual,wanted):
|
||||
from dx_productivity.trial_task_center.netlist_verification import _reference_plan
|
||||
before=graph('$1I1');before['graph']['components']['$1I1'][field]=actual
|
||||
target={'uid':'$1I1','refdes':'R1', 'sheet_number' if field=='source_page' else 'schematic_name':wanted}
|
||||
with pytest.raises(ValueError):
|
||||
_reference_plan({'rows':[{'enabled':True,'target':target}]},before,{})
|
||||
|
||||
|
||||
def test_snapshot_scope_disambiguates_unselected_raw_uid_collisions(tmp_path):
|
||||
from dx_productivity.trial_task_center import native_helper as helper
|
||||
path=tmp_path/'snapshot.tsv'
|
||||
path.write_text('C\tselected\tR1\t0\t0\t0\t1\t\nU\tduplicate\tR2\tOTHER\tx\tn\tl\nU\tduplicate\tR3\tOTHER\tx\tn\tl\n')
|
||||
with helper.snapshot_identity_scope(path):
|
||||
identities=[helper._component_uid(SimpleNamespace(UID='duplicate',Refdes=ref)) for ref in ('R2','R3')]
|
||||
assert identities==['@REFDES|R2','@REFDES|R3']
|
||||
|
||||
|
||||
def unit(uid,x,part='NEW'):
|
||||
attributes={'Part Number':part,'Part Name':'LIB:NEW','Part Label':'LABEL','Value':'1K'}
|
||||
component=SimpleNamespace(UID=uid,Refdes='U1',Orientation=0,Scale=1,
|
||||
GetLocation=lambda:SimpleNamespace(X=x,Y=0),
|
||||
FindAttribute=lambda name:SimpleNamespace(Value=attributes[name],InstanceValue='') if name in attributes else None)
|
||||
component._oleobj_=component
|
||||
return component
|
||||
|
||||
|
||||
def test_multiunit_readback_keeps_original_unit_keys_after_uid_churn(tmp_path):
|
||||
from dx_productivity.trial_task_center import native_helper as helper
|
||||
path=tmp_path/'snapshot.tsv'
|
||||
path.write_text('C\told1\tU1\t0\t0\t0\t1\t\nC\told2\tU1\t100\t0\t0\t1\t\n')
|
||||
records=[['C','old1','U1','0','0','0','1',''],['C','old2','U1','100','0','0','1','']]
|
||||
with helper.snapshot_identity_scope(path):
|
||||
current=[unit('new2',100),unit('new1',0)]
|
||||
assert [helper._component_uid(component) for component in current]==['old2','old1']
|
||||
assert helper.replacement_matches(current,records,'NEW',True,library_row=['','NEW','','','LIB:NEW','LABEL'])==['old1','old2']
|
||||
assert helper.replacement_matches([unit('new1',0),unit('new2',100,'WRONG')],records,'NEW',True) is None
|
||||
with pytest.raises(ValueError):
|
||||
helper.replacement_matches([unit('new1',0),unit('new2',0)],records,'NEW',True)
|
||||
|
||||
|
||||
def test_vbs_unit_aliases_survive_uid_changes_and_keep_physical_selector(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
output=guards.run_vbs(tmp_path,['QPCommandTargetKey'],'''
|
||||
Dim Rows,A,B,Selectors
|
||||
Set Rows=CreateObject("Scripting.Dictionary")
|
||||
Set A=New FakeComponent:A.UID="old1":A.X=0
|
||||
Set B=New FakeComponent:B.UID="old2":B.X=100
|
||||
Rows.Add A,True:Rows.Add B,True
|
||||
QPBindSheetReferences Rows
|
||||
A.UID="changed1":B.UID="changed2"
|
||||
Set Selectors=CreateObject("Scripting.Dictionary")
|
||||
Selectors.Add "@REFDES|U1",Array("U1","1K")
|
||||
WScript.Echo QPComponentIdentityKey(A)
|
||||
WScript.Echo QPComponentIdentityKey(B)
|
||||
WScript.Echo QPCommandTargetKey(B,Selectors)
|
||||
''',prelude='''
|
||||
Class FakePoint
|
||||
Public X,Y
|
||||
End Class
|
||||
Class FakeComponent
|
||||
Public UID,X,Refdes,Orientation,Scale
|
||||
Private Sub Class_Initialize()
|
||||
Refdes="U1":Orientation=0:Scale=1
|
||||
End Sub
|
||||
Public Function GetLocation()
|
||||
Dim P:Set P=New FakePoint:P.X=X:P.Y=0:Set GetLocation=P
|
||||
End Function
|
||||
End Class
|
||||
Function QPSheetIdentity()
|
||||
QPSheetIdentity="MAIN:1"
|
||||
End Function
|
||||
''')
|
||||
assert output.splitlines()==['old1','old2','@REFDES|U1']
|
||||
|
||||
|
||||
def test_multiunit_value_evidence_covers_physical_selector_without_dropping_units(tmp_path):
|
||||
from test_value_readback import case
|
||||
from dx_productivity.trial_task_center.value_readback import capture_value_readback
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field as enc
|
||||
row, raw=case(tmp_path)
|
||||
row['target_selectors']='@REFDES|U1\x1fU1\x1f10nF'
|
||||
snapshot='C\tu1\tU1\t0\t0\t0\t1\tNO_PINS\r\nC\tu2\tU1\t100\t0\t0\t1\tNO_PINS\r\n'
|
||||
(tmp_path/'snapshot-job.tsv').write_bytes(snapshot.encode())
|
||||
header=['V','2','task','job','saved_ok',row['project_path'],'MAIN','1','OLD','NEW','custom','10nF','1','CELL',row['target_selectors'],snapshot]
|
||||
records=[header,['C','u1','U1','NEW','10nF','1','CELL'],['C','u2','U1','NEW','10nF','1','CELL']]
|
||||
raw.write_text('\n'.join('\t'.join(map(enc,record)) for record in records)+'\n',encoding='utf8')
|
||||
result=capture_value_readback(tmp_path,row,('saved_ok','batch_closed','verified'))
|
||||
assert result['value_readback_status']=='verified',result
|
||||
assert {item['uid'] for item in result['value_readback']}=={'u1','u2'}
|
||||
raw.write_text('\n'.join('\t'.join(map(enc,record)) for record in records[:-1])+'\n',encoding='utf8')
|
||||
assert capture_value_readback(tmp_path,row,('saved_ok','batch_closed','verified'))['value_readback_status']=='rejected'
|
||||
|
||||
|
||||
def test_real_vbs_capture_records_all_units_of_one_physical_selector(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
names=['QPCaptureAndSelect','QPSnapshotComponent','QPCommandTargets','QPReadSnapshot','QPReadUtf8','QPWriteUtf8',
|
||||
'QPSnapshotAttribute','QPSortSnapshotLines','QPOptionalAttributeField','QPFindCurrentAttribute','QPInvalidatePinIndex',
|
||||
'QPBooleanText','QPLogField','QPTargetCell','QPRequireUniqueAttributeNames','QPAttributeObservations']
|
||||
path=tmp_path/'units.tsv'
|
||||
output=guards.run_vbs(tmp_path,names,'''
|
||||
Dim A,B,Attr,Command,Records,Comp
|
||||
Set QP_App=New FakeApp
|
||||
Set A=New FullComponent:A.UID="u1":A.Refdes="U1":A.X=0
|
||||
Set B=New FullComponent:B.UID="u2":B.Refdes="U1":B.X=100
|
||||
For Each Comp In Array(A,B)
|
||||
Set Attr=New FullAttribute:Attr.Name="Part Number":Attr.Value="OLD":Attr.InstanceValue="OLD"
|
||||
Comp.Attributes.Add Attr,True
|
||||
Set Attr=New FullAttribute:Attr.Name="Value":Attr.Value="1K":Attr.InstanceValue="1K"
|
||||
Comp.Attributes.Add Attr,True
|
||||
QP_App.ActiveView.Components.Add Comp,True
|
||||
Next
|
||||
Set Command=CreateObject("Scripting.Dictionary")
|
||||
Command.Add "job_id","test":Command.Add "old_part_number","OLD":Command.Add "new_part_number","NEW"
|
||||
Command.Add "schematic_name","MAIN":Command.Add "sheet_number","1"
|
||||
Command.Add "target_scope","targets":Command.Add "target_selectors",QPEncode("@REFDES|U1" & Chr(31) & "U1" & Chr(31) & "1K")
|
||||
QPCaptureAndSelect Command,"'''+str(path)+'''"
|
||||
Set Records=QPReadSnapshot("'''+str(path)+'''")
|
||||
WScript.Echo Records("components").Count
|
||||
WScript.Echo CapturedState
|
||||
''',prelude=guards.LOG_STUB+guards.FULL_FAKE+'''
|
||||
Dim CapturedState,QP_SavedNetlistMode
|
||||
Sub QPBuildComponentIndex(ByRef Components)
|
||||
End Sub
|
||||
Function QPComponentPins(ByVal Uid)
|
||||
Err.Raise 59001,"Forbidden","pin query"
|
||||
End Function
|
||||
Function QPUseNativeSnapshot()
|
||||
Err.Raise 59002,"Forbidden","native CAD read"
|
||||
End Function
|
||||
Function QPCurrentSemanticState()
|
||||
Err.Raise 59003,"Forbidden","whole page"
|
||||
End Function
|
||||
Sub QPVerifyUntouched(ByRef Records)
|
||||
Err.Raise 59004,"Forbidden","unselected validation"
|
||||
End Sub
|
||||
Function QPValueMode(ByRef Command)
|
||||
QPValueMode="preserve"
|
||||
End Function
|
||||
Function QPCustomValue(ByRef Command)
|
||||
QPCustomValue=""
|
||||
End Function
|
||||
Function QPSyncNc(ByRef Command)
|
||||
QPSyncNc=False
|
||||
End Function
|
||||
Sub QPWriteStatus(ByVal Root,ByVal Job,ByVal State,ByVal Code,ByVal Message)
|
||||
CapturedState=State
|
||||
End Sub
|
||||
''')
|
||||
assert output.splitlines()==['2','awaiting_native_replace']
|
||||
rows=[line.split('\t') for line in path.read_text(encoding='utf-8-sig').splitlines() if line]
|
||||
assert [row for row in rows if row[0]=='X']==[['X','saved_netlist_v1']]
|
||||
assert [row for row in rows if row[0]=='C']==[
|
||||
['C','u1','U1','0','0','0','1','NETLIST'],
|
||||
['C','u2','U1','100','0','0','1','NETLIST']]
|
||||
assert {(row[1],row[2],row[3],row[4]) for row in rows if row[0]=='A'}=={
|
||||
(uid,name,value,value) for uid in ('u1','u2') for name,value in (('Part Number','OLD'),('Value','1K'))}
|
||||
assert not any(row[0] in {'U','G'} for row in rows)
|
||||
|
||||
|
||||
def test_fresh_vbs_recovery_restores_interleaved_unselected_unit_aliases(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field as enc
|
||||
path=tmp_path/'recovery.tsv'
|
||||
text='M\tC:\\board.prj\tOLD\tNEW\tMAIN:1\r\nC\tselected\tR1\t0\t0\t0\t1\t\r\nA\tselected\tValue\t1\t1\t0\t0\t0\t0\t10\t0\t1\r\n'
|
||||
for uid,x in [('unit1','0'),('unit2','100')]:
|
||||
state=f'C\t{uid}\tU1\t{x}\t0\t0\t1\tNO_PINS\r\n'
|
||||
text+=f'U\t{uid}\tU1\tOTHER\tv\tn\tl\r\nG\t{uid}\t{enc(state)}\r\n'
|
||||
path.write_bytes(text.encode())
|
||||
output=guards.run_vbs(tmp_path,['QPReadSnapshot','QPReadUtf8'],'''
|
||||
Dim QP_Fso,Records,A,B
|
||||
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
|
||||
Set Records=QPReadSnapshot("'''+str(path)+'''")
|
||||
Set A=New Unit:A.X=0:Set B=New Unit:B.X=100
|
||||
WScript.Echo QPComponentIdentityKey(A)
|
||||
WScript.Echo QPComponentIdentityKey(B)
|
||||
''',prelude='''
|
||||
Class Point
|
||||
Public X,Y
|
||||
End Class
|
||||
Class Unit
|
||||
Public X
|
||||
Public Property Get UID()
|
||||
UID="changed"
|
||||
End Property
|
||||
Public Property Get Refdes()
|
||||
Refdes="U1"
|
||||
End Property
|
||||
Public Function GetLocation()
|
||||
Dim P:Set P=New Point:P.X=X:P.Y=0:Set GetLocation=P
|
||||
End Function
|
||||
Public Property Get Orientation()
|
||||
Orientation=0
|
||||
End Property
|
||||
Public Property Get Scale()
|
||||
Scale=1
|
||||
End Property
|
||||
End Class
|
||||
Function QPSheetIdentity()
|
||||
QPSheetIdentity="MAIN:1"
|
||||
End Function
|
||||
''')
|
||||
assert output.splitlines()==['unit1','unit2']
|
||||
|
||||
|
||||
def test_all_match_pre_submit_rejects_reused_uid_with_changed_ref(monkeypatch):
|
||||
from dx_productivity.trial_task_center import native_helper as helper
|
||||
component=unit('original',0,'OLD');component.Refdes='R2'
|
||||
app=SimpleNamespace(ActiveView=SimpleNamespace(Query=lambda *args:[component]))
|
||||
monkeypatch.setattr(helper,'check_active_context',lambda *args:None)
|
||||
with pytest.raises(ValueError,match='reference|Reference'):
|
||||
helper.check_context(app,SimpleNamespace(IVdComp=lambda value:value),
|
||||
['M','board','OLD','NEW','MAIN:1','preserve'],[['C','original','R1','0','0','0','1','']],require_selection=False)
|
||||
|
||||
|
||||
def test_vbs_snapshot_does_not_alias_distinct_blank_reference_annotations(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
from dx_productivity.trial_task_center.quick_run_protocol import _encode_vbs_field as enc
|
||||
path=tmp_path/'annotations.tsv'
|
||||
text='M\tC:\\board.prj\tOLD\tNEW\tMAIN:1\r\nC\tselected\tR1\t0\t0\t0\t1\t\r\nA\tselected\tValue\t1\t1\t0\t0\t0\t0\t10\t0\t1\r\n'
|
||||
for uid in ('annotation1','annotation2'):
|
||||
state=f'C\t{uid}\t\t0\t0\t0\t1\tNO_PINS\r\n'
|
||||
text+=f'U\t{uid}\t\t\t\t\t\r\nG\t{uid}\t{enc(state)}\r\n'
|
||||
path.write_bytes(text.encode())
|
||||
output=guards.run_vbs(tmp_path,['QPReadSnapshot','QPReadUtf8'],'''
|
||||
Dim QP_Fso,Records,A,B
|
||||
Set QP_Fso=CreateObject("Scripting.FileSystemObject")
|
||||
Set Records=QPReadSnapshot("'''+str(path)+'''")
|
||||
Set A=New Annotation:A.UID="annotation1"
|
||||
Set B=New Annotation:B.UID="annotation2"
|
||||
WScript.Echo QPComponentIdentityKey(A)
|
||||
WScript.Echo QPComponentIdentityKey(B)
|
||||
WScript.Echo CStr(QP_ReferenceAliases.Exists("main:1|"))
|
||||
''',prelude='''
|
||||
Class Annotation
|
||||
Public UID,Refdes
|
||||
End Class
|
||||
Function QPSheetIdentity()
|
||||
QPSheetIdentity="MAIN:1"
|
||||
End Function
|
||||
''')
|
||||
assert output.splitlines()==['annotation1','annotation2','False']
|
||||
|
||||
|
||||
def test_semantic_capture_reports_each_completed_component_without_changing_baseline(tmp_path):
|
||||
import test_accuracy_vbs_guards as guards
|
||||
output=guards.run_vbs(tmp_path,['QPCurrentSemanticState','QPSortSnapshotLines'],'''
|
||||
Dim QP_App,State
|
||||
Set QP_App=New App
|
||||
State=QPCurrentSemanticState()
|
||||
WScript.Echo Replace(State,vbCrLf,";")
|
||||
''',prelude='''
|
||||
Const VDM_COMP=128
|
||||
Const VD_ALL=0
|
||||
Class App
|
||||
Property Get ActiveView()
|
||||
Set ActiveView=Me
|
||||
End Property
|
||||
Function Query(ByVal Kind,ByVal Scope)
|
||||
Dim Rows:Set Rows=CreateObject("Scripting.Dictionary")
|
||||
Rows.Add "C3",True:Rows.Add "C1",True:Rows.Add "C2",True
|
||||
Set Query=Rows
|
||||
End Function
|
||||
End Class
|
||||
Sub QPInvalidatePinIndex()
|
||||
End Sub
|
||||
Sub QPInvalidateComponentIndex()
|
||||
End Sub
|
||||
Function QPUntouchedFullState(ByVal Comp)
|
||||
QPUntouchedFullState=Comp & vbCrLf
|
||||
End Function
|
||||
Sub QPProgress(ByVal Stage,ByVal Done,ByVal Total)
|
||||
WScript.Echo Stage & ":" & Done & "/" & Total
|
||||
End Sub
|
||||
''')
|
||||
assert output.splitlines()[:3]==['capture_semantic_state:1/3','capture_semantic_state:2/3','capture_semantic_state:3/3']
|
||||
assert output.splitlines()[3]=='S\tfull_sheet_v1;C1;C2;C3;'
|
||||
|
||||
|
||||
def test_semantic_component_progress_refreshes_business_watchdog():
|
||||
from dx_productivity.trial_task_center.worker_monitor import ProgressWatch
|
||||
watch=ProgressWatch('token','job',150,0)
|
||||
for sequence,now in enumerate((100,200,300,400),1):
|
||||
assert watch.accept(f'DX_PROGRESS\ttoken\tjob\t{sequence}\tcapture_semantic_state\t{sequence}\t104\t0\n'.encode(),now)
|
||||
assert watch.action(now+100) is None
|
||||
assert not watch.stop_requested
|
||||
Loaded 100 of 117 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user